Netflix has published details of the internal platform it uses to run large language models in production, according to a report from InfoQ carried on Google News.
The report describes the system as an in-house LLM serving platform built around two named components: Triton and vLLM. Both are established pieces of AI infrastructure tooling, and Netflix's contribution is the layer it wrapped around them — a company-specific platform for actually delivering model responses to applications at scale.
That distinction is the heart of the story. "Serving" is the unglamorous half of AI: not training a model, but answering requests from it quickly, reliably, and cheaply, thousands of times a second. It is where GPU costs pile up and where latency becomes something users can feel. Companies operating at Netflix's scale increasingly conclude that off-the-shelf hosted AI services do not give them enough control over performance and spend, and build their own plumbing instead.
The available source is a single summary item, so the specifics — which Netflix products the platform powers, what it costs to run, how it performs — are not detailed here. InfoQ's write-up is the primary account of the architecture.
Why it matters: when a company of Netflix's size decides the AI infrastructure it can buy is not good enough and builds its own serving stack on open components, it is a signal that running AI, not just making it, has become a core engineering discipline.