GPUs & Inference · Explainer
What is NVIDIA NVKE?Think of a wedding hall.
NVIDIA released something called a Kubernetes Engine. It isn't what I first thought it was.
When I first saw the name “NVIDIA Kubernetes Engine”, I put it in the wrong box. My first question was whether it's like vLLM, SGLang, Google AX or Agent Substrate. I assumed it was one more tool in that family. I also assumed I'd need a GPU just to try it.
Both guesses were wrong. NVKE doesn't serve models and it doesn't run agents. It makes Kubernetes clusters. And you can try it on one Linux machine with no GPU hardware. [4]
This is the explanation I wanted that day. I'll use one picture all the way through: a wedding-hall complex. Everything about NVKE here comes from NVIDIA's own docs and README. I haven't installed NVKE myself yet.
01 / The wrong box
Is NVKE another tool like vLLM?
No. Here's what the tools I mixed it up with actually do.
vLLM is a library for LLM inference and serving. [6] SGLang is an inference framework for large language models. [7] You give them a model, and they answer requests with it.
Agent Substrate is a runtime for sandboxed agents, and it runs on standard Kubernetes. [9] Google AX is an orchestrator for agent workloads that runs on top of Agent Substrate. [8]
Their jobs are serving models and running agents. NVKE's job is different. It creates the Kubernetes clusters, and then looks after them. [1]
Why does that matter? NVIDIA's docs say AI factories use Kubernetes to operate their GPU fleets, and as the foundation for services like model endpoints. [1] So NVKE sits below the tools I confused it with. It makes the cluster they run on.
Figure 1 / Where NVKE sits
Tap a floor to see its job.
NVKE: the operator asks for a cluster, and NVKE provisions the nodes, forms the cluster and keeps operating it. It doesn't serve models or run agents.
- In the wedding hall
- The caterer and the band work inside a hall. Neither of them gets the hall ready.
- In the system
- vLLM and AX are workloads that can run inside a cluster. NVKE gets the cluster ready.
02 / In one sentence
So what is NVKE?
NVKE is open-source software that a company with its own NVIDIA GPU machines installs, so it can create and run Kubernetes clusters for its customers. [1]
NVIDIA calls that company an AI factory operator. The docs name NVIDIA Cloud Partners as one example. [1]
If you've used GKE or EKS, you already know this idea from the customer's side. Both are managed Kubernetes services. [10] [11] You ask for a cluster, and you get one.
NVKE is for the people on the other side of that counter. The project's README describes it as a way to run a managed Kubernetes service on your own NVIDIA GPU infrastructure. [2]
Now the wedding hall. An owner has land and several halls. Families want a hall for a wedding. They don't want to build one. Between the two sits a booking office, with staff who prepare a hall for each family. NVKE is that booking office and its staff, given to the owner as open-source software to set up.
Three facts to keep with you:
- It's open source. NVKE is licensed under Apache 2.0 and accepts outside contributions. [1]
- It's a preview. NVIDIA says APIs, configuration and behavior can change between releases, and it doesn't recommend NVKE for production workloads. [1]
- It's ordinary Kubernetes underneath. NVKE uses upstream Kubernetes, kubeadm and Cluster API, and the CNCF has certified that the clusters it creates are conformant. [1] The README says NVKE is not a Kubernetes distribution and not a Cluster API flavour. [2]
- In the wedding hall
- The owner has the halls. The booking office turns them into a service.
- In the system
- The operator has the GPU machines. NVKE turns them into a Kubernetes service.
03 / Making a cluster
What happens when a customer needs a cluster?
First you need to know that an NVKE installation has two kinds of cluster. [2]
- The management cluster runs the NVKE management plane: the API and the controllers. [1] Think of it as the booking office building.
- Workload clusters run the customers' applications. [2] These are the halls.
The operator sets up the management cluster first, with a command-line tool called
nvkectl. It starts a temporary seed cluster, uses it to form the management
cluster, and deletes the seed when the bootstrap succeeds.
[1]
Once the management plane is installed and connected to NICo, a new customer cluster is made in five steps.
Figure 2 / From request to kubeconfig
Follow one cluster being made.
- 1 · AskThe operator requests a cluster through the NVKE API or nvkectl.
- 2 · Get machinesThe management plane asks NICo for machines, through Cluster API.
- 3 · Prepare themNICo provisions each machine from the operator's node OS image.
- 4 · Form the clusternvke-agent on each machine runs kubeadm and installs the components.
- 5 · Hand overThe customer receives a kubeconfig and uses kubectl with their cluster.
Two names in that picture need a line each. NICo is NVIDIA Infra Controller. It looks after the physical side: finding the machines, updating their firmware, setting up their networks and provisioning them. [1] nvke-agent is NVKE's program that runs on each node. [1]
This is the bit that's easy to miss. NVKE doesn't manage the bare metal itself. It asks NICo for machines, and NICo does that work. [1]
There's also a check at the door. On bare metal, a node has to prove its identity first. It does that with its TPM, through SPIFFE/SPIRE, and only then gets the credential it needs to join. [1] In hall terms, a guard checks every staff member's ID before letting them in.
And each workload cluster has its own control plane, its own etcd and its own node pools. [1] Every family gets its own hall, with its own manager and its own register.
The work doesn't stop at handover. NVKE also drives scaling node pools, upgrading Kubernetes, replacing nodes and renewing certificates. [1] With NVSentinel installed, NVKE reboots or replaces the nodes that NVSentinel reports as faulty. [1] In the preview, some of this still needs manual steps. [2]
- In the wedding hall
- You book at the office. The staff prepare your hall and hand you the key.
- In the system
- The operator calls the NVKE API. NVKE forms the cluster, and the customer receives a kubeconfig.
04 / What the owner gains
Why would a GPU owner want this?
Because GPU machines alone aren't a product. To offer Kubernetes as a managed service, an operator has to build a whole cluster management system. That means an API to request clusters, a separate control plane for each customer, upgrades, replacing broken nodes, checking each node's identity, and monitoring. [1]
NVIDIA's point is that, without NVKE, the operator has to put that system together from separate open-source and commercial parts, and then keep it working as Kubernetes and GPUs change. [1] NVKE is that system, already built and open source. [1]
That's the main benefit, and it's a big one. A hall owner doesn't have to design a booking system from nothing. The software already exists.
The software doesn't run the business, though. The software comes from NVIDIA. Running the service is the operator's job. [1] Here's a shortened version of how the docs split the work. [1]
| Who | Looks after |
|---|---|
| NVKE (the software) | Provisioning nodes, forming clusters, node identity, upgrades, node replacement and certificate renewal. |
| The operator | The infrastructure and NICo, the management cluster, building and publishing the artifacts, the accelerated runtime, cluster availability, and the access customers receive. |
| The customer | Their workloads, images, namespaces and application secrets, and access rules for their own users. |
NVIDIA is also clear about what NVKE doesn't do. [1]
- It doesn't manage bare metal. That's NICo.
- It doesn't ship OS images or Kubernetes binaries. The operator builds and hosts them, with a companion project called NVKR or from the NVKE repository.
- It doesn't run the customers' workloads. The customer owns what runs in the cluster.
One last piece of context. NVKE is one layer of a bigger NVIDIA stack for AI factories called DSX OS. It's the layer that manages Kubernetes. [1] The accelerated runtime, such as the GPU Operator, comes from a separate project called AICR. The operator installs it into each workload cluster. [1]
- In the wedding hall
- You get the booking system. You still own and run the halls.
- In the system
- NVIDIA supplies the software. The operator runs the service.
05 / Where to be careful
What isn't ready yet?
Quite a lot, and NVIDIA lists it openly. These are the limits I'd read first. [3]
- The management cluster doesn't manage itself. It doesn't upgrade, scale or replace its own nodes.
- No management-plane upgrade and no backup. There's no procedure to move the management plane to a later NVKE release, and no procedure to back up or restore it.
- The API starts unprotected. After install, the NVKE API serves plain HTTP with authentication off. The operator has to turn authentication on, admit only the networks that need the API, and put TLS in front of it.
- No authorization. Every caller that authentication admits can call every endpoint, including the ones that create and delete workload clusters.
- One key per hall. Customers get only the admin kubeconfig, and one that has been handed over can't be revoked. It stays valid until its client certificate expires.
My view: read NVKE to learn how this layer works, and test it. I wouldn't put customers on it yet. NVIDIA says the same thing in its own words. [1]
06 / Should you care?
Does this matter if you don't own GPUs?
If you don't own GPU machines, you won't install NVKE. When I need a cluster on Google Cloud or AWS, I'd use GKE or EKS. They're already managed for me.
I'd still spend time on it. I think more companies are going to run their own GPU datacenters, and they'll need engineers who understand this layer. Reading how NVKE splits the job between the management cluster, NICo and the node agent is a cheap way to learn it.
And you don't need a GPU to look around. NVIDIA has a quickstart that runs NVKE on one Linux machine, with no GPU hardware and no NVIDIA account. [4] It replaces NICo with mock-NICo, a simulator that starts each workload cluster node as a Docker container. [4]
The machine needs at least 6 CPUs, 12 GiB of free memory and 100 GiB of free disk, plus Docker and the tools the quickstart lists. [4] The first step is to get the code:
git clone https://github.com/dsx-ai-factory/nvke.git
cd nvke
From there, follow NVIDIA's quickstart page. [4] I haven't run it yet, so I won't paste steps I haven't tested. One warning from that page: the example passwords and the simulated NICo are for local evaluation only. [4]
- In the wedding hall
- You may never own a hall. Knowing how a good one runs still helps you.
- In the system
- You may never install NVKE. Knowing this layer still makes you a better engineer.
Keep these names
A small glossary
- AI factory operator
- NVIDIA's name for the company that owns the GPU infrastructure and installs NVKE to offer Kubernetes to its customers. [1]
- Management cluster
- The Kubernetes cluster that runs the NVKE management plane. [2]
- Workload cluster
- A cluster NVKE creates for a customer's applications, with its own control plane, etcd and node pools. [1]
- Control plane
- The components that manage the overall state of a Kubernetes cluster. [12]
- etcd
- The key-value store that holds all of the Kubernetes API server's data. [12]
- Node pool
- A group of nodes that share an instance type, a release and a Kubernetes version. [1]
- NICo
- NVIDIA Infra Controller. It handles the bare-metal machines and their networks. [1]
- nvke-agent
- The NVKE node agent. On each machine it runs kubeadm and installs the cluster's components. [1]
- kubeadm
-
A Kubernetes tool for creating clusters, built around
kubeadm initandkubeadm join. [13] - Cluster API
- A Kubernetes sub-project that provides declarative APIs and tooling for provisioning, upgrading and operating multiple Kubernetes clusters. [14]
- kubeconfig
-
The file
kubectluses to find a cluster and talk to its API server. [15]
Check the concept
Can you explain it back?
1. Does NVKE serve models, like vLLM does?
No. NVKE creates and operates Kubernetes clusters. Model serving is a workload that runs in a cluster.
2. Who installs NVKE: the customer or the GPU owner?
The GPU owner, which NVIDIA calls the AI factory operator. The customer only receives a cluster.
3. Two customers use the same operator. Do they share a control plane?
No. Each workload cluster has its own control plane, etcd and node pools.
4. Which part prepares the physical machines?
NICo. NVKE asks NICo for machines and then forms the cluster on them.
5. Do you need a GPU to try NVKE?
No. The quickstart runs on one Linux machine and simulates NICo, with each node started as a Docker container.
Continue the thread
Now look one floor up.
NVKE gives you the cluster. These articles cover what runs in one: how inference works with vLLM, what Agent Substrate is, and how Google AX works with it.
Read further
Sources
NVKE references use the dsx-ai-factory/nvke repository at commit
487f511 (10 November 2026). Sources accessed 11 November 2026.
- NVIDIA NVKE docs: What is NVKE. Also published at docs.nvidia.com.
- NVIDIA NVKE: project README.
- NVIDIA NVKE docs: Preview status and known limitations.
- NVIDIA NVKE docs: Quickstart, evaluate locally with mock-NICo.
- NVIDIA NVKE: product-family diagram (Apache 2.0, copyright NVIDIA Corporation and affiliates). Figure 2 is redrawn from it.
- vLLM: project README.
- SGLang: project README.
- Google AX: project README.
- Agent Substrate: project README.
- Google Cloud: GKE overview.
- AWS: What is Amazon EKS?
- Kubernetes documentation: Kubernetes components.
- Kubernetes documentation: kubeadm.
- The Cluster API Book: introduction.
- Kubernetes documentation: Organizing cluster access using kubeconfig files.