NVIDIA just shipped a small open model aimed at a boring-sounding job, and it might matter more than the next frontier flagship. Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model with 3 billion active parameters, built for the high-volume execution layer of always-on AI agents.
The idea is a system of models. A frontier reasoning model like Nemotron 3 Ultra or GPT-5.6 does the planning, while a smaller, cheaper model like Lightning handles the routine work that eats most of an agent’s token budget: tool calls, result validation, subagent delegation.
The best AI coding agents of 2026 all burn tokens on exactly this kind of busywork, and the cost is real. The more capable the agent, the more of its runtime goes to execution tasks nobody headlines, which is precisely the gap NVIDIA is trying to fill.
Why an open model for the agent execution layer
Most of what a long-running agent does is high-volume execution, not deep reasoning. Using a frontier model for every step adds latency and cost. NVIDIA’s answer is a model trained for harnesses like OpenClaw and Hermes Agent, with speculative decoding and harness-optimized training baked in.
The result is speed without the usual accuracy tradeoff. NVIDIA says Nemotron 3.5 Lightning delivers up to 4x the output speed of similar-sized models, and it wins the accuracy-versus-speed Pareto frontier on the Artificial Analysis Intelligence Index.
On PinchBench it reaches 86 percent accuracy while completing 10,000 tasks 30 percent faster than Qwen3.6 35B at similar accuracy.
That kind of claim always deserves a skeptical look, since it comes straight from the vendor, but the direction it points is what matters here: a small model that does the grunt work faster, not a big one that does it slightly smarter.
The pitch only works if the execution layer stays accurate enough that you don’t notice the swap, and that is exactly the tradeoff NVIDIA is optimizing for with this release.
NeMo Switchyard decides which model runs what
The model alone is only half the story. NVIDIA also released NeMo Switchyard, an open source library that routes each request to the most capable and efficient model for the job. Plans route up to the frontier; execution routes down to Lightning.
That routing is where the cost savings show up. NVIDIA’s internal benchmarks say Switchyard maintains frontier-level accuracy while cutting task completion cost to nearly one-third of Opus 4.8 alone.
Partners report similar wins, including a 74 percent lower cost for LangChain on a 145-task Deep Agents batch and a 58 percent cost cut for Ramp in SWE-Bench. Those are still vendor-attributed numbers, but the pattern is consistent: routing work to a cheaper model for the easy stuff is where the savings live.
Local AI gets a real option
Because Lightning is small, it runs where a frontier model can’t. NVIDIA says it works on local systems including DGX Spark, GeForce RTX 5090, and Jetson, and you can run it with the usual tooling: LM Studio, llama.cpp, Ollama, and Unsloth.
This matters for privacy and cost. OpenAI just cut prices across its GPT-5.6 lineup, and the broader open-weight wave keeps pushing what a small open model can do. Lightning is NVIDIA’s bid to own the cheap execution layer instead of renting a frontier API for every task.
The local story is the sleeper here. When a model fits on hardware you already own, the marginal cost of an agent tick approaches zero, which changes how casually you can let agents run. That is a bigger deal for hobbyists and small teams than for anyone renting capacity by the token.
The play underneath the release
Here’s what I see. NVIDIA doesn’t need Lightning to be the best open model in the world. It needs Lightning to be the model that runs cheaply and fast on NVIDIA silicon, inside a routing stack NVIDIA controls. The open weights are the wedge, exactly like the open-weight strategy behind Qwen3.8-Max.
The partner list is telling. Nous Research integrated Switchyard into Hermes, and LangChain, LiteLLM, Cognition, and a string of cloud providers are already routing through it. That is NVIDIA building the default plumbing for agentic workloads before the category settles.
Bottom line
Nemotron Lightning is not a frontier model, and it isn’t trying to be. It is an open model built for the unglamorous work that makes agents affordable, paired with a router that decides when to use it. The benchmark numbers are NVIDIA’s own, so treat them as vendor claims until independent runs confirm them.
The direction, though, is clear. Agents are getting cheaper to run not because frontier models got cheaper, but because developers now have a reason to route work to a small model. NVIDIA is betting it owns that execution layer, and it just shipped the default cheap option.
