NVIDIA’s Nemotron 3.5 Lightning is aimed at a problem that gets more important as AI agents become less like chatbots: one model rarely makes sense for every step of a job.
Sending every agent call to a frontier model is now a choice, not a default.
The company has released Nemotron 3.5 Lightning alongside NeMo Switchyard, an open source routing library for agent tools. The pitch is practical rather than flashy: use a larger model when a task needs it, send simpler work to a faster specialist, and keep the application interface stable while the models underneath change.
Nemotron 3.5 is built for specialized agent work
NVIDIA’s announcement describes Nemotron 3.5 Lightning as a 30-billion-parameter mixture-of-experts model for long-running agentic workloads. It’s not positioned as a universal replacement for every frontier model. Instead, NVIDIA says it’s designed to handle targeted work inside a larger system of models.
That distinction matters. A planning model can decide what an agent should do, while a smaller specialist handles code review, tool use, security alerts, or routine customer questions. In theory, that division lets a team spend expensive inference only where it improves the result.
NVIDIA says Nemotron 3.5 Lightning delivers up to 4x faster output and 30% faster agentic task completion than other models in its class, based on the company’s PinchBench comparisons. Those are vendor-reported results, so I’d treat them as a performance claim to test rather than a guarantee for every workload.
The model is open and customizable, according to NVIDIA. Organizations can post-train it with their own domain data, tools, and workflows using NVIDIA NeMo, which could make it more useful for narrow jobs than a larger general-purpose model that knows less about a company’s actual process.
NVIDIA also says the model can run across local AI systems, workstations, data centers, and the cloud. That deployment range is one of the more interesting parts of the announcement: teams can keep high-volume specialized tasks closer to their own infrastructure instead of sending every request to a hosted frontier model.
NeMo Switchyard puts routing in the stack
The companion NeMo Switchyard release addresses the other half of the problem. Its router chooses a model for each step of an agent workflow based on priorities such as quality, latency, and cost, while letting developers keep their application code largely unchanged.
NVIDIA reports that internal testing reduced task-completion cost to nearly one-third of Opus 4.8 alone while maintaining frontier-level accuracy. That comparison is also an internal benchmark, not an independent evaluation, and the result will depend heavily on the workload, routing policy, and models made available to the router.
The partner examples show how NVIDIA wants Switchyard to be used. The company says LangChain cut cost by 74% across 145 multi-turn Deep Agents tasks by sending only 7% of calls to a frontier model, with a 6% accuracy tradeoff. Ramp, meanwhile, reportedly cut cost by 58% and runtime by 33% in its SWE-Bench testing.
Those numbers sound useful, but they also show the tradeoff plainly. Routing isn’t free magic. A system can lower cost by using a cheaper model more often, yet the quality loss may be acceptable for one task and unacceptable for another. Any serious deployment needs its own task-level evaluation, not just a blended average.
The library is designed to work with existing agent tooling. NVIDIA lists integrations or work with Boomi, Cadence, Cognition, Kong, LangChain, LiteLLM, Ramp, Siemens, and Nous Research. That matters because a router only saves engineering time if it fits the tools a team already uses.
What developers can actually get
NVIDIA says Nemotron 3.5 Lightning is available through Hugging Face, ModelScope, OpenRouter, build.nvidia.com, and NVIDIA NIM, while NeMo Switchyard is available on GitHub.
The announcement doesn’t turn these into one-click production infrastructure, though. Teams still need to choose an inference runtime, check licenses, measure latency, and decide what data can leave their environment.
The training-data disclosure is worth watching too. NVIDIA says it publishes as much of the training data and techniques as licensing permits, and it’s releasing the Nemotron-RL-Agentic-Terminal-Pivot dataset used to post-train the model for coding-agent capabilities.
More transparency can help teams audit and adapt an open model, but the phrase as licensing permits leaves room for important gaps.
For a broader view of where this fits, Tony’s best AI coding agents guide covers the application layer, while the Hermes Agent release coverage shows why configurable agent infrastructure is becoming its own product category.
My read is that Nemotron 3.5 Lightning is more compelling as a component than as a headline model. The useful idea is the combination: a customizable specialist for repetitive agent work, plus a router that can reserve stronger models for the steps that genuinely need them.
If you’re building an agent, I wouldn’t replace your current model stack on the strength of NVIDIA’s benchmarks.
I’d take one recurring workflow, record quality, latency, and cost for each step, then compare a fixed model against a routed setup.
If Switchyard can preserve the decisions that matter while cutting routine inference, Nemotron 3.5 could earn a place in production without pretending that one model solves everything.
