For a while, the standard AI-agent architecture has been very simple: take the smartest cloud model you can afford, give it tools, and hope your finance team doesn’t notice how many “quick little steps” your agent takes before it finally files a ticket.
NVIDIA’s new combo of Nemotron 3.5 Lightning and NeMo Switchyard is aimed directly at that problem.
Lightning is a 30B-parameter mixture-of-experts model with 3B active parameters, designed to be customized and run locally across NVIDIA hardware. Switchyard is the companion traffic controller: an open-source routing layer that can decide which model handles each step of an agent workflow.
Neither product is trying to convince the world that one model will win every task. In fact, quite the opposite.
NVIDIA’s actual bet is the next generation of serious agents will use a portfolio: a fast, private model for routine work; a specialized model where it has an edge; and a frontier model only when the job truly earns its expensive taste.
The model is not “small.” It's strategically small.
Lightning still has 30B total parameters, so it is not a tiny model you can casually toss onto a phone. Those weights still affect storage and memory requirements. But it activates only a fraction of the model on each token, which is the mixture-of-experts trick: less per-token computation without reducing the total pool of learned capability quite so brutally.
That makes Lightning a plausible default worker for agent systems that need to run constantly: document triage, retrieval follow-ups, tool selection, extraction, structured outputs, support actions, internal knowledge work, and the dozens of small calls that pile up inside a multi-step workflow.
The model is available in optimized variants on Hugging Face, and NVIDIA is explicitly positioning it for deployment across DGX Spark, DGX Station, RTX PRO, and RTX systems. In plain English: it wants organizations to keep more routine agent work close to their data, their hardware, and their control plane.
This fits the strategy NVIDIA has been telegraphing across the broader Nemotron family. As we wrote in our Nemotron 3 Ultra review, the company’s real argument is about intelligence per dollar, per second, per workflow; not winning a single chatbot beauty contest.
Lightning is the “do the normal work all day” member of that family.
Its closest rival has basically the same nametag
The most direct competitive comparison is Alibaba’s Qwen3-30B-A3B. It, too, is a 30B MoE model with 3B active parameters, open weights, and broad local deployment support.
That is not a coincidence. This is becoming a meaningful category: models large enough to be useful in agent loops, but efficient enough that you can consider making them the default rather than the emergency credit-card expense.
Google’s Gemma models and Mistral’s smaller and coding-focused offerings are also relevant alternatives. Gemma’s stack is especially interesting for teams that want tiny local models to execute routine actions and larger models to tackle harder work. Mistral’s Devstral line matters for teams whose agents live mostly in codebases, terminals, and developer workflows.
But NVIDIA’s advantage is the surrounding machine.
If you already run NVIDIA infrastructure, Lightning arrives with a relatively coherent story: hardware, optimized inference, model customization, deployment tooling, and agent software in one ecosystem. The pitch is less compelling if you want a radically hardware-neutral stack. It is much more compelling if you are building an internal agent program and would rather not stitch together six vendors.
Switchyard is the more consequential release
Lightning is the worker. Switchyard is the manager who decides whether the worker is qualified for the assignment.
Switchyard sits between an agent and its model backends. It can preserve the agent’s native API shape, like OpenAI Chat, OpenAI Responses, or Anthropic Messages, while routing requests across local and hosted backends. It can use fixed splits, classifier-driven routing, or stage-aware routing, and it can run as a local proxy, a shared HTTP service, or embedded Python.
NVIDIA’s Switchyard documentation is refreshingly clear about the intended use: different requests, or different moments within one request, deserve different model choices.
That matters because agents do not make one clean inference call. A decent agent might plan, search, summarize, inspect a result, call a tool, validate the answer, and write a final response. Sending every step to a frontier reasoning model is like hiring a partner at a law firm to alphabetize receipts.
Switchyard’s ideal workflow looks something like this:
- Lightning handles routine retrieval, classification, extraction, and tool calls locally.
- A stronger model handles difficult planning, complex code, ambiguous exceptions, or high-risk reasoning.
- A verifier checks whether the low-cost answer is actually good enough.
- The app keeps working even if a provider is unavailable or a team changes models.
It's basically a manager for a smart AI model policy that will likely save you money in short order.
That's a more durable architecture than choosing one favorite model and building your whole product around its current API, price sheet, and good mood.
But the routing market is already crowded
Switchyard is entering a competitive space, not creating one from scratch.
The open-source side includes LiteLLM, the Swiss Army knife of provider normalization, fallbacks, load balancing, logging, and routing. RouteLLM is an especially close conceptual competitor because it focuses on routing easier prompts to cheaper models while preserving quality on harder work.
The key question is: where does the routing intelligence live, and how much control does the buyer retain?
Microsoft and Amazon are strong choices for teams already committed solely to their clouds.
Switchyard’s opening is the hybrid environment: teams that want to mix local NVIDIA-backed models, self-hosted endpoints, and closed APIs without making any one provider the permanent center of gravity.
That's a powerful option, and here's why.
Where this stack is the right answer
The Nemotron-Lightning-plus-Switchyard pairing makes the most sense for organizations that have three things:
- Lots of agent calls. Cost savings are boring until an agent workflow turns one user request into 20 model calls. Then boring becomes budget.
- Sensitive or proprietary data. Keeping routine work local is useful in regulated, sovereign, internal-knowledge, and operational settings, even if some tasks still escalate to cloud models.
- A willingness to evaluate. Routing is only as good as the policy and test suite behind it. If the system cannot tell when a local model failed, it will efficiently deliver incorrect answers at an excellent price.
That third point is the caveat. Model routing is not magic. You need task-level evaluations, clear escalation rules, logging, and some humility about what should never be delegated to the cheap tier.
High-stakes legal, medical, financial, or irreversible operational decisions need much stricter controls than “the router thought this prompt looked easy.”
Still, the overall direction is right.
The market is moving away from the fantasy of one all-powerful model and toward systems that resemble an actual organization: lower-cost workers handle the repeatable stuff, specialists tackle the tricky bits, and expensive expertise gets involved when it matters.
NVIDIA is trying to own the local default in that organization.
Lightning gives teams a credible worker model. Switchyard gives them a way to avoid marrying it, or any other model, forever. And that combination may be more useful than another claim to the top of a leaderboard.
Related Reading
- NVIDIA’s Nemotron 3 Ultra is fast, open, and surprisingly good at weird instructions — A look at the higher-capability reasoning side of NVIDIA’s growing agent-model strategy.
- NVIDIA’s Nemotron 3 Nano Omni gives AI agents eyes and ears — How NVIDIA is framing efficient, specialized models as sub-agents rather than standalone chatbots.
- NVIDIA’s open model super panel made a strong case for open agents — The broader case for open models, agent harnesses, and multi-model AI systems.
