Contents
A Purpose-Built Executor That Wakes 3 Billion Parameters per Token
Released as an open model, Nemotron 3.5 Lightning features a hybrid mixture-of-experts (MoE) architecture that activates 3 billion of its 30 billion total parameters per token ¹³. Distilled from Nemotron 3 Ultra, the model was developed in collaboration with the Nemotron Coalition, whose members contributed evaluation methodologies, datasets, and inference software ¹³. It supports context windows of up to 1 million tokens and ships in BF16 and NVFP4 checkpoints, with deployment targets that NVIDIA says span edge devices like Jetson through to H100 and B300 datacenter GPUs ³. Open-source inference engine vLLM announced day-0 support, serving the model through an OpenAI-compatible API ³.
A Routing Layer That Segments Requests by Cost Tier
NeMo Switchyard, released alongside the model, is an open-source routing library that runs directly inside popular agent tools. It intelligently directs each request to the most suitable model across a developer’s own mix of open, proprietary, and NVIDIA models — without requiring application rewrites ¹². The design rests on the idea that agent architectures are evolving into multi-model systems divided by role: frontier reasoning models such as Nemotron 3 Ultra or GPT-5.6 plan and orchestrate workflows, while compact executors like Lightning handle numerous, well-scoped steps — including code review, tool use, security alert monitoring, and billing questions ¹². NVIDIA says companies including CrowdStrike, Harvey, and CodeRabbit are already customizing the model for their own domains ¹.
4x Speed Claims and a Verification Gap
NVIDIA reports up to 4x higher output speed than similarly sized models, achieved through speculative decoding, multi-token prediction, and NVFP4/BF16 quantization, and claims this translates into 30% faster agentic task completion versus models in its class ¹². Those figures come directly from the vendor’s own PinchBench measurements. Independent third-party latency and quality benchmarks for long-running agent workloads have not yet been published, and the hardware conditions behind the 4x figure remain to be verified externally ¹².
Sources
- ¹ · NVIDIA Nemotron 3.5 Lightning and NeMo Switchyard Deliver Faster, Smarter, More Efficient Agentic AI | NVIDIA Blog
- ² · NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents | NVIDIA Technical Blog
- ³ · Announcing Day-0 Support for NVIDIA Nemotron 3.5 Lightning on vLLM | vLLM Blog