NVIDIA Nemotron 3.5 Lightning: Compact MoE Model for Agentic Execution

ArticleChris Alexiuk and Chintan PatelAnnouncement

NVIDIA has announced Nemotron 3.5 Lightning, an open 30B mixture-of-experts (MoE) model featuring 3B active parameters. It is tailored for the high-volume execution layer of long-running AI agents, targeting routine tasks such as tool calling and result validation with low latency.

Key points

  1. MoE design with 30B total parameters and 3B active parameters per token, balancing capacity and compute cost.
  2. Features speculative decoding via multi-token prediction (MTP) baked in during pretraining, alongside DSpark and DFlash draft models.
  3. Released under the OpenMDW-1.1 licence with weights, training data, and recipes in BF16 and NVFP4 quantized formats.
  4. Deployable across local setups (RTX 5090, DGX Spark) and data centres (NVIDIA Ampere, Hopper, Blackwell).
  5. Scores 86% accuracy on PinchBench while finishing 10,000 tasks 30% faster than Qwen3.6 35B.
  6. Integrates with NeMo Switchyard for intelligent request routing between frontier planning models and execution-focused models.

Why it matters

Persistent AI agents spend most of their token budgets on high-frequency execution calls rather than multi-step planning. Using expensive frontier reasoning models for routine actions creates latency and budget bottlenecks. Nemotron 3.5 Lightning provides an open, efficient execution layer designed to run economically both locally and in data centres.

Analysis

NVIDIA is targeting a modular 'system-of-models' paradigm with Nemotron 3.5 Lightning and NeMo Switchyard: heavy reasoning models orchestrate, while compact MoEs handle the heavy lifting of tool execution. Leveraging native NVFP4 kernels and speculative decoding allows high inference throughput without sacrificing accuracy. Its permissive open-source release makes it a strong contender against rivals like Qwen for production-grade agent harnesses.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Link:Apodex AI

    Apodex-1.1-mini: Multimodal Agentic Model Released on Hugging Face

    Apodex AI has released Apodex-1.1-mini under the Apache-2.0 licence on Hugging Face. Based on the qwen3_5_moe architecture, the model is designed for long-horizon research and agentic workflows with native tool execution.

    AI & LLMs· Tool

  • Article:Zhidx

    Ant Group open-sources compact MoE model Ling-3.0-tiny

    Ant Group's Bailian team has released Ling-3.0-tiny, an open-source lightweight mixture-of-experts model. It leverages DeepSeek's MLA and Kimi's KDA architectures with 7.9B total parameters, activating just 1.3B parameters per token for local edge deployment on hardware like Apple Silicon Macs and DGX systems.

    AI & LLMs· Announcement

  • Link:inclusionAI

    Ling-3.0-tiny: Lightweight Hybrid Reasoning MoE Model by inclusionAI

    inclusionAI has introduced Ling-3.0-tiny, a compact MoE model with 7.9B total parameters activating only 1.3B parameters per token. Utilizing a hybrid architecture of linear and latent attention mechanisms, it aims to deliver efficient reasoning and agentic workloads on local hardware.

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.