Bottom line: Nvidia combines autoregressive and diffusion decoding in a tri-modal model that achieves 6x higher token generation in the 8B variant compared to comparable open-source models.
Nvidia has developed Nemotron-Labs-Diffusion, a language model that unites autoregressive, diffusion, and self-speculative decoding in a single architecture. The model can switch between modes to achieve high throughput depending on the deployment scenario and load.
Nemotron-Labs-Diffusion was developed with a combined training objective for autoregressive and diffusion approaches. The key finding is that these two approaches complement each other: diffusion improves multi-token lookahead, while autoregressive training provides classical left-to-right language priorities.
In self-speculative mode, the diffusion component acts as a draft generator, while the autoregressive component serves as verification. According to the developers, this configuration outperforms multi-token prediction methods both in acceptance rate and real device efficiency. Theoretical analysis shows that diffusion can theoretically achieve up to 76.5% more tokens per forward pass than self-speculative mode under optimal samplers.
The family includes variants with 3 billion, 8 billion, and 14 billion parameters, including base, instruct, and vision-language models. According to manufacturer specifications, the 8B model decodes 6x more tokens per forward pass than Qwen3-8B with comparable accuracy and achieves 4x higher throughput on SPEED-Bench with SGLang on a GB200 GPU.
Source: arxiv.org · Published 6 July 2026
Lumi AI News — AI-assisted curation pursuant to Art. 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.7.3.