NVIDIA just dropped fresh numbers on its latest Blackwell Ultra AI hardware, and they’re the kind that make CFOs sit up straight.
The company says its new Blackwell Ultra GB300 NVL72 systems deliver up to 50x more performance per megawatt and 35x lower cost per token compared to its Hopper platform. That’s a phase change.
And it’s happening right as AI agents, especially coding assistants, are devouring compute at record speed.
The real story: AI inference is exploding
According to OpenRouter’s State of Inference report, software-related AI queries jumped from 11% to nearly 50% of all traffic last year. Coding copilots, multi-step agents, and long-context assistants are now core workloads.
These systems need two things:
- Low latency (every millisecond compounds across multi-step workflows)
- Long context windows (reading entire codebases, not snippets)
That’s a brutal combo. Fast and memory-hungry.
Blackwell Ultra is NVIDIA’s answer.
50x more throughput per megawatt
The GB300 NVL72 system, powered by the Blackwell Ultra GPU and tightly integrated software, pushes throughput per megawatt up to 50x higher than Hopper, according to NVIDIA.
That translates into up to 35x lower cost per million tokens, especially at low-latency settings where agents operate.
This is full-stack codesign:
- Optimized GPU kernels for low latency
- NVLink Symmetric Memory for direct GPU-to-GPU access
- Programmatic dependent launch to reduce idle time
- Continuous improvements from TensorRT-LLM, Dynamo, Mooncake, and SGLang
In fact, NVIDIA says TensorRT-LLM improvements alone delivered up to 5x better low-latency performance in just four months on earlier Blackwell systems.
Translation: software is squeezing more juice from the same hardware every quarter.
Independent benchmarking from firms like SemiAnalysis (see their InferenceX v2 comparison) has also shown major gains for Blackwell over prior generations, reinforcing the broader direction of these efficiency improvements.
Long context = real-world agentic coding
The economics get even more interesting with large contexts.
For workloads using 128,000-token inputs and 8,000-token outputs, think AI reading an entire repository, GB300 delivers up to 1.5x lower cost per token than GB200.
Blackwell Ultra boosts:
- 1.5x higher NVFP4 compute
- 2x faster attention processing
That matters because context length scales cost aggressively. The more an agent reads, the more compute explodes. Faster attention reduces that tax.
Cloud providers are already deploying
Microsoft, CoreWeave, and Oracle Cloud Infrastructure are deploying GB300 NVL72 systems for production use cases like agentic coding.
CoreWeave’s Chen Goldberg put it plainly: as inference becomes the center of AI production, token efficiency and long-context performance become critical.
This is the shift: training used to be the glamour event. Now inference is the economic battlefield.
Efficiency isn’t optional anymore
There’s a broader pattern here.
People worry about AI’s environmental footprint—and they’re not wrong to. But look at the incentive landscape:
- Power is scarce.
- Data center real estate is limited and increasingly expensive.
- Local communities are pushing back.
- Token demand is exploding.
- Models are getting larger.
Efficiency used to be a nice-to-have. Now it’s survival.
If you need 35x fewer dollars per token and 50x more throughput per megawatt to stay competitive, you optimize or you die.
That pressure creates a powerful force toward better hardware, smarter software, and tighter system integration.
It's also a win for those with well-deserved environmental impact concerns surrounding AI.
And this isn’t the end
NVIDIA’s next platform, Rubin, is already teased to deliver up to 10x higher throughput per megawatt than Blackwell for mixture-of-experts inference and train frontier models using one-fourth the GPUs.
Working theory: we are entering the “efficiency decade” of AI.
The first wave was scale at any cost. The next wave is scale constrained by physics, power grids, and balance sheets.
Blackwell Ultra is evidence that AI infrastructure is maturing from brute force to disciplined engineering.
In the long run, that discipline may matter more than raw model size.
