AI Is Getting Cheaper. My Token Usage Is Going Through the Roof.

A year ago, my daily AI usage was in the low thousands of tokens. Today, it can reach hundreds of millions.

That sentence would have sounded ridiculous to me a year ago. It still looks a little ridiculous written down.

Tokens are the units AI models use to process information. In text, they’re often words or pieces of words. A usage total can include your prompts, documents, conversation history, tool results, reasoning and generated answers.

My numbers are a personal observation, not a controlled study. How those tokens are counted matters, especially when an agent repeatedly uses information it has already processed.

Still, the change has me wondering how much AI one person might consume a year from now.

New research on NVIDIA’s Vera Rubin platform makes that question more interesting. The computing infrastructure is becoming substantially better at handling the long, complicated AI assignments that drive enormous usage.

And the pattern feels familiar.

We once connected to the internet to check email and browse a few pages. Then faster connections and better applications gave us reasons to spend far more time, and move far more data, online.

AI could follow a similar path. As the cost of doing useful work falls, we may discover much more work we want it to do.

From H100 to Rubin: How much more AI can the hardware deliver?

There isn’t one universal number for how much AI a chip can produce. It depends on the model, the software, the amount of context, how many requests arrive together and how quickly each user needs an answer.

But individual comparisons show how far the economics have moved.

NVIDIA’s H100 performance page cites April 2026 InferenceX results putting H100 at approximately $0.09 per million tokens for GPT-OSS-120B using vLLM, at 66 tokens per second per user. It lists Blackwell B200 at approximately $0.02 per million tokens using TensorRT-LLM, at 55 tokens per second per user.

Under those assumptions, a dollar of serving cost buys roughly 11 million tokens on H100 versus 50 million on B200.

Those are infrastructure economics, not API prices. The software and response speeds differ, so this is a comparison of complete serving configurations rather than a pure chip comparison.

There’s also a useful historical checkpoint: in its August 2024 MLPerf debut, Blackwell delivered up to four times H100’s performance on the Llama 2 70B inference workload.

Now Rubin is extending the progression toward much heavier agent workloads.

The lesson is that newer systems can serve substantially more AI within a given operating budget. Exactly how much depends on what you ask them to run.

What Vera Rubin changes

Vera Rubin NVL72 connects 72 Rubin GPUs into a rack-scale system. Its processors, memory and networking work together to support sustained inference, the computation involved in running a trained model. NVIDIA’s architecture explanation emphasizes keeping information moving efficiently across many reasoning steps.

SemiAnalysis’s September 14 analysis reports approximately 67 times the throughput per ownership dollar of GB300 running TensorRT-LLM at a 170-token-per-second interactivity target.

That is a specific operating point near the comparison configuration’s speed limit.

At 60–100 tokens per second, the range the researchers identify as more representative for providers serving the tested model, Rubin delivers 1.4–3 times the throughput per ownership dollar. With three-year rental assumptions, it produces 62% more tokens per dollar at 80 tokens per second.

Against the older H200, a later member of the H100’s Hopper generation, Rubin delivers 18 times as many tokens per ownership dollar at 80 tokens per second on the tested agentic workload.

At 100 tokens per second, Rubin also delivers approximately 2.09 times the throughput per megawatt of the stronger GB300 software configuration.

These are early, workload-specific results. The different comparisons cannot be multiplied into a universal H100-to-Rubin improvement.

Even the more modest gains matter. They can change which AI services are economical to offer.

Why agents need so much more

A chatbot exchange is relatively easy to picture: ask a question, receive an answer.

An agent can turn one instruction into dozens of operations. It reads material, calls tools, examines results and keeps going. Sometimes it launches additional agents to handle parts of the assignment.

Each step can bring earlier information back into the model’s input.

The vLLM team’s analysis of AgentX reports a median of 43 turns per session, with median inputs of 142,000 tokens and outputs of just 444 tokens. Forty-four percent of sessions contain at least one subagent.

Much of that input repeats. The reported prefix-cache hit rate exceeds 96%, allowing extensive reuse of earlier computation.

Caching is a little like keeping your research open on the desk. You still need space for it and still refer to it, but you avoid starting from scratch every time you write another paragraph.

It also explains why enormous usage totals need context. A workflow making 1,000 calls with 200,000 input tokens each would register 200 million input tokens. It would not have written 200 million tokens of new material.

The AgentX methodology uses traces gathered from power users and replays their approximate traffic patterns. It is evidence about demanding infrastructure workloads, rather than a representative picture of everyday consumers or a guarantee of successful task completion.

Still, it illustrates how one person’s request can create a substantial amount of activity behind the scenes.

This feels like the internet all over again

Email offers a useful starting point.

Sending a message electronically was already valuable. But the internet’s growth eventually encompassed video calls, streaming, cloud storage, large downloads and applications constantly exchanging information in the background.

Faster connections expanded what people could reasonably do online.

Cisco’s historical traffic analysis estimated that global internet traffic grew more than 564 times between 2000 and 2014. It attributed traffic growth to a combination of more users and devices, changing applications, increased usage and greater bandwidth availability.

That combination is the useful parallel for AI.

A short chat answer might be the equivalent of an email: helpful, relatively lightweight and easy to understand. An agent handling a substantial assignment resembles a more demanding application built on the same underlying service.

The consumer doesn’t have to become fascinated by tokens any more than a person watching a movie needs to study network packets. The application creates the demand.

There is an important difference. Moving data and generating an AI response are different operations. Tokens also vary in their computational cost depending on the model, context and caching. They are not interchangeable units like bytes.

But the behavioral comparison holds: what looks like abundant capacity today can become ordinary consumption once useful applications arrive.

And email supplies a second lesson. Making something cheap and easy does not guarantee that every additional unit is valuable. Anyone with an inbox has encountered that problem.

Email is free because the cost of serving one person is neglible, or dare I quote Sam Altman, "too cheap to meter."

What consumers could actually notice

Most people will never choose which GPU runs their assistant. They will notice whether it finishes the job, how long it takes and when they hit a usage limit.

My expectation is that some efficiency gains will arrive as more work included in the same subscription.

An assistant could spend more computation researching a question, comparing alternatives or checking its answer. Providers could allow longer assignments or more simultaneous projects.

Some companies may cut prices. Others may keep prices steady and improve the service or retain the savings.

Imagine an assistant comparing travel options around several people’s schedules. The useful improvement would be a finished comparison that accounts for the constraints. Faster inference could help make that practical, although websites, external services and approvals would still introduce delays.

The same monthly payment could buy considerably more completed work.

That is the consumer benefit I would watch for, and if you compare 2026 to 2023, it's already happened in dramatic fashion.

Businesses should measure the finished job

Businesses have a different calculation.

Their cost includes model usage, connected tools, infrastructure, human review and fixing mistakes. An inexpensive answer that requires an hour of cleanup can become an expensive result.

An April study of agentic coding tasks found substantial variation in token consumption, including runs of the same task differing by as much as 30 times. More tokens did not reliably produce greater accuracy.

So businesses need to measure cost per accepted result.

Lower serving costs could make it economical to run a useful process across more customers, documents or projects. A company might expand an automation after a smaller deployment proves worthwhile.

That creates a perfectly plausible outcome: its cost per task falls while its total AI spending rises.

If comparable token prices fall threefold and consumption rises ninefold, the bill triples.

That can still be a good deal. The business needs to know what the extra spending accomplished.

Labs get more choices, and more pressure

For AI labs, better inference efficiency creates room to serve more customers, lower prices, improve margins or spend more computation on difficult requests.

They can explore approaches that generate multiple candidate answers and evaluate them before returning a result. Better hardware does not automatically make an unchanged model smarter, but it can make computationally expensive techniques more practical.

For operators limited by available electricity, improved efficiency also creates additional capacity within an existing power allocation.

Capacity still needs customers. Profit depends on utilization, selling prices and operating costs.

A lab might want to keep the savings while a competitor uses similar improvements to offer more generous limits. The competition will help determine how much reaches end users.

As we explored in our look at AI becoming infrastructure, cheaper model access leaves plenty of value—and expense—in the infrastructure and services around it.

Demand is already growing quickly

At Google I/O in May 2026, Sundar Pichai said Google was processing more than 3.2 quadrillion tokens per month across its surfaces, up from roughly 480 trillion a year earlier. That’s about sevenfold growth.

Google also reported that more than 375 Cloud customers each processed over a trillion tokens during the preceding 12 months.

Separately, OpenRouter reported in May that its weekly volume had grown from 5 trillion to 25 trillion tokens in six months.

Those are platform totals. They reflect changes in customer numbers, products and workloads, so they cannot establish how much more the typical person is using.

OpenRouter’s usage study also identified programming as a major driver of longer prompts. Its developer-heavy audience makes it useful for understanding intensive adoption, but a poor substitute for a representative consumer survey.

I would be careful about assuming everyone is a year away from my usage level.

What seems plausible is that many people could consume much more AI through familiar software without ever seeing a token counter.

Where could we be a year from now?

Extending my personal growth rate would produce a dramatic number and a lousy forecast.

Moving to a different way of working can create a jump that does not repeat at the same rate forever.

But several changes can multiply together: more users, more tasks per person, more model calls per task and more information in each call.

Imagine the user base growing 50%, tasks per user doubling, calls per task doubling and tokens per call increasing 50%. Together, those changes produce ninefold token growth.

That is arithmetic, not a prediction. It shows how consumption can grow without anyone typing nine times as much.

Better software can push in the other direction. Smaller models, more selective context and more effective workflows can reduce the tokens needed for a job. Eliminating wasted agent activity would be progress.

Ultimately, I care about the useful work that comes back.

But the internet comparison keeps sticking with me. We found plenty to do with connections that once seemed impossibly fast. Entire categories of everyday activity moved onto them.

Vera Rubin points toward more computing capacity for a similar expansion in AI.

A year from now, hundreds of millions of tokens may still be unusual for an individual. The more interesting, and in my opinion likely, possibility is that the amount of work we expect from an ordinary AI subscription will have changed just as dramatically.

Related reading

• AI Is Turning Intelligence Into Infrastructure. Who Owns the Pipes? — How falling model costs change where companies capture value.
• Sam Altman Says AI Has Quietly Crossed an Important Threshold — The broader argument for AI becoming everyday infrastructure.