Cheaper tokens, same bill

Between November 2022 and October 2024, the price of running a model at a fixed capability level fell more than 280-fold — Stanford's AI Index measured it, and I unpack exactly what that number does and does not mean below. Ask whoever owns your cloud bill whether it fell by anything like that.

More price cuts landed last week. The announcement that mattered was titled "Advancing the price-performance frontier with GPT-5.6" (OpenAI, 2026), and the title was the entire message: not a smarter model, a cheaper one per unit of smart. If you hold a GenAI budget line, you read that and did the obvious arithmetic for next year.

I want to argue that two things are true at once. The price cuts are real and large. Your bill is still not going to fall. The gap between those is not vendors being sneaky, and it is not an accounting error at your end. It is that the price they keep cutting is not the number that governs what you pay.

> How to read the evidence tags. Every load-bearing claim below carries its source in the prose and a confidence label: triangulated (two or more independent, admissible origins), single-source (one origin — argued well, but not corroborated), contested (credible sources disagree), or my-judgment (my inference, not a finding). Price and efficiency figures date fast and vary by provider, so each is marked with the date it was measured. All of them are snapshots.

What we argued last week, and what the market has now conceded

A week ago I argued here, in "The proprietary premium stopped buying capability. It didn't drop to zero.", that the premium had stopped paying for capability on most tasks, that a real gap persisted on long-horizon agentic work, and that the model itself was only a modest share of what a production GenAI application actually costs — so the decision belonged to total cost of ownership, lock-in and control rather than to headline price (NivaLogic, 2026). That argument carried its own sourcing, which I have not re-run here; I am building on it, not re-proving it. (prior work; treat as that piece's evidence, not a fresh measurement.)

The market has since conceded the first half of that in public. When a leading lab names a release after the price-performance frontier rather than after a benchmark score, the argument about capability is effectively over — not because capability stopped improving, but because it stopped being the thing worth selling on. (OpenAI's own announcement, 30 July 2026 — a dated record of what the vendor said, not independent evidence of what it delivers.)

So the prior piece was right about the axis. It also left a question it did not ask, and last week made that question urgent. If price per token is the smallest line on the bill, and price per token is now the line falling fastest, does any of that reach the buyer at all?

The collapse is real. Read what is being held fixed.

Concede the strong part first, because it is genuinely enormous. Stanford's Institute for Human-Centered AI, in its 2025 AI Index, found that the price of running a model at a fixed capability level — scoring 64.8 on MMLU, roughly GPT-3.5 class — fell from $20.00 per million tokens in November 2022 to $0.07 by October 2024. More than 280-fold in about eighteen months (Stanford HAI, 2025). Single-source. Snapshot: the measurement window closes in October 2024 and was published in April 2025. It is not a 2026 re-measurement, and nothing in it describes what any specific provider charges today.

Read it slowly, because the operative word is fixed. What fell 280-fold is the price of buying yesterday's capability. It measures how cheap the old thing got.

And the rate is far softer than any single headline suggests. Stanford's own figures span two orders of magnitude — annual declines of anywhere from 9x to 900x, depending purely on which capability milestone you pin them to. Gundlach et al. (2026) examined a later window, April 2024 to November 2025, deliberately excluding the earliest period as unusually rapid early-stage change, and landed near the bottom of that span: roughly 5x to 10x a year on their headline measure, and about 3x a year once genuine algorithmic efficiency is isolated from hardware gains and competitive pricing. They position that estimate explicitly against the extreme multi-hundred-fold claims. Contested — not because the two flatly contradict each other (Stanford's range already reaches down near Gundlach's number) but because the central estimate and the window are genuinely disputed between two Tier-1 sources.

Be careful what you take from that, including from me. It does not license blanket cynicism about every price figure — I am relying on precise figures myself, a few lines above and several times below. What it licenses is narrower and more useful: a decline rate is a function of the window and the milestone chosen, so any single percentage — last week's included — tells you about a measurement before it tells you anything about your bill.

The unit that got cheaper is not the unit you buy

Here is the hinge, and it is almost embarrassingly simple. A bill is a price multiplied by a quantity. The industry cut the price. Nobody cut the quantity — and the quantity is where the last two years actually happened.

Two findings point at it. They measure different things, so I am not going to pretend they corroborate each other.

The first is about what you can see. Wang et al. (2025) built a method for auditing the reasoning tokens that commercial APIs bill for but do not show you, and their motivation is the finding that matters here: commercial services "often conceal internal reasoning traces while still charging users for every generated token, including those from hidden intermediate steps, raising concerns of token inflation and potential overbilling." Single-source. Note what I am not claiming: I have seen the figure that over 90% of billed tokens are hidden reasoning tokens quoted in several places, and I could not find it in that paper. So I am not using it. What is established is the structure — you are billed for deliberation you cannot inspect.

The second is about how much. Bai et al. (2026), at Stanford's Digital Economy Lab, measured token consumption across agentic coding tasks and found "agentic tasks are uniquely expensive, consuming 1000x more tokens than code reasoning and code chat". And the extra spend does not buy proportionate quality: "higher token usage does not translate into higher accuracy; instead, accuracy often peaks at intermediate cost and saturates at higher costs." Single-source. Snapshot: published April 2026, measured on agentic coding tasks specifically.

The price of each token fell. For this class of work, the number of tokens a task consumes rose by orders of magnitude. The unit that got cheaper is not the unit you buy.

Elasticity is a setting, not a property of the work

Everything above was measured on frontier reasoning models and agentic coding, so the obvious next move is to sort your estate into two piles. Elastic work — agent loops, extended reasoning, anything that retries until it passes — where consumption floats. And fixed work — classification, extraction, embedding, routine single-turn inference — where a document is the same length this year as last, so the price cut lands on your bill intact.

That sorting is useful, and I nearly left it there. It is also not safe, because which pile a task sits in is not a property of the task. It is a property of a setting.

Reasoning effort is a dial, and increasingly it is a dial the vendor sets on your behalf and turns up by default. (my-judgment — I am describing a direction of travel across providers, not a measured trend; check your own provider's current defaults rather than taking my word for it.) Zhou et al. (2026) found that on test-time compute scaling "marginal returns diminish substantially at higher budgets and that models exhibit 'overthinking', where extended reasoning is associated with abandoning previously correct answers." Single-source — and note that Zhou et al. and Bai et al. are separate teams reaching compatible conclusions, but both are arXiv preprints, and under the evidence standard I hold to here two papers on one platform count as one origin, not two. I am not going to call that triangulated.

The practical consequence: a classification job that was cheap and fixed last quarter can quietly become elastic this quarter, without you changing a line of code, because a default moved underneath you. So the useful question is not "which pile is my work in" but "who is setting my reasoning defaults, and when did anyone last look". (my-judgment.)

Meanwhile adoption keeps widening — 88% of surveyed organisations report AI in at least one business function, up from 78% the year before (Stanford HAI, 2026a). Single-source; Stanford is reproducing McKinsey's survey instrument here, so this is one origin, not two. I have no measurement of how that adoption splits across elastic and fixed work, and I am not going to guess at it.

Where the saving went — and how hard I am allowed to push this

The efficiency is not being handed to you. It is being spent, on inference-time compute: bigger models, longer deliberation, more autonomous loops. Two separate groups point the same way — Bai et al. (2026) from the consumption side, Gundlach et al. (2026) from the pricing side. I am going to hold myself to the rule I set a moment ago: both are arXiv preprints, so by the standard I am using they count as one origin, and I will call the direction single-source rather than triangulated. Worth noting that they come from genuinely different institutions — Stanford's Digital Economy Lab and MIT — which is why I find the convergence persuasive even though my own rule will not let me score it as corroboration.

A stronger version of that claim is available, and I want to be explicit about how much weight it can carry. Gundlach et al. (2026) argue that once you control for competitive pricing and for hardware cost declines, genuine algorithmic efficiency is improving only around 3x per year — and that the price of running true frontier capability has been rising, somewhere between 3x and 18x per year, because labs keep redirecting efficiency into larger models and heavier reasoning. Single-source. It is a striking finding and it fits everything else here, but I went looking for a second admissible origin and did not find one. Hold it as a well-argued single-source finding, not as settled fact.

There is a real counter-case, and it deserves its strongest form. Epoch AI argues the effect is transient rather than structural: because the cost of reaching any given capability level keeps falling quickly, "the inference cost burden is more transient than it might appear from looking only at frontier models at launch" (Epoch AI, 2026). Contested — and Epoch is not an institutional source under the evidence standard I hold myself to here, so it is argument rather than proof. If it is right, today's expensive frontier is next year's commodity and patience is a legitimate strategy. I think it is a serious argument. I also think it is no help with the invoice arriving this quarter.

And here I have to hold myself to the evidence rather than to the tidy story. The tidy story is Jevons: efficiency rises, consumption explodes, total spending goes up. It is satisfying, and the literature does not support it at that strength. The best-studied empirical work on efficiency and consumption — energy economics — finds that direct and indirect rebound effects are likely to "shave 5-30% off intended energy savings", perhaps 60% once economy-wide effects are counted, and that outright backfire is unlikely (Borenstein, 2013; Gillingham et al., 2013). Triangulated, across two independent teams.

So the defensible claim is narrower than the exciting one, and I will state it at the confidence the evidence carries: a substantial share of the unit-price saving is being absorbed by rising consumption. Not necessarily all of it. Not a guarantee your bill grows. Just a firm expectation that the discount you were quoted is not the discount you will bank. (my-judgment, built on the triangulated findings above.)

What actually moves the bill now

If price is the vendor's variable and volume is yours, every lever that matters sits on your side of the contract: which work you commission, which model runs it, and whether anyone is watching the volume at all. (my-judgment.)

None of them is free, and pretending otherwise would be the same error pointing the other way.

Routing cheap models under hard work carries a measured tax. Wang and Brorsson (2026) compared small-model deployments across agent paradigms and found task-completion falling from 99.67% when the model is called directly, to 79.92% inside a single-agent loop, to 72.01% multi-agent — while tokens per task rose roughly three- to six-fold. Their own summary is that "single-agent systems achieve the best balance between performance and cost, while multi-agent setups add overhead with limited gains." Single-source, so hold it loosely. But the mechanism matters: a cheap model in a loop is not a cheaper version of the expensive one. It fails more often, and failure inside an agent loop is expensive, because the retries are billed too.

Cascades genuinely work. Chen et al. (2023) showed that escalating to a stronger model only when a cheaper one is not confident can match a much stronger single model at a fraction of its cost. Single-source, and dated 2023 — treat it as an established technique, not a current benchmark. A production router is also not a one-time build; it is a standing maintenance obligation, because, as Taberner-Miller (2026) puts it, "providers revise pricing, model quality can shift or regress without notice, and new models arrive regularly." Single-source.

There are also real levers on the procurement side, and it would be dishonest to write as though there were none. Prompt and context caching, batch or asynchronous pricing tiers for work that does not need an immediate answer, committed-use agreements, and self-hosting an open-weight model all reduce what you pay without redesigning anything. Several are substantial. (Vendor-published discounts, varying by provider and changing often — check current terms rather than any figure quoted in an article, including this one.) But note their shape: each is a one-time step-down. You take the caching discount once. It does not repeat next year, and it does not slow the rate at which your loops consume tokens. (my-judgment.)

And there is a third lever, sitting between the discount and the rewrite, which I very nearly missed. You can turn the dial down. Capping reasoning effort, bounding maximum output tokens, limiting retries, switching extended thinking off for work that never needed it — these are configuration changes, not architecture projects, and they act on quantity directly. Zhou et al. (2026) found that "stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy" — which is to say the default is often simply set higher than your task requires. Single-source.

But I am not going to sell that as free, because the same paper is clear that the right budget depends on how hard the task is — cap too aggressively and you will underthink genuinely difficult work and pay for that failure instead. The dial is not something you set once and forget; it needs a measurement behind it, per class of work. That is real effort. It is still far less effort than re-architecting. If you do one thing after reading this, make it finding out what your reasoning defaults currently are, and who last chose them.

And the binding constraint may not be money at all. MIT Sloan Management Review's (2025) account of why GenAI stalls between pilot and scale reports causes that are organisational rather than technical — momentum fading part-way through the rollout, teams never trained for their specific function, a workforce that never took the tool up. Stanford's 2026 Index points the same way, putting the leading cited obstacle to implementation at knowledge gaps, 59%, with budget constraints a close second at 48% and regulatory uncertainty at 41% (Stanford HAI, 2026b). To be precise about what is corroborated: the direction — that the binding constraints are more organisational than technical — is triangulated across the two sources. Those specific percentages come from Stanford alone and are single-source.

That last one cuts both ways and I do not want to oversell it. Budget at 48% is not a small number. Cost has not stopped mattering. What has changed is that the part of it that is growing is no longer the part you set by negotiation.

One note on evidence hygiene, in a field awash with confident statistics. You have probably met the claim that 95% of enterprise GenAI pilots fail to deliver measurable P&L impact. I am not using it. It traces to a single report from July 2025 that is not peer-reviewed, and I could not retrieve the primary document to check what it measured (MIT Project NANDA, 2025). Unverified — which is exactly why it is not carrying any weight here. A number that famous ought to be easier to verify than that.

The negotiation moved

So the efficiency race did not hand you a discount. It cut the price of a token and, in the same movement, handed the quantity dial to somebody else.

For three years the AI cost conversation happened where procurement conversations happen: across a table, against a price sheet, opposite someone whose job was to offer you a better rate. Have that conversation. It still pays — genuinely so on work whose token count is stable, where the declines arrive on their own and switching keeps the pressure on. (my-judgment; this follows from the pass-through logic above, and I have no independent measurement of realised enterprise savings.)

But it is no longer the conversation that decides your bill, because the quantity is not on the price sheet. It is in a reasoning-effort default someone at your vendor chose. It is in an engineer picking the thinking model over the plain one because it looked better in a demo. It is in a loop that retries until it passes, written by someone who has never seen the invoice.

If your AI arrives as per-seat licences inside software you did not build, do not assume this misses you. Seat pricing is exactly where usage-based billing is being reintroduced, and the same elastic dynamic reaches you through credits and tiers rather than through an API bill. You have fewer levers than a team that builds its own — but you have more than none: which tier you buy, what caps you set, and what you insist on seeing about consumption before you renew. (my-judgment — check your own contract rather than taking my word for the direction.)

So: buy the step down. Take the caching discount, move batch work to batch pricing, sign the committed-use deal. Then turn the dial down, because that is the one lever that acts on quantity and costs you nothing but attention.

What is not for sale is the trend. The trend is set by how much deliberation your organisation switches on by default — and at the moment, in most organisations, nobody owns that setting at all.

---

Sources

Tiers follow NivaLogic's source standard: tier 1 (primary) — peer-reviewed, preprint, or official academic/government record; tier 2 (institutional) — established research institution or professional body; untiered — first-party vendor, aggregator or independent analyst material, admissible as dated context but never as load-bearing evidence.

Bai, L., Huang, Z., Wang, X., Sun, J., Mihalcea, R., Brynjolfsson, E., Pentland, A., & Pei, J. (2026, April 24). How do AI agents spend your money? Analyzing and predicting token consumption in agentic coding tasks. arXiv:2604.22750. Also published via Stanford Digital Economy Lab. https://arxiv.org/abs/2604.22750 — tier1_primary

Borenstein, S. (2013, May). A microeconomic framework for evaluating energy efficiency rebound and some implications (NBER Working Paper No. 19044). National Bureau of Economic Research. https://www.nber.org/papers/w19044 — tier1_primary

Chen, L., Zaharia, M., & Zou, J. (2023, May 9). FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv. https://arxiv.org/abs/2305.05176 — tier1_primary

Epoch AI. (2026, February 16). How persistent is the inference cost burden? Epoch AI Gradient Updates. https://epoch.ai/gradient-updates/how-persistent-is-the-inference-cost-burden — untiered (independent analyst; context only)

Gillingham, K., Kotchen, M. J., Rapson, D. S., & Wagner, G. (2013, January 24). Energy policy: The rebound effect is overplayed. Nature, 493, 475–476. https://www.nature.com/articles/493475a — tier1_primary

Gundlach, H., Lynch, J., Mertens, M., & Thompson, N. (2026, March 23). The price of progress (v2; v1 November 28, 2025). arXiv. https://arxiv.org/abs/2511.23455 — tier1_primary

MIT Project NANDA. (2025, July). The GenAI divide: State of AI in business 2025. https://nanda.media.mit.edu/ai_report_2025.pdf — unverified (primary document could not be retrieved; cited only to document the provenance problem, not as evidence)

MIT Sloan Management Review. (2025, July 8). How to scale GenAI in the workplace. https://sloanreview.mit.edu/article/how-to-scale-genai-in-the-workplace/ — tier2_institutional

NivaLogic. (2026, July 23). The proprietary premium stopped buying capability. It didn't drop to zero. — prior NivaLogic article (self-citation)

OpenAI. (2026, July 30). Advancing the price-performance frontier with GPT-5.6. https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6 — untiered (vendor first-party; dated record of the announcement only)

Stanford HAI. (2025, April 9). AI Index report 2025: Research and development. Stanford Institute for Human-Centered AI. https://hai.stanford.edu/ai-index/2025-ai-index-report/research-and-development — tier1_primary

Stanford HAI. (2026a). AI Index report 2026: Economy. Stanford Institute for Human-Centered AI. https://hai.stanford.edu/ai-index/2026-ai-index-report/economy — tier1_primary

Stanford HAI. (2026b). AI Index report 2026: Responsible AI. Stanford Institute for Human-Centered AI. https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai — tier1_primary

Wang, Z., Sun, G., He, Y., Shen, Z., Tian, B., & Li, A. (2025, July 29). Predictive auditing of hidden tokens in LLM APIs via reasoning length estimation. arXiv:2508.00912. https://arxiv.org/abs/2508.00912 — tier1_primary

Taberner-Miller, A. (2026, March 31). ParetoBandit: Budget-paced adaptive routing for non-stationary LLM serving. arXiv:2604.00136. https://arxiv.org/abs/2604.00136 — tier1_primary

Zhou, S., Ling, R., Chen, J., Wang, X., Fan, T., & Wang, H. (2026, April 12). When more thinking hurts: Overthinking in LLM test-time compute scaling. arXiv:2604.10739. https://arxiv.org/abs/2604.10739 — tier1_primary

Wang, X., & Brorsson, M. (2026, April 21). Rethinking scale: Deployment trade-offs of small language models under agent paradigms. arXiv:2604.19299. https://arxiv.org/abs/2604.19299 — tier1_primary

Figures in this piece are point-in-time snapshots, dated where they appear. Provider pricing changes frequently and varies by model tier, by input versus output tokens, and by region; nothing here should be read as a current quote for any specific provider.