The Flat-Rate AI Era Is Over. Here Is the Math That Replaces It. | CloudWeld

AI Costs

The Flat-Rate AI Era Is Over. Here Is the Math That Replaces It.

Claudia StanJuly 6, 20269 min read

If your AI bill looked strange in June 2026, it was not just you. In the space of two weeks, the pricing model that the whole industry had quietly depended on stopped working, and a lot of budgets broke on the same schedule.

On June 1, GitHub Copilot moved every plan to usage-based credits. Heavy users watched bills jump from $29 to nearly $750, and from $50 to around $3,000 (UsageBox kept a good timeline). Two weeks later, Anthropic split subscription usage into separate pools, after already blocking third-party tools from consuming flat-rate plans back in April. This was not a round of greedy price hikes. It was the end of a subsidy that could never last.

Those numbers are not ours; they come from people who reconstructed the unit economics (Stephen Adams did the arithmetic, and Business Insider reported a single user pulling $35,000 of inference on a $200 plan). The point is that flat-rate AI was a promotional period, and promotional periods end.

The companies that got hurt were not the reckless ones. They were the ones who treated a metered utility like a fixed cost. Uber reportedly burned through its entire 2026 AI budget by April. A Microsoft division pulled engineers off Claude Code mid-year after token billing hit roughly $2,000 per engineer per month. These are organizations with finance teams. They still got surprised, because nobody was watching the meter that had quietly started running.

If AI is your product, this is a margin problem

For most companies, AI tooling is an expense line. Annoying when it spikes, survivable. If your product itself runs on model calls, the same shift lands somewhere more dangerous: your gross margin.

We ask every AI product team the same question, and it is remarkable how often it goes quiet:

What does one active user cost you, per month, in tokens?

Most teams cannot answer it. And if you cannot answer it, your pricing is a guess. You almost certainly have profitable users subsidizing users who cost you three times their subscription, and no instrument that tells you which is which. While inference was subsidized, that blindness was harmless. Now that every layer is metered, it compounds every month you leave it alone.

Where the money actually goes

Before optimizing anything, you have to see the shape of the spend. When we instrument a product’s LLM calls for the first time, the distribution is almost always lopsided: a large majority of requests are simple, and a large majority of the cost sits in a small number of expensive calls routed to a frontier model that never needed to be there.

The reason that pattern is so common is the price gap between model tiers. It is not 20 percent. It is an order of magnitude or more:

When every request goes to the frontier tier by default, you are paying the bottom-row price for top-row work. That single default is usually the biggest line item on the bill, and the easiest to fix without a user ever noticing.

The five moves

Token bills respond to the same discipline that tamed cloud bills a decade ago. None of this is exotic. It is unglamorous, and it works.

1. Meter first. Put a gateway or proxy in front of every LLM call and tag each request by feature and user tier. Until you can read the bill broken down that way, every other decision is a guess dressed up as a strategy.

2. Cache the repeats. Identical and near-identical requests are far more common than teams expect, especially inside RAG pipelines and agent loops that re-ask the same thing. Prompt and response caching is usually the cheapest money you will ever save.

3. Route by difficulty. Send each request to the cheapest model that can actually do the job, and fall back to a stronger one only when a quality check says so.

4. Batch the background. Anything a user is not actively waiting on belongs on batch pricing and smaller models. Interactive requests keep their latency; the overnight work takes the discount.

5. Right-size, then revisit. Last year’s frontier model is this year’s mid tier at a fraction of the price. The correct model for a task changes every few months, which is exactly why this is a habit and not a project.

There is one rule that keeps routing safe rather than reckless: you measure output quality on your own tasks before and after every change, and anything that regresses goes straight back to the stronger model. The goal is never to be cheaper at the cost of being worse. It is to stop paying frontier prices for work a small model does identically.

The self-hosting question, answered honestly

The moment the bill hurts, someone proposes self-hosting open-weight models. Sometimes that is right. Usually it is early. The deciding factor is volume, and the hidden operations cost, not the sticker price of a GPU.

Below roughly 20 million tokens a month, managed APIs win once you price in the engineers who keep a GPU cluster alive; the operations overhead is a multiple of the raw hardware (this 2026 decision guide walks the thresholds). Above it, the answer is rarely all-or-nothing. F5’s 2026 research found organizations running or evaluating seven models on average, with 78 percent operating some inference themselves. The winning shape is almost always hybrid: cheap, steady, high-volume work on your own infrastructure, spiky and hard work on the API.

How we run our own

We are not describing this from the outside. ooky.ai, our own product, runs its inference through the same gateway we build for clients: every call tagged by feature, the cheap-and-common requests routed to small models, prompt caching on the hot paths, and a dashboard that shows cost per feature rather than one number at the end of the month. We hold our own bill to the same bar we sell to clients, and we retune it as prices move, because they always do.

That is the honest reason we can offer the guarantee we do: if a cost teardown cannot find at least 25 percent in credible savings, it is free. Unmetered LLM usage almost always hides more than that, and we have yet to meet a product that was already routing, caching, and right-sizing on day one.

The counterweight you should not forget

One variable keeps this whole exercise from ever being finished: at fixed capability, per-token prices have fallen roughly 10x per year. That cuts two ways. It is the strongest argument against self-hosting too early, because the API you would be escaping gets cheaper while your GPUs depreciate. And it means your routing table has a shelf life, since a model that was too expensive for a task last quarter may be the obvious choice for it now.

The subsidy that hid all of this is gone, and it is not coming back. But the underlying trend is friendly, if you are paying attention. The teams that win the next two years are not the ones who spend the least on inference. They are the ones who know, to the cent, what they are spending it on.

AI Cost Engineering

Want the breakdown for your own bill?

Our cost teardown meters your LLM usage and delivers a savings plan with real numbers attached. If we cannot find at least 25 percent, the teardown is free.

Book a Bill ReviewHow Cost Engineering Works

Continue Reading

Related Articles

[

ai-codebases-investor-diligence

Production

Your Codebase Is 95 Percent AI-Generated. Investors Now Check.

CloudWeld

AI-generated codebases stopped being a curiosity and became a due-diligence line item. What technical reviewers actually look for, what it does to your valuation, and how to be ready before the data room opens.

10 min read

](/blog/ai-codebases-investor-diligence)[

mcp-servers-that-survive-production

MCP & Agents

Most MCP Servers Die Before Production. Yours Does Not Have To.

CloudWeld

Audits show more than half of public MCP servers are dead and fewer than one in five are production-ready. The reasons are consistent: authentication, tool design, and a spec that keeps moving. Here is what production-grade actually requires.

11 min read

](/blog/mcp-servers-that-survive-production)[

what-actually-breaks-in-ai-built-apps

Security

What Actually Breaks in AI-Built Apps: The Incident Record

CloudWeld

Missing access rules, leaked keys, and deleted databases. A tour of the documented failures in AI-built software, why it is always the same five things, and the short list of fixes that would have prevented most of them.

11 min read

](/blog/what-actually-breaks-in-ai-built-apps)