The figure circulates privately among AI executives, usually in the form of a question: do you know what one of your users is costing you? A report this week by 36Kr, a Chinese technology media outlet, cited an example that has been making the rounds in the industry: a single user at one major company consumed $500,000 worth of tokens in a single month. The number — half a million dollars of model output from one person in thirty days — has become shorthand for the cost structure now bearing down on the AI industry.
Training the frontier models that power the current boom costs hundreds of millions of dollars per run, and industry estimates put the next generation of models higher still. But inference — the act of running a model to answer a user — is now the bigger and faster-growing bill. Every query, every image, every generated video burns compute, and as products scale, token consumption compounds: a user who chats all day, or a developer whose coding agent runs thousands of tasks, can consume what used to be a company’s entire monthly cloud bill. The $500,000 user is the extreme end of a distribution that is shifting fast.
The question the figure sharpens is whether the industry’s founding logic still holds: that buying more compute buys more intelligence, and that intelligence pays for itself. Inside Silicon Valley, that logic is now being questioned with unusual sharpness, according to people involved in the discussions. Some argue the constraint is not demand but cost — that the AI industry can sell its products, but at current marginal costs, serving heavy users at scale can mean losing money on every one of them.
The economics have a cloud-era shape with a steeper slope. Token prices have fallen by orders of magnitude since the early days of large models, but usage has grown faster than prices have fallen. Unit costs down, total bills up: the classic cloud story, repeated at a more punishing angle. The industry’s capital spending has reached levels that make earlier tech buildouts look small — the largest cloud companies now spend hundreds of billions of dollars a year on data centers and chips — and that money has to be earned back through usage, which means even more tokens sold, at whatever price the market will bear.
The two camps inside the industry disagree on the slope. Bulls argue that cost per token keeps falling, model efficiency keeps improving, and the companies that solve the efficiency problem will be among the most valuable in history. Bears reply that the industry is subsidizing its own growth with capital: if training and inference costs rise as fast as revenue, the AI boom is a transfer of money from investors to cloud providers and chipmakers, with the middle ground — the AI applications themselves — squeezed on both sides. Both sides agree on the numbers; they disagree on the math that follows.
The responses are taking shape in several directions. Model makers are pushing efficiency: smaller models trained to match larger ones, distillation, mixture-of-experts architectures, aggressive caching of repeated computations, and cheaper hardware. Enterprises are pushing back on pricing: large customers are renegotiating contracts, capping usage and building their own inference stacks to cut API bills. Investors are asking harder questions about gross margins, and companies that burn cash on inference face valuation scrutiny. Each response reduces the burn a little; none removes it.
For the industry’s customers, the practical effects are already visible: higher prices, usage caps, tiered plans, and a growing market for open-weight models that run on a company’s own hardware for a fraction of the API price. Coding agents, the fastest-growing use of AI, are also among the most token-hungry; a team that automates heavily can watch its monthly bill climb into six figures. The token meter has become a line item in software budgets that did not exist three years ago, and finance departments are learning to read it.
The 36Kr report is one data point in a broader reckoning, but it captures the mood. The phrase that keeps coming up is the same one the industry used during the last infrastructure boom: build now, figure out the economics later. The difference is that the previous boom’s costs were predictable — servers, bandwidth, power — while AI costs scale with intelligence itself. Nobody is certain how much intelligence the money buys, or whether the buyers will keep paying.
For Silicon Valley, the $500,000 month is a number that will keep being quoted until someone proves the math wrong. The industry’s most valuable companies were built on the assumption that computing power is a rising tide that lifts every model with it. The token bill is the tide turning from tailwind to line item, and the anxiety is not about whether AI works — it is about whether the people paying for it can keep paying. That question, at the moment, has no confident answer.
