We sent the same document to AI models from seven companies. When the bills came back, the number of input tokens was different for every company. We had not changed a single character. It was the same document every time.
A token is the unit an AI uses to break text into pieces as it reads, and it is also the unit you are charged by. Grammar does not decide where those breaks fall. The breaks come from the huge amount of text the model was trained on, where certain chunks of characters kept appearing together. A token is roughly a word or a piece of a word.
Every AI company measures tokens with its own ruler
The catch is that every company draws up its own list of chunks, and with it its own tokenizer, the tool that counts tokens. A single company may also write a new list when it releases a new generation of models. Paste one sample sentence into OpenAI's public token counter and the newest generation counts 53 tokens, a model one or two generations older counts 57, and an even older one counts 64. If the count moves that much inside one company, the gap between companies is no surprise.
Length has the meter and weight has the gram, shared units everyone agrees on. Tokens have nothing like that. Each company is measuring the same text with a different ruler.
The price per token stayed the same, but the bill went up
We first noticed the problem in a talk at a FinOps event. FinOps is the practice of tracking and managing cloud and AI spending. The speaker explained why it is so hard to estimate what AI will cost, and pointed out that because companies count tokens differently, the same price list can lead to very different amounts actually paid.
There is a real case of this. In April, Anthropic released a new model and changed the way it counts tokens. The listed price per token was the same as for the previous model, yet the same text now produced up to 35% more tokens. Not a cent was added to the price, and still the bill grew.
We counted the tokens in one Korean document across sixteen models from seven companies
So we measured it ourselves at Polora. We attached the same report, written mostly in Korean, to the same question and sent it several times to sixteen models from seven companies. Then we collected the token counts each company returned as the basis for its charge.
Taking OpenAI's count as 1.00, Google came to 1.05, Mistral to 1.06, DeepSeek to 1.09, xAI to 1.15 and Moonshot to 1.19, all fairly close to one another. Anthropic alone stood far apart, at 1.55.
These figures are not chance variation from one request to the next. We sent the same input 121 times to two models of the same generation from the same company, and the token count did not change once.
To see what an AI really costs, multiply the price by the token count
A difference in token counts is a difference in money because AI price lists are written per token. If two companies both list $2 per million tokens, but one counts the same text as 1.55 times as many tokens, you pay 1.55 times as much. It is like comparing price tags from two countries with different currencies while ignoring the exchange rate.
None of this says anything directly about quality. Splitting text into smaller pieces can help a model perform better, and companies may have set their prices with that in mind. The point is that putting prices side by side is not enough. You only see what you will actually pay once you multiply by how many tokens each company counts for your own documents. Keep in mind that this measurement used a document written mostly in Korean, so a different mix of languages will give different ratios.
When you compare quotes, you check that both sides count quantities the same way before you compare unit prices. In our measurement, the companies counted the same Korean document differently. AI pricing may one day need a shared unit, the way length has the meter.
In the next piece, we measure how far the bill drifts apart when the same question is asked in English and in Korean. For people who write in Korean, the numbers look a little unfair.
