The invoice is three times the estimate
In one line
The estimate used "four characters per token"; the workload was JSON and identifiers, where the real ratio is closer to two — and the only reliable estimator is the tokenizer itself.
The situation
A pipeline classifies incoming webhooks — JSON payloads, a few kilobytes each — by sending them to a model. The cost model in the planning doc was simple: 3KB per payload, four characters per , so about 750 tokens each. At volume, a few hundred dollars a month.
The first invoice is three times that. Nothing in the code changed, the volume is as forecast, and the per-token price is exactly what was quoted. The estimate was wrong on the one number nobody thought to check.
Where the ratio comes from#
"Four characters per token" is a real average — for English prose, on a whose vocabulary was built mostly from English prose. The process that builds the vocabulary merges the most frequent character sequences in the training corpus into single tokens. "the", "ing", " and" get one token each because they were everywhere.
JSON is not prose. Braces, quotes, colons and newlines each tend to be their
own token. Keys like customer_id split into three or four. Values are the
worst case: a UUID is a run of characters that appeared in no common sequence,
so it tokenizes at roughly one token per two characters, and a base64 blob is
worse still. Ten thousand characters of prose is about 2,500 tokens; the same
bytes as a webhook payload is 5,000 to 7,000.
The same effect hits code (identifiers), logs (timestamps and ids), and any language that was under-represented in the corpus. It is not a bug — the tokenizer is doing what it was built to do — but it is invisible unless you measure.
The habit that prevents it#
Every provider ships their tokenizer as a library. Run it. Take a hundred real payloads, count tokens, and use that distribution as the estimate. It takes ten minutes and replaces a rule of thumb with a measurement.
Then look at what is actually being sent. The classifier probably does not need the full payload. Strip the fields that carry no signal — internal ids, timestamps, signatures — before the call, and the token count falls with them. The same trimming that cuts the bill also leaves more of the free, which becomes the binding constraint long before cost does on larger inputs.
The wider lesson is about estimates in this domain generally. The units are tokens, the conversion from anything human-readable to tokens is content-dependent, and every heuristic is a lower bound. Anything that will be budgeted, capped or paid for should be counted in tokens from the start.
Questions to ask
- Have I run the actual tokenizer on actual inputs, or applied a ratio?
- What fraction of each input carries signal the model needs?
- Is the cost dominated by input tokens or output tokens? (Usually input.)
- Will this workload's shape change — more code, more languages — and re-price itself?
Mindset
Tokens are the unit of cost and capacity, and the conversion from bytes to tokens depends on what the bytes are. Measure with the tokenizer, trim what the model does not need, and treat any characters-per-token rule as the optimistic case.
Where it connects
Got the shape of it?
Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.