hello·ai
ExampleBest after Phase 13 min

Why it cannot count the r's in "strawberry"

#tokenization#debugging#mindset

In one line

The letters were destroyed before the model ran — and once you see that, a whole family of "the AI is bad at simple things" complaints becomes one bug with one fix.

The situation

The team's assistant is generally excellent and occasionally embarrassing. Asked to count the r's in "strawberry", it says two. Asked to reverse a product code, it drops a character. Asked whether two SKUs differ by one digit, it confidently says they are identical when they are not.

Someone files it as "the model is bad at reasoning". It is not a reasoning problem, and treating it as one — bigger model, better prompt, chain of thought — wastes a week. It is a data problem: the information being asked about was never in the input.

"strawberry"strawberry← what the model receivesquestion: how many r's?the letters were never in the input — the model is guessing from what "berry" usually containsdo character work in code: "strawberry".count("r") → 3
The model receives pieces, not letters. Asking it to count letters is asking it about data it never saw.

What the model actually received#

Before any model runs, the splits the string into pieces from a fixed vocabulary. "strawberry" becomes something like str + aw + berry, and each piece is replaced by an integer id. The model sees three ids. It has never seen the letter "r" as a unit in that word — the pieces are atomic to it, the way a byte is atomic to a program that only has byte-level operations.

So "how many r's" is a question about the internal spelling of a , and the model's only route to an answer is a statistical guess about what "berry"-shaped pieces usually contain. Sometimes the guess is right. It is a guess either way, and the confident delivery is the same in both cases.

The same reasoning explains the other complaints. Reversing a string means reversing characters the model does not have. Comparing two SKUs digit by digit means comparing sub-token structure. None of these are hard problems; they are problems the model cannot see.

The fix is to route around it#

Anything character-level belongs in code. If the assistant needs to count, reverse, diff or validate a string, give it a tool — a function it can call — and let ordinary code do the work. That is , and it is the general shape of the fix: keep the model for judgement on text, and hand exactness to something built for exactness.

The mindset shift is in the diagnosis. When a model fails at something a child could do, the first question is not "why is it dumb" but "did it ever receive the information this task needs?" For letters, digits and exact identifiers, the answer is usually no.

Mindset

A model reads pieces, not characters. When it fails at something trivial, check whether the trivial thing was visible to it at all — and if it was not, the fix is a tool, not a better prompt.

Where it connects

Got the shape of it?

Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.