ExampleBest after Phase 14 min

Finding duplicate tickets that share no words

In one line

Two tickets say the same thing in different words; embeddings put them nearly on top of each other, and the only real decision left is where you draw the line.

The situation

The support queue fills with the same bug reported forty ways. "App crashes when I upload." "Upload makes the app close." "Attaching a file kills it." Keyword search finds none of them from the others, because they share almost no words. An engineer proposes "use AI to find duplicates", and the question is what that would concretely mean.

No shared words, near-identical vectors. The threshold is the product decision; the geometry is just arithmetic.

What an embedding gives you#

Run each ticket through an model and it comes back as a vector — a list of a few hundred numbers — chosen so that texts with similar meaning point in similar directions. "App crashes when I upload" and "upload makes the app close" land at a of 0.98. "How do I change my email" lands at 0.31 from either.

That is the whole mechanism. No rules about synonyms, no stemming, no list of crash-related words. The model learned, from an enormous corpus, that those phrasings occur in the same contexts, and geometry now encodes it.

Two engineering details make it work in practice. the vectors once at write time so similarity is a plain dot product. And store them in a so "find the nearest tickets to this one" is a lookup rather than a scan over the whole queue.

The decision the model does not make#

Similarity is a number between −1 and 1. Nothing about the number says "duplicate". The threshold is yours, and it is a product decision with a cost on each side.

Set it high — 0.95 — and you merge only near-paraphrases, missing many real duplicates. Set it lower — 0.85 — and you start merging tickets that are related but distinct: "upload crashes the app" and "upload is slow" are close in meaning-space and different in fact. The right value depends on your corpus and your embedding model, and the only way to find it is to label a few hundred pairs and look at where the mistakes fall.

Note what the threshold cannot do. "The app crashes on upload" and "the app does not crash on upload" are almost identical vectors, because negation barely moves an embedding. Exact identifiers — an error code, a build number — carry almost no signal. For those cases, a keyword check beside the vector check is not optional.

The math

Cosine similarity between two normalised vectors is just their dot product:

\cos(\theta) = \frac{a \cdot b}{\lVert a \rVert\,\lVert b \rVert}

For two tiny illustrative vectors, $a = [0.81, 0.12, 0.57]$ and $b = [0.79, 0.15, 0.59]$ :

a \cdot b = 0.640 + 0.018 + 0.336 = 0.994, \quad \lVert a \rVert \approx 0.998,\ \lVert b \rVert \approx 0.998

\cos(\theta) = \frac{0.994}{0.998 \times 0.998} \approx 0.998

Nearly one, from three multiplications and two additions. The scale is the real version with 768 dimensions and a million tickets, and that is an index problem, not a mathematical one.

Questions to ask

What similarity score does a human-labelled duplicate pair actually get on my corpus?
Which mistakes are worse — merging distinct tickets, or missing duplicates?
Do my tickets carry identifiers that a vector will not distinguish?
Do I need a suggestion ("possible duplicate") or an action (auto-merge)?

Mindset

Embeddings turn "does this mean the same thing" into "how close are these two points" — which is the easy part. The judgement that remains is where to draw the line, and that is decided with labelled examples, not with a better model.

Where it connects

Got the shape of it?

Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.

Next: Replacing keyword search on the internal wiki