Vector indexes

An index over an ordered key range partitions cleanly: a value is on one side of a boundary or the other. High-dimensional space has no such partition, because every region borders too many others — the geometry of the space simply does not admit it.

So approximate nearest neighbour structures navigate rather than partition. They start somewhere and move toward the query, examining a bounded number of candidates, and stop. If the true nearest neighbour sits in a region the walk never entered, it is missed. Silently, with no error and no signal.

That makes Recall a first-class operational number: of the genuinely nearest k vectors, how many did the index actually return? Ninety per cent recall means one relevant chunk in ten never reaches the model, and downstream this presents as "the assistant doesn't know things" rather than as a search fault.

Measuring it is straightforward and almost nobody does it: take a few hundred sample queries, run a brute-force exact scan to get ground truth, compare. Once you have that number you can tune against it, and the shape of the curve is always the same — recall rises steeply at first and then asymptotes, so the last few points cost disproportionately.

Flat is no index: compare against everything. Exact, trivially correct, and entirely appropriate below roughly 100,000 vectors, where a scan is milliseconds. A surprising number of production systems should be using this and are not.

HNSW builds a layered proximity graph. The top layer is sparse with long edges for covering distance quickly; each layer down is denser; search descends until it converges. It is the default in most vector databases because it gives excellent recall at high Throughput. Its costs are memory — the graph itself is often comparable to the vectors — and awkward updates, since deletes are tombstoned and the graph degrades until rebuilt.

IVF clusters vectors and searches only the nearest few clusters. Much lighter on memory, needs a training pass over a sample to pick centroids, and recall depends on how many clusters you probe. It pairs naturally with product quantization, which compresses each vector into a short code and lets billions of vectors fit in memory at the cost of some precision.

Disk-based graphs such as DiskANN keep most of the index on SSD and are the right answer at a scale where RAM is the binding cost.

The two dials on HNSW that you will actually touch: M, the edges per node, traded against memory; and ef_search, the candidate list size at query time, traded directly against latency. ef_search is a query-time parameter, which means you can raise it for a slow path and lower it for an interactive one without rebuilding anything.

A vector index is a derived artefact, and treating it as a database is the source of most operational pain.

Rebuilds are routine. Changing Embedding model, Chunking strategy or Dimensionality means re-embedding everything. Build the new index alongside, verify recall on a sample, then swap. Store the model id and dimension in index metadata so a mismatched write fails loudly instead of quietly corrupting results.

Deletes are lazy. Graph indexes tombstone rather than remove, so heavy churn degrades both recall and latency until a rebuild. If your corpus turns over often, schedule rebuilds rather than waiting for a complaint.

Filtering needs care. "Nearest neighbours where team = platform" can be done by filtering after search, which may return nothing if the top k were all other teams, or before search, which can be expensive. Modern engines support filtered search that constrains the graph walk; check that yours does before designing around it.

Normalise at write time. With unit vectors, cosine similarity and dot product agree and the index can use whichever is faster. Doing it per query is wasted work and one more place for a mismatch to creep in.

And the boring point worth repeating: most systems do not need a dedicated vector database. Postgres with pgvector handles millions of vectors, keeps your metadata joins in the same transaction, and removes an entire piece of infrastructure from your diagram. Reach for a specialised engine when you have measured that you need it.

Two million chunks, 768 dimensions, float32.

raw vectors     2e6 x 768 x 4 bytes           = 6.1 GB
HNSW graph      M=16, ~2 x 16 x 4 x 2e6       = 0.3 GB
total in memory                               ~ 6.4 GB

Fits on one machine comfortably. Now measure the recall-latency curve by sweeping ef_search against a 300-query ground-truth set:

ef_search   recall@10    p50 latency
   16         0.82          1.1 ms
   40         0.93          2.4 ms
  100         0.98          5.6 ms
  200         0.99         10.8 ms

The decision falls out of the table. Going from 40 to 100 buys five points of recall for 3ms — clearly worth it in a pipeline where the model call takes two seconds. Going from 100 to 200 buys one point for another 5ms, which is not.

Then apply Quantization to the stored vectors:

int8 vectors    2e6 x 768 x 1 byte            = 1.5 GB   (-75%)
recall@10 at ef_search=100                    = 0.97     (-1 point)

A quarter of the memory for one point of recall, which for most corpora is the right trade — and is only visible as a trade because the measurement existed.