hello·ai

Track 2 of 2 · 6 modules · 1 of 27 topics written · about 1 h

Classifying with small models

Pick a label, or decline to, with models that fit on a CPU.

Small encoders, prototypes, calibrated confidence and honest metrics. It builds on Phase 1 (embeddings, vector space, encoders) and runs alongside the rest of the fundamentals.

Opens after AI fundamentals · Phase 1, then runs in parallel with the rest of it. Modules gate each other at the same 80% mark.

Modules

Module A · Framing

🔒 Unlocks at 4 of 4 topics in Phase 1 — 0 done.

Should this be a model at all, and when should it stay quiet?

Before any vectors: what a classifier is for, when "no answer" is the right output, and which traffic never needed a model.

  1. A1

    Classification and abstention Planned

    Picking a label, where "no answer" is a valid and often safer output.

  2. A2

    Rules before models Planned

    Send structured or repeated inputs to lookups, and profile traffic before optimising.

Module B · Labels and representation

🔒 Unlocks at 4 of 4 topics in Phase 1 — 0 done.

What does the model see, and what does it promise to return?

Labels as an API contract, the text features that came before embeddings, and how to pick an encoder without trusting a leaderboard.

  1. B1

    Labels and label encoding Planned

    Labels are the model’s API contract; encoding turns them into numbers.

  2. B2

    TF-IDF Planned

    Word weights that favour rare words, with no notion of synonyms.

  3. B3

    Choosing an encoder Planned

    Four families, the raw-BERT trap, contrastive training, retrieval vs similarity.

  4. B4

    Model cards and bake-offs Planned

    Filter by hard limits, then test on your own data.

Module C · Making a decision

0/1 done🔒 Unlocks at 4 of 4 topics in Phase 1 — 0 done.

How do scores become a confident answer, or an honest "not sure"?

From vectors to a label: prototypes, a small learned head, softmax, temperature, blending and per-class thresholds.

  1. C1

    Prototypes Planned

    Average a class’s example texts into one vector; descriptions act as weights.

  2. C2

    The classifier head Planned

    Logistic regression on frozen vectors, with learned weights per class.

  3. C3

    Softmax and temperature

    6 minGuided

    Softmax turns scores into probabilities that sum to one; temperature decides how sure those probabilities sound, and calibration makes them honest.

  4. C4

    Combining two scorers Planned

    Blend calibrated probabilities; the blend weight shifts as labelled data grows.

  5. C5

    Thresholds Planned

    Per-class confidence bars decide whether to answer at all.

  6. C6

    Frozen encoder vs fine-tuning Planned

    Train the smallest part that works.

Module D · Knowing it works

🔒 Unlocks at 1 of 1 topics in Module C — 0 done.

How do I know it is right, and right for whom?

Precision, recall and the metrics built on them, slicing, honest data splits, small samples, and where labels come from.

  1. D1

    Precision and recall Planned

    "Did we find them all?" versus "when we said it, were we right?"

  2. D2

    F1 and macro-F1 Planned

    The harmonic mean punishes imbalance; macro weights every class equally.

  3. D3

    Slicing Planned

    Report metrics per slice, because one number hides everything.

  4. D4

    Train, validation, test Planned

    Textbook, practice tests, final exam.

  5. D5

    Small samples Planned

    Many knobs on little data overfit; confidence intervals say how much.

  6. D6

    Where labels come from Planned

    Humans, LLMs and weak labels, and never grading a system with its own answers.

Module E · Exploring data by hand

🔒 Unlocks at 1 of 1 topics in Module C — 0 done.

What is actually in the data before I label it?

Projection, clustering, neighbours and topic models as tools for looking, plus a labelling workflow that uses them.

  1. E1

    Dimensionality reduction Planned

    PCA, t-SNE and UMAP: look, don’t measure.

  2. E2

    Clustering Planned

    K-means versus HDBSCAN; clusters are not labels.

  3. E3

    k-nearest neighbours Planned

    Label by the neighbours’ vote, and use it to audit mislabels.

  4. E4

    Topic modelling with NMF Planned

    Readable word topics from TF-IDF; weak on short text.

  5. E5

    Gaussian mixture models Planned

    Soft clustering via EM, BIC to pick the count, and novelty detection.

  6. E6

    A labelling workflow Planned

    Read raw data, map, guide, seed, scale, audit, lock the test set.

Module F · Beyond bi-encoders, and shipping

🔒 Unlocks at 1 of 1 topics in Module C — 0 done.

What do I reach for when similarity is not enough, and how do I ship it?

Reading two texts together, two-stage retrieval, and getting a small model onto real hardware without the answers drifting.

  1. F1

    NLI and cross-encoders Planned

    Read both texts together; zero-shot labels via hypotheses.

  2. F2

    Retrieve, then rerank Planned

    A fast filter and a careful judge, measured by recall@k.

  3. F3

    Shipping small models Planned

    ONNX, int8 quantization and parity fixtures.