Skip to content
System One

Classification model

A classification model assigns an input to one of a fixed set of labels, and usually returns a probability for each label. The classic version is an encoder such as BERT or DeBERTa fine-tuned on labelled examples. Jev's Choice question does the same job with labels supplied at request time.

Full guide: Is Jev just a zero-shot classifier?

Textbooks use the phrase for anything from logistic regression to decision trees. Here it means the working version: a model your code calls to sort text into labels.

The classic build is an encoder, a model that reads the whole input at once and outputs a vector instead of writing text. BERT and DeBERTa are the usual starting points. You fine-tune one on labelled examples for one label set, and it returns a probability per label. It is small, fast and cheap to run. Its weak spot is that the labels are baked in. A new label means new training data and a new training run.

A Choice question in a System One model is classification with the labels passed in with each request, which makes it a zero-shot classifier. Score and Noul are not classification. Score places an input on a numbered scale and Noul answers yes or no. What the decision-model vendors add is training aimed at calibration, so a stated 80% is right about 80% of the time.

A trained classifier still wins when you have labelled data for a fixed label set. In the Jev benchmarks, independent authors who trained a plain classifier on a few hundred to a few thousand labelled examples found it usually matched or beat zero-shot Jev. Jev’s advantage is that you can start without labels. For models built for the zero-shot case, see zero-shot classification model.

Related terms