layman/terms
AI words

Training vs inference

tray · ning vs in · fer · enss

In shortTraining is when an AI learns from examples. Inference is when it uses what it learned to answer you.

Learning happens once, slowly and at great cost. Using what was learned happens every time you ask a question.

Before an exam, you spend years at school: reading, practising, getting things wrong and learning from it. Then, on the day, you sit down and answer the questions using what you learned. No new lessons in the exam room.

AI has the same two stages. Training is the schooling: the model learns from enormous amounts of examples. Inference is the exam: it uses what it learned to answer you.

Training

Inference

What it isn’t

Inference isn’t learning. When you correct an AI in a chat, it adjusts for the rest of that chat, but the model itself hasn’t changed.

Where the comparison breaks down

You only sit an exam once. An AI sits millions of “exams” a day, for millions of people, all with the same schooling.

How the words fit together

Machine learning is the method. Training builds the model. Fine-tuning is extra training for a special job. Inference is using it.

Good to know

Chatting with an AI doesn't train it on the spot. Your messages may be saved and used later for training, depending on the app's settings.

When you'll hear it

“Training costs millions.” Teaching a big model takes huge numbers of computers running for weeks.

“Inference is cheap per question.” Each answer costs very little compared with the training.

“Is our data used for training?” Will the company use what we type to teach future versions?

Say it in a meeting

“Will anything we type into this AI be used to train it? Can we switch that off?”