A concrete example
When I type a question into Claude, that's inference. The training already happened months ago.
Why it matters
You pay in inference every time you use a model. Understanding that cost means steering your AI usage.
It's what you pay for on every API call or every message handled by a hosted model.
Don't mix it up with
Latency: Latency is the total delay you feel between your request and the answer.
Token: A token is a small chunk of text the AI counts to measure what it reads, writes, and bills.
Common mistakes
- Mixing up training time and inference time.
- Thinking a model learns from your prompts during inference (untrue for most).
- Forgetting that inference costs more on reasoning models.
Quick checklist
- First I check whether the word names a concept, a tool, a risk, or a metric.
- I tie it to a concrete case: When I type a question into Claude, that's inference. The training already happened months ago.
- I keep the main trap in mind: Mixing up training time and inference time.
Quick questions
What is Inference in AI?
Inference is the moment when the model produces an answer from your request.
Where will I run into Inference?
It's what you pay for on every API call or every message handled by a hosted model.
Which word should I read next?
Start with Latency, Token, AI model.