A concrete example
When I type a question to Claude, it takes 800 ms before the first word. That wait is the latency.
Why it matters
Latency decides whether an experience is usable. On a voice agent, 2 seconds is dead on arrival.
You'll see it in chat interfaces, voice agents, and real-time automations.
Don't mix it up with
Streaming: Streaming shows you the answer as the model generates it.
TTFT: TTFT measures the time between sending your request and the first token showing up.
Common mistakes
- Confusing latency (first word) with total time (the complete answer).
- Thinking a big model is always slower (depending on the provider, that's false).
- Forgetting that streaming cuts perceived latency, not actual latency.
Quick checklist
- First I check whether the word names a concept, a tool, a risk, or a metric.
- I tie it to a concrete case: When I type a question to Claude, it takes 800 ms before the first word. That wait is the latency.
- I keep the main trap in mind: Confusing latency (first word) with total time (the complete answer).
Quick questions
What is Latency in AI?
Latency is the total delay you feel between your request and the answer.
Where will I run into Latency?
You'll see it in chat interfaces, voice agents, and real-time automations.
Which word should I read next?
Start with Streaming, TTFT, TPS.