A concrete example
I hand Claude a photo of my invoice, and it gives me the figures back as JSON. That's multimodal in action.
Why it matters
Multimodal blows open what you can automate: reading scanned docs, analyzing video, transcribing audio.
You'll see it when an AI analyzes a screenshot, transcribes a voice, or understands an image.
Don't mix it up with
Vision model: A vision model understands or describes images.
TTS / STT: TTS turns text into speech. STT turns speech into text.
Common mistakes
- Thinking a multimodal model sees as well as a human (still a long way off).
- Forgetting that images eat up a lot of tokens.
- Mixing up multimodal (inputs) and multimodal generation (outputs).
Quick checklist
- First I check whether the word names a concept, a tool, a risk, or a metric.
- I tie it to a concrete case: I hand Claude a photo of my invoice, and it gives me the figures back as JSON. That's multimodal in action.
- I keep the main trap in mind: Thinking a multimodal model sees as well as a human (still a long way off).
Quick questions
What is Multimodal in AI?
A multimodal model accepts several types of input, like text, images, audio, or video.
Where will I run into Multimodal?
You'll see it when an AI analyzes a screenshot, transcribes a voice, or understands an image.
Which word should I read next?
Start with Vision model, TTS / STT, AI model.