Karpathy's
work,
explained simply.
Andrej Karpathy is probably the best AI teacher in the world right now. Former head of AI at Tesla, co-founder of OpenAI. But above all: the only one who takes the time to explain how all of this actually works, no jargon. I sum up what he teaches you — even if you're not a dev.
What you'll learn
- Who Karpathy is and why he matters so much in the world of AI
- The 3 key concepts he explains and that you can grasp without being a dev
- The 4 essential resources to watch / read (in what order, how long)
- An LLM = a text simulator that predicts the next word. That's why it can hallucinate: it makes up whatever "sounds right" even when it doesn't know.
- A token = a unit of counting. French uses 1.5× to 2× more tokens than English to say the same thing. Your AI bill depends on this.
- Prompt engineering = a discipline, not magic. A good prompt on a mid-tier model beats a bad prompt on the best model.
- Transparency: article written with Claude (Anthropic), reviewed and validated by me. If you spot an error or a passage that rings false, write to me, I'll fix it.
Who is Andrej Karpathy?
Karpathy has the most impressive resume in modern AI. PhD at Stanford (a doctorate in artificial intelligence) under Fei-Fei Li. Co-founder of OpenAI in 2015. Head of AI at Tesla for 5 years (in charge of Autopilot). Then a return to OpenAI before leaving in 2024 to found his startup Eureka Labs, dedicated to AI education.
But what makes him unique isn't his CV. It's his obsession: making AI understandable for everyone. While the others keep their secrets, he publishes free 4-hour YouTube courses, opens up the code on GitHub (the code-sharing platform), and explains how to train a GPT from scratch with a notebook and a pen.
Why I follow him
I'm not a dev. But when Karpathy explains how an LLM works, I get it. Everything becomes clear. He's the only one who made me really grasp what's happening under the hood of ChatGPT. After his videos, you don't prompt the same way anymore — you understand why one prompt works and another doesn't.
What he actually teaches you.
Karpathy says a lot of things. Here are the 3 ideas that I take away — the ones that struck me most and that changed how I see AI.
An LLM is a "text simulator"
Karpathy's favorite metaphor: an LLM doesn't "think," it imitates. It has read billions of web pages and, for each word, it guesses what comes next. That's it. No consciousness, no reasoning in the human sense — just a machine that predicts the next word, incredibly well calibrated.
Why it's useful for you: it explains why an LLM hallucinates (it makes up whatever "sounds right" even if it's false), why it's better on topics that are well represented in its training data, and why a good prompt sets the right tone from the very start.
Why "tokens" matter
An LLM doesn't see words. It sees tokens — chunks of words. "cat" is 1 token, "antidisestablishmentarianism" is several tokens. Each token has a cost (time + money).
Why it's useful for you: you understand why some languages cost more (French uses more tokens than English), why a long prompt with a lot of context blows up your bill, and why it's often better to summarize than to spell everything out when you prompt.
Software 3.0: you program AI in plain English
His flagship concept since 2025 (YC AI Startup School talk, June 2025 — YC = Y Combinator, the startup incubator). He splits the history of software into 3 eras: Software 1.0 = we write code, Software 2.0 = we train neural networks (we write weights, not code), Software 3.0 = we program LLMs in plain English. A prompt (the instruction you type into the AI) that's well written is a program.
Why it's useful for you: you don't need to know how to code to "program" an AI. A structured prompt — context, role, format, examples — is literally code in plain English. The people who master this get 10× more value out of it than those who fire off vague requests.
What he's saying in 2025-2026 (beyond these 3 ideas)
Karpathy moves fast. His recent concepts are worth the detour:
- Vibe coding (February 2025) — "forget the code, let the AI code, you just vibe" (in plain terms: you describe what you want, the AI spits out the code): the concept that made Cursor, Lovable and Bolt blow up.
- LLM OS — imagining the LLM as the heart of a new operating system à la Windows or macOS (context = RAM, embeddings = hard drive — the embeddings being the compressed memory of the texts the AI has read, tools = peripherals).
- Jagged intelligence — AIs are "brilliant at X, useless at Y," and unable to consolidate what they learn. Useful for cooling down the hype (the overblown buzz).
- "Decade of agents, not year of agents" — autonomous AI agents are a decade of work ahead, not a year. It pushes you to build with human-in-the-loop (the human stays in the loop to validate) à la Cursor, not to automate everything.
What it changes for your business.
The 3 ideas above, turned into actions you can launch this week — whatever your line of work.
Coach / consultant
Stop sending 3-line prompts. Write them like a brief: context, role, expected format, examples. Your Claude results will jump a level. A well-structured 20-line prompt gets you a deliverable that's 10× better than a vague request.
E-commerce seller
Before writing your 500 product descriptions, do the token math. 500 descriptions × 2,000 tokens = 1M tokens. At $3 per million (Claude Sonnet), that costs you $3. Not $300. Stop being scared of the price — test, measure, scale.
Agency / creative
Your team says Claude "hallucinates"? Normal: it simulates. Give it sources + ask it to cite them. It hallucinates much less when it knows it has to back things up. House rule: no answer without citing the source in the prompt.
The 4 essential resources.
Here's his content, from the most accessible to the most advanced. You don't need to watch all of it — pick what speaks to you.
No time for 15h of videos?
Just watch Deep Dive into LLMs like ChatGPT (3h31, February 2025) — it's his most up-to-date and complete video for a general audience. If you only have an hour, go for the Intro to LLMs (1h, 2023). If it grabs you, you'll move on to the others naturally.
Here are the 4 resources at a glance, so you can choose based on your time and your level.
| Resource | Length | Level | Prerequisites | Format |
|---|---|---|---|---|
| Intro to LLMs | 1 h | Beginner | None | Accessible talk |
| Let's build GPT from scratch | ~2 h | Intermediate | Reading Python | Code-along |
| Neural Networks Zero to Hero (8 videos) | ~14 h 40 | Advanced | Python + basic math | Code-along (series) |
| X account @karpathy | Ongoing | All levels | None | Posts + threads |
1h of general-audience explanation on "what an LLM is, how it works, where it's going." Not a single line of code. Just to understand the basics. If you only watch one Karpathy video, make it this one.
Watch on YouTube →2h where he codes a mini-GPT in Python from scratch, line by line, explaining each concept. You come out with a technical understanding of how a GPT actually works. Accessible if you can read simple code — you don't need to be a senior.
Watch on YouTube →A series of 8 videos (56 min to 2h25 each, ~14h40 total) that goes from "what is a neuron" all the way to building a full GPT-2. The absolute reference for understanding modern AI. Intense but exceptional.
Watch the playlist →Karpathy regularly posts thoughts on AI, threads on what he's testing, takes a stance. One of the most signal-dense accounts in the ecosystem. He's also on GitHub with nanoGPT, llm.c and more recently nanochat (October 2025) — a repo (a public code repository) that trains a full ChatGPT for ~$100 of cloud compute (renting computing power remotely).
See his X →My suggested order
(1) Watch Deep Dive into LLMs (3h31, 2025) first — it's the most up-to-date general-audience video. Or Intro to LLMs (1h, 2023) if you have less time. (2) If it clicks, dive into Let's build GPT (1h56) to understand it technically. (3) And if you're truly hooked, take on the Zero to Hero series. No need to watch all of it — each video is useful on its own.
What he's building now.
Since 2024, Karpathy has left OpenAI to found Eureka Labs, a startup building "the first AI-native school." The idea: an AI teacher who guides you through a course, adapted to your pace, with personalized feedback.
Their first course, LLM101n, is being built in public on GitHub. The goal: learn to build an LLM from scratch, with an AI tutor alongside you. A vision that lines up perfectly with his obsession — explaining AI simply, at scale.
Why it interests me
If Eureka Labs succeeds, it changes education. An AI teacher available 24/7, free or nearly so, that adapts to your level — that's exactly what a lot of people are waiting for. Karpathy is currently building LLM101n: Let's Build a Storyteller, a full course on how to build an LLM from A to Z. He shares the progress in public on his GitHub. No official release date, but all the work is visible live on the repo.
To dig deeper: 2 companion articles
If you want to see how I concretely apply Karpathy's ideas: self-improving agents (a workflow inspired by his approach) and the LLM Wiki, your second brain (the external memory of an agent, Karpathy-style). And if you want to run his ideas locally: how to install an AI on your own machine without coding.
Karpathy FAQ.
Who is Andrej Karpathy in two sentences?
An AI researcher and educator, a Stanford PhD under Fei-Fei Li, co-founder of OpenAI in 2015 and the former head of AI at Tesla for five years (in charge of Autopilot). Since 2024, he founded Eureka Labs, a startup building the first AI-native school with a built-in AI tutor.
Which video should I start with as a beginner?
If you have an hour, watch Intro to Large Language Models (1h, 2023): no code, just understanding what an LLM is. If you have three hours, go for Deep Dive into LLMs like ChatGPT (3h31, February 2025): it's his most up-to-date and complete video for a general audience.
How long for the Zero to Hero series?
The full series is 8 videos for about 14h40 total (from 56 minutes to 2h25 per video). Each video is useful on its own: you don't need to watch all of them. To understand a transformer in depth, the first 4 are enough (~ 8 hours).
What is Software 3.0?
It's his flagship concept since the YC AI Startup School talk (June 2025). Three eras: Software 1.0 = writing code, Software 2.0 = training neural networks, Software 3.0 = programming LLMs in plain English. A well-written prompt is literally a program.
What is the LLM OS?
His metaphor for thinking of the LLM as the heart of a new operating system. The context plays the role of RAM, the embeddings the role of the hard drive, the tools the model can reach the role of peripherals. A useful framing for building AI agents that look like an OS rather than a chatbot.
What is vibe coding?
A concept launched by Karpathy in February 2025 on X: you describe what you want, the AI spits out the code, and you check it by eye and by feel. It's the concept that made Cursor, Lovable and Bolt blow up. The developer becomes a conductor more than a pianist.
Why did Karpathy leave OpenAI?
He left OpenAI in February 2024 to found Eureka Labs, a startup dedicated to AI education. His vision: a course where an AI tutor guides each student at their own pace, with personalized feedback. First course being built in public: LLM101n, Let's Build a Storyteller.
Does Karpathy speak French?
No, all his videos are in English. But YouTube generates good-quality automatic subtitles in many languages. Turn them on with the CC button at the bottom of the player, then the settings gear → subtitles → auto-translate → your language.
What is nanochat, his October 2025 project?
A GitHub repo that shows how to train a full ChatGPT for ~100 dollars of cloud compute. The follow-up to nanoGPT and llm.c. Educational goal: demystify training an LLM by showing every line of code.
Why does he explain things better than a university course?
Three ingredients: (1) he codes live instead of showing slides, (2) he always starts from a concrete case before the theory (an LLM predicting a word, not an abstract transformer), (3) he embraces the breakages, redoes them, explains why. University aims for exhaustive rigor; Karpathy aims for intuitive understanding.
Spotted an error?
An outdated fact, a number that's moved, a stale source? Write to me at sagnier.jeremy@gmail.com · I fix it within 48h max and note the update date at the top of the article. Feedback from the field is worth a thousand times more than the articles — I read everything, I reply.
Shall we keep going?
I test AI for real and share what works, no jargon and no hype. If this article helped you, the easiest way to never miss anything is my Friday letter. And if you have a question or a doubt: reply to me, I read everything.