Run a real AI
on your own
machine.

Why it's gotten serious.

Two years ago, running an AI at home was the business of a bearded geek who spent weekends in the terminal. Today, it's become simpler than installing Photoshop. You download an app, you double-click, you pick a model from a list, and it's done.

What changed in 2025-2026: open models caught up with closed ones. Mistral, Llama, DeepSeek, Qwen — you can download AIs for free that rival ChatGPT for most daily uses. And on January 27, 2025, an unknown Chinese company (DeepSeek) released a model that wiped $589 billion off NVIDIA in a single day. The whole "you need billions and €100,000 servers to do AI" thesis collapsed.

If you've already read my article on open source, you know where this movement comes from. Here, we move to the next step: how you put it to work, concretely, on your computer, this weekend.

Five concrete reasons to get into it

The two others, subtler but they change the game:

No commercial filter. Cloud AIs sometimes refuse to help you "because it's legal advice" or slap a disclaimer on you every 3 lines. A local AI does what you ask, full stop.

You can plug in your own documents. You drop your PDFs, your notes, your contracts into the tool — and your AI answers based on your documents, not its general memory. We'll cover that in detail at the end of the article.

Why you might give up.

I'd rather tell you right away, instead of you finding out after two hours of installation. Local AI is great, but it's not magic. Here are the four things that could frustrate you if you don't know them from the start.

My honest take: if you're after a single AI that does everything, the cloud is still ahead for now. But if you're after an AI that respects your data, works without wifi, and costs you nothing after install — and you're OK keeping ChatGPT on the side for the 10% of cases where local falls short — then you're going to love what follows. The right answer in 2026 is local + cloud, not local OR cloud.

What you really need.

Before going any further, you need to know whether your computer can do it. Good news: most MacBooks and PCs bought since 2022 are eligible. Here's the equation, explained simply.

The equation in two words: memory

An AI model is a big file (between 4 and 40 GB depending on its size) that has to be loaded into your computer's memory to run. If your memory is too small, your computer will "write" the rest to the hard drive — and then everything gets painfully slow. An answer that should take 3 seconds takes 3 minutes.

So question No. 1 is: how much RAM do you have? You can see it in your computer's settings (Mac: "About This Mac," Windows: right-click "This PC" then Properties).

Three budgets, three options

I've tested, compared, asked the community. Here's what I concretely recommend depending on what you can spend.

The clever option that changes everything: grab a desktop PC + a used RTX 3090 graphics card (€650 to €800) + 64 GB of RAM, total around €1,500. It's what the community that knows its stuff (on Reddit, r/LocalLLaMA) has recommended for 18 months as the best value on the market.

Mac or PC?

Macs with an Apple Silicon chip (M1, M2, M3, M4) have a magical quirk called "unified memory." Plainly: RAM AND video memory share the same pot. If your MacBook has 48 GB, all 48 GB can serve the AI. On a PC, RAM and video memory are separate — which makes things a little more complicated.

In practice: if you're already on Mac, you don't have to change anything. If you're on PC, you can do the same — it's just a bit more technical at first. If you're a total beginner and still on the fence, go Mac: the install simplicity is unbeatable.

The four tools to know.

There are about thirty tools for running an AI locally. You don't need to know them all. These four cover 95% of needs, from the absolute beginner to the curious one who wants to dig deeper.

The winning 2026 combo for non-devs

If I had to recommend only one combination, it would be Ollama + Open WebUI. It's what gives you:

• The simplicity of Ollama for downloading and managing models
• The ChatGPT interface of Open WebUI (history, multiple conversations, syntax-highlighted code)
• The ability to chat with your own documents (drag and drop a PDF and ask questions)
• Sharing with your family or your team if you want (everyone gets their own account, their own private chats)
• All of it, 100% free and entirely on your machine

It's what we'll install together below, step by step.

Which model for what.

Once your tools are in place, you'll have to pick a "model" to download. It's the equivalent of a brain you install into the engine. There are dozens, sorted into six big families. Here's the overview, without going into the technical weeds.

How to read the names (in two sentences)

When you see "Mistral 7B," the "7B" means 7 billion parameters. The bigger the number, the smarter the model — and the more memory it eats. A "7B" model weighs about 4 GB and fits in 8 GB of memory. A "70B" weighs 40 GB and needs 64 GB. That's it, that's all you need to know to get started.

The six families to know

The five models to try first

If you're starting today, download these five in this order. Total: about 30 GB, and you'll form an opinion in one afternoon.

My blunt advice: start with Mistral 7B. It's enough for 80% of uses — writing, summarizing, brainstorming, translating, general questions. You'll quickly see whether you need something bigger or more specialized. No point filling your hard drive on day one.

We install together.

Here's the procedure, validated and tested. I'll use Mac as the running example because it's the simplest, but I give the Windows and Linux variants at each step. Count 15 minutes the first time, half of it waiting for downloads.

Six examples that justify it all.

"OK it's installed, but what do I do with it?" Here are six concrete use cases, by profession, that on their own justify the install. If one of them sounds like you, then it'll be worth your while.

Chat with your own documents.

Here's the feature that changes your life, and that nobody ever sells you properly. It's what's called "RAG." The word is scary, the concept is very simple.

The analogy that clicks

Imagine you hire a brilliant assistant. Without RAG, you ask them questions and they answer with what they learned in school — general knowledge, in other words. With RAG, you give them your personal library and you tell them "answer me based on these books, and cite the exact pages."

In practice: you drop your PDFs, your Word notes, your Markdown into Open WebUI. The tool chops them into little chunks, indexes them, and when you ask a question, it goes and finds the relevant passages in your documents and answers based on them.

How to turn it on in Open WebUI

You've already installed everything by following the tutorial above. Here are the 3 steps to use RAG:

1. Create a knowledge base: in Open WebUI, click "Workspace" on the left, then "Knowledge," then "+ New Knowledge." Give it a name (for example "My client contracts").
2. Drag and drop your documents: you can add PDFs, Word files, Markdown, notes. The tool indexes them automatically (count a few minutes for 50 documents).
3. Ask your questions: create a new chat, select your knowledge base from the menu, ask your question. The model answers while citing the exact sources (page X of document Y).

The six cases where RAG is unbeatable

The limit to know about: quality depends on your documents. An unreadable scanned PDF, a chaotic Excel file, badly structured notes — that'll give mediocre results. The cleaner your sources, the better the answers.

Local vs cloud: the verdict.

The "local OR cloud" debate is framed wrong. The right answer in 2026 is local AND cloud, depending on the need. Here's my honest decision matrix.

My personal workflow (the honest version)

Concretely, here's how I organize day to day, in April 2026:

• Local: everything that touches my professional documents, my contracts, my sensitive drafts. Plus audio transcription and daily chat when I can wait 2 extra seconds for an answer.
• Cloud (ChatGPT, Claude): current web search, heavy complex reasoning, modern agentic work, and the urgent stuff where every second counts.

Three apps handle both in the same interface if you want to switch in one click: Msty (the most polished), Open WebUI (open source, you can plug OpenAI or Claude into it on top of local), Cherry Studio (an open-source alternative to Msty).

The five beginner mistakes.

And after that?

Once you've mastered the Ollama + Open WebUI + RAG combo, you've unlocked 95% of the uses. Here are the avenues to go further if you want to dig in.

• Continue.dev — a free VS Code extension that plugs your Ollama into your editor. You swap GitHub Copilot (€10/month) for free, local. 10 minutes of setup for anyone who codes.
• n8n + Ollama — n8n is the open-source equivalent of Zapier. You can plug your local AI into workflows: "when an email arrives, summarize it, post to Slack." For the tinkerers.
• Home Assistant + Ollama — control your home by voice, locally, without Alexa listening to everything. A setup for enthusiasts (one weekend).
• Image generation with FLUX — install ComfyUI and you generate your images at home. Count 2 days to learn the interface, then you do whatever you want.
• Fine-tune your own model — the next step, ambitious but accessible. You take an existing model and re-teach it your writing style or your domain. For the diehards, but doable at home on an M4 Mac.

What's coming in 2026-2028 that's going to speed everything up: Apple Intelligence built into macOS and iOS, the NPU chips in new Windows PCs (Copilot+ PC), models that keep getting smaller and more capable. By 2027, models that fit on a phone will rival today's ChatGPT-4. The trajectory is clear: what was impossible at home in 2023 will be trivial in 2027.

If you haven't yet read my article that lays the foundations — what open source is, how companies make a living giving things away for free, why open AI changes everything — it's right here: Open source explained for those who aren't devs. It's the natural companion to this one. And if you want the story of the moment DeepSeek made NVIDIA collapse, I tell it in episode 4 of my AI Wars podcast.

Three things to do this weekend

1. Install Ollama and have your first chat. 10 minutes flat. You'll see with your own eyes that an AI can run on your computer with no subscription.
2. Push on to Open WebUI if you want the pretty ChatGPT interface and RAG. 15 more minutes.
3. Load your last 10 work PDFs into a knowledge base and ask them a real professional question. That's the moment you understand why it's non-negotiable for certain professions.

If you want me to send you more tutorials like this (on AI, open source, the tools I test for myself first), sign up for AI Playbook — it's my weekly watch, I share with you the same thing I keep for myself. And if you get stuck on a step, or you think I've got something wrong somewhere, write to me. I read everything, I don't take it badly.

FAQ on local AI.