Almost every local-AI guide assumes you own a graphics card. Most of us reading “just get a 24GB GPU” are quietly thinking: I don’t have one, and I’m not buying one right now. So here’s the question nobody seems to answer honestly — can you run a local LLM with no dedicated GPU at all?
I can, because that’s exactly my setup. This site runs on the philosophy of testing on the hardware you actually have, so this one is personal: an AMD Ryzen 5 5600G, its integrated Radeon graphics, and 16GB of RAM. No discrete card. Here’s what that machine can really do.
The setup
Nothing exotic. The relevant parts:
| Component | Spec |
|---|---|
| CPU | AMD Ryzen 5 5600G (6 cores, integrated Radeon) |
| Dedicated VRAM | ~0.5 GB (the iGPU barely counts) |
| RAM | 16 GB DDR4-2666 |
| Runtime | Ollama (CPU inference) |
With no real VRAM to speak of, Ollama runs the model on the CPU and keeps the weights in system RAM. That single fact decides everything about the experience: speed is bound by your CPU and, more than anything, your memory.
What I actually ran
Two models, both through Ollama, both usable:
Gemma 2B — small, fast, and genuinely fine for quick tasks. On a CPU this is the comfortable zone. Responses feel responsive and it barely dents the RAM.
Mistral 7B — this is the one I leaned on. I wired it into a personal voice assistant project so it could run fully local, without sending anything to a cloud API. And it worked. Not flawlessly, but well enough to be genuinely useful.
The honest detail: on this machine, Mistral 7B takes roughly 2 to 4 seconds to start responding to a prompt. Once it gets going, the text streams at a readable pace. Compare that to a cloud API, where the first words appear near-instantly, and you feel the gap. But for a local, private, $0-running-cost assistant, a 2–4 second pause is a trade I’ll take.
The thing nobody tells you: your RAM is the bottleneck
Here’s the most useful thing I learned, and it’s not in most guides.
When you run an LLM on the CPU, the model has to stream its weights from RAM on every token. That means memory bandwidth — not CPU clock speed — is usually what limits your tokens per second. And memory bandwidth depends heavily on whether your RAM runs in dual channel.
A lot of budget builds (mine included, at first) ship with a single RAM stick. That’s single channel, and it roughly halves your memory bandwidth. Adding a second matched stick to run in dual channel can give you a large, almost free speed-up for CPU inference — often close to double the generation speed — for the price of one RAM module.
If you’re running LLMs on CPU and only have one stick installed, that’s the first upgrade to make. It’s cheaper than a GPU and the impact is real.
What fits in 16GB
A rough guide for CPU + 16GB RAM, using 4-bit quantized models:
| Model size | Runs? | Feel |
|---|---|---|
| 2B–3B (Gemma 2B, Phi mini) | Yes, easily | Snappy |
| 7B–8B (Mistral 7B, Llama 8B) | Yes | 2–4s to start, then readable |
| 13B–14B | Technically | Slow, getting painful |
| 30B+ | Don’t | Not worth it on this hardware |
The 7B class is the sweet spot here — large enough to be genuinely capable, small enough to stay usable. Push past it and the wait between prompt and response stops being worth it.
So, is it worth running LLMs without a GPU?
Yes — with honest expectations. If you want instant, cloud-speed answers, you’ll be frustrated. If you want a private, free, always-available model that runs on the PC you already own, a 7B model on a modern CPU absolutely delivers. I built a working local voice assistant on exactly this hardware.
If you do one thing after reading this: check whether your RAM is running in dual channel. It’s the highest-impact, lowest-cost change you can make for local AI on a GPU-less machine. After that, stick to 7B-class models, be patient with the first couple of seconds, and enjoy running AI that never leaves your desk.