Choosing a Local LLM Runtime: Ollama, LM Studio, or llama.cpp

Most people asking “which local LLM tool should I use” are really asking how much of the machinery they want to touch. Ollama, LM Studio, and llama.cpp mostly run the same models with the same underlying math; what differs is how much is decided for you. Pick based on which layer you want to own, not on which one is “best.”

A useful thing to know up front: llama.cpp is the engine underneath a lot of the ecosystem. Several friendlier tools are, in effect, packaging and workflow built on top of that same inference core. That’s why the quality of a given model at a given quantization is broadly similar across them — you are choosing an interface and an operational model.

llama.cpp: the engine, exposed

llama.cpp is the low-level end. You get a compiled binary (or you build it yourself with the acceleration backend your hardware needs), you point it at a GGUF file you downloaded yourself, and you control the run with flags.

Choose it when you want the knobs. Layer offload, context size, cache behaviour, thread counts, unusual hardware backends — the granular controls live here first, and new features land here before they surface in the wrappers. It is also the leanest option if you’re embedding inference into something else and don’t want a background service.

The cost is that nothing is decided for you. You manage the build, you find and verify model files, you remember which flags a given model wants. If you are still working out how much VRAM and RAM a model needs, doing that while also debugging a build is two problems at once.

Ollama: a service and a registry

Ollama’s bet is that model management should feel like package management. It runs as a background service, pulls models by short name from a curated library, keeps them in its own store, and exposes a local HTTP API. One command gets you from nothing to a chat.

Choose it when you want local models to be infrastructure. Because it exposes an HTTP endpoint, it becomes the thing your scripts and side projects call — see serving a local model over HTTP for what that looks like in practice. It’s also the easiest to automate: pull, run, and query are all scriptable, so a fresh machine is a short setup script rather than an afternoon.

The cost is indirection. Models live in a managed store with its own layout, defaults are chosen for you, and when you want an unusual quantization or a model that isn’t in the library you have to go around the front door. That’s doable — see running a GGUF that isn’t in the library — but it’s a step outside the smooth path.

LM Studio: a desktop application

LM Studio is a GUI. You browse and download models in-app, load one with visible controls for the settings that matter, and chat in a window. It generally also offers a local server mode so other tools can call the loaded model.

Choose it when you want to see what you’re doing. A GUI is genuinely better for the exploratory phase: comparing two quantizations of the same model, watching whether a load actually fits in memory, or fiddling with sampling settings until output looks right. For someone whose main use is “chat with a private assistant,” it may be the whole answer.

The cost is that a desktop app is awkward as a dependency. It’s harder to script, harder to put on a headless box, and it wants to be running for anything else to work.

The comparison that actually decides it

Rather than a feature grid, ask three questions in order:

1. Is a human or a program going to talk to this model? A program wants a service with a stable endpoint. A human wants a window. That single answer eliminates one of the three most of the time.

2. Will this run on a machine with a screen? A home server, an old workstation in a closet, or a cloud box you SSH into rules out the GUI immediately.

3. Do you need a setting the wrappers don’t expose? If you already know you want specific offload or cache behaviour, you’ll end up at llama.cpp eventually. Going there first saves a migration.

They are not mutually exclusive

A pattern worth naming: use the GUI to evaluate and the service to deploy. Download a couple of candidate models in the desktop app, compare them on your real work, decide, and then have the service pull the winner for everything automated. You are not signing up for one tool forever — the models are the durable asset, and GGUF files are portable between runtimes.

The migration cost is genuinely low, which is the strongest argument against agonising over this. Your prompts, your evaluation notes, and your downloaded weights all move. What doesn’t move is time spent researching instead of running something.

A practical takeaway

If you have no strong preference: start with the managed service if you write code and want local inference available to it, and start with the desktop app if you mainly want to chat privately. Reach for llama.cpp when a wrapper blocks you, not before.

One caveat that applies to all three: this tooling changes quickly. Installation methods, default ports, supported hardware backends, and library contents all move, sometimes between minor versions. Treat any specific command you read — here or anywhere — as illustrative, and check the tool’s own current documentation before assuming a flag or endpoint still exists.