Tagged “llama-cpp”
-
Choosing a Local LLM Runtime: Ollama, LM Studio, or llama.cpp
How the main local LLM runtimes differ in practice — packaging, control, and API surface — and a decision framework for picking one and moving on.
-
GPU Layer Offload, and Why Partial Is the Worst Place to Be
How local runtimes split a model between GPU and CPU, why a few spilled layers cost so much speed, and how to decide what to cut instead.
-
CPU-Only Inference: When It's Genuinely Enough
No GPU is not the same as no local LLM. Where CPU inference works well, what governs its speed, and which workloads suit it best.