Tagged “getting-started”
-
Choosing a Local LLM Runtime: Ollama, LM Studio, or llama.cpp
How the main local LLM runtimes differ in practice — packaging, control, and API surface — and a decision framework for picking one and moving on.
-
How to Pick a Local Model for the Job You Have
A repeatable way to choose an open-weight model for a specific task — filter by what fits, shortlist by design intent, then decide on your own work.
-
Serving a Local Model Over HTTP to Your Own Apps
How local LLM runtimes expose an HTTP endpoint, why the OpenAI-compatible shape matters, and how to wire your own scripts and tools into a local model.
-
Local LLMs on Apple Silicon: What Unified Memory Changes
Why Macs punch above their weight for local inference, where the bandwidth ceiling bites, and how to think about memory when CPU and GPU share a pool.
-
Understanding GGUF and Quantization Levels
What GGUF is, how to read quantization labels like Q4_K_M, and how to choose a quant that balances quality and memory for your hardware.
-
How Much VRAM and RAM Do You Actually Need to Run an LLM?
A practical guide to sizing local LLMs: how to estimate VRAM and RAM from parameters and quantization, plus what happens when a model doesn't fit.
-
Run Your First Local LLM with Ollama
A hands-on getting-started guide to running an open-weight LLM locally with Ollama — install, pull a model, chat, and call the local API.
-
CPU-Only Inference: When It's Genuinely Enough
No GPU is not the same as no local LLM. Where CPU inference works well, what governs its speed, and which workloads suit it best.
-
Running a Local LLM in a Container: What's Worth the Trouble
Containers make a local model reproducible and portable — but GPU passthrough and model storage are the two things that decide whether it's worth it.
-
Keeping a Local Model Library From Eating Your Disk
Model files accumulate fast and quietly. Where runtimes store them, how to audit what you have, and a simple policy for what to keep.