Blog
-
Feeding Web Pages to a Local Model Without Wrecking the Context
Raw HTML is mostly not content, and locally every token costs VRAM and wall-clock time. Why markdown is the format a small model can actually read.
-
Giving a Local Model Access to the Live Web
A local model's knowledge stops at its training cutoff. Wiring search into it closes the gap — and quietly ends the guarantee that nothing leaves your box.
-
What a Tool Call Looks Like From the Model's Side
Local models don't run code or reach the network. Tool calling is a text protocol plus a loop you write — and small models fail at it in specific ways.
-
Choosing a Local LLM Runtime: Ollama, LM Studio, or llama.cpp
How the main local LLM runtimes differ in practice — packaging, control, and API surface — and a decision framework for picking one and moving on.
-
How to Pick a Local Model for the Job You Have
A repeatable way to choose an open-weight model for a specific task — filter by what fits, shortlist by design intent, then decide on your own work.
-
Why Your Local Model Is Slow: A Diagnostic Order
Local inference feels sluggish for a handful of reasons. Check them in this order — memory spill first — and you'll find the cause without guessing.
-
Serving a Local Model Over HTTP to Your Own Apps
How local LLM runtimes expose an HTTP endpoint, why the OpenAI-compatible shape matters, and how to wire your own scripts and tools into a local model.
-
What to Look For in a GPU for Local Inference
The GPU properties that actually govern local LLM performance — capacity first, then bandwidth — and why the spec people shop on matters least.
-
Working Within 8 GB of VRAM
8 GB is enough for real local LLM work if you spend it deliberately. How to budget weights against context, and which trade-offs to make first.
-
Local vs Cloud LLMs: Privacy, Cost, and When Each Wins
An honest comparison of running LLMs locally versus using cloud APIs — privacy, cost, latency, capability, and which to choose for real use cases.
-
Local LLMs on Apple Silicon: What Unified Memory Changes
Why Macs punch above their weight for local inference, where the bandwidth ceiling bites, and how to think about memory when CPU and GPU share a pool.
-
Understanding GGUF and Quantization Levels
What GGUF is, how to read quantization labels like Q4_K_M, and how to choose a quant that balances quality and memory for your hardware.
-
GPU Layer Offload, and Why Partial Is the Worst Place to Be
How local runtimes split a model between GPU and CPU, why a few spilled layers cost so much speed, and how to decide what to cut instead.
-
What a Long Context Window Costs on Local Hardware
Context isn't free locally — it takes memory for the KV cache and time on every turn. How to size a context window to what you actually use.
-
How Much VRAM and RAM Do You Actually Need to Run an LLM?
A practical guide to sizing local LLMs: how to estimate VRAM and RAM from parameters and quantization, plus what happens when a model doesn't fit.
-
Prompt Processing vs Token Generation: Two Different Speeds
Local inference has two phases with opposite bottlenecks. Knowing which one you're waiting on tells you whether to change the model or the prompt.
-
Run Your First Local LLM with Ollama
A hands-on getting-started guide to running an open-weight LLM locally with Ollama — install, pull a model, chat, and call the local API.
-
CPU-Only Inference: When It's Genuinely Enough
No GPU is not the same as no local LLM. Where CPU inference works well, what governs its speed, and which workloads suit it best.
-
Is That Quantization Good Enough? Test It, Don't Guess
Stepping down a quant level to fit your GPU is usually worth it — but the only way to know for your work is a small repeatable check you run yourself.
-
Running a GGUF Your Runtime's Library Doesn't Offer
Curated model libraries are a subset. How to find, vet, and load an arbitrary GGUF file yourself when the quant or model you want isn't listed.
-
Model Load Time, Cold Starts, and Keeping a Model Warm
The first request after an idle period pays to read gigabytes from disk. Why that happens, when to prevent it, and what keeping a model resident costs.
-
What Happens When Two Requests Hit Your Local Model at Once
Local runtimes handle concurrency very differently from hosted APIs. What queueing and batching cost you, and how to plan for more than one caller.
-
Running a Local LLM in a Container: What's Worth the Trouble
Containers make a local model reproducible and portable — but GPU passthrough and model storage are the two things that decide whether it's worth it.
-
Exposing a Local Model on Your Network Without Regretting It
Local runtimes bind to loopback for a reason. What changes when you make one reachable from other machines, and how to do it deliberately.
-
Running Locally: What Actually Leaves Your Machine
Inference staying local doesn't mean nothing goes out. The traffic a local LLM setup really generates, and how to verify rather than assume.
-
Keeping a Local Model Library From Eating Your Disk
Model files accumulate fast and quietly. Where runtimes store them, how to audit what you have, and a simple policy for what to keep.
-
Updating Models and Runtimes Without Breaking What Works
Local LLM tooling moves fast, and updates change behaviour. How to upgrade deliberately, verify nothing regressed, and keep a way back.