Keeping a Local Model Library From Eating Your Disk

A local model library grows faster than anyone expects. Each model is measured in gigabytes, trying a few quantizations of the same model multiplies that, and every runtime keeps its own store in its own place — so the total is spread across directories you didn’t choose and don’t think about. The usual discovery is a full disk, and the usual cause is a dozen models you tried once.

Where the files actually are

Each runtime has its own store, and that’s the first thing to establish. Broadly you’ll have up to three kinds of location:

A managed store. Service-style runtimes keep models in a directory of their own, often with a content-addressed layout — meaning filenames are hashes rather than model names. This is why the directory looks inscrutable and why you should use the runtime’s own commands to list and remove things rather than deleting files by hand.
A scanned directory. Desktop applications typically watch a folder and show whatever GGUF files are in it. Easy to audit, since it’s ordinary files with readable names.
Wherever you put them. Anything you downloaded yourself, per running a GGUF that isn’t in the library, lives where you saved it — commonly a downloads folder, forgotten.

Most runtimes let you relocate their store via configuration, which is the fix if your system drive is small. Consult current documentation for the mechanism, and do the move with the runtime stopped.

Auditing what you have

A worthwhile twenty minutes:

List through each runtime’s own interface first. It knows what it has, what it’s using, and in a content-addressed store it’s the only thing that can map hashes back to names.

Then look at total disk usage per directory. A recursive size summary of each store tells you the real number, which is usually larger than the mental estimate. Compare it against the list — a gap means orphaned files, which happens when a download was interrupted or a model was partially removed.

Look for the specific patterns that waste the most:

Multiple quantizations of one model. Left over from a comparison you already concluded. Keep the winner and delete the others — this is typically the single biggest recovery.
Old versions of a model you’ve since updated. Some runtimes keep the previous version when you pull a new one.
The same model in two runtimes. Duplicate weights in two stores, doing the same job.
Models you tried once. Downloaded during an evening of curiosity, never used again.
Interrupted downloads. Partial files that no longer correspond to anything.

A keep-or-delete policy

Deleting is safe and reversible in the only sense that matters — you can download it again. So bias toward removal, with a few exceptions.

Keep:

Your one or two working models, at the quantization you settled on.
A small fast model. Cheap in space, and genuinely useful for quick tasks and for testing that a setup works — see picking a model for the job.
Anything hard to re-obtain: a model you converted or quantized yourself, or one whose publication you don’t expect to persist.

Delete without much thought:

Losing quantizations from a comparison.
Anything unused for months.
Duplicates across runtimes — pick one store and consolidate.
Large models you concluded were too slow on this hardware.

Note the exception to “just re-download it”: weights do occasionally disappear. Publishers remove repositories, licences change, community fine-tunes go away. If a model matters to you and isn’t from a major publisher, that’s a reason to keep it rather than re-fetch it later.

If you use more than one runtime, some can be pointed at the same directory of GGUF files, and that’s worth setting up — one copy, one place to audit. Support varies and a managed content-addressed store generally can’t participate, so the practical arrangement is a plain directory that the flexible tools read.

The same directory is what you’d mount into a container so the store survives image upgrades, per running a local LLM in a container.

Habits that keep it from recurring

Decide where models live once, ideally on your largest drive, and configure each runtime to use it.
Delete losing candidates at the end of a comparison, while you still remember which was which. A week later you won’t, and they’ll survive by ambiguity.
Check the file size before downloading. Both for whether it fits in memory and for whether it fits on disk. The habit from the sizing arithmetic pays off twice.
Audit when you update. Runtime and model updates are a natural moment to prune — updating models and runtimes safely.

A practical takeaway

Find out where each runtime stores models, measure the real total, and delete the losing quantizations from every comparison you’ve already settled. Most people recover a large fraction of their model directory in one pass, and the models worth keeping turn out to be two or three rather than the fifteen on disk.