Tagged “gguf”
-
Understanding GGUF and Quantization Levels
What GGUF is, how to read quantization labels like Q4_K_M, and how to choose a quant that balances quality and memory for your hardware.
-
Is That Quantization Good Enough? Test It, Don't Guess
Stepping down a quant level to fit your GPU is usually worth it — but the only way to know for your work is a small repeatable check you run yourself.
-
Running a GGUF Your Runtime's Library Doesn't Offer
Curated model libraries are a subset. How to find, vet, and load an arbitrary GGUF file yourself when the quant or model you want isn't listed.
-
Keeping a Local Model Library From Eating Your Disk
Model files accumulate fast and quietly. Where runtimes store them, how to audit what you have, and a simple policy for what to keep.