Quantization

IST-DASLab ships Qwen3.8-27B as GGUF in four sizes — 8.4 to 11.8 GB. IQ3_S keeps 99.8 % of base performance at a 4.6× reduction and fits on 16-GB cards. With the file table, measured token rates, and a decision rule.

From 8 GB to 512 GB VRAM: which local AI models run on your hardware — with the production recipes and benchmarks from Mia's AI Lab.

DeepSeek-V4-Flash activates 13B parameters per token but keeps 284B resident. Its experts ship in FP4, and that one choice sets almost the whole memory bill.