FP4 Quantization

FP4 quantization represents model weights or activations in a four-bit floating-point format. Instead of storing values with 16 or 32 bits, the model uses a much smaller representation while preserving a limited exponent structure, which can be more useful for neural networks than plain integer-only formats. The main operational benefit is lower memory use and reduced bandwidth pressure. For very large models, especially mixture-of-experts systems with many resident parameters, FP4 can be the reason a deployment fits on a given GPU class at all. The trade-off is precision. Four-bit formats can affect edge cases, rare capabilities, numerical stability, or domain-specific tasks in ways that headline benchmarks may not reveal. That makes calibration, comparison against higher-precision baselines, and production monitoring essential. For business buyers, FP4 matters because many attractive cost and speed claims depend on aggressive low-precision serving. Understanding the format helps teams ask the right procurement questions: what was quantized, what quality loss was measured, and whether the serving stack matches their workload.