FP8 Mixed Precision

Training or inference that uses 8-bit floating-point formats for much of the compute, cutting memory and speeding up large model training when done carefully.

How it works

FP8 uses only 8 bits to represent floating-point numbers, compared to 16 bits for BF16 and 32 bits for FP32. Two FP8 variants exist: E4M3 (4 exponent bits, 3 mantissa bits), which has higher precision and is good for weights and activations, and E5M2 (5 exponent bits, 2 mantissa bits), which has a wider range and is used for gradients. Mixed precision keeps sensitive operations in higher precision while running most matrix multiplications in FP8, halving memory use and doubling throughput on compatible hardware.

Why it matters

Training frontier models like DeepSeek-V3 with FP8 was one of the key reasons their training was so cost-efficient. Halving the memory footprint means you can fit larger batches, train larger models, or use fewer GPUs for the same workload. As model sizes continue to grow, FP8 (and even lower precision formats) are becoming essential tools for keeping training and inference economically viable.