Sparse vs Dense (FP4)

Sparse and dense describe whether a headline compute figure counts skipped-zero operations or every operation. It matters because NVIDIA's GB10 '1 PFLOP FP4' rating is a sparse number — dense compute is roughly half, ~500 TFLOPS, about RTX 5070-class. FP4 is a 4-bit floating-point format used for that peak rating. When you compare AI accelerators, a sparse figure and a dense one aren't the same measurement, and the sparse one is always the bigger, more marketable number.

Read the fine print on any headline AI-performance spec: 'sparse' peaks roughly double the dense reality. Compare boxes on dense figures — and remember bandwidth caps LLM speed regardless of either number.