Quantization is a bandwidth trick that happens to save memory

Lesson 5 · absmax, INT4 and NF4 · this is Netra Task A

This is the lesson your short-term goal actually needs. Netra Task A is NF4 dequantization, and Lesson 2 already told you why anyone bothers: decode time is weight_bytes ÷ bandwidth. Quarter the bytes, quarter the time. Everything below is about what that costs you in accuracy — and where the cost is hiding.

Get the motivation the right way round

Most explanations lead with "quantization lets you fit a big model on a small card." True, and second in importance. The first-order effect is the one from Lesson 2:

Decode re-reads every weight, for every token. It is memory-bound. So the bytes per weight is the latency. FP16 → INT4 is not a 4× memory saving that happens to be fast; it is a 4× speedup that happens to save memory.

That reframing tells you where to look for wins and, just as usefully, where not to: quantizing weights barely helps prefill, which is compute-bound. Same lesson, same simulator, already proven.

Absmax quantization, in four lines

Symmetric absmax quantization and its inverse, for a block of weights.

Listen first: four steps. One, take a block of weights and find the largest absolute value in it — the absmax. Two, divide that by the largest integer code you can store, giving a scale; for signed four-bit that largest code is seven. Three, quantize by dividing each weight by the scale and rounding to the nearest integer. Four, dequantize by multiplying the integer back by the scale. The stored artefact is the small integers plus one scale per block. Notice that the scale is shared: one weight sets it for everybody.

absmax = max(|w|) over the block          # ONE value shared by the whole block
scale  = absmax / 7                       # 7 = largest code in signed 4-bit
q      = round(w / scale)                 # store this  (4 bits each)
w'     = q * scale                        # dequantize: this is what the kernel rebuilds

Read line 1 again, because it is the whole game. One weight sets the step size for the entire block. If a block contains a single huge outlier, the scale inflates and every other weight in that block is rounded with a coarser ruler. Outliers do not damage themselves — they damage their neighbours.

What NF4 changes

INT4-absmax spaces its 15 codes uniformly between −absmax and +absmax. But neural network weights are not uniform — they are roughly normally distributed, clustered near zero. Uniform levels spend most of their resolution on tail values that barely occur.

NF4 keeps 4 bits but moves the 16 levels to the quantiles of a standard normal, so each level carries roughly equal probability mass. Dense near zero, sparse in the tails — matching where the weights actually are.

The 16 NF4 levels, normalized to [-1, 1], as used by bitsandbytes.

Listen first: sixteen fixed levels running from minus one to plus one, with an exact zero in the middle. The spacing is the point: between zero and roughly zero-point-one-six there are two levels, while the entire stretch from zero-point-seven-two to one contains only one. Resolution is concentrated where normally-distributed weights actually live. These numbers are a fixed lookup table, not something computed per model — which is exactly why dequantizing NF4 is a table lookup and a multiply, and why it makes a good kernel exercise.

NF4 = [-1.0,      -0.6961928, -0.5250731, -0.3949175,
       -0.2844414, -0.1847734, -0.0910500,  0.0,
        0.0795803,  0.1609302,  0.2461123,  0.3379152,
        0.4407098,  0.5626170,  0.7229568,  1.0]

dequantize:  w' = NF4[code] * absmax_of_block     # lookup + multiply

The honest scorecard, measured on the 64 normally-distributed weights in the micro-world below:

Format, block 64	RMS error	Worst-case error	Bits/weight
INT8 absmax	0.0062	0.0103	8.25
INT4 absmax	0.1026	0.1849	4.25
NF4	0.0815	0.2324	4.25

NF4 wins on average error — about 1.26× better RMS at identical bits — and loses on worst case. That is not a flaw, it is the trade it was designed to make: its levels are sparse in the tails, so the rare large weight is reconstructed less precisely, in exchange for better accuracy on the many small ones. Anyone who tells you NF4 is strictly better is skipping the second column.

Where the 0.25 bits come from

You store 4 bits per weight plus one fp16 scale per block. At block size 64 that is 4 + 16/64 = 4.25 bits per weight. Shrink the block to 16 and error drops — but the overhead triples to 4 + 16/16 = 5.0 bits. That tension is the dial this lesson is really about. (QLoRA's "double quantization" then quantizes the scales themselves, clawing 4.25 back to about 4.127.)

Micro-world: quantize a real block and watch it break

64 weights, one scale per block

The weights are a fixed pseudo-random normal sample — identical on every load, so you can compare runs honestly. Bars show absolute reconstruction error per weight.

Format

Block size

Weight distribution

Absolute reconstruction error, per weight (all 64, in order)

Inside block 0 — the first eight weights

Compare INT4 and NF4 at block 64, normal weights. Same 4.25 bits/weight. NF4's RMS is lower — but check the worst-case row and watch it go the other way. Both facts are real.
Switch the distribution to "one big outlier" and keep block 64. One weight changes, and the error bars go up across the whole block. Nothing else about the data moved. This is the misconception this simulator exists to break: quantization error is not a property of a weight, it is a property of the company it keeps.
Now shrink the block to 16, outlier still on. The damage collapses to the one block that actually contains the outlier — the other three are untouched. That is why block size exists.
Watch the bits/weight line as you shrink the block. 4.25 → 4.5 → 5.0 → 6.0. You are buying accuracy with bandwidth, which is the exact currency Lesson 2 said you were short of.
Set INT8 at block 8 and read the decode-speedup line. You have spent 10 bits/weight to get excellent accuracy and thrown away most of the speedup. Every quantization decision is this trade.

Check yourself

Cold-recall defense (one breath):
Decode is memory-bound, so bytes per weight is latency — quantization is a speed technique first. Absmax: scale = absmax(block)/max_code, store round(w/scale), rebuild q*scale. One weight sets the scale for the whole block, so an outlier degrades its neighbours, and block size is the dial: smaller block = less outlier damage, more scale overhead (bits + 16/block; NF4 at block 64 = 4.25 bits/weight). NF4 keeps 4 bits but places its 16 levels at normal quantiles — better average error than uniform INT4, worse tail error. Dequantizing NF4 is a table lookup times the block's absmax.

Where this points, for Netra Task A

You now have the whole shape of the kernel before writing a line of it. A dequantization kernel per program instance: load a tile of packed 4-bit codes, load that block's absmax, unpack two codes per byte, look each up in the 16-entry NF4 table, multiply by absmax, store fp16. Three things to think about while you write it, all of which are earlier lessons:

The packed codes are two weights per byte — your offsets are no longer element-indexed. That is a Lesson 3 problem.
Reading them must stay coalesced, and fp16 has a lower coalescing floor than fp32. That is a Lesson 4 problem.
The kernel is memory-bound, so benchmark it in GB/s of output, and compare against bitsandbytes rather than against nothing. That is Lesson 9.

Primary source

Read QLoRA: Efficient Finetuning of Quantized LLMs §3 — the sections defining 4-bit NormalFloat, double quantization, and blockwise absmax (~15 min for those sections; skip the finetuning results for now). It is the paper NF4 comes from. Then skim bitsandbytes for the reference implementation you will be benchmarking against.

💬 I'm your teacher for this — ask me followups any time. Two good ones: "why is the scale fp16 and not fp32?" and "what would activation quantization change?" Ask either and it becomes a lesson.