Curriculum

The lesson sequence for this lab, and why it is in this order.

Every lesson is one tightly-scoped idea with one tangible win, an embedded micro-world you can drive, and an in-page quiz. Lessons teach; they are not proof of mastery. The proof is in records/ and experiments/.

The arc

The sequence answers one question in stages: what makes a fixed pile of arithmetic go fast on a GPU, and how do I prove I made it faster?

| # | Lesson | The one win | Type | | - | ------ | ----------- | ---- | | 0 | What inference actually is | Training is not your job; inference is a hot path | orientation | | 1 | Triton deletes a level | You program a block, not a thread | mechanism | | 2 | Inference is the half you pay for forever | Serving is a memory-bandwidth amortization game | mechanism | | 3 | A tensor is a pointer plus arithmetic | Shape is a lie; strides are the truth | mechanism | | 4 | The GPU fetches more than you asked for | Access pattern, not byte count, sets your bandwidth | mechanism | | 5 | Quantization: buying bandwidth with precision | Block size is the accuracy/​compression dial — Netra Task A | mechanism | | 6 | Matmul earns its intensity by tiling | Reuse is what makes matmul compute-bound | mechanism | | 7 | Softmax, overflow, and the online trick | Why attention needs a running max | mechanism | | 8 | Attention and the KV cache | The cache is what caps your batch size | mechanism | | 9 | Benchmarking a kernel honestly | Most speedups are measurement bugs | mechanism |

Why this order

• 0–2 build the frame. What the job is, how the hardware is addressed, and what actually costs money. Lesson 2 is the economic argument the rest serves.
• 3–4 are the memory pair. Layout decides addresses; addresses decide coalescing; coalescing decides the bandwidth you actually get. Every memory-bound kernel — which is most of them — is won or lost here.
• 5 is the Netra forcing function. NF4 dequantization is Task A. It only makes sense once you believe the memory-bound argument from Lesson 2 and understand blocks from Lesson 1.
• 6–8 are the model. Matmul, softmax, attention, KV cache — the ops that eat the time, in the order they depend on each other. Lesson 8 answers the question Lesson 2 deliberately left open: what stops me raising the batch?
• 9 closes every loop. A benchmark you cannot defend is not evidence, and every experiment in this lab is supposed to be evidence.

Reference docs

• CUDA ↔ Triton cheat-sheet — execution-model vocabulary
• The inference cost model on one page — roofline formulas and the lever table

Not covered yet

Deliberately deferred, in the order they are most likely to be needed next:

• Kernel fusion as a bandwidth technique, and what torch.compile fuses for you
• Model loading: safetensors, mmap, and cold-start cost
• Continuous batching and the throughput/latency tradeoff in a real server
• Paged and quantized KV cache
• QLoRA / FSDP2, once the quantization lesson has an experiment behind it

Ask for any of these and it becomes the next lesson.