Zain's AI Inference Lab

Zain Fathoni's public lab notebook for AI inference runtime engineering — from Triton kernels to deployable inference systems.

Start here

The short-term forcing function is readiness for Netra Runtime-style inference puzzles: six technical tasks covering Triton kernels, quantization/dequantization, torch.compile, QLoRA/FSDP2, and benchmark-driven explanations.

The long-term direction is AI runtime engineering: understanding how models move from PyTorch code to fast, observable, deployable inference systems.

• Mission · the learning and career strategy behind this lab. raw
• Resources · the sources, codebases, and constraints guiding the work. raw

Lessons

• Lesson 0 — What inference actually is · the training-vs-inference split, for a fullstack dev.
• Lesson 1 — Triton deletes a level · how the CUDA thread→block→grid model re-draws itself in Triton.
• Lesson 2 — Inference is the half you pay for forever · prefill vs decode, the roofline, and why serving is a memory-bandwidth problem.
• Lesson 3 — A tensor is a pointer plus arithmetic · strides, views, and why transpose is free but not cheap.
• Lesson 4 — The GPU fetches more than you asked for · 32-byte sectors, and the 8× tax on a strided read.
• Lesson 5 — Quantization: buying bandwidth with precision · absmax, NF4, block size. This is Netra Task A.
• Lesson 6 — Matmul earns its intensity by tiling · why naive matmul runs at 0.3% of peak.
• Lesson 7 — Softmax, overflow, and the online trick · the arithmetic behind FlashAttention.
• Lesson 8 — Attention and the KV cache · the real ceiling on batch size.
• Lesson 9 — Benchmarking a kernel honestly · how a measurement becomes evidence.

See the full curriculum — the arc, why the order, and what is deliberately not covered yet. raw

Artifact types

• Lessons teach one concept with the AI-assisted /teach style.
• Learning records capture my recall, corrections, and what actually clicked.
• Experiments run code, capture output, measure behavior, and document failures.
• Portfolio artifacts are curated later from the strongest records and experiments.

Near-term plan

1. Launch this lab framing and point ai.zainf.dev here.
2. Run Experiment 0001: a first Triton kernel on a free T4 via Kaggle or Colab.
3. Publish the correctness check, benchmark table, notebook link, and failure notes.
4. Use that loop to attack Netra Task A-style NF4 dequantization.

Learning records

• Record 0001 — Triton program instance recall · waiting for learner recall and Experiment 0001 output.

Reference

• CUDA ↔ Triton cheat-sheet · the execution-model vocabulary, compressed.
• The inference cost model on one page · prefill/decode formulas, machine balance, and the lever table.

💬 This is a teaching lab first and a polished portfolio later. Lessons are allowed; mastery is tested through recall, correction, execution, and measurement.

github.com/zainfathoni/ai-inference-lab