Inference optimization: study hub

Prep for an inference-optimization role at a frontier lab, Modal, Baseten, Fireworks or similar.

1. Where you stand

Strengths (lead with these)	Gaps screeners probe (close these)
Production inference, latency optimization, caching	Custom GPU kernel development
Hardware migrations, cost/performance decisions	Deep GPU architecture and memory optimization
Model certification against serving SLAs	Lower-level PyTorch/JAX, strong C++, Nsight profiling
Deployment and platform ownership	Microsecond trading workloads (a different domain; skip unless you target it)

Check the actual job posting Each company weights these differently, and I'm describing them generally, not from inside knowledge. Serverless-GPU platforms tend to stress cold starts, scheduling and multi-tenancy; model-API companies and labs tend to stress kernels, quantization, parallelism and throughput per GPU. Match your depth to the posting.

2. The pages

Page	What it gives you	Validated how
GPU + Kubernetes + Nsight	Pods/clusters from zero, GPUs in k8s, a real lab pod, `nsys` profiling, memory- vs compute-bound.	Re-checked against the live stg cluster by `validate.sh` (16 checks).
Inference optimization	The core: roofline, KV cache, batching, FlashAttention, quantization, speculation, parallelism, kernels, capacity planning, interview Qs.	Math computed in code; demos measured on the stg GPU.
LLM training	Pre-training, SFT, RLHF/PPO, DPO, LoRA, RLAIF, GRPO, eval, and how each affects serving.	Numbers computed; loss/LoRA code run.
Statistics & classic ML	Latency statistics, CIs, A/B for performance, linear algebra, core ML, metrics.	Simulations run with fixed seeds.
C++, Python, pandas, TF	Systems C++, CUDA primer, Python perf, pandas/numpy, TF vs PyTorch vs JAX, libs, coding questions.	Code compiled and run.

3. Suggested order (about 6 weeks, adjust to your time)

Week 1: foundations you can finish fast. Do the GPU + Kubernetes + Nsight page end to end, then the inference page's metrics and roofline sections. These turn your production intuition into numbers you can quote.
Week 2: KV cache, batching, attention. Inference page sections 3-5. Re-derive the KV-cache size formula for your own model until it's automatic.
Week 3: quantization, speculation, parallelism. Inference page 6-9, plus the training page's architecture section (GQA, MoE) since it explains why those costs exist.
Week 4: kernels. Triton first (the lab pod has it), then the CUDA C++ section in the languages page. Goal: write a fused kernel and compare it with PyTorch under nsys.
Week 5: breadth. Training page (SFT/RLHF/DPO/LoRA) and stats-ml page. These come up as "do you understand the model lifecycle" and "can you benchmark rigorously".
Week 6: interviews. Do every interview-prep section out loud, then one full system-design mock: "serve a 70B model at p99 TTFT under 500 ms; walk me through it".

4. Telling your story

You are not starting from zero. Frame it as: "I own production inference and certification today, and I've deliberately gone deep on the layer below: GPU memory, kernels and profiling." Concrete anchors from your own work that map to what these teams do:

Moving to INT4 AWQ on an RTX PRO 6000 instead of an L4 is a real quantization-plus-hardware decision; you can quantify why (memory-bound decode, bandwidth ratio).
The tokenizer CPU bottleneck and the 2-Triton-per-pod fix is a real end-to-end bottleneck-shifting story.
Certification against SLAs is benchmarking methodology: realistic length distributions, p99, open-loop load. Learn to say it that way.

5. How to use the pages

Every section ends with a Check yourself quiz. Your score shows bottom-right. Answers are saved in this browser.
Select any text to Highlight it or Ask Claude (copies a formatted question to paste into the chat).
Numbers shown as measured come from real runs. Anything marked "spec" or flagged in a yellow box was not verified here.
The lab pod from the GPU tutorial self-deletes after 4 hours; recreate it with kubectl --context apply -f pod.yaml (see the tutorial for context and safety notes).