Skip to content

AI Infrastructure Track

LLM Inference, GPU Serving and Performance Engineering

Turn model artifacts into dependable services through prefill and decode mechanics, KV cache, continuous batching, quantization, GPU topology, distributed serving, SLOs, capacity, cost, and release safety.

Backend Engineer to AI Infrastructure Architect
About 150 min · 3 labs
52 authored sections

Open before purchase

A useful preview, not a teaser

Read these complete sections to judge the technical depth, teaching style, and architecture standard before buying.

Preview 1

Course Contract: Serve Outcomes, Not Tokens

A model checkpoint has no latency, availability, or cost guarantee until an inference system gives it one.

Preview 2

Follow One Request from Text to Stream

The public API hides a pipeline of deterministic and probabilistic stages.

Preview 3

Understand the Inference Engine as a Control Plane and Data Plane

Optimized kernels matter, but scheduling and memory ownership decide whether they stay useful.

Start the free preview

Complete curriculum

52 sections · 5 parts + inference studio

Course orientation

  1. Course ContractFree

Part 1 · Understand the Runtime

  1. Request LifecycleFree
  2. Prefill and DecodeCourse Pass
  3. Transformer StepCourse Pass
  4. Latency VocabularyCourse Pass
  5. Workload ContractCourse Pass
  6. Memory AccountingCourse Pass
  7. KV CacheCourse Pass
  8. Compute or Memory BoundCourse Pass
  9. Model and HardwareCourse Pass

Part 2 · Engineer the Engine

  1. Engine ArchitectureFree
  2. Continuous BatchingCourse Pass
  3. Paged KV CacheCourse Pass
  4. Scheduling and AdmissionCourse Pass
  5. Chunked PrefillCourse Pass
  6. Prefix CachingCourse Pass
  7. Structured DecodingCourse Pass
  8. Speculative DecodingCourse Pass
  9. QuantizationCourse Pass
  10. ParallelismCourse Pass
  11. MoE ServingCourse Pass
  12. Multimodal ServingCourse Pass

Part 3 · Build the Platform

  1. API BoundaryCourse Pass
  2. Streaming and CancellationCourse Pass
  3. RoutingCourse Pass
  4. Multi-Model ServingCourse Pass
  5. LoRA ServingCourse Pass
  6. Multi-TenancyCourse Pass
  7. AutoscalingCourse Pass
  8. Kubernetes and GPUsCourse Pass
  9. Deployment and StartupCourse Pass
  10. Disaggregated ServingCourse Pass
  11. Batch InferenceCourse Pass
  12. Cache TiersCourse Pass

Part 4 · Operate Performance

  1. Benchmark DesignCourse Pass
  2. Capacity PlanningCourse Pass
  3. SLO and GoodputCourse Pass
  4. ObservabilityCourse Pass
  5. GPU TelemetryCourse Pass
  6. Cost and EnergyCourse Pass
  7. ReliabilityCourse Pass
  8. SecurityCourse Pass
  9. Incident ResponseCourse Pass
  10. Worked CaseCourse Pass

Part 5 · Architect the System

  1. CapstoneCourse Pass

Inference Studio

  1. Capacity LabCourse Pass
  2. Scheduling LabCourse Pass
  3. Release Gate LabCourse Pass
  4. AssessmentCourse Pass
  5. Interview QuestionsCourse Pass
  6. Cheat SheetCourse Pass
  7. ReferencesCourse Pass