The course's worked case is an enterprise support copilot. Twelve thousand support agents get cited draft answers to product and policy questions. A human sends or edits every answer, and refunds, case closure and account changes are out of scope. The workload, prices, targets and organizational decisions are constructed for design practice, so read the numbers as a method rather than a benchmark.
Targets first
The design commits to numbers before drawing boxes: 99.9% usable-outcome availability, p95 first token under 1.5 seconds and a complete draft under six seconds, groundedness of at least 92%, zero critical tenant leak, an effective policy searchable within 15 minutes, and p95 served cost under $0.035.
From requests a day to tokens a minute
250,000 requests a day, 4x peak
-> about 12 peak requests per second
2.5 s mean service time, 2% one-extra-attempt retries
-> about 29.5 mean in-flight attempts at steady peak
1,200 input + 350 output tokens, 35% of input cached
-> about 1.10 million tokens a minute (TPM)
Each line feeds the next. Requests per second and service time give concurrency, which is what you provision for. Tokens per request and retries give the quota to ask the provider for, before any extra reserve.
Common Mistake
Buying quota from averages. The course says to recalculate using p95 token length before procurement, and to validate any provisioning proxy under load.
What would reopen the design
The case lists its own triggers. Retries above the assumed 2% raise tokens per minute above the 1.10 million planned. A cached share below 35% raises the cost per attempt. Any critical tenant leak in testing reopens it, and so does a proposal to let the copilot send answers itself, which moves authority to the model and needs a new design review.
Writing down what would change the design is part of the design. It tells the reviewer which of your assumptions to attack first.
