API Design

When an adjuster opens a claim, the claims assistant from Chapter 1 runs its intake job: a summary and a field extraction in parallel, then retrieval, a coverage recommendation and a customer letter draft. That is about 2,200 output tokens on the critical path. With output streaming at 70 tokens a second and a first token in under a second (illustrative figures, inside the case's 2-second target), the simulation in this chapter's program puts the median intake at 36.6 seconds and the 99th percentile at 53.2.