Section 3

Probability & Statistics

Section 3 treats a model's outputs as probability distributions and its evaluation results as estimates with errors, working from published cases such as the GPT-4 report's calibration plots, Meta's Prompt Guard 2 model card, the HumanEval results in OpenAI's Codex paper and llama.cpp's measurements of quantized Llama 3 8B. It ends with the volume's Review chapter, which runs one training step of a 100-parameter attention head through all three sections and judges it with a confidence interval.