Section 2

Calculus

Section 2 follows one question through six chapters: for each of GPT-2 small's 124,439,808 parameters, which way does the loss move, and how far should the weight step? It builds the derivative and its numerical checks, the rules behind a framework's backward functions, Jacobians and the backward pass, the gradient of cross-entropy, and the optimizers that turn gradients into a training run. Each chapter works its mechanism at the scale of GPT-2 small or Llama 3.1 8B and tells a failure story from a real training stack.