Section 2

Calculus

Section 1 answered "what is this piece of data" — a vector, a matrix, a transformation. Section 2 answers a different question: "how does a model get better at its job?" For differentiable models, calculus supplies a central part of the answer. An objective assigns a cost to current behavior. A derivative measures local sensitivity to one variable. A gradient collects those sensitivities, and a gradient-based optimizer uses them to propose updates. Whether those updates improve held-out behavior depends on the step rule, data, objective, parameterization, and evaluation contract.