Section 3

Attention & the Transformer Revolution

The recurrent architectures in the previous section processed a sequence one step at a time. Attention creates direct, content-dependent interactions between positions, and self-attention can compute those interactions in parallel during training when the full sequence is available. The transformer assembles attention, position-wise transformations, residual paths, normalization, and positional information into an architecture that became central to modern language and multimodal systems.