Section 1

Representation and Attention Foundations

Before a Transformer can compute over text, raw strings become token IDs, token IDs map to learned vectors, position information enters the computation, and attention forms context-dependent mixtures. This section builds that path from scratch while separating what each mechanism computes from what a trained system may learn to represent.