Section 3

Attention & the Transformer Revolution

Section 3 builds attention from Bahdanau's 2014 translation model to the scaled dot-product form every transformer layer runs, assembles it into the transformer block, and traces that block through GPT-2 small and Llama 3.1 8B with every shape and parameter count. It then follows the two halves apart: BERT's encoder, now the retrieval and classification layer beside LLMs, and the GPT decoder lineage from 2018 to September 2026. Chapter 13 closes the volume with a design review that sizes an LSTM, two pretrained encoders and Llama 3.1 8B for one task and recommends one with numbers.