Transformers

When OpenAI released GPT-2 in February 2019, it described the smallest model as having 117 million parameters and the next as 345 million. The released repository now carries a correction: "our original parameter counts were wrong due to an error". The models are 124 million and 355 million. A 6% error in the headline number of a famous model is a useful warning. A model's size is not a label someone assigns; it is arithmetic on five numbers from its configuration, and an engineer who can do that arithmetic can check a vendor's claim, predict a model's memory before downloading it, and see where a design spends its budget.