End of Volume Review & Interview Questions

Twice in 2024, four months apart, llama.cpp, the engine under Ollama and other tools for running models on your own hardware, turned correct Llama weights into a quietly worse model. In April, days after Llama 3's release, llama.cpp split text with GPT-2's pre-tokenizer expression instead of Llama 3's, so the model received token sequences it had never seen in training; pull request 6920 fixed it on 29 April. In July, converted Llama 3.1 files carried the RoPE base but not the frequency scaling the model was trained with, and answers degraded beyond 8,192 tokens until pull request 8676 was merged on 27 July. Neither bug raised an error, and neither was in the weights. Chapter 1 explains the first and Chapter 3 the second.