Fine-tuning

Give the base Llama 3.1 8B the text Color of sky is blue but sometimes can also be and it keeps writing: a list of colours, then a fresh sentence about why the sky is blue. It is completing a document; the example comes from Meta's own prompt-format documentation for Llama 3.1. Wrap a question in the chat template and give it to Llama 3.1 8B Instruct, and it answers, emits the special token <|eot_id|> and stops. Both checkpoints have the same architecture and the same 8,030,261,248 parameters. What separates them is post-training: six rounds of supervised fine-tuning and preference optimisation, according to Meta's Llama 3 paper.