Distillation

In January 2025 DeepSeek released R1, a reasoning model that matched OpenAI's o1 on several mathematics and coding benchmarks, and alongside it six much smaller models. Those six were ordinary open models, Qwen2.5 and Llama checkpoints from 1.5 to 70 billion parameters, fine-tuned on 800,000 examples curated with R1. No reinforcement learning, no new architecture. DeepSeek-R1-Distill-Qwen-32B scored 72.6 on the AIME 2024 mathematics exam, against 63.6 for o1-mini; even the 1.5B model scored 28.9, three times GPT-4o's 9.3. The same paper reported something more uncomfortable for anyone planning to train small models from scratch: large-scale reinforcement learning applied directly to the 32B base model reached only 47.0.