Self-Consistency

One model, three scores: 74%, 83% and 93% on the 2024 AIME maths competition, all reported by OpenAI for o1 when it introduced the model in September 2024. With one sample per problem, o1 solved 74% of problems. Taking the most common answer across 64 samples raised that to 83%. Re-ranking 1,000 samples with a learned scoring function reached 93%. The model did not change between those numbers; only the way its outputs were combined did.