One-shot Prompting

When OpenAI evaluated GPT-3 in 2020, the first example did most of the work. On Natural Questions, going from no examples to one raised the 175-billion-parameter model's accuracy from 14.6% to 23.0%; going on to a full few-shot prompt, with as many examples as the context held, added only another 6.9 points, to 29.9%. More than half of the entire gain came from a single demonstration. On LAMBADA, a benchmark of predicting the last word of a passage, the same single example made the model worse: 76.2% zero-shot, 72.5% one-shot, before a few-shot prompt lifted it to 86.4%.