Types of Artificial Intelligence

GPT-4 passed a simulated bar exam "with a score around the top 10% of test takers", OpenAI's technical report for the model said in March 2023: 298 out of 400 on the Uniform Bar Exam, the test most US states use to license lawyers. That is about the 90th percentile, meaning it beat about 90% of the people who sat the exam. The claim travelled fast, because a 90th-percentile lawyer sounds like expert-level, even general, intelligence. A year later Eric Martínez, then at MIT, re-examined it in Artificial Intelligence and Law. The 90th-percentile figure came from comparing GPT-4 with people taking the February exam, a group weighted towards candidates who had failed before. Against first-time takers the same score sat at about the 62nd percentile, and 42nd on the essay sections. Against people who passed, about the 48th percentile overall and the 15th on essays.