End of Volume Review & Interview Questions

This volume has taken apart one real failure after another. A chatbot told a customer he could claim a bereavement fare after travelling, which its airline's policy did not allow, and a tribunal made the airline pay. GPT-4 kept its name while its accuracy on a fixed prime-number test fell from 84.0% to 51.1% in three months. A pneumonia model that scored an AUC of 0.931 on its own hospitals' images dropped to 0.815 at another hospital, because it had learned to recognise the hospitals. A speech model that sometimes writes sentences nobody said sat inside a note-taking tool that had transcribed about seven million medical visits, and the tool erased the audio a clinician would have needed to check them. In none of these cases was the model the whole story. Each time the model did what models of its kind do, and the system around it lacked the control that would have caught the result.