Why RAG?

The company in this volume keeps 150,000 internal documents, about 360 million tokens of policies, runbooks, product manuals and support macros, and its 5,000 employees put 12,000 questions to them on a working day. Claude Sonnet 5, the model this volume prices against, reads at most a million tokens per request. So for every question something has to choose a few thousand tokens out of 360 million for the model to read, and that choice decides more about the answer than the model does: whether it is right, whether it is current, and whether the person asking may see it.