What is an Agent?

A chatbot that states the wrong policy costs its operator an apology and, now and then, a small-claims judgment. An agent that applies the wrong policy moves money. How often a capable model applies a written policy correctly is therefore the first number to know about an agent, and in June 2024 Sierra published a benchmark that measures it.