AI Agent Test Purchase

This is the author's method of Sergei Ponomarev, PhD, set out in full. A test purchase is an old discipline: write down what a service is supposed to do, then go through it as an ordinary customer and record every gap between the promise and the practice. This guide carries that method over to AI agents, where the human shopper becomes an AI shopper, the clerk becomes the company's agent, and the analysis becomes a reconciliation between what the agent said and what the system actually recorded. It sets out the six steps of a check — standard, script, run, reconciliation, report, fix and re-test — and the three documents that serve as the norm: a public AI Policy, an AI Service Passport per service, and an AI Receipt per interaction. It covers the five axes along which a check is designed (external or internal, manual or automated, partial or full, comprehensive or targeted, standard or specialised), how evidence is captured so the results can be believed, and how findings are scored. It is equally clear about the boundaries: one run documents a failure but never establishes how often it happens, an external check answers what the customer got rather than why, and the analyst is an AI too and can be wrong. The method is carried over from twenty years of the author's own work on service quality: nationwide monitoring of public services, hundreds of independent assessments and test purchases. The reference pilot, run jointly with the Swiss AI metrology company NeoMundi, is published in full as open source, code, prompts, transcripts and defects included.