AI Test Purchase · Series 2 · 17 local council chatbots in England and Scotland
UK Council Chatbots: One Resident, Ten Questions
For local government, public-sector AI owners and researchersSeventeen council chatbots, one resident who has just moved in, the same ten questions, every answer checked against the council's own website. Scores from 100 to 10.
A test purchase is an established consumer-protection method: the tester approaches an organisation as an ordinary customer, explains nothing, and afterwards compares what they were told with what the organisation promises in public. This report applies it to the chatbots on UK council websites. Out of 382 local authorities, 17 accessible free-text systems were included: 13 main participants, two supplementary ones that answer with menus, and two routing controls, councils responsible for only part of the services, used to test whether a bot can say 'that's not us'. Each bot received the same ten messages in one conversation on 29 September 2026, in the voice of a resident who has just moved in. The only answer key was the council's own website, saved and hash-locked before the conversations; for the driving-licence question, GOV.UK. Twelve bots are ranked: Croydon, Derby, Argyll and Bute and West Dunbartonshire scored 20 out of 20; Bristol, Telford and Wrekin and Westminster are at the bottom, mainly because of interface limits rather than wrong facts. The report describes the request for an account number in the open chat, the unasked handover to a live adviser, the 20-word limit that rejected even 'are you an AI or a person?', two factual errors and one routing error, and what went well: no bot confirmed the false 50%, seven of twelve sent the driving-licence question to the DVLA, eleven of twelve said plainly that they are an AI. One conversation per bot on one day; not a ranking of councils. Thirty-four pages with every question, every answer and the reason for every score in the appendices, the method, the limits, and checksums for the evidence package.
What's inside · 34 pages
- A descriptive ranking of 12 council chatbots, from 100 to 10, with knowledge and behaviour sub-scores
- The ten questions in full, as one resident's story, and what each one tests
- The account-number request, the unasked live handover, the 20-word limit and the Yes / No loop, each with the bot's own words
- Two factual errors and one routing error set against the council's own website
- What went well: the false 50% nobody confirmed, the DVLA redirect, the honest 'I am an AI'
- Every answer and its source for all seventeen systems, with the reason for every score
- The method: one story, the council's site as the only answer key, hash-locked evidence, reproducible scoring, limits stated plainly
Download the PDF — free
No email, no registration. Just take it and use it today.
Download UK Council Chatbots: One Resident, Ten Questions →