Gemeentelijke Chatbots Getest — an Independent Audit Finds Dutch Municipal AI Chatbots Answer Only 1 in 10 Questions Correctly

Since June 2025, independent researchers Wiep Hamstra and Jules Ernst have run an unfunded audit of municipal chatbots deployed across Dutch local government, after early evidence suggested these AI-branded tools might not perform as advertised.

To systematically and repeatably test how accurately municipal chatbots answer real citizen questions, and track whether performance improves over time.

For the second edition (fieldwork December 2025-early February 2026, published 9 March 2026), the researchers tested 39 municipal chatbots with the same eight citizen questions each - a mix of routine service queries (bulky-waste collection, permit status) and election-related questions (replacing a lost voting pass) - for 312 total question-answer exchanges.

Results were largely unchanged from the first edition: 64% of answers were wrong, 23% returned only a hyperlink (often to a page without the answer), 10% were correct, and 3% produced no response. Not one of the 39 chatbots correctly answered a basic household waste-collection question unaided. The researchers estimate at most 10 of the 39 tools are actually powered by a language model; most rely on hand-maintained, scripted question-answer pairs. Because nearly 30 municipalities license the same ‘Gem’ chatbot product, identical wrong answers propagate across all of them simultaneously.

VNG, the association representing Dutch municipalities, publicly acknowledged the findings, stating the new AI chatbots “are still no finished products” and that data quality determines whether a bot gives an outdated or hallucinated answer; it also noted chatbots have reduced call-centre volume, a claim the audit did not independently test. As a recurring, methodologically consistent, independently run check on a widely deployed public-facing AI tool, the audit is a rare case of rigorous accountability for municipal AI - even though what it evidences is mostly failure, not success.