Auditing Machine Learning Algorithms — A White Paper and Audit Catalogue Co-Developed by Five Supreme Audit Institutions

In November 2020 the Office of the Auditor General of Norway, working with data scientists from the supreme audit institutions (SAIs) of Finland, Germany, the Netherlands and the United Kingdom, published 'Auditing Machine Learning Algorithms: A White Paper for Public Auditors'. It was written because limited guidance existed on how auditors should examine AI and machine-learning systems used in public services.
The paper summarises key risks, including automated and institutionalised unequal treatment and mass production of incorrect or discriminatory decisions, and proposes an audit catalogue with methodological approaches in three areas: project management and governance, data considerations (quality, bias, privacy, GDPR), and model development (documentation, reproducibility, fairness). It is guidance, not a fixed checklist. It is published openly at auditingalgorithms.net, and the INTOSAI Journal and the UK National Audit Office have both described it.
Evidence limits: neither source reports how many audits have used the catalogue or what they found, so impact and replication are undocumented. Its value is as an open, multi-institution method that other SAIs can reuse.

Read the full analysis: https://www.auditingalgorithms.net