ClassifAI & Survey Assist — the UK ONS's AI Pipeline That Halved Occupation-Coding Time

National statistics offices spend significant staff time manually coding survey respondents' free-text answers into standard classifications such as the Standard Occupational Classification (SOC) and Standard Industrial Classification (SIC), a bottleneck as survey volumes grow. The UK's Office for National Statistics (ONS) Data Science Campus built ClassifAI, an open-source Python package that combines semantic search over previously coded examples with retrieval-augmented generation.

ClassifAI aims to assign free text automatically to classifications including SOC, SIC, COICOP and their international equivalents (ISCO, ISIC). Building on the same pipeline, the ONS is separately piloting 'Survey Assist' on the Transformed Labour Force Survey, using AI-generated follow-up questions to sharpen SIC/SOC coding.

ClassifAI, according to the ONS's own account, is now coding labour market data in production, including a large-language-model application used on the Annual Survey of Hours and Earnings (ASHE). Survey Assist, which could cover around 90,000 households (about 2.25 million survey instances a year) once deployed, was published on 30 October 2025 under the UK's Algorithmic Transparency Record Standard (v3.0), documenting its pilot status, data flows, human clerical review and a planned evaluation against gold-standard coded data.

The ONS's own account states that the large-language-model application built on ClassifAI halved processing time for occupation coding on ASHE. Survey Assist, the newer follow-up-question tool, has not yet published accuracy or consistency results, as testing was still under way through 2025.

The Survey Assist Transparency Record sets a materially higher public-transparency bar than most comparable AI deployments in this dataset, but the well-evidenced part of the impact claim remains limited to the ASHE processing-time figure, with Survey Assist itself still unproven in production.