PNLD Image-Quality Triage — A Randomised Trial of a CNN Assistant With 76 of Brazil's Textbook Analysts

The Programa Nacional do Livro e do Material Didático (PNLD), created in 1937 and run by the Fundo Nacional de Desenvolvimento da Educação (FNDE) with the Ministry of Education, evaluates and distributes textbooks to Brazil's public schools. Its assessment cycle can take at least two years and involves hundreds of professionals doing manual review work, including judging the technical quality of images printed in candidate textbooks.

Researchers at the Nucleus of Excellence in Social Technologies (NEES) at the Federal University of Alagoas, with co-authors at several Brazilian and international universities, built a convolutional neural network to sort each textbook image as sharp, defocused-blurred or motion-blurred, so that analysts can concentrate on the cases the model flags.

The team ran a parallel two-arm study with 1:1 randomisation and 76 textbook analysts, people who actually do PNLD assessment work rather than a convenience sample. Participants took a baseline test and a post-test after the initial assessment; the primary outcome was the post-test score, and secondary outcomes were the analysts' productivity and the quality of their assessments.

The AI component significantly increased productivity: the experimental group assessed substantially more images than the control group, and assessment quality was preserved rather than traded away for speed. The primary outcome was weaker, the difference between pre-test and post-test was modest, though the authors read the direction as evidence that working with the AI component improved analysts' own ability to tell image-defect categories apart.

The trial tests one narrow component, blur classification on images, not the pedagogical evaluation at the heart of PNLD, and the productivity finding should not be read as a claim about the assessment cycle as a whole. A randomised study of a component is also not the same as evidence of a production rollout; FNDE and MEC have a broader PNLD modernisation effort under way with NEES, but this specific trial establishes efficacy, not deployment at scale. Randomised evaluation of a government AI tool, with the actual public officials who use it, published open access including a null-ish primary outcome, is rare among public-sector AI productivity claims.