NUMI AI Math Tutor — A 6,997-Student Trial in Chattanooga, Tennessee Finds Modest, Mostly Post-Error Gains

NUMI is an AI math tutor built by researchers Philip Oreopoulos and Michael Liut, designed deliberately to avoid giving students direct answers and instead provide structured, error-focused support. It was tested in a randomized field experiment across 20 middle schools and about 100 teachers in Hamilton County Schools, which serves Chattanooga, Tennessee, with 6,997 students randomized to AI versus computer-assisted-learning-only support, and to mastery versus non-mastery progression, during a single ~50-minute practice session in March 2026.
The results were deliberately reported as mixed rather than a clean win. AI-assisted students progressed more slowly and attempted fewer questions, but were 8.5 percentage points more likely to answer correctly on their next attempt after a mistake, needed 0.96 fewer attempts to recover, and spent more time engaging with structured feedback. On a delayed assessment given the following week, students who had used AI scored 40.2% versus 37.0% for controls on practiced material -- a 3.2 percentage point gain that was only marginally significant (p=0.065) -- rising to a 5.9 point advantage on more difficult topics, while mastery-based progression alone did not improve delayed learning.
The authors and independent education press coverage are explicit about the limits: this was a single short session, the delayed test had only four questions, and the working paper had not yet been peer-reviewed at publication, with the authors describing their own findings as 'suggestive rather than definitive.'

Read the full analysis: https://www.michaelliut.ca/projects.html