evidoria

← Back to browse

Good practice Imported

One Click Away — A Two-Year Cluster RCT Finds Khanmigo Access Alone Doesn't Move the Needle in Tennessee

United States of America · Chattanooga · See the United States of America profile · See the Chattanooga profile

Evidence: Randomised controlled trial Top 56% 47/100 · Ask Evidence Copilot about this practice

A two-year cluster-randomized trial across 18 Hamilton County, Tennessee middle schools found Khanmigo access raised math scores by 0.06–0.08 SD, but the median student engaged on only a third of practice days — access alone did not guarantee use.

0.06–0.08 SD
Math achievement gain from Khanmigo access (over a school year)
about 1.3 national percentile ranks per term
Math achievement gain, percentile-rank terms
about 0.14 SD
Implied effect at full active participation
96 %
Students who tried Khanmigo at least once
17 %
Exercise sessions with a mistake in which the student messaged Khanmigo
18 schools
Middle schools in the cluster RCT
One Click Away — A Two-Year Cluster RCT Finds Khanmigo Access Alone Doesn't Move the Needle in Tennessee

Details

Promoter
Hamilton County Schools with Philip Oreopoulos & Nina Low (Annenberg Institute at Brown University / NBER)
Period
2024–2026 (published August 2026)
Keywords
K-12 education, mathematics, AI tutoring, education research

Context

Economists Philip Oreopoulos and Nina Low ran a two-year cluster-randomized controlled trial in 18 middle schools in Hamilton County, Tennessee, assigning low-performing students to use Khan Academy's AI tutor Khanmigo — configured to coach rather than hand out answers — during existing daily remedial mathematics sessions.

Objectives

Test whether AI-tutor access during remedial math sessions improves math achievement, and understand what drives, or limits, that impact.

Activities

The working paper 'One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment' was released as NBER Working Paper 35620 and EdWorkingPaper ai26-1551 in August 2026, based on the two-year cluster-randomized design across the 18 schools.

Results

Assignment to the Khanmigo condition raised math achievement by about 1.3 national percentile ranks per term, or roughly 0.06 to 0.08 standard deviations over a school year; extrapolating to full active participation, the implied effect reached about 0.14 standard deviations, comparable to gains from Khan Academy practice without AI assistance. The binding constraint was engagement, not access: 96% of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced, and in just 17% of the exercise sessions in which they made a mistake.

Conclusions

The authors conclude that realizing AI tutoring's promise requires getting students to actually use the tool, not simply granting them access to it — a cautionary finding against assuming rollout alone drives outcomes.

Implementation
Indicative cost
Medium (€50k–€500k) — Not disclosed; a research-grade cluster RCT across 18 schools over two years using the commercially available Khanmigo tool within Khan Academy's existing remedial-math sessions.
Time to results
Medium (1–3 years) — Two-year cluster-randomized experiment; working paper released August 2026.
Staffing & skills
Hamilton County Schools, Philip Oreopoulos and Nina Low (Annenberg Institute at Brown University / NBER)

Conditions for success

  • Configuring Khanmigo to coach rather than hand out direct answers
  • Embedding tutor access inside existing daily remedial mathematics sessions rather than as a stand-alone add-on

Common failure modes

  • Engagement, not access, was the binding constraint: the median student messaged Khanmigo on only about a third of practice days and in just 17% of sessions where they erred
  • The measured effect (0.06–0.08 SD) is modest even under a strong causal design

Commonly funded by

National / regional programmes

Indicative funding routes for practices of this type — always check each programme's current calls and eligibility rules.

Replication kit

Reusable artefacts from this practice — as published by their sources.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Attachments

Similar practices you may find useful