A within-subjects RCT with 194 Harvard undergraduates found a custom GPT-4 tutor built on evidence-based teaching strategies produced more than double the learning gains of live active-learning physics classes on the same topics, in less time, published in Scientific Reports.
2x+
Learning gain, AI tutor vs active-learning classroom (Spring 2024)
Details
Promoter
Harvard University, Department of Physics
Period
Spring 2024 (published June 2025)
Keywords
higher education, physics, generative AI tutoring, learning science
Context
In a within-subjects randomised trial, 194 Harvard undergraduates each experienced both an in-person active-learning physics class (group work plus instructor support) and a take-home lesson using a custom GPT-4 tutor engineered around evidence-based teaching strategies — brief responses, frequent checks for understanding, and safeguards against cognitive overload — covering surface tension and fluid flow.
Results
Students using the AI tutor learned more than twice as much as in the active-learning classroom, in less time, and reported higher engagement and motivation.
Conclusions
Published in Scientific Reports in June 2025, the study is among the strongest peer-reviewed evidence to date that a carefully engineered AI tutor — not a generic chatbot — can outperform a well-established, research-backed classroom method, though the evidence to date covers only two physics lessons at a single elite university.
Implementation
Indicative cost
Low (< €50k) — Small-scale classroom study; no separate implementation budget is published.
Time to results
Short (< 1 year) — Two lessons delivered in Spring 2024; published June 2025.
Staffing & skills
Harvard University Department of Physics research team
Conditions for success
Tutor prompts deliberately engineered around evidence-based teaching strategies rather than left as a raw chatbot
Where it fits
Governance type
university research study
Scale
194 students, single university
Income level
high income
Do you run this practice?
Claim it —
verified implementers get a public contact pathway and can propose corrections.
Data sources
Where this practice's information was retrieved from, and when.
A quasi-experimental study at Indonesia's 550,000-student Open University compared 18,713 students in ChatGPT-assisted tutorials against 19,030 in human-tutored classes, finding …
A 13-week controlled study at four Hungarian universities found a ChatGPT-based adaptive tutor using Bayesian Knowledge Tracing nearly doubled programming …
