Cognitive Tutor Algebra I — RAND's Randomised Effectiveness Trial Finds a Delayed Gain of About 0.2 SD in U.S. High Schools

Cognitive Tutor Algebra I pairs an AI-based intelligent tutoring system with a blended classroom curriculum. RAND's effectiveness trial (Pane et al., 2014, Educational Evaluation and Policy Analysis) randomly assigned schools to use it or continue their usual algebra curriculum.
In the first year of implementation, posttest differences were not statistically significant. In the second year, high schools using the tutor outscored control schools by about 0.2 standard deviations, which the authors describe as moving a typical student from the 50th to the 58th percentile. The second-year middle-school estimate was of similar size but not statistically significant.
Caveats: treatment schools had slightly lower pretest scores at baseline, Evidence for ESSA's weighted average across three qualifying high-school studies is much smaller (+0.03), and later analyses suggest the gain may not stem from mastery learning itself. The case shows that tutoring software can take time and teacher adaptation to pay off.

Read the full analysis: https://ies.ed.gov/ncee/WWC/study/82101