evidoria

← Back to browse

Good practice Imported

Personalized Problem Sequencing Lifts Python Exam Scores — A 10-School RCT in Taipei

Taiwan · Taipei · See the Taiwan profile · See the Taipei profile

Evidence: Randomised controlled trial Top 40% 53/100 · Ask Evidence Copilot about this practice

A randomized trial across 10 Taipei high schools and about 800 students gave every student the same AI chatbot for a five-month Python course, varying only the sequence of practice problems. Personalized, difficulty-adapted sequencing raised exam scores by 0.15 standard deviation

0.15 SD
Exam score improvement, personalized sequencing vs control
~800
Students enrolled
~60 vs ~30 minutes/module
Time on task, treatment vs control
Personalized Problem Sequencing Lifts Python Exam Scores — A 10-School RCT in Taipei

Details

Maturity
Pilot
Promoter
Wharton School, University of Pennsylvania & National Taiwan University
Period
2025-2026 (five-month course; draft paper posted March 2026)
Keywords
secondary education, computer science education, EdTech research

Context

Researchers from Wharton, Carnegie Mellon and National Taiwan University ran a randomized controlled trial embedded in an after-school Python certification course across 10 Taipei high schools, randomising only the sequencing of practice problems (fixed vs AI-personalized difficulty) while all students had identical access to the same AI tutor.

Results

Students in the personalized-sequencing group scored 0.15 SD higher on final exams — roughly equivalent to six to nine months of additional learning — and engaged nearly twice as long per module; gains were larger for students new to Python and from less academically elite schools. The paper is a March 2026 working-paper draft, not yet peer-reviewed, and the sample skewed toward highly motivated students with educated parents.

Implementation
Indicative cost
Low (< €50k)
Time to results
Short (< 1 year)
Staffing & skills
Research team (Wharton, Carnegie Mellon, National Taiwan University)

Conditions for success

  • No additional teacher time or workload required — only problem sequencing changed

Common failure modes

  • Sample skewed toward highly motivated students with educated parents, limiting generalizability

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Attachments

Similar practices you may find useful