Macro Buddy — A Socratic AI Tutor Combined with Peer Study Groups Lifts Exam Scores in Undergraduate Macroeconomics

Undergraduate macroeconomics students often work through problem sets either alone or in study groups, with instructors unable to be present for every session; instructors at two US universities tested whether a purpose-built AI tutor could substitute for some of that missing guidance.

Test whether a custom Socratic-style AI tutor, restricted to course materials, could improve exam performance alone or combined with peer study groups, without simply giving away answers.

Researchers at the University of Wisconsin-La Crosse and Appalachian State University built 'Macro Buddy', a custom ChatGPT-based tool trained only on the course's own lecture transcripts, slides and homework questions, with general internet access disabled, responding with follow-up questions in a Socratic style rather than direct answers. In a randomized trial, 140 students across four course sections were assigned to one of four conditions: individual study, group study, individual study with Macro Buddy, or group study with Macro Buddy.

Scores on the second exam declined across all four groups. By the third exam, several weeks into the course, the combined condition (peer discussion plus Macro Buddy) scored highest, individual Macro Buddy users outperformed students working alone without AI, and the group-only (no-AI) condition showed the smallest gains.

The authors note the study covered a single course at a single institution, which limits how far the results can be generalized, but deliberately designed the low-cost, no-code tool to be replicable by other instructors.