Grading at Scale — A Randomized Trial of AI vs. Human Graders Across Four Political-Science Programs

Grading and providing personalised feedback on short-answer questions is time-consuming, which often pushes instructors of large classes toward multiple-choice assessments and reduces students' opportunities to develop critical-thinking skills.

Test whether large-language-model (GPT-4) assistance can grade short-answer responses as accurately as human graders and let instructors give large classes the kind of individualised feedback usually reserved for much smaller classes.

Researchers led by Heinrich and colleagues ran a randomized controlled trial across four undergraduate political-science programmes (University of Houston, University of South Carolina, University of Michigan, and Academia Sinica in Taiwan) in 2023-2024. Across 26 tests covering 271 students, roughly ten student responses per short-answer question were randomly assigned to human or GPT-4 grading, with the AI guided by instructor-built rubrics and "gold" example answers.

The study, published in PLoS One in August 2025, found no statistically significant difference in grade accuracy between AI and human graders (mean difference -0.04). Regrade requests were only marginally higher for human-graded work, and human feedback was rated "helpful" about two percentage points more often than AI feedback. The clearest gain was productivity: AI-assisted grading let instructors deliver individualised feedback to classes of roughly 70 students at a depth typically reserved for classes a third to a quarter that size.

The authors are explicit about limits: the trial measured grading accuracy and feedback ratings, not actual student learning gains, results varied across the four courses, and reliance on a single proprietary model (GPT-4) raises replicability concerns as models change.