GPT-3.5 Feedback Raises Text Revision and Motivation — A Randomised Trial with German Upper-Secondary Students

Researchers at the IPN – Leibniz Institute for Science and Mathematics Education in Kiel, Germany, ran a randomised controlled trial testing whether large-language-model-generated feedback helps secondary-school students revise their writing.
459 Grade 10 students in academic-track schools studying English as a foreign language wrote argumentative essays. In the experimental condition, students revised their texts using feedback generated by GPT-3.5-turbo and tuned to evidence-based feedback principles, while control-group students revised without this AI feedback. Revision quality was scored with automated essay scoring, alongside self-reported motivation and emotion measures.
Published in Computers and Education: Artificial Intelligence (December 2023), the trial found the AI feedback group achieved higher text-revision performance (Cohen's d = 0.19), task motivation (d = 0.36) and positive emotions (d = 0.34) than the no-feedback control. These are modest-to-small effect sizes from a single-school, single-subject RCT — real but limited evidence that should be read as an initial proof of concept rather than a system already operating at scale.

Read the full analysis: https://www.leibniz-ipn.de/en/the-ipn/about-us/staff/jennifer-meyer-1