evidoria

← Back to browse

Good practice Imported

GPT-3.5 Feedback Raises Text Revision and Motivation — A Randomised Trial with German Upper-Secondary Students

Germany · Kiel · See the Germany profile

Evidence: Randomised controlled trial Top 89% 27/100 · Ask Evidence Copilot about this practice

A randomised trial with 459 German Grade 10 students found GPT-3.5-turbo-generated writing feedback modestly improved essay revision (d=0.19), motivation (d=0.36) and positive emotion (d=0.34) versus no AI feedback — real but small, single-school effects.

459 students
Student sample size (2023)
0.19
Text-revision performance effect (Cohen's d)
0.36
Task motivation effect (Cohen's d)
0.34
Positive emotion effect (Cohen's d)

Details

Promoter
IPN – Leibniz Institute for Science and Mathematics Education, Kiel
Period
2023
Region (NUTS)
DEF0
Keywords
assessment & feedback, natural language processing, secondary education, EFL writing instruction

Context

Researchers at the IPN - Leibniz Institute for Science and Mathematics Education in Kiel, Germany, ran a randomised controlled trial testing whether large-language-model-generated feedback helps secondary-school students revise their writing.

Objectives

The trial tested whether GPT-3.5-turbo feedback, tuned to evidence-based feedback principles, improves essay-revision quality, motivation and emotion compared with no AI feedback.

Activities

459 Grade 10 students in academic-track schools studying English as a foreign language wrote argumentative essays; the experimental group revised their texts using GPT-3.5-turbo feedback while the control group revised without it. Revision quality was scored with automated essay scoring, alongside self-reported motivation and emotion measures.

Results

The AI-feedback group achieved higher text-revision performance (Cohen's d = 0.19), task motivation (d = 0.36) and positive emotions (d = 0.34) than the no-feedback control.

Conclusions

These are modest-to-small effect sizes from a single-school, single-subject RCT -- real but limited evidence that should be read as an initial proof of concept rather than a system already operating at scale.

Implementation
Indicative cost
Low (< €50k)
Time to results
Short (< 1 year)
Staffing & skills
IPN - Leibniz Institute for Science and Mathematics Education research team (Kiel)

Conditions for success

  • AI feedback tuned to evidence-based feedback principles rather than generic LLM output
  • Single subject/grade focus enabling a controlled comparison

Commonly funded by

National / regional programmes

Indicative funding routes for practices of this type — always check each programme's current calls and eligibility rules.

Replication kit

Reusable artefacts from this practice — as published by their sources.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Similar practices you may find useful