evidoria

← Back to browse

Good practice Imported

Letrus AI Writing Feedback — Closing Brazil's ENEM Essay Gap in Espírito Santo

Brazil · Vitória · See the Brazil profile

Evidence: Randomised controlled trial Top 7% 73/100 · Ask Evidence Copilot about this practice

Espírito Santo's public schools used the Letrus AI platform to give ENEM essay writers rapid feedback; a J-PAL randomized evaluation of ~19,000 students found scores rose 0.09 SD, closing about 9% of the public-private achievement gap, with submissions up 28-32%.

19000 students (approx.)
Students in the randomized evaluation
178
Schools in the evaluation
110
Schools with Letrus access
68
Comparison schools
0.09 SD (~17 points on a 1,000-point scale)
Gain in official ENEM essay scores
9 %
Public-private achievement gap closed
0.14-0.16 SD
Gain in the policy-proposal sub-grade
28-32 %
Increase in essay submissions
35 %
Increase in individual essay discussions with teachers
28000
Active student users in Espírito Santo (2024)
100000 users (over)
Total users since 2020
Letrus AI Writing Feedback — Closing Brazil's ENEM Essay Gap in Espírito Santo

Details

Maturity
Scaling
Promoter
Secretaria de Estado da Educação do Espírito Santo (Sedu), with Letrus, J-PAL and Google.org
Period
2019–present (statewide since 2022)
Keywords
AI writing feedback, natural language processing, public education, exam preparation

Context

Espírito Santo's State Department of Education (Sedu) piloted Letrus, an automated writing evaluation platform combining natural-language processing with human review, to help public high-school seniors prepare for the essay component of Brazil's ENEM university-entrance exam.

Objectives

Economists Bruno Ferman, Lycia Lima and Flavio Riva evaluated the 2019 rollout as a three-arm randomized study covering roughly 19,000 seniors across 178 public schools, 110 with Letrus access and 68 comparison schools.

Activities

Students wrote practice essays on the platform and received rapid, rubric-aligned feedback across ENEM's five official competencies, with human correctors refining the AI's semantic feedback in a subset of schools.

Results

Official ENEM essay scores rose by about 0.09 standard deviations (roughly 17 points on the 1,000-point scale), closing an estimated 9% of Brazil's public-private school achievement gap in the essay component; the policy-proposal sub-grade rose 0.14-0.16 SD, essay submissions increased 28-32%, and students discussed about 35% more essays individually with teachers, with both the AI-only and AI-plus-human versions producing similar score gains.

Conclusions

Following the pilot, Espírito Santo rolled Letrus out statewide in 2022 and it has since been adopted by education departments in Mato Grosso do Sul, Alagoas, Paraíba, Pará, Mato Grosso, Goiás and Ceará, with USAID Development Innovation Ventures funding a 2024-2026 follow-up evaluation and around 28,000 active student users in Espírito Santo alone by 2024.

Implementation
Indicative cost
Medium (€50k–€500k)
Time to results
Long (> 3 years)
Staffing & skills
Delivered jointly by Espírito Santo's State Department of Education (Sedu), the Letrus platform team, J-PAL researchers and Google.org, with human correctors reviewing AI feedback in a subset of schools.

Conditions for success

  • A human-review layer refining the AI's semantic feedback in some schools produced similar gains to the AI-only version.
  • Teachers redirected time from routine error-flagging to individualized coaching once the AI handled routine feedback.
  • External funding (Google.org, later USAID DIV) supported evaluation and scale-up.

Replication kit

Reusable artefacts from this practice — as published by their sources.

Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.

Data sources

Where this practice's information was retrieved from, and when.

Attachments

Similar practices you may find useful