Canada · Vancouver · See the Canada profile · See the Vancouver profile
Evidence: Randomised controlled trial Top 56% 47/100 · Ask Evidence Copilot about this practice
A blinded RCT of 258 first-year UBC medical students found AI-generated exam questions were made 5.6x faster than student-written ones, with similar scores and acceptability, though human items discriminated slightly better and a gender gap emerged only on the AI exam.
Writing a single high-quality multiple-choice exam question for medical education can take instructors or item-writers well over an hour and cost an estimated US$200 to over US$2,000 per usable item, making large question banks expensive to build and maintain.
Test whether AI-generated exam questions can match human-written ones on exam performance, item quality and student acceptability, while quantifying the time savings.
The University of British Columbia Faculty of Medicine ran the AHEAD Trial on 8-9 December 2024, randomizing 258 first-year MD students 1:1 to a blinded 112-item mock exam built either by senior medical students (human-generated) or by a ChatGPT-4-plus-Google-Gemini validation pipeline (AI-generated), with all authorship indicators stripped before delivery.
AI-generated items took 4.2 minutes each to produce versus 19.6 minutes for human-written items, a 5.6-fold efficiency gain (p<0.0001). Overall exam scores were statistically similar (73.4% human-generated vs 71.0% AI-generated, p=0.083). Human-written items had modestly but significantly higher item-discrimination scores (small effect), and student-rated acceptability favoured the human exam on four of nine measures with small effect sizes (Cohen's d <=0.32). Neither exam meaningfully changed students' self-rated exam readiness. An exploratory subgroup analysis found male students outperforming female students on the AI-generated exam (rrb=0.26) but not on the human-generated exam (rrb=0.07).
AI-generated items matched human-written ones on most measures at a fraction of the production time, but the trial was single-center, first-year-only, used senior students rather than faculty as question-writers, was registered retrospectively, and flags the gender-gap signal as requiring demographic monitoring rather than a confirmed effect.
Do you run this practice? Claim it — verified implementers get a public contact pathway and can propose corrections.
Where this practice's information was retrieved from, and when.
United States of America
After Brown University economics professor Roberto Serrano moved a midterm to a take-home format, grades spiked implausibly — 40 of …
Australia
Instead of an unwinnable AI-detection arms race, Sydney redesigned assessment into two lanes — secure in-person assessment of core capability, …
Hong Kong
In June 2023 HKU's Senate named GenAI a "fifth literacy" and gave every student and staff member free ChatGPT-class access. …
Vietnam
British University Vietnam piloted the AI Assessment Scale, a 5-level framework for permitted GenAI use in coursework; the peer-reviewed evaluation …
Open full copilot Grounded in cited practices — always check the sources.