AI-Powered Oral Exams — NYU Stern's Cheat-Resistant Alternative to Written Assessment

At NYU Stern School of Business, professor Panos Ipeirotis noticed that written submissions in his 'AI/ML Product Management' course had started reading like polished consulting memos; when he probed authors with follow-up questions, many could not explain their own submitted work.

Test whether students actually understood what they had submitted, without relying on AI-detection software, by building an AI-run oral exam with co-instructor Konstantinos Rizakos.

Built on ElevenLabs' conversational voice AI alongside Anthropic Claude, Google Gemini and OpenAI models, the system reads each student's submitted project in advance and dynamically tailors questions to their specific claims. It was tested across a 36-student cohort over nine days, averaging 25 minutes per student.

The exam cost $15 in total, about 42 cents per student, versus an estimated $750 for two human graders at $25/hour. In student surveys, 83% rated the AI oral exam more stressful than a written test, yet 70% agreed it better measured their real understanding, and only 13% said they would prefer a human examiner. Three independent AI models grading the same responses agreed exactly 29% of the time and were within one point 60% of the time. Students scored higher on 'problem framing' (3.39/4 average) than on 'experimentation' (1.94/4), with three unable to discuss their own experimentation choices at all.

This is a single-course pilot self-reported by the instructor via NYU's student newspaper and independent tech press, not an independently audited or peer-reviewed study, so its small, one-off scale should temper expectations of generalisability.