NYU Grossman School of Medicine's Machine-Learning Screener for Medical-School Applications

NYU Grossman School of Medicine built a machine-learning tool to help screen the roughly 14,500 applications it receives each admissions cycle, a process that previously consumed more than 6,000 hours of faculty time a year. The model was trained on 14,555 applications from 2013–2017 and the human screening decisions faculty made on them.

The tool aimed to reduce faculty screening workload while producing interview recommendations statistically comparable to human screeners, without replacing human judgment on final decisions.

The model went through staged validation: retrospective validation on 2,910 applications, prospective validation on 2,715 applications in the 2018 cycle, and finally a formal randomized trial in the 2019 cycle that split 3,700 applications between faculty-only screening and algorithm-assisted screening. It reads only structured data — not essays or letters — and its output remains a recommendation reviewed by humans, not a final decision.

Published in the peer-reviewed journal Academic Medicine, the randomized trial found no statistically significant difference between the algorithm's and faculty's interview-recommendation rates overall, including for underrepresented-in-medicine applicants (61 of 65 vs 70 of 71 recommended, P=.14) and female applicants (220 of 227 vs 224 of 229, P=.55). The same trial found a statistically significant gap for economically disadvantaged applicants: the algorithm recommended 25 of 156 for interview versus 44 of 178 recommended by faculty (P=.048).

The authors are explicit that a model trained on past human decisions 'by definition' inherits the biases of the faculty who made them, that it only reads structured data, and that its output remains a recommendation reviewed by humans, not a final decision — the economically-disadvantaged-applicant gap is a genuine, disclosed equity caveat rather than a hidden flaw.