to the editor: Our discussions in the medical education community have revealed that some medical schools have administered multiple-choice (MC) summative assessments of competence unproctored, or with web-based remote proctoring not involving high-security proctoring protocols during the COVID-19 pandemic. While this may be justified under these unprecedented circumstances, a key question is what do schools that typically reuse summative test items across cohorts do to minimize threats to validity of test scores from future administrations of summative assessments of competence with possibly compromised items? The beneficiaries would take the exam under standard conditions in the future, yet their exam would be suspect. As it is rarely possible to replenish an entire summative MC test form with valid high-quality items on a yearly basis, maintaining a secure item database is a key measure (1). This is also an issue that directly relates to public accountability of medical schools.A legal defense of the validity of unproctored high-stakes online testing in the face of arguably unknowable effects of cheating would be problematic (2). As documented earlier (3), it is possible to obtain screenshots of test items with smart phones or other devices, potentially compromising test item security. Even with web-based remote monitoring, these actions may escape detection.We suggest that, during the pandemic (while test item security is potentially compromised), faculty committees select 50–70% of the questions of a given exam from the proven items (with acceptable item performance data) in their secure database, with new items to make up the remainder. Additionally, a transient 15–25% reduction in the number of summative MC items for a course held during the pandemic might be considered, if it would still allow for an adequate sampling of learning from across various aspects of a course and enable valid pass-fail adjudications in each course/program.After the pandemic, the same exam might be composed of ~70% protected, uncompromised, proven items from the secure database and ~30% newly written ones. This approach would allow comparison of cohorts of examinees, validate the exam, and maximize exam security, but might be wasteful and would not enable the institution to ascertain the necessity to discard valuable test items.Alternatively, ~25% of test items used during the pandemic under nonstandard proctoring conditions would be reused, with ~50% of items drawn from the secure database, and newly written items making up the rest. This approach enables a statistical screen for a “boost” in scores suggestive of significant irregular behavior connected to unprotected (potentially compromised) reused items at the level of a cohort of examinees, or potentially at the level of individual examinees. To be clear, this letter does not pertain to deliberately designed, authentic summative assessments of competence intended to be of the “open-book” type.
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the authors.
DISCLAIMERS
The views expressed herein are strictly the personal views of the authors.
AUTHOR CONTRIBUTIONS
E.S.P. and B.K. conceived and designed research; E.S.P. drafted manuscript; E.S.P. and B.K. edited and revised manuscript; E.S.P. and B.K. approved final version of manuscript.