Ben Ewald1. 1. Centre for Clinical Epidemiology, University of Newcastle, Maddison Building , Level 3, NSW, Australia. Ben.Ewald@newcastle.edu.au [corrected]
Abstract
BACKGROUND AND OBJECTIVE: To examine the extent of bias introduced to diagnostic test validity research by the use of post hoc data driven analysis to generate an optimal diagnostic cut point for each data set. METHODS: Analysis of simulated data sets of test results for diseased and nondiseased subjects, comparing data driven to prespecified cut points for various sample sizes and disease prevalence levels. RESULTS: In studies of 100 subjects with 50% prevalence a positive bias of five percentage points of sensitivity or specificity was found in 6 of 20 simulations. For studies of 250 subjects with 10% prevalence a positive bias of 5% was observed in 4 of 20 simulations. CONCLUSION: The use of data-driven cut points exaggerates test performance in many simulated data sets, and this bias probably affects many published diagnostic validity studies. Prespecified cut points, when available, would improve the validity of diagnostic test research in studies with less than 50 cases of disease.
BACKGROUND AND OBJECTIVE: To examine the extent of bias introduced to diagnostic test validity research by the use of post hoc data driven analysis to generate an optimal diagnostic cut point for each data set. METHODS: Analysis of simulated data sets of test results for diseased and nondiseased subjects, comparing data driven to prespecified cut points for various sample sizes and disease prevalence levels. RESULTS: In studies of 100 subjects with 50% prevalence a positive bias of five percentage points of sensitivity or specificity was found in 6 of 20 simulations. For studies of 250 subjects with 10% prevalence a positive bias of 5% was observed in 4 of 20 simulations. CONCLUSION: The use of data-driven cut points exaggerates test performance in many simulated data sets, and this bias probably affects many published diagnostic validity studies. Prespecified cut points, when available, would improve the validity of diagnostic test research in studies with less than 50 cases of disease.
Authors: Brooke Levis; Andrea Benedetti; Alexander W Levis; John P A Ioannidis; Ian Shrier; Pim Cuijpers; Simon Gilbody; Lorie A Kloda; Dean McMillan; Scott B Patten; Russell J Steele; Roy C Ziegelstein; Charles H Bombardier; Flavia de Lima Osório; Jesse R Fann; Dwenda Gjerdingen; Femke Lamers; Manote Lotrakul; Sonia R Loureiro; Bernd Löwe; Juwita Shaaban; Lesley Stafford; Henk C P M van Weert; Mary A Whooley; Linda S Williams; Karin A Wittkampf; Albert S Yeung; Brett D Thombs Journal: Am J Epidemiol Date: 2017-05-15 Impact factor: 4.897
Authors: Mariska M G Leeflang; Jonathan J Deeks; Constantine Gatsonis; Patrick M M Bossuyt Journal: Ann Intern Med Date: 2008-12-16 Impact factor: 25.391
Authors: Elba C Etchebehere; Brian P Hobbs; Denái R Milton; Osama Malawi; Shreyaskumar Patel; Robert S Benjamin; Homer A Macapinlac Journal: Eur J Nucl Med Mol Imaging Date: 2015-12-03 Impact factor: 9.236
Authors: Charity L Washam; Stephanie D Byrum; Kim Leitzel; Suhail M Ali; Alan J Tackett; Dana Gaddy; Suzanne E Sundermann; Allan Lipton; Larry J Suva Journal: Cancer Epidemiol Biomarkers Prev Date: 2013-03-05 Impact factor: 4.254