Brice Ozenne1, Fabien Subtil1, Delphine Maucort-Boulch2. 1. Université de Lyon, Département Biomaths-Santé, Lyon, 92 Rue Pasteur, 69007, France; Université Lyon 1, Département Biomaths-Santé, Villeurbanne, 43 boulevard du 11 Novembre 1918, 69622, France; Hospices Civils de Lyon, Service de Biostatistique Lyon, 165 Chemin du Grand Revoyet, Pierre-Bénite F-69310, France; CNRS UMR5558, Laboratoire de Biométrie et Biologie Evolutive, Equipe Biostatistique-Santé, Villeurbanne F-69100, France. 2. Université de Lyon, Département Biomaths-Santé, Lyon, 92 Rue Pasteur, 69007, France; Université Lyon 1, Département Biomaths-Santé, Villeurbanne, 43 boulevard du 11 Novembre 1918, 69622, France; Hospices Civils de Lyon, Service de Biostatistique Lyon, 165 Chemin du Grand Revoyet, Pierre-Bénite F-69310, France; CNRS UMR5558, Laboratoire de Biométrie et Biologie Evolutive, Equipe Biostatistique-Santé, Villeurbanne F-69100, France. Electronic address: delphine.maucort-boulch@chu-lyon.fr.
Abstract
OBJECTIVES: Compare the area under the receiver operating characteristic curve (AUC) vs. the area under the precision-recall curve (AUPRC) in summarizing the performance of a diagnostic biomarker according to the disease prevalence. STUDY DESIGN AND SETTING: A simulation study was performed considering different sizes of diseased and nondiseased groups. Values of a biomarker were sampled with various variances and differences in mean values between the two groups. The AUCs and the AUPRCs were examined regarding their agreement and vs. the positive predictive value (PPV) and the negative predictive value (NPV) of the biomarker. RESULTS: With a disease prevalence of 50%, the AUC and the AUPRC showed high correlations with the PPV and the NPV (ρ > 0.95). With a prevalence of 1%, small PPV and AUPRC values (<0.2) but high AUC values (>0.9) were found. The AUPRC reflected better than the AUC the discriminant ability of the biomarker; it had a higher correlation with the PPV (ρ = 0.995 vs. 0.724; P < 0.001). CONCLUSION: In uncommon and rare diseases, the AUPRC should be preferred to the AUC because it summarizes better the performance of a biomarker.
OBJECTIVES: Compare the area under the receiver operating characteristic curve (AUC) vs. the area under the precision-recall curve (AUPRC) in summarizing the performance of a diagnostic biomarker according to the disease prevalence. STUDY DESIGN AND SETTING: A simulation study was performed considering different sizes of diseased and nondiseased groups. Values of a biomarker were sampled with various variances and differences in mean values between the two groups. The AUCs and the AUPRCs were examined regarding their agreement and vs. the positive predictive value (PPV) and the negative predictive value (NPV) of the biomarker. RESULTS: With a disease prevalence of 50%, the AUC and the AUPRC showed high correlations with the PPV and the NPV (ρ > 0.95). With a prevalence of 1%, small PPV and AUPRC values (<0.2) but high AUC values (>0.9) were found. The AUPRC reflected better than the AUC the discriminant ability of the biomarker; it had a higher correlation with the PPV (ρ = 0.995 vs. 0.724; P < 0.001). CONCLUSION: In uncommon and rare diseases, the AUPRC should be preferred to the AUC because it summarizes better the performance of a biomarker.
Authors: D Alexander Perry; Daniel Shirley; Dejan Micic; Pratish C Patel; Rosemary Putler; Anitha Menon; Vincent B Young; Krishna Rao Journal: Clin Infect Dis Date: 2022-06-10 Impact factor: 20.999
Authors: Rebecca A Clark; Sogol Mostoufi-Moab; Yutaka Yasui; Ngoc Khanh Vu; Charles A Sklar; Tarek Motan; Russell J Brooke; Todd M Gibson; Kevin C Oeffinger; Rebecca M Howell; Susan A Smith; Zhe Lu; Leslie L Robison; Wassim Chemaitilly; Melissa M Hudson; Gregory T Armstrong; Paul C Nathan; Yan Yuan Journal: Lancet Oncol Date: 2020-02-14 Impact factor: 41.316
Authors: Akash A Shah; Sai K Devana; Changhee Lee; Amador Bugarin; Elizabeth L Lord; Arya N Shamie; Don Y Park; Mihaela van der Schaar; Nelson F SooHoo Journal: World Neurosurg Date: 2021-05-28 Impact factor: 2.210