Garrett S Bullock1,2, Joseph Mylott3,4, Tom Hughes5,6, Kristen F Nicholson3, Richard D Riley7, Gary S Collins8,9. 1. Department of Orthopaedic Surgery, Wake Forest School of Medicine, 475 Vine St, Bowman Gray Medical Building, Winston-Salem, NC, 27101, USA. gbullock@wakehealth.edu. 2. Centre for Sport, Exercise and Osteoarthritis Research Versus Arthritis, University of Oxford, Oxford, UK. gbullock@wakehealth.edu. 3. Department of Orthopaedic Surgery, Wake Forest School of Medicine, 475 Vine St, Bowman Gray Medical Building, Winston-Salem, NC, 27101, USA. 4. Baltimore Orioles Baseball Club, Baltimore, USA. 5. Manchester United Football Club, Manchester, UK. 6. Department of Health Professions, Manchester Metropolitan University, Manchester, UK. 7. Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK. 8. Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, UK. 9. Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
Abstract
BACKGROUND: An increasing number of musculoskeletal injury prediction models are being developed and implemented in sports medicine. Prediction model quality needs to be evaluated so clinicians can be informed of their potential usefulness. OBJECTIVE: To evaluate the methodological conduct and completeness of reporting of musculoskeletal injury prediction models in sport. METHODS: A systematic review was performed from inception to June 2021. Studies were included if they: (1) predicted sport injury; (2) used regression, machine learning, or deep learning models; (3) were written in English; (4) were peer reviewed. RESULTS: Thirty studies (204 models) were included; 60% of studies utilized only regression methods, 13% only machine learning, and 27% both regression and machine learning approaches. All studies developed a prediction model and no studies externally validated a prediction model. Two percent of models (7% of studies) were low risk of bias and 98% of models (93% of studies) were high or unclear risk of bias. Three studies (10%) performed an a priori sample size calculation; 14 (47%) performed internal validation. Nineteen studies (63%) reported discrimination and two (7%) reported calibration. Four studies (13%) reported model equations for statistical predictions and no machine learning studies reported code or hyperparameters. CONCLUSION: Existing sport musculoskeletal injury prediction models were poorly developed and have a high risk of bias. No models could be recommended for use in practice. The majority of models were developed with small sample sizes, had inadequate assessment of model performance, and were poorly reported. To create clinically useful sports musculoskeletal injury prediction models, considerable improvements in methodology and reporting are urgently required.
BACKGROUND: An increasing number of musculoskeletal injury prediction models are being developed and implemented in sports medicine. Prediction model quality needs to be evaluated so clinicians can be informed of their potential usefulness. OBJECTIVE: To evaluate the methodological conduct and completeness of reporting of musculoskeletal injury prediction models in sport. METHODS: A systematic review was performed from inception to June 2021. Studies were included if they: (1) predicted sport injury; (2) used regression, machine learning, or deep learning models; (3) were written in English; (4) were peer reviewed. RESULTS: Thirty studies (204 models) were included; 60% of studies utilized only regression methods, 13% only machine learning, and 27% both regression and machine learning approaches. All studies developed a prediction model and no studies externally validated a prediction model. Two percent of models (7% of studies) were low risk of bias and 98% of models (93% of studies) were high or unclear risk of bias. Three studies (10%) performed an a priori sample size calculation; 14 (47%) performed internal validation. Nineteen studies (63%) reported discrimination and two (7%) reported calibration. Four studies (13%) reported model equations for statistical predictions and no machine learning studies reported code or hyperparameters. CONCLUSION: Existing sport musculoskeletal injury prediction models were poorly developed and have a high risk of bias. No models could be recommended for use in practice. The majority of models were developed with small sample sizes, had inadequate assessment of model performance, and were poorly reported. To create clinically useful sports musculoskeletal injury prediction models, considerable improvements in methodology and reporting are urgently required.
Authors: Karel G M Moons; Andre Pascal Kengne; Mark Woodward; Patrick Royston; Yvonne Vergouwe; Douglas G Altman; Diederick E Grobbee Journal: Heart Date: 2012-03-07 Impact factor: 5.994
Authors: Evangelia Christodoulou; Jie Ma; Gary S Collins; Ewout W Steyerberg; Jan Y Verbakel; Ben Van Calster Journal: J Clin Epidemiol Date: 2019-02-11 Impact factor: 6.437
Authors: W A Knaus; D P Wagner; E A Draper; J E Zimmerman; M Bergner; P G Bastos; C A Sirio; D J Murphy; T Lotring; A Damiano Journal: Chest Date: 1991-12 Impact factor: 9.410
Authors: Paula Dhiman; Jie Ma; Constanza Andaur Navarro; Benjamin Speich; Garrett Bullock; Johanna Aa Damen; Shona Kirtley; Lotty Hooft; Richard D Riley; Ben Van Calster; Karel G M Moons; Gary S Collins Journal: J Clin Epidemiol Date: 2021-06-29 Impact factor: 6.437
Authors: He Cai; Garrett S Bullock; Maria T Sanchez-Santos; Nicholas Peirce; Nigel K Arden; Stephanie R Filbay Journal: BMC Musculoskelet Disord Date: 2019-12-12 Impact factor: 2.362
Authors: Ben Van Calster; Laure Wynants; Dirk Timmerman; Ewout W Steyerberg; Gary S Collins Journal: J Am Med Inform Assoc Date: 2019-12-01 Impact factor: 4.497