Nancy A Dreyer1, Christina D Mack2, Robert B Anderson3,4, Edward M Wojtys5, Elliott B Hershman6, Allen Sills7. 1. Center for Advanced Evidence Generation, Real-World & Analytic Solutions, IQVIA, Cambridge, Massachusetts. 2. Center for Advanced Evidence Generation, Real-World & Analytic Solutions, IQVIA, Research Triangle Park, North Carolina. 3. Foot and Ankle Institute, OrthoCarolina, Charlotte, North Carolina. 4. Sports Foot/Ankle, Titletown Sports Medicine, Green Bay, Wisconsin. 5. MedSport, Department of Orthopaedic Surgery, University of Michigan, Ann Arbor, Michigan. 6. Department of Orthopaedic Surgery, Lenox Hill Hospital, North Shore-LIJ Orthopaedic Institute, North Shore-LIJ Healthcare System, New York, New York. 7. National Football League, New York, New York.
Abstract
BACKGROUND: "Research-ready" evidence platforms that link sports data with anonymized electronic health records (EHRs) or other data are important tools for evaluating injury occurrence in response to changes in games, training, rules, and other factors. While there is agreement that high-quality data are essential, there is little evidence to guide data curation. HYPOTHESIS: We hypothesized that an EHR used in the course of clinical care and curated for research readiness can provide a robust evidence platform. Our purpose was to describe the data curation used for active injury surveillance by the National Football League (NFL). STUDY DESIGN: Dynamic cohort study. LEVEL OF EVIDENCE: Level 2. METHODS: Players provide informed consent for research activities through the collective bargaining process. A league-wide EHR is used to record injuries that come to the attention of the teams' athletic trainers and physicians, NFL medical spotters, or unaffiliated neurotrauma consultants. Information about football activities and injuries are linkable by player, setting, and event to other sports-related data, including game statistics and game-day stadium quality measures, using a unique player identification designed to protect player privacy. Ongoing data curation is used to review data completeness and accuracy and is adjusted over time in response to findings. RESULTS: The core data curation activities include monthly injury summaries to team staff, queries to resolve incomplete reporting, and periodic external checks. Experiences derived from producing more than 100 reports per year on diverse topics are used to update coding training and related guidance documents in response to missing data or inconsistent coding that is observed. Roughly 20% more injuries were recorded for the same "reportable" injuries after switching from targeted reporting to an EHR. CONCLUSION: Research-ready databases need systematic curation for quality and completeness, along with related action plans. More injuries were reported through EHR than through targeted reporting. CLINICAL RELEVANCE: Evidence-driven decision-making thrives on reliable data fine-tuned through systematic use, review, and ongoing adjustments to the curation process.
BACKGROUND: "Research-ready" evidence platforms that link sports data with anonymized electronic health records (EHRs) or other data are important tools for evaluating injury occurrence in response to changes in games, training, rules, and other factors. While there is agreement that high-quality data are essential, there is little evidence to guide data curation. HYPOTHESIS: We hypothesized that an EHR used in the course of clinical care and curated for research readiness can provide a robust evidence platform. Our purpose was to describe the data curation used for active injury surveillance by the National Football League (NFL). STUDY DESIGN: Dynamic cohort study. LEVEL OF EVIDENCE: Level 2. METHODS: Players provide informed consent for research activities through the collective bargaining process. A league-wide EHR is used to record injuries that come to the attention of the teams' athletic trainers and physicians, NFL medical spotters, or unaffiliated neurotrauma consultants. Information about football activities and injuries are linkable by player, setting, and event to other sports-related data, including game statistics and game-day stadium quality measures, using a unique player identification designed to protect player privacy. Ongoing data curation is used to review data completeness and accuracy and is adjusted over time in response to findings. RESULTS: The core data curation activities include monthly injury summaries to team staff, queries to resolve incomplete reporting, and periodic external checks. Experiences derived from producing more than 100 reports per year on diverse topics are used to update coding training and related guidance documents in response to missing data or inconsistent coding that is observed. Roughly 20% more injuries were recorded for the same "reportable" injuries after switching from targeted reporting to an EHR. CONCLUSION: Research-ready databases need systematic curation for quality and completeness, along with related action plans. More injuries were reported through EHR than through targeted reporting. CLINICAL RELEVANCE: Evidence-driven decision-making thrives on reliable data fine-tuned through systematic use, review, and ongoing adjustments to the curation process.
Entities:
Keywords:
data curation; electronic health records; epidemiology surveillance; football
Authors: Joshua J Jacobs; Toby R W King; John H Klippel; Sigurd H Berven; David B Burr; Paul M Caskey; Ann L Elderkin; Paul W Esposito; Eric P Gall; Steven R Goldring; Andrew N Pollak; Christy I Sandborg; Kimberly J Templeton Journal: J Bone Joint Surg Am Date: 2013-09-04 Impact factor: 5.284
Authors: Robert M Califf; Melissa A Robb; Andrew B Bindman; Josephine P Briggs; Francis S Collins; Patrick H Conway; Trinka S Coster; Francesca E Cunningham; Nancy De Lew; Karen B DeSalvo; Christine Dymek; Victor J Dzau; Rachael L Fleurence; Richard G Frank; J Michael Gaziano; Petra Kaufmann; Michael Lauer; Peter W Marks; J Michael McGinnis; Chesley Richards; Joe V Selby; David J Shulkin; Jeffrey Shuren; Andrew M Slavitt; Scott R Smith; B Vindell Washington; P Jon White; Janet Woodcock; Jonathan Woodson; Rachel E Sherman Journal: N Engl J Med Date: 2016-12-15 Impact factor: 91.245
Authors: Elliott B Hershman; Robert Anderson; John A Bergfeld; James P Bradley; Michael J Coughlin; Robert J Johnson; Kurt P Spindler; Edward Wojtys; John W Powell Journal: Am J Sports Med Date: 2012-09-12 Impact factor: 6.202
Authors: Rachel E Sherman; Steven A Anderson; Gerald J Dal Pan; Gerry W Gray; Thomas Gross; Nina L Hunter; Lisa LaVange; Danica Marinac-Dabic; Peter W Marks; Melissa A Robb; Jeffrey Shuren; Robert Temple; Janet Woodcock; Lilly Q Yue; Robert M Califf Journal: N Engl J Med Date: 2016-12-08 Impact factor: 91.245
Authors: Ann M Bailey; Timothy L McMurry; Joseph M Cormier; James R Funk; Jeff R Crandall; Christina D Mack; Barry S Myers; Kristy B Arbogast Journal: Ann Biomed Eng Date: 2020-10-06 Impact factor: 3.934
Authors: Christina D Mack; Gary Solomon; Tracey Covassin; Nicholas Theodore; Javier Cárdenas; Allen Sills Journal: Sports Health Date: 2021-04-19 Impact factor: 3.843
Authors: Joseph D Lamplot; Dean Wang; Leigh J Weiss; Michael Baum; Kristina Zeidler; Christina Mack; Ronnie P Barnes; Russell F Warren; Samuel A Taylor; Scott A Rodeo Journal: Sports Health Date: 2021-01-11 Impact factor: 3.843
Authors: Riann M Palmieri-Smith; Christina D Mack; Robert H Brophy; Brett D Owens; Mackenzie M Herzog; Bruce D Beynnon; Kurt P Spindler; Edward M Wojtys Journal: Am J Sports Med Date: 2021-04-30 Impact factor: 7.010
Authors: Roald Bahr; Ben Clarsen; Wayne Derman; Jiri Dvorak; Carolyn A Emery; Caroline F Finch; Martin Hägglund; Astrid Junge; Simon Kemp; Karim M Khan; Stephen W Marshall; Willem Meeuwisse; Margo Mountjoy; John W Orchard; Babette Pluim; Kenneth L Quarrie; Bruce Reider; Martin Schwellnus; Torbjørn Soligard; Keith A Stokes; Toomas Timpka; Evert Verhagen; Abhinav Bindra; Richard Budgett; Lars Engebretsen; Uğur Erdener; Karim Chamari Journal: Orthop J Sports Med Date: 2020-02-18
Authors: Roald Bahr; Ben Clarsen; Wayne Derman; Jiri Dvorak; Carolyn A Emery; Caroline F Finch; Martin Hägglund; Astrid Junge; Simon Kemp; Karim M Khan; Stephen W Marshall; Willem Meeuwisse; Margo Mountjoy; John W Orchard; Babette Pluim; Kenneth L Quarrie; Bruce Reider; Martin Schwellnus; Torbjørn Soligard; Keith A Stokes; Toomas Timpka; Evert Verhagen; Abhinav Bindra; Richard Budgett; Lars Engebretsen; Uğur Erdener; Karim Chamari Journal: Br J Sports Med Date: 2020-02-18 Impact factor: 13.800
Authors: Jakob Ackermann; Vishal Saxena; Jim Whalen; Christina D Mack; Mackenzie Herzog; Jessica R Probst; Mark D Price Journal: Orthop J Sports Med Date: 2022-01-03
Authors: Derrick M Knapik; Christina D Mack; Rebecca Lee; William Heersink; Joseph E Tanenbaum; Robert J Wetzel; James E Voos Journal: Orthop J Sports Med Date: 2021-11-11