Joel J Gagnier1, Patrick J Kellam. 1. Department of Orthopaedic Surgery, University of Michigan, 24 Frank Lloyd Wright Drive, MedSport, Domino's Farms, Ann Arbor, MI 48106-0391, USA. jgagnier@umich.edu
Abstract
BACKGROUND: Properly designed and conducted systematic reviews can reliably produce valid pooled treatment-effect estimates and are an important resource for clinical decision-making. The purpose of this report was to assess the reporting and methodological quality of systematic reviews in orthopaedic journals. METHODS: With use of the 2010 Institute for Scientific Information (ISI) Thomson Reuters Journal Citation Reports, the five orthopaedic surgery journals with the highest impact factors were searched by one individual over a five-year period (from 2006 to 2010) for systematic reviews and meta-analyses. The two authors separately and independently assessed the included studies. The reporting quality was assessed with use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, and the methodological quality was assessed with use of the Assessment of Multiple Systematic Reviews (AMSTAR) guidelines, both of which are accepted instruments. We calculated the proportions of each item reported within and across journals. RESULTS: Seventy-six systematic reviews and meta-analyses were included. Of the five journals that were examined, articles from The Journal of Bone and Joint Surgery (American Volume) had the best reporting. Articles from The American Journal of Sports Medicine fulfilled the most methodological quality items. The papers from all of the journals reported an average of only 68% of the PRISMA items and only 54% of the AMSTAR quality items. CONCLUSIONS: Both reporting and methodological quality in the top five orthopaedic journals were poor; the reporting quality was slightly superior to the methodological quality. Although there was a wide range of reporting quality and methodological quality scores across the journals, the included articles demonstrated inadequate adherence to accepted standards of quality. The use of PRISMA and AMSTAR guidelines in designing, implementing, and writing systematic reviews is recommended to improve the quality of systematic reviews and meta-analyses in orthopaedic journals. CLINICAL RELEVANCE: The validity of published systematic reviews in orthopaedics is questionable, and their contribution to clinical decision-making is suboptimal. Clinicians should be careful when interpreting and applying findings of current orthopaedic systematic reviews.
BACKGROUND: Properly designed and conducted systematic reviews can reliably produce valid pooled treatment-effect estimates and are an important resource for clinical decision-making. The purpose of this report was to assess the reporting and methodological quality of systematic reviews in orthopaedic journals. METHODS: With use of the 2010 Institute for Scientific Information (ISI) Thomson Reuters Journal Citation Reports, the five orthopaedic surgery journals with the highest impact factors were searched by one individual over a five-year period (from 2006 to 2010) for systematic reviews and meta-analyses. The two authors separately and independently assessed the included studies. The reporting quality was assessed with use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, and the methodological quality was assessed with use of the Assessment of Multiple Systematic Reviews (AMSTAR) guidelines, both of which are accepted instruments. We calculated the proportions of each item reported within and across journals. RESULTS: Seventy-six systematic reviews and meta-analyses were included. Of the five journals that were examined, articles from The Journal of Bone and Joint Surgery (American Volume) had the best reporting. Articles from The American Journal of Sports Medicine fulfilled the most methodological quality items. The papers from all of the journals reported an average of only 68% of the PRISMA items and only 54% of the AMSTAR quality items. CONCLUSIONS: Both reporting and methodological quality in the top five orthopaedic journals were poor; the reporting quality was slightly superior to the methodological quality. Although there was a wide range of reporting quality and methodological quality scores across the journals, the included articles demonstrated inadequate adherence to accepted standards of quality. The use of PRISMA and AMSTAR guidelines in designing, implementing, and writing systematic reviews is recommended to improve the quality of systematic reviews and meta-analyses in orthopaedic journals. CLINICAL RELEVANCE: The validity of published systematic reviews in orthopaedics is questionable, and their contribution to clinical decision-making is suboptimal. Clinicians should be careful when interpreting and applying findings of current orthopaedic systematic reviews.