OBJECTIVE: Clinical decisions based on a meta-analysis that is based on an ineffective retrieval strategy may have serious negative consequences for patients. The study objective was to investigate the extent to which meta-analyses report proof of their retrieval strategies' effectiveness. METHODS: The authors examined a random sample (n = 100) of articles in the 1996 to 2002 full-text subset of Ovid MEDLINE indexed as "meta-analysis." We classified the articles in three ways: the article (A) reported both a retrieval strategy in sufficient detail (such that it could be repeated) and with evidence of the strategy's effectiveness, (B) reported a retrieval strategy in sufficient detail but not with evidence of the strategy's effectiveness, or (C) neither reported a strategy in detail nor evidence of the strategy's effectiveness. Articles classified as (A) were further classified according to the level of evidence reported. RESULTS: Of the eighty-nine articles in our final analysis, six (6.7%) were classified as category (A), fifty-seven (64%) as (B), and twenty-six (29%) as (C). Articles in category (A) reported a previously validated search, a published strategy, or strategy based on expert opinion. CONCLUSION: Peer-review standards must be developed that require authors of meta-analyses to report evidence for the effectiveness of their retrieval strategies.
OBJECTIVE: Clinical decisions based on a meta-analysis that is based on an ineffective retrieval strategy may have serious negative consequences for patients. The study objective was to investigate the extent to which meta-analyses report proof of their retrieval strategies' effectiveness. METHODS: The authors examined a random sample (n = 100) of articles in the 1996 to 2002 full-text subset of Ovid MEDLINE indexed as "meta-analysis." We classified the articles in three ways: the article (A) reported both a retrieval strategy in sufficient detail (such that it could be repeated) and with evidence of the strategy's effectiveness, (B) reported a retrieval strategy in sufficient detail but not with evidence of the strategy's effectiveness, or (C) neither reported a strategy in detail nor evidence of the strategy's effectiveness. Articles classified as (A) were further classified according to the level of evidence reported. RESULTS: Of the eighty-nine articles in our final analysis, six (6.7%) were classified as category (A), fifty-seven (64%) as (B), and twenty-six (29%) as (C). Articles in category (A) reported a previously validated search, a published strategy, or strategy based on expert opinion. CONCLUSION: Peer-review standards must be developed that require authors of meta-analyses to report evidence for the effectiveness of their retrieval strategies.
Authors: D F Stroup; J A Berlin; S C Morton; I Olkin; G D Williamson; D Rennie; D Moher; B J Becker; T A Sipe; S B Thacker Journal: JAMA Date: 2000-04-19 Impact factor: 56.272