Mike Clarke1, Sally Hopewell, Iain Chalmers. 1. UK Cochrane Centre, NHS R&D Programme, Summertown Pavilion, Middle Way, Oxford OX2 7LG, UK. mclarke@cochrane.co.uk
Abstract
OBJECTIVE: Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported. DESIGN: Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals. SETTING: Reports of randomized trials in five general medical journals. PARTICIPANTS: Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine. INTERVENTIONS: None. MAIN OUTCOME MEASURES: The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed. RESULTS: We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration - quantitative or qualitative - of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials. CONCLUSIONS: There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.
OBJECTIVE: Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported. DESIGN: Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals. SETTING: Reports of randomized trials in five general medical journals. PARTICIPANTS: Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine. INTERVENTIONS: None. MAIN OUTCOME MEASURES: The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed. RESULTS: We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration - quantitative or qualitative - of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials. CONCLUSIONS: There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.
Authors: Harriet L MacMillan; B Helen Thomas; Ellen Jamieson; Christine A Walsh; Michael H Boyle; Harry S Shannon; Amiram Gafni Journal: Lancet Date: 2005 May 21-27 Impact factor: 79.321
Authors: Anna Bill-Axelson; Lars Holmberg; Mirja Ruutu; Michael Häggman; Swen-Olof Andersson; Stefan Bratell; Anders Spångberg; Christer Busch; Stig Nordling; Hans Garmo; Juni Palmgren; Hans-Olov Adami; Bo Johan Norlén; Jan-Erik Johansson Journal: N Engl J Med Date: 2005-05-12 Impact factor: 91.245
Authors: A M Grant; A Avenell; M K Campbell; A M McDonald; G S MacLennan; G C McPherson; F H Anderson; C Cooper; R M Francis; C Donaldson; W J Gillespie; C M Robinson; D J Torgerson; W A Wallace Journal: Lancet Date: 2005 May 7-13 Impact factor: 79.321
Authors: Pierre J Guillou; Philip Quirke; Helen Thorpe; Joanne Walker; David G Jayne; Adrian M H Smith; Richard M Heath; Julia M Brown Journal: Lancet Date: 2005 May 14-20 Impact factor: 79.321
Authors: Herbert L DuPont; Zhi-Dong Jiang; Pablo C Okhuysen; Charles D Ericsson; Francisco Javier de la Cabada; Shi Ke; Margaret W DuPont; Francisco Martinez-Sandoval Journal: Ann Intern Med Date: 2005-05-17 Impact factor: 25.391
Authors: Bramah N Singh; Steven N Singh; Domenic J Reda; X Charlene Tang; Becky Lopez; Crystal L Harris; Ross D Fletcher; Satish C Sharma; J Edwin Atwood; Alan K Jacobson; H Daniel Lewis; Dennis W Raisch; Michael D Ezekowitz Journal: N Engl J Med Date: 2005-05-05 Impact factor: 91.245
Authors: B S Chong; A M Berman; G Billis; A M Quinn; C D Emery; G Di Filippo; H F Duncan; E Schäfer; N P Chandler Journal: Br Dent J Date: 2016-02-12 Impact factor: 1.626
Authors: David Moher; Sally Hopewell; Kenneth F Schulz; Victor Montori; Peter C Gøtzsche; P J Devereaux; Diana Elbourne; Matthias Egger; Douglas G Altman Journal: BMJ Date: 2010-03-23
Authors: Alessandro Liberati; Douglas G Altman; Jennifer Tetzlaff; Cynthia Mulrow; Peter C Gøtzsche; John P A Ioannidis; Mike Clarke; P J Devereaux; Jos Kleijnen; David Moher Journal: BMJ Date: 2009-07-21
Authors: Alessandro Liberati; Douglas G Altman; Jennifer Tetzlaff; Cynthia Mulrow; Peter C Gøtzsche; John P A Ioannidis; Mike Clarke; P J Devereaux; Jos Kleijnen; David Moher Journal: PLoS Med Date: 2009-07-21 Impact factor: 11.069
Authors: Nynke R van den Broek; Sarah A White; Mark Goodall; Chikondi Ntonya; Edith Kayira; George Kafulafula; James P Neilson Journal: PLoS Med Date: 2009-12-01 Impact factor: 11.069