OBJECTIVE: To investigate the credibility of authors' claims of subgroup effects using a representative sample of recently published randomised controlled trials. DESIGN: Systematic review. DATA SOURCE: Core clinical journals, as defined by the National Library of Medicine, in Medline. STUDY SELECTION: Randomised controlled trials published in 2007. Using prespecified criteria, teams of trained reviewers independently judged whether authors claimed subgroup effects and the strength of their claims. Reviewers assessed each of these claims against 10 predefined criteria, developed through a search of existing criteria and a consensus process. RESULTS: Of 207 randomised controlled trials reporting subgroup analyses, 64 (31%) made claims for the primary outcome. Of those, 20 were strong claims and 28 claims of a likely effect. Authors included subgroup variables measured at baseline in 60 (94%) trials, used subgroup variable as a stratification factor at randomisation in 13 (20%), clearly prespecified their hypotheses in 26 (41%), correctly prespecified direction in 4 (6%), tested a small number of hypotheses in 28 (44%), carried out a test of interaction that proved statistically significant in 6 (9%), documented replication of a subgroup effect with previous related studies in 21 (33%), identified consistency of a subgroup effect across related outcomes in 19 (30%), and provided a compelling indirect evidence for the effect in 14 (22%). In the 19 trials making more than one claim, only one (5%) checked the independence of the interaction. Of the 64 claims, 54 (84%) met four or fewer of the 10 criteria. For strong claims, more than 50% failed each of the individual criteria, and only three (15%) met more than five criteria. CONCLUSION: Authors often claim subgroup effects in their trial report. However, the credibility of subgroup effects, even when claims are strong, is usually low. Users of the information should treat claims that fail to meet most criteria with scepticism. Trial researchers should report the conduct of subgroup analyses and provide sufficient evidence when claiming a subgroup effect or suggesting a possible effect.
OBJECTIVE: To investigate the credibility of authors' claims of subgroup effects using a representative sample of recently published randomised controlled trials. DESIGN: Systematic review. DATA SOURCE: Core clinical journals, as defined by the National Library of Medicine, in Medline. STUDY SELECTION: Randomised controlled trials published in 2007. Using prespecified criteria, teams of trained reviewers independently judged whether authors claimed subgroup effects and the strength of their claims. Reviewers assessed each of these claims against 10 predefined criteria, developed through a search of existing criteria and a consensus process. RESULTS: Of 207 randomised controlled trials reporting subgroup analyses, 64 (31%) made claims for the primary outcome. Of those, 20 were strong claims and 28 claims of a likely effect. Authors included subgroup variables measured at baseline in 60 (94%) trials, used subgroup variable as a stratification factor at randomisation in 13 (20%), clearly prespecified their hypotheses in 26 (41%), correctly prespecified direction in 4 (6%), tested a small number of hypotheses in 28 (44%), carried out a test of interaction that proved statistically significant in 6 (9%), documented replication of a subgroup effect with previous related studies in 21 (33%), identified consistency of a subgroup effect across related outcomes in 19 (30%), and provided a compelling indirect evidence for the effect in 14 (22%). In the 19 trials making more than one claim, only one (5%) checked the independence of the interaction. Of the 64 claims, 54 (84%) met four or fewer of the 10 criteria. For strong claims, more than 50% failed each of the individual criteria, and only three (15%) met more than five criteria. CONCLUSION: Authors often claim subgroup effects in their trial report. However, the credibility of subgroup effects, even when claims are strong, is usually low. Users of the information should treat claims that fail to meet most criteria with scepticism. Trial researchers should report the conduct of subgroup analyses and provide sufficient evidence when claiming a subgroup effect or suggesting a possible effect.
Authors: Georgina G Gurney; Robert L Pressey; Joshua E Cinner; Richard Pollnac; Stuart J Campbell Journal: Philos Trans R Soc Lond B Biol Sci Date: 2015-11-05 Impact factor: 6.237
Authors: Kanwal Pratap Singh Raghav; Sminil Mahajan; James C Yao; Brian P Hobbs; Donald A Berry; Rebecca D Pentz; Alda Tam; Waun K Hong; Lee M Ellis; James Abbruzzese; Michael J Overman Journal: J Clin Oncol Date: 2015-08-24 Impact factor: 44.544
Authors: Vangie A Foshee; Thad Benefield; Kimberly S Dixon; Ling-Yin Chang; Virginia Senkomago; Susan T Ennett; Kathryn E Moracco; J Michael Bowling Journal: J Youth Adolesc Date: 2015-03-17
Authors: Puey Ling Chia; Craig Gedye; Paul C Boutros; Paul Wheatley-Price; Thomas John Journal: J Natl Cancer Inst Date: 2016-05-31 Impact factor: 13.506
Authors: Patrick S Parfrey; Geoffrey A Block; Ricardo Correa-Rotter; Tilman B Drüeke; Jürgen Floege; Charles A Herzog; Gerard M London; Kenneth W Mahaffey; Sharon M Moe; David C Wheeler; Glenn M Chertow Journal: Clin J Am Soc Nephrol Date: 2015-11-27 Impact factor: 8.237