Mark J Bolland1, Greg D Gamble2, Alison Avenell3, Andrew Grey2, Thomas Lumley4. 1. Department of Medicine, University of Auckland, Private Bag 92 019, Auckland 1142, New Zealand. Electronic address: m.bolland@auckland.ac.nz. 2. Department of Medicine, University of Auckland, Private Bag 92 019, Auckland 1142, New Zealand. 3. Health Services Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK. 4. Department of Statistics, University of Auckland, Private Bag 92 019, Auckland 1142, New Zealand.
Abstract
OBJECTIVE: Comparing observed and expected distributions of baseline variables in randomized controlled trials (RCTs) has been used to investigate possible research misconduct, although the validity of this approach has been questioned. We explored this technique and introduced a novel metric to compare P values from baseline variables between treatment arms. STUDY DESIGN AND SETTING: We compared observed with expected distributions of baseline P values using a one-way chi-square test and by comparing the area under the curve (AUC) of the cumulative distribution function in 13 RCTs conducted by our group, two groups of RCTs known to contain fabricated data, and simulations. RESULTS: In our 13 RCTs, the distribution of P values from baseline continuous variables was consistent with the expected theoretical uniform distribution (P = 0.19, difference from expected AUC -0.03, 95% confidence interval [-0.04, 0.04]). For categorical variables, the P value distribution was not uniform. The distributions of P values from RCTs with fabricated data were highly unusual and not consistent with the uniform distribution for continuous variables, nor with the expected distribution for categorical variables, nor with the distribution of P values in genuine RCTs. CONCLUSIONS: Assessing baseline P values in groups of RCTs can identify highly unusual distributions that might raise or reinforce concerns about randomization and data integrity.
OBJECTIVE: Comparing observed and expected distributions of baseline variables in randomized controlled trials (RCTs) has been used to investigate possible research misconduct, although the validity of this approach has been questioned. We explored this technique and introduced a novel metric to compare P values from baseline variables between treatment arms. STUDY DESIGN AND SETTING: We compared observed with expected distributions of baseline P values using a one-way chi-square test and by comparing the area under the curve (AUC) of the cumulative distribution function in 13 RCTs conducted by our group, two groups of RCTs known to contain fabricated data, and simulations. RESULTS: In our 13 RCTs, the distribution of P values from baseline continuous variables was consistent with the expected theoretical uniform distribution (P = 0.19, difference from expected AUC -0.03, 95% confidence interval [-0.04, 0.04]). For categorical variables, the P value distribution was not uniform. The distributions of P values from RCTs with fabricated data were highly unusual and not consistent with the uniform distribution for continuous variables, nor with the expected distribution for categorical variables, nor with the distribution of P values in genuine RCTs. CONCLUSIONS: Assessing baseline P values in groups of RCTs can identify highly unusual distributions that might raise or reinforce concerns about randomization and data integrity.
Authors: Colby J Vorland; Andrew W Brown; John A Dawson; Stephanie L Dickinson; Lilian Golzarri-Arroyo; Bridget A Hannon; Moonseong Heo; Steven B Heymsfield; Wasantha P Jayawardene; Chanaka N Kahathuduwa; Scott W Keith; J Michael Oakes; Carmen D Tekwe; Lehana Thabane; David B Allison Journal: Int J Obes (Lond) Date: 2021-07-29 Impact factor: 5.095