Mike J Bradburn1, Ellen C Lee2, David A White2, Daniel Hind2, Norman R Waugh3, Deborah D Cooke4, David Hopkins5, Peter Mansell6, Simon R Heller7. 1. Clinical Trials Research Unit, University of Sheffield, Sheffield, UK. Electronic address: m.bradburn@sheffield.ac.uk. 2. Clinical Trials Research Unit, University of Sheffield, Sheffield, UK. 3. Warwick Medical School, University of Warwick, UK. 4. School of Health Sciences, University of Surrey, UK. 5. Institute of Diabetes, Endocrinology & Obesity, King's Health Partners, London, UK. 6. Department of Diabetes and Endocrinology, Nottingham University Hospitals NHS Trust, UK. 7. Department of Oncology and Metabolism, University of Sheffield, Sheffield, UK.
Abstract
OBJECTIVES: Randomized controlled trials (RCTs) have been criticized for lacking external validity. We assessed whether a trial in people with type I diabetes mellitus mirrored the wider population and applied sample-weighting methods to assess the impact of differences on our trial's findings. STUDY DESIGN AND SETTING: The Relative Effectiveness of Pumps over MDI and Structured Education trial was nested within a large UK cohort capturing demographic, clinical, and quality of life data for people with type I diabetes mellitus undergoing structured diabetes-specific education. We first assessed whether our RCT participants were comparable with this cohort using propensity score modeling. After this, we reweighted the trial population to better match the wider cohort and re-estimated the treatment effect. RESULTS: Trial participants differed from the cohort in regard to sex, weight, HbA1c, and also quality of life and satisfaction with current treatment. Nevertheless, the treatment effects derived from alternative model weightings were similar to that of the original RCT. CONCLUSION: Our RCT participants differed in composition to the wider population, but the original findings were unaffected by sampling adjustments. We encourage investigators take steps to address criticisms of generalizability but doing so is problematic; external data, even if available, may contain limited information and analyses can be susceptible to model misspecification.
OBJECTIVES: Randomized controlled trials (RCTs) have been criticized for lacking external validity. We assessed whether a trial in people with type I diabetes mellitus mirrored the wider population and applied sample-weighting methods to assess the impact of differences on our trial's findings. STUDY DESIGN AND SETTING: The Relative Effectiveness of Pumps over MDI and Structured Education trial was nested within a large UK cohort capturing demographic, clinical, and quality of life data for people with type I diabetes mellitus undergoing structured diabetes-specific education. We first assessed whether our RCT participants were comparable with this cohort using propensity score modeling. After this, we reweighted the trial population to better match the wider cohort and re-estimated the treatment effect. RESULTS: Trial participants differed from the cohort in regard to sex, weight, HbA1c, and also quality of life and satisfaction with current treatment. Nevertheless, the treatment effects derived from alternative model weightings were similar to that of the original RCT. CONCLUSION: Our RCT participants differed in composition to the wider population, but the original findings were unaffected by sampling adjustments. We encourage investigators take steps to address criticisms of generalizability but doing so is problematic; external data, even if available, may contain limited information and analyses can be susceptible to model misspecification.
Authors: Federica Di Guardo; A Racca; G Coticchio; A Borini; P Drakopoulos; S Mackens; H Tournaye; G Verheyen; C Blockeel; L Van Landuyt Journal: J Assist Reprod Genet Date: 2022-07-20 Impact factor: 3.357