Jacopo Olivieri1, Lucia Manfredi2, Laura Postacchini2, Silvia Tedesco2, Pietro Leoni1, Armando Gabrielli2, Alessandro Rambaldi3, Andrea Bacigalupo4, Attilio Olivieri5, Giovanni Pomponio6. 1. Clinica di Ematologia, Università Politecnica Delle Marche, Ancona, Italy. 2. Clinica Medica, Università Politecnica Delle Marche, Ancona, Italy. 3. Unità di Ematologia e Trapianto di Midollo, Azienda Ospedaliera Papa Giovanni XXIII, Bergamo, Italy. 4. Divisione Ematologia e Trapianto di Midollo, IRCCS San Martino, Genova, Italy. 5. Clinica di Ematologia, Università Politecnica Delle Marche, Ancona, Italy. Electronic address: a.olivieri@univpm.it. 6. Clinica Medica, Azienda Ospedaliera-Universitaria "Ospedali Riuniti", Ancona, Italy.
Abstract
BACKGROUND: Consensus recommendations are used to improve the methodology of research about rare disorders, but their uptake is unknown. We studied the uptake of consensus recommendations in steroid-refractory chronic graft-versus-host disease (SR-cGVHD). Although in 2006 the National Institutes of Health (NIH) cGVHD consensus project produced recommendations for clinical trials, guidelines have emphasised the scarcity of valuable evidence for all tested interventions. METHODS: We searched Medline (PubMed) between Jan 1, 1998, and Oct 1, 2013, for non-randomised studies of systemic treatment for SR-cGVHD. To measure adherence to NIH recommendations, we applied a 61 item checklist derived from the NIH consensus document. We did a meta-analysis to measure pooled effect size for overall response rate (ORR) and meta-regression analyses to measure the effect of deviations from NIH recommendations on pooled effect size. FINDINGS: We included 82 studies related to nine interventions. Conformity to NIH recommendations was evenly low across the analysed timeframe (1998-2013), and did not change significantly after publication of NIH recommendations. The pooled effect size for ORR for systemic treatment of SR-cGVHD was 0.66 (95% CI 0.62-0.70). Increased adherence to NIH recommendations in a score of items defining correct response assessment was associated with a significant reduction in ORR (-4.2%, 95% CI -6.6 to -1.9; p=0.001). We recorded no significant association between ORR and sets of items related to correct diagnostic definition of SR-cGVHD (change in ORR -3.1%, 95% CI -7.7 to 1.5), specification of primary intervention (0, -3.8 to 3.6), or concomitant treatments (-1.6%, -5.4 to 2.3). The score of items defining correct response assessment increased after publication of NIH recommendations. INTERPRETATION: Our findings show evidence of bias in the reported efficacy of treatment of SR-cGVHD. The overall effect of NIH recommendations in scientific literature is scarce; however, NIH recommendations improved assessment of response, possibly reducing the overestimation bias. Better implementation of NIH recommendations might reduce false expectations about new interventions, and thus prevent clinical studies with ineffective treatments. FUNDING: None.
BACKGROUND: Consensus recommendations are used to improve the methodology of research about rare disorders, but their uptake is unknown. We studied the uptake of consensus recommendations in steroid-refractory chronic graft-versus-host disease (SR-cGVHD). Although in 2006 the National Institutes of Health (NIH) cGVHD consensus project produced recommendations for clinical trials, guidelines have emphasised the scarcity of valuable evidence for all tested interventions. METHODS: We searched Medline (PubMed) between Jan 1, 1998, and Oct 1, 2013, for non-randomised studies of systemic treatment for SR-cGVHD. To measure adherence to NIH recommendations, we applied a 61 item checklist derived from the NIH consensus document. We did a meta-analysis to measure pooled effect size for overall response rate (ORR) and meta-regression analyses to measure the effect of deviations from NIH recommendations on pooled effect size. FINDINGS: We included 82 studies related to nine interventions. Conformity to NIH recommendations was evenly low across the analysed timeframe (1998-2013), and did not change significantly after publication of NIH recommendations. The pooled effect size for ORR for systemic treatment of SR-cGVHD was 0.66 (95% CI 0.62-0.70). Increased adherence to NIH recommendations in a score of items defining correct response assessment was associated with a significant reduction in ORR (-4.2%, 95% CI -6.6 to -1.9; p=0.001). We recorded no significant association between ORR and sets of items related to correct diagnostic definition of SR-cGVHD (change in ORR -3.1%, 95% CI -7.7 to 1.5), specification of primary intervention (0, -3.8 to 3.6), or concomitant treatments (-1.6%, -5.4 to 2.3). The score of items defining correct response assessment increased after publication of NIH recommendations. INTERPRETATION: Our findings show evidence of bias in the reported efficacy of treatment of SR-cGVHD. The overall effect of NIH recommendations in scientific literature is scarce; however, NIH recommendations improved assessment of response, possibly reducing the overestimation bias. Better implementation of NIH recommendations might reduce false expectations about new interventions, and thus prevent clinical studies with ineffective treatments. FUNDING: None.
Authors: David Miklos; Corey S Cutler; Mukta Arora; Edmund K Waller; Madan Jagasia; Iskra Pusic; Mary E Flowers; Aaron C Logan; Ryotaro Nakamura; Bruce R Blazar; Yunfeng Li; Stephen Chang; Indu Lal; Jason Dubovsky; Danelle F James; Lori Styles; Samantha Jaglowski Journal: Blood Date: 2017-09-18 Impact factor: 22.113