Nicole Wyman Engen1, Kathy Huppler Hullsiek1, Waldo H Belloso2, Elizabeth Finley3, Fleur Hudson4, Eileen Denning1, Catherine Carey5, Mary Pearson6, Jonathan Kagan7. 1. Division of Biostatistics, University of Minnesota, Minneapolis, MN, USA. 2. CICAL and Hospital Italiano de Buenos Aires, Buenos Aires, Argentina. 3. Washington Veterans Affairs Medical Center, Washington, DC, USA. 4. Medical Research Council Clinical Trials Unit, University College London, London, UK. 5. The Kirby Institute, University of New South Wales, Sydney, NSW, Australia. 6. Rigshospitalet, University of Copenhagen, Copenhagen, Denmark. 7. National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, MD, USA.
Abstract
BACKGROUND: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. METHODS: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. RESULTS:In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1-2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8-85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1-3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. CONCLUSION: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.
RCT Entities:
BACKGROUND: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. METHODS: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant-level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios and 95% confidence intervals comparing on-site monitoring versus no on-site monitoring are cited. RESULTS: In total, 99 sites (2107 participants) were randomized to receive annual on-site monitoring and 97 sites (2264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (odds ratio = 1.7; 95% confidence interval: 1.1-2.7; p = 0.03). Informed consent violations accounted for most outcomes in each group (56 vs 41 participants). The largest odds ratio was for eligibility violations (odds ratio = 12.2; 95% confidence interval: 1.8-85.2; p = 0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 vs 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (odds ratio = 2.0; 95% confidence interval: 1.1-3.7; p = 0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over US$2 million. CONCLUSION: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.
Entities:
Keywords:
On-site monitoring; central monitoring; cluster randomization; fixed effects; generalized estimating equation; hierarchical logistic regression; local monitoring; proc genmod
Authors: Alan R Lifson; Frank S Rhame; Waldo H Belloso; Ulrik B Dragsted; Wafaa M El-Sadr; Jose M Gatell; Jennifer F Hoy; Eric A Krum; Ray Nelson; Court Pedersen; Sarah L Pett; Richard T Davey Journal: HIV Clin Trials Date: 2006 May-Jun
Authors: Eric L Eisenstein; Philip W Lemons; Barbara E Tardiff; Kevin A Schulman; M King Jolly; Robert M Califf Journal: Am Heart J Date: 2005-03 Impact factor: 4.749
Authors: Sally P Stenning; William J Cragg; Nicola Joffe; Carlos Diaz-Montana; Rahela Choudhury; Matthew R Sydes; Sarah Meredith Journal: Clin Trials Date: 2018-08-22 Impact factor: 2.486
Authors: Catrin Tudur Smith; Deborah D Stocken; Janet Dunn; Trevor Cox; Paula Ghaneh; David Cunningham; John P Neoptolemos Journal: PLoS One Date: 2012-12-12 Impact factor: 3.240
Authors: Kia E Bryant; Yan Yuan; Melissa Engle; Ekaterina V Kurbatova; Cynthia Allen-Blige; Kumar Batra; Nicole E Brown; Kuo Wei Chiu; Howard Davis; Mascha Elskamp; Melissa Fagley; Pamela Fedrick; Kimberley N C Hedges; Kim Narunsky; Joanita Nassali; Mimi Phan; Ha Phan; Anne E Purfield; Jessica N Ricaldi; Kathleen Robergeau-Hunt; William C Whitworth; Erin E Sizemore Journal: Contemp Clin Trials Date: 2021-03-10 Impact factor: 2.226