Monica Taljaard1, Steve McDonald2, Stuart G Nicholls3, Kelly Carroll3, Spencer P Hey4, Jeremy M Grimshaw5, Dean A Fergusson5, Merrick Zwarenstein6, Joanne E McKenzie2. 1. Clinical Epidemiology Program, Ottawa Hospital Research Institute (OHRI), The Ottawa Hospital, General Campus, 501 Smyth Road, Ottawa, Canada, K1H 8L6; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada. Electronic address: mtaljaard@ohri.ca. 2. School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, Victoria 3004, Australia. 3. Clinical Epidemiology Program, Ottawa Hospital Research Institute (OHRI), The Ottawa Hospital, General Campus, 501 Smyth Road, Ottawa, Canada, K1H 8L6. 4. Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, 1620 Tremont Street, Boston, MA, USA, 02120; Center for Bioethics, Harvard Medical School, Boston, MA, USA. 5. School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute (OHRI), The Ottawa Hospital, General Campus, 501 Smyth Road, Ottawa, Canada, K1H 8L6; Department of Medicine, University of Ottawa, Ottawa, Canada. 6. Centre for Studies in Family Medicine, Department of Family Medicine, Schulich School of Medicine & Dentistry, Western University, 1151 Richmond Street, London, Ontario, Canada, N6A 3K7.
Abstract
OBJECTIVES: Identifying pragmatic trials from among all randomized trials is challenging because of inconsistent reporting. Our objective was to develop and validate a search filter to identify reports of pragmatic trials from Ovid MEDLINE. STUDY DESIGN AND SETTING: Two sets of known and probable pragmatic trial records were analyzed using text mining to generate candidate terms. Two large population sets comprising clinical trials and explanatory trials were used to select discriminating terms. Various combinations of terms were tested iteratively to achieve optimal search performance. Two externally derived sets were used to validate sensitivity and specificity of the derived filters. RESULTS: Our validated sensitivity-maximizing filter (combines trial design terms with terms relating to attributes of pragmatic trials) retrieves over 42,000 records in MEDLINE and has sensitivity of 46.4% (95% confidence interval (CI) 37.2 to 55.7%) and estimated specificity of 98.1% (95% CI 93.4 to 99.8%). Search performance is superior to other ad hoc filters for pragmatic trials. The Cochrane search for randomized trials has much better sensitivity (98.2%), but poorer specificity (1.9%) and retrieves 4.5 million records. CONCLUSION: A highly specific filter (low false positive rate) with moderate sensitivity is available for identifying reports of trials more likely to be pragmatic.
OBJECTIVES: Identifying pragmatic trials from among all randomized trials is challenging because of inconsistent reporting. Our objective was to develop and validate a search filter to identify reports of pragmatic trials from Ovid MEDLINE. STUDY DESIGN AND SETTING: Two sets of known and probable pragmatic trial records were analyzed using text mining to generate candidate terms. Two large population sets comprising clinical trials and explanatory trials were used to select discriminating terms. Various combinations of terms were tested iteratively to achieve optimal search performance. Two externally derived sets were used to validate sensitivity and specificity of the derived filters. RESULTS: Our validated sensitivity-maximizing filter (combines trial design terms with terms relating to attributes of pragmatic trials) retrieves over 42,000 records in MEDLINE and has sensitivity of 46.4% (95% confidence interval (CI) 37.2 to 55.7%) and estimated specificity of 98.1% (95% CI 93.4 to 99.8%). Search performance is superior to other ad hoc filters for pragmatic trials. The Cochrane search for randomized trials has much better sensitivity (98.2%), but poorer specificity (1.9%) and retrieves 4.5 million records. CONCLUSION: A highly specific filter (low false positive rate) with moderate sensitivity is available for identifying reports of trials more likely to be pragmatic.
Authors: Monica Taljaard; Fan Li; Bo Qin; Caroline Cui; Leyi Zhang; Stuart G Nicholls; Kelly Carroll; Susan L Mitchell Journal: Clin Trials Date: 2021-11-29 Impact factor: 2.486
Authors: Stuart G Nicholls; Steve McDonald; Joanne E McKenzie; Kelly Carroll; Monica Taljaard Journal: J Clin Epidemiol Date: 2022-01-23 Impact factor: 7.407
Authors: Jennifer Zhe Zhang; Stuart G Nicholls; Kelly Carroll; Hayden Peter Nix; Cory E Goldstein; Spencer Phillips Hey; Jamie C Brehaut; Paul C McLean; Charles Weijer; Dean A Fergusson; Monica Taljaard Journal: J Med Ethics Date: 2021-11-15 Impact factor: 5.926
Authors: Pascale Nevins; Shelley Vanderhout; Kelly Carroll; Stuart G Nicholls; Seana N Semchishen; Jamie C Brehaut; Dean A Fergusson; Bruno Giraudeau; Monica Taljaard Journal: J Clin Epidemiol Date: 2021-12-08 Impact factor: 7.407
Authors: Stuart G Nicholls; Kelly Carroll; Spencer Phillips Hey; Merrick Zwarenstein; Jennifer Zhe Zhang; Hayden P Nix; Jamie C Brehaut; Joanne E McKenzie; Steve McDonald; Charles Weijer; Dean A Fergusson; Monica Taljaard Journal: J Clin Epidemiol Date: 2021-03-28 Impact factor: 6.437
Authors: Stuart G Nicholls; Kelly Carroll; Hayden P Nix; Fan Li; Spencer Phillips Hey; Susan L Mitchell; Charles Weijer; Monica Taljaard Journal: Alzheimers Dement (N Y) Date: 2022-05-02