Kirsty Loudon1, Merrick Zwarenstein2, Frank M Sullivan3, Peter T Donnan4, Ildikó Gágyor5, Hans J S M Hobbelen6, Fernando Althabe7, Jerry A Krishnan8, Shaun Treweek9. 1. Nursing Midwifery and Allied Health Professions Research Unit, Unit 13 Scion House, University of Stirling Innovation Park, Stirling FK9 4NF, UK. Electronic address: kirsty.loudon@stir.ac.uk. 2. Centre for Studies in Family Medicine, Department of Family Medicine, Schulich School of Medicine & Dentistry, Western University, Western Centre for Public Health and Family Medicine, 1151 Richmond St, London, Ontario N6A 4K7, Canada. 3. School of Medicine, Medical & Biological Sciences, North Haugh, St Andrews KY16 9ST, UK; North York General Hospital, 4001 Leslie Street, Toronto, Ontario M2K 1E1, Canada. 4. Division of Population Health Sciences, School of Medicine, University of Dundee, The Mackenzie Building, Kirsty Semple Way, Dundee DD2 4BF, UK. 5. Department of General Practice, University Medical Center Göttingen, Humboldtallee 38, D-37073 Göttingen, Germany. 6. Ageing and Healthcare Research Group, Healthy Ageing, Allied Health Care and Nursing, Centre of Expertise Healthy Ageing, Hanze University of Applied Sciences Groningen, Eyssoniusplein 18, Kamer A0.17, Postbus 3109 9701 DC Groningen, The Netherlands. 7. Departamento de Investigación en Salud de la Madre y el Niño, Instituto de Efectividad Clínica y Sanitaria (IECS), Dr Emilio Ravignani 2024 (C1414CPV), Buenos Aires, Argentina. 8. Population Health Sciences, Office of the Vice Chancellor for Health Affairs, Medicine and Public Health, University of Illinois at Chicago, 1200 West Harrison St., Chicago, IL 60607, USA. 9. Health Services Research Unit, University of Aberdeen, 3rd Floor, Health Sciences Building, Foresterhill, Aberdeen AB25 2ZD, UK.
Abstract
OBJECTIVES: PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. RESULTS: Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. CONCLUSION: We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders.
OBJECTIVES: PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. STUDY DESIGN AND SETTING: Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. RESULTS: Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. CONCLUSION: We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders.
Authors: Matthieu Legrand; Sean M Bagshaw; Jay L Koyner; Ivonne H Schulman; Michael R Mathis; Juliane Bernholz; Steven Coca; Martin Gallagher; Stéphane Gaudry; Kathleen D Liu; Ravindra L Mehta; Romain Pirracchio; Abigail Ryan; Dominik Steubl; Norman Stockbridge; Fredrik Erlandsson; Alparslan Turan; F Perry Wilson; Alexander Zarbock; Michael P Bokoch; Jonathan D Casey; Patrick Rossignol; Michael O Harhay Journal: J Am Soc Nephrol Date: 2022-07-13 Impact factor: 14.978