Jeanne A Teresi1,2, Xiaoying Yu3, Anita L Stewart4, Ron D Hays5. 1. Columbia University Stroud Center at New York State Psychiatric Institute, New York. 2. Research Division, Hebrew Home at Riverdale, Riverdale, NY. 3. Department of Preventive Medicine and Population Health, University of Texas Medical Branch at Galveston, Galveston, TX. 4. Department of Social and Behavioral Sciences, University of California, San Francisco, Institute for Health & Aging, San Francisco, CA. 5. Division of General Internal Medicine and Health Services Research, University of California, Los Angeles, Los Angeles, CA.
Abstract
BACKGROUND: Pilot studies test the feasibility of methods and procedures to be used in larger-scale studies. Although numerous articles describe guidelines for the conduct of pilot studies, few have included specific feasibility indicators or strategies for evaluating multiple aspects of feasibility. In addition, using pilot studies to estimate effect sizes to plan sample sizes for subsequent randomized controlled trials has been challenged; however, there has been little consensus on alternative strategies. METHODS: In Section 1, specific indicators (recruitment, retention, intervention fidelity, acceptability, adherence, and engagement) are presented for feasibility assessment of data collection methods and intervention implementation. Section 1 also highlights the importance of examining feasibility when adapting an intervention tested in mainstream populations to a new more diverse group. In Section 2, statistical and design issues are presented, including sample sizes for pilot studies, estimates of minimally important differences, design effects, confidence intervals (CI) and nonparametric statistics. An in-depth treatment of the limits of effect size estimation as well as process variables is presented. Tables showing CI around parameters are provided. With small samples, effect size, completion and adherence rate estimates will have large CI. CONCLUSION: This commentary offers examples of indicators for evaluating feasibility, and of the limits of effect size estimation in pilot studies. As demonstrated, most pilot studies should not be used to estimate effect sizes, provide power calculations for statistical tests or perform exploratory analyses of efficacy. It is hoped that these guidelines will be useful to those planning pilot/feasibility studies before a larger-scale study.
BACKGROUND: Pilot studies test the feasibility of methods and procedures to be used in larger-scale studies. Although numerous articles describe guidelines for the conduct of pilot studies, few have included specific feasibility indicators or strategies for evaluating multiple aspects of feasibility. In addition, using pilot studies to estimate effect sizes to plan sample sizes for subsequent randomized controlled trials has been challenged; however, there has been little consensus on alternative strategies. METHODS: In Section 1, specific indicators (recruitment, retention, intervention fidelity, acceptability, adherence, and engagement) are presented for feasibility assessment of data collection methods and intervention implementation. Section 1 also highlights the importance of examining feasibility when adapting an intervention tested in mainstream populations to a new more diverse group. In Section 2, statistical and design issues are presented, including sample sizes for pilot studies, estimates of minimally important differences, design effects, confidence intervals (CI) and nonparametric statistics. An in-depth treatment of the limits of effect size estimation as well as process variables is presented. Tables showing CI around parameters are provided. With small samples, effect size, completion and adherence rate estimates will have large CI. CONCLUSION: This commentary offers examples of indicators for evaluating feasibility, and of the limits of effect size estimation in pilot studies. As demonstrated, most pilot studies should not be used to estimate effect sizes, provide power calculations for statistical tests or perform exploratory analyses of efficacy. It is hoped that these guidelines will be useful to those planning pilot/feasibility studies before a larger-scale study.
Authors: Anna M Nápoles; Jasmine Santoyo-Olsson; Carmen Ortiz; Steven Gregorich; Howard E Lee; Ysabel Duron; Kristi Graves; Judith A Luce; Peggy McGuire; Marynieves Díaz-Méndez; Anita L Stewart Journal: Clin Trials Date: 2014-02-26 Impact factor: 2.486
Authors: Usha Kaul; Clara Scher; Charles R Henderson; Patricia Kim; Mette Dyhrberg; Vanessa Rudin; Millie Lytle; Nicole Bundy; M Carrington Reid Journal: Front Pain Res (Lausanne) Date: 2022-07-25
Authors: Armando Armas-Salazar; Noe Téllez-León; Ana Isabel García-Jerónimo; Francisco Alberto Villegas-López; José Luis Navarro-Olvera; José Damián Carrillo-Ruiz Journal: Pain Res Manag Date: 2022-08-02 Impact factor: 2.667