Timothy C Guetterman1, John W Creswell, Marsha Wittink, Fran K Barg, Felipe G Castro, Britt Dahlberg, Daphne C Watkins, Charles Deutsch, Joseph J Gallo. 1. Dr. Guetterman: Department of Family Medicine, University of Michigan, Ann Arbor, Michigan. Dr. Creswell: Department of Family Medicine, University of Michigan, Ann Arbor, MI. Dr. Wittink: Department of Psychiatry and Department of Family Medicine, University of Rochester, Rochester, NY. Dr. Barg: Department of Family Medicine and Community Health, University of Pennsylvania, Philadelphia, PA. Dr. Castro: College of Nursing & Health Innovation, Center for Health Promotion and Disease Prevention, Arizona State University, Phoenix, AZ. Dr. Dahlberg: Institute for Research, Chemical Heritage Foundation, Philadelphia, PA. Dr. Watkins: School of Social Work, University of Michigan, Ann Arbor, MI. Dr. Deutsch: Harvard Catalyst Population Health Research Program, Harvard University, Boston, MA. Dr. Gallo: Department of Mental Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD.
Abstract
INTRODUCTION: Demand for training in mixed methods is high, with little research on faculty development or assessment in mixed methods. We describe the development of a self-rated mixed methods skills assessment and provide validity evidence. The instrument taps six research domains: "Research question," "Design/approach," "Sampling," "Data collection," "Analysis," and "Dissemination." Respondents are asked to rate their ability to define or explain concepts of mixed methods under each domain, their ability to apply the concepts to problems, and the extent to which they need to improve. METHODS: We administered the questionnaire to 145 faculty and students using an internet survey. We analyzed descriptive statistics and performance characteristics of the questionnaire using the Cronbach alpha to assess reliability and an analysis of variance that compared a mixed methods experience index with assessment scores to assess criterion relatedness. RESULTS: Internal consistency reliability was high for the total set of items (0.95) and adequate (≥0.71) for all but one subscale. Consistent with establishing criterion validity, respondents who had more professional experiences with mixed methods (eg, published a mixed methods article) rated themselves as more skilled, which was statistically significant across the research domains. DISCUSSION: This self-rated mixed methods assessment instrument may be a useful tool to assess skills in mixed methods for training programs. It can be applied widely at the graduate and faculty level. For the learner, assessment may lead to enhanced motivation to learn and training focused on self-identified needs. For faculty, the assessment may improve curriculum and course content planning.
INTRODUCTION: Demand for training in mixed methods is high, with little research on faculty development or assessment in mixed methods. We describe the development of a self-rated mixed methods skills assessment and provide validity evidence. The instrument taps six research domains: "Research question," "Design/approach," "Sampling," "Data collection," "Analysis," and "Dissemination." Respondents are asked to rate their ability to define or explain concepts of mixed methods under each domain, their ability to apply the concepts to problems, and the extent to which they need to improve. METHODS: We administered the questionnaire to 145 faculty and students using an internet survey. We analyzed descriptive statistics and performance characteristics of the questionnaire using the Cronbach alpha to assess reliability and an analysis of variance that compared a mixed methods experience index with assessment scores to assess criterion relatedness. RESULTS: Internal consistency reliability was high for the total set of items (0.95) and adequate (≥0.71) for all but one subscale. Consistent with establishing criterion validity, respondents who had more professional experiences with mixed methods (eg, published a mixed methods article) rated themselves as more skilled, which was statistically significant across the research domains. DISCUSSION: This self-rated mixed methods assessment instrument may be a useful tool to assess skills in mixed methods for training programs. It can be applied widely at the graduate and faculty level. For the learner, assessment may lead to enhanced motivation to learn and training focused on self-identified needs. For faculty, the assessment may improve curriculum and course content planning.
Authors: Maryellen E Gusic; Robert J Milner; Elizabeth J Tisdell; Edward W Taylor; David A Quillen; Luanne E Thorndyke Journal: Acad Med Date: 2010-09 Impact factor: 6.893
Authors: Russell E Glasgow; Cynthia Vinson; David Chambers; Muin J Khoury; Robert M Kaplan; Christine Hunter Journal: Am J Public Health Date: 2012-05-17 Impact factor: 9.308
Authors: Timothy C Guetterman; Rae V Sakakibara; Vicki L Plano Clark; Mark Luborsky; Sarah M Murray; Felipe González Castro; John W Creswell; Charles Deutsch; Joseph J Gallo Journal: PLoS One Date: 2019-11-15 Impact factor: 3.240