Josep-Maria Losilla1, Isabel Oliveras1, Juan A Marin-Garcia2, Jaume Vives3. 1. Department of Psychobiology and Methodology of Health Sciences, Psychology Faculty, Universitat Autònoma de Barcelona, Carrer de la Fortuna, Edifici B, Bellaterra, Barcelona 08193, Spain. 2. Department of Business Management, School of Industrial Engineering, Universitat Politècnica de València, Dept. Organización de Empresas, Edificio 7D, Camino de Vera s/n, Valencia 46022, Spain. 3. Department of Psychobiology and Methodology of Health Sciences, Psychology Faculty, Universitat Autònoma de Barcelona, Carrer de la Fortuna, Edifici B, Bellaterra, Barcelona 08193, Spain. Electronic address: Jaume.Vives@uab.cat.
Abstract
OBJECTIVES: The aim of this study was to assess the agreement and compare the performance of three different instruments in assessing risk of bias (RoB) of comparative cohort studies included in a health psychology meta-analysis. STUDY DESIGN AND SETTING: Three tools were applied to 28 primary studies included in the selected meta-analysis: the Newcastle-Ottawa Scale, quality of cohort studies (Q-Coh), and risk of bias in nonrandomized studies of interventions (ROBINS-I). RESULTS: Interrater agreement varied greatly from tool to tool. For overall RoB, 75% of the studies were rated as low RoB with the Newcastle-Ottawa Scale, 11% of the studies with Q-Coh, and no study was found to be at low RoB using ROBINS-I. No influence of quality ratings on the meta-analysis results was found for any of the tools. CONCLUSION: Assessing RoB using the three tools may lead to opposite conclusions, especially at low and high levels of RoB. Domain-based tools (Q-Coh and ROBINS-I) provide a more comprehensive framework for identifying potential sources of bias, which is essential to improving the quality of future research. Both further guidance on the application of RoB tools and improvements in the reporting of primary studies are necessary.
OBJECTIVES: The aim of this study was to assess the agreement and compare the performance of three different instruments in assessing risk of bias (RoB) of comparative cohort studies included in a health psychology meta-analysis. STUDY DESIGN AND SETTING: Three tools were applied to 28 primary studies included in the selected meta-analysis: the Newcastle-Ottawa Scale, quality of cohort studies (Q-Coh), and risk of bias in nonrandomized studies of interventions (ROBINS-I). RESULTS: Interrater agreement varied greatly from tool to tool. For overall RoB, 75% of the studies were rated as low RoB with the Newcastle-Ottawa Scale, 11% of the studies with Q-Coh, and no study was found to be at low RoB using ROBINS-I. No influence of quality ratings on the meta-analysis results was found for any of the tools. CONCLUSION: Assessing RoB using the three tools may lead to opposite conclusions, especially at low and high levels of RoB. Domain-based tools (Q-Coh and ROBINS-I) provide a more comprehensive framework for identifying potential sources of bias, which is essential to improving the quality of future research. Both further guidance on the application of RoB tools and improvements in the reporting of primary studies are necessary.
Authors: Katsiaryna Bykov; Mengdong He; Jessica M Franklin; Elizabeth M Garry; John D Seeger; Elisabetta Patorno Journal: Diabetes Obes Metab Date: 2019-05-29 Impact factor: 6.577
Authors: Maya M Jeyaraman; Rasheda Rabbani; Nameer Al-Yousif; Reid C Robson; Leslie Copstein; Jun Xia; Michelle Pollock; Samer Mansour; Mohammed T Ansari; Andrea C Tricco; Ahmed M Abou-Setta Journal: Syst Rev Date: 2020-01-13
Authors: Lidia Sanchez-Ruiz; Beatriz Blanco; Juan A Marin-Garcia; Elsa Diez-Busto Journal: Int J Environ Res Public Health Date: 2020-11-09 Impact factor: 3.390
Authors: Kyle Steenland; M K Schubauer-Berigan; R Vermeulen; R M Lunn; K Straif; S Zahm; P Stewart; W D Arroyave; S S Mehta; N Pearce Journal: Environ Health Perspect Date: 2020-09-14 Impact factor: 9.031
Authors: Paras Karmacharya; Ravi Shahukhal; Cynthia S Crowson; M Hassan Murad; John M Davis; Pragya Shrestha; Delamo Bekele; Kerry Wright; Rikesh Chakradhar; Maureen Dubreuil Journal: Rheumatol Ther Date: 2020-11-10