Susan Armijo-Olivo1, Jorge Fuentes, Maria Ospina, Humam Saltaji, Lisa Hartling. 1. Postdoctoral Fellow, CLEAR (Connecting Leadership and Research) Outcomes Research Program, University of Alberta, 5-115A Edmonton Clinic Health Academy (ECHA), 11405 - 87 Avenue, Edmonton, Alberta T6G 1C9 Canada. susanarmijo@gmail.com.
Abstract
BACKGROUND: Assessing the risk of bias of randomized controlled trials (RCTs) is crucial to understand how biases affect treatment effect estimates. A number of tools have been developed to evaluate risk of bias of RCTs; however, it is unknown how these tools compare to each other in the items included. The main objective of this study was to describe which individual items are included in RCT quality tools used in general health and physical therapy (PT) research, and how these items compare to those of the Cochrane Risk of Bias (RoB) tool. METHODS: We used comprehensive literature searches and a systematic approach to identify tools that evaluated the methodological quality or risk of bias of RCTs in general health and PT research. We extracted individual items from all quality tools. We calculated the frequency of quality items used across tools and compared them to those in the RoB tool. Comparisons were made between general health and PT quality tools using Chi-squared tests. RESULTS: In addition to the RoB tool, 26 quality tools were identified, with 19 being used in general health and seven in PT research. The total number of quality items included in general health research tools was 130, compared with 48 items across PT tools and seven items in the RoB tool. The most frequently included items in general health research tools (14/19, 74%) were inclusion and exclusion criteria, and appropriate statistical analysis. In contrast, the most frequent items included in PT tools (86%, 6/7) were: baseline comparability, blinding of investigator/assessor, and use of intention-to-treat analysis. Key items of the RoB tool (sequence generation and allocation concealment) were included in 71% (5/7) of PT tools, and 63% (12/19) and 37% (7/19) of general health research tools, respectively. CONCLUSIONS: There is extensive item variation across tools that evaluate the risk of bias of RCTs in health research. Results call for an in-depth analysis of items that should be used to assess risk of bias of RCTs. Further empirical evidence on the use of individual items and the psychometric properties of risk of bias tools is needed.
BACKGROUND: Assessing the risk of bias of randomized controlled trials (RCTs) is crucial to understand how biases affect treatment effect estimates. A number of tools have been developed to evaluate risk of bias of RCTs; however, it is unknown how these tools compare to each other in the items included. The main objective of this study was to describe which individual items are included in RCT quality tools used in general health and physical therapy (PT) research, and how these items compare to those of the Cochrane Risk of Bias (RoB) tool. METHODS: We used comprehensive literature searches and a systematic approach to identify tools that evaluated the methodological quality or risk of bias of RCTs in general health and PT research. We extracted individual items from all quality tools. We calculated the frequency of quality items used across tools and compared them to those in the RoB tool. Comparisons were made between general health and PT quality tools using Chi-squared tests. RESULTS: In addition to the RoB tool, 26 quality tools were identified, with 19 being used in general health and seven in PT research. The total number of quality items included in general health research tools was 130, compared with 48 items across PT tools and seven items in the RoB tool. The most frequently included items in general health research tools (14/19, 74%) were inclusion and exclusion criteria, and appropriate statistical analysis. In contrast, the most frequent items included in PT tools (86%, 6/7) were: baseline comparability, blinding of investigator/assessor, and use of intention-to-treat analysis. Key items of the RoB tool (sequence generation and allocation concealment) were included in 71% (5/7) of PT tools, and 63% (12/19) and 37% (7/19) of general health research tools, respectively. CONCLUSIONS: There is extensive item variation across tools that evaluate the risk of bias of RCTs in health research. Results call for an in-depth analysis of items that should be used to assess risk of bias of RCTs. Further empirical evidence on the use of individual items and the psychometric properties of risk of bias tools is needed.
Authors: L Arrivé; R Renard; F Carrat; A Belkacem; H Dahan; P Le Hir; L Monnier-Cholley; J M Tubiana Journal: Radiology Date: 2000-10 Impact factor: 11.105
Authors: Anna Santos Salas; Jorge Fuentes Contreras; Susan Armijo-Olivo; Humam Saltaji; Sharon Watanabe; Thane Chambers; Lori Walter; Greta G Cummings Journal: Support Care Cancer Date: 2015-11-10 Impact factor: 3.603
Authors: Susan Armijo-Olivo; Bruno R da Costa; Greta G Cummings; Christine Ha; Jorge Fuentes; Humam Saltaji; Matthias Egger Journal: PLoS One Date: 2015-07-10 Impact factor: 3.240
Authors: Humam Saltaji; Susan Armijo-Olivo; Greta G Cummings; Maryam Amin; Bruno R da Costa; Carlos Flores-Mir Journal: BMC Med Res Methodol Date: 2018-05-18 Impact factor: 4.615