Hyun-Ju Seo1, Soo Young Kim2, Yoon Jae Lee3, Bo-Hyoung Jang4, Ji-Eun Park5, Seung-Soo Sheen6, Seo Kyung Hahn7. 1. Department of Nursing, College of Medicine, Chosun University, Gwangju, Republic of Korea. 2. Department of Family Medicine, College of Medicine, Hallym University, Seoul, Republic of Korea. Electronic address: hallymfm@gmail.com. 3. Department of Oriental Gynecology, CHA Bundang Medical Center, CHA University, Seongnam, Republic of Korea. 4. Department of Preventive Medicine, College of Korean Medicine, Kyung Hee University, Seoul, Republic of Korea. 5. Department of Health Care Management and Policy, School of Public Health, Seoul National University, Seoul, Republic of Korea. 6. Department of Pulmonology, Ajou University Hospital, Suwon, Republic of Korea. 7. College of Medicine, Seoul National University, Seoul, Republic of Korea.
Abstract
OBJECTIVE: To develop a study Design Algorithm for Medical Literature on Intervention (DAMI) and test its interrater reliability, construct validity, and ease of use. STUDY DESIGN AND SETTING: We developed and then revised the DAMI to include detailed instructions. To test the DAMI's reliability, we used a purposive sample of 134 primary, mainly nonrandomized studies. We then compared the study designs as classified by the original authors and through the DAMI. Unweighted kappa statistics were computed to test interrater reliability and construct validity based on the level of agreement between the original and DAMI classifications. Assessment time was also recorded to evaluate ease of use. RESULTS: The DAMI includes 13 study designs, including experimental and observational studies of interventions and exposure. Both the interrater reliability (unweighted kappa = 0.67; 95% CI [0.64-0.75]) and construct validity (unweighted kappa = 0.63, 95% CI [0.52-0.67]) were substantial. Mean classification time using the DAMI was 4.08 ± 2.44 minutes (range, 0.51-10.92). CONCLUSIONS: The DAMI showed substantial interrater reliability and construct validity. Furthermore, given its ease of use, it could be used to accurately classify medical literature for systematic reviews of interventions although minimizing disagreement between authors of such reviews.
OBJECTIVE: To develop a study Design Algorithm for Medical Literature on Intervention (DAMI) and test its interrater reliability, construct validity, and ease of use. STUDY DESIGN AND SETTING: We developed and then revised the DAMI to include detailed instructions. To test the DAMI's reliability, we used a purposive sample of 134 primary, mainly nonrandomized studies. We then compared the study designs as classified by the original authors and through the DAMI. Unweighted kappa statistics were computed to test interrater reliability and construct validity based on the level of agreement between the original and DAMI classifications. Assessment time was also recorded to evaluate ease of use. RESULTS: The DAMI includes 13 study designs, including experimental and observational studies of interventions and exposure. Both the interrater reliability (unweighted kappa = 0.67; 95% CI [0.64-0.75]) and construct validity (unweighted kappa = 0.63, 95% CI [0.52-0.67]) were substantial. Mean classification time using the DAMI was 4.08 ± 2.44 minutes (range, 0.51-10.92). CONCLUSIONS: The DAMI showed substantial interrater reliability and construct validity. Furthermore, given its ease of use, it could be used to accurately classify medical literature for systematic reviews of interventions although minimizing disagreement between authors of such reviews.
Authors: J W Song; S C Guiry; H Shou; S Wang; W R Witschey; S R Messé; S E Kasner; L A Loevner Journal: AJNR Am J Neuroradiol Date: 2019-11-14 Impact factor: 3.825
Authors: Eric Sven Kroeber; Lucas Adam; Adamu Addissie; Alexander Bauer; Thomas Frese; Eva Johanna Kantelhardt; Susanne Unverzagt Journal: BMJ Open Date: 2020-09-13 Impact factor: 2.692