Susanne Brogaard Krogh1, Tue Secher Jensen2,3,4, Nanna Rolving5, Malene Laursen6, Janus Nikolaj Laust Thomsen7, Casper Brink Hansen3, Christoffer Høj Werenberg3, Erik Rasmussen3, Rune Carlson3, Rikke Krüger Jensen3,4. 1. Department of Diagnostic Imaging, Silkeborg Regional Hospital, Silkeborg, Denmark. susako@rm.dk. 2. Department of Diagnostic Imaging, Silkeborg Regional Hospital, Silkeborg, Denmark. 3. Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark. 4. Chiropractic Knowledge Hub, Odense, Denmark. 5. DEFACTUM, Central Denmark Region, Aarhus, Denmark. 6. Research Unit, Centre of Elective Surgery, Regional Hospital of Silkeborg, Silkeborg, Denmark. 7. Center for General Practice, Department of Clinical Medicine, Aalborg University, Aalborg, Denmark.
Abstract
BACKGROUND: Managing low back pain (LBP) often involves MRI despite the fact that international guidelines do not recommend routine imaging. To allow us to explore the topic and use this knowledge in further research, a reliable method to review the MRI referrals is needed. Consequently, this study aimed to assess the inter-rater reliability of a method evaluating lumbar spine MRI referrals' appropriateness. METHODS: Four inexperienced students (chiropractic master's students) and a senior clinician (chiropractor) were included as independent raters in this inter-rater reliability study. Lumbar spine MRI referrals from primary care on patients (> 18 years) with LBP with or without leg pain were included. The referrals were classified using a modified version of the American College of Radiology (ACR) imaging appropriateness criteria for LBP. Categories of appropriate referrals included; fractures, cancer, previous surgery, candidate for surgery or suspicion of cauda equina. Inappropriate referrals included lacking information on previous non-surgical treatment, no word on non-surgical treatment duration, or "other reasons" for inappropriate referrals. After two rounds of training and consensus sessions, 50 lumbar spine MRI referrals were reviewed independently by the five raters. Inter-rater reliability was quantified using unweighted Kappa statistics, and the observed agreement was calculated with both a pairwise comparison and an overall five-rater comparison. RESULTS: Inter-rater reliability was substantial, with a Kappa value for appropriate vs. inappropriate referrals of 0.76 (95% CI: 0.55-0.89). When six and eight subcategories were evaluated, the Kappa values were 0.77 (95% CI: 0.58-0.91) and 0.82 (95% CI: 0.72-0.92), respectively. The overall percentage of agreement for appropriate and inappropriate referrals was 92% and ranged from 88 to 98% for the pairwise comparisons of the five raters' results. For the six and eight subcategories, the overall agreement was 92 and 88%, respectively, ranging from 88 to 98% and 84-92%, respectively, for the pairwise comparisons. CONCLUSION: The inter-rater reliability of the evaluation of the appropriateness of lumbar spine MRI referrals, according to the modified ACR-appropriateness criteria, was found to range from substantial to almost perfect and can be used for research and quality assurance purposes.
BACKGROUND: Managing low back pain (LBP) often involves MRI despite the fact that international guidelines do not recommend routine imaging. To allow us to explore the topic and use this knowledge in further research, a reliable method to review the MRI referrals is needed. Consequently, this study aimed to assess the inter-rater reliability of a method evaluating lumbar spine MRI referrals' appropriateness. METHODS: Four inexperienced students (chiropractic master's students) and a senior clinician (chiropractor) were included as independent raters in this inter-rater reliability study. Lumbar spine MRI referrals from primary care on patients (> 18 years) with LBP with or without leg pain were included. The referrals were classified using a modified version of the American College of Radiology (ACR) imaging appropriateness criteria for LBP. Categories of appropriate referrals included; fractures, cancer, previous surgery, candidate for surgery or suspicion of cauda equina. Inappropriate referrals included lacking information on previous non-surgical treatment, no word on non-surgical treatment duration, or "other reasons" for inappropriate referrals. After two rounds of training and consensus sessions, 50 lumbar spine MRI referrals were reviewed independently by the five raters. Inter-rater reliability was quantified using unweighted Kappa statistics, and the observed agreement was calculated with both a pairwise comparison and an overall five-rater comparison. RESULTS: Inter-rater reliability was substantial, with a Kappa value for appropriate vs. inappropriate referrals of 0.76 (95% CI: 0.55-0.89). When six and eight subcategories were evaluated, the Kappa values were 0.77 (95% CI: 0.58-0.91) and 0.82 (95% CI: 0.72-0.92), respectively. The overall percentage of agreement for appropriate and inappropriate referrals was 92% and ranged from 88 to 98% for the pairwise comparisons of the five raters' results. For the six and eight subcategories, the overall agreement was 92 and 88%, respectively, ranging from 88 to 98% and 84-92%, respectively, for the pairwise comparisons. CONCLUSION: The inter-rater reliability of the evaluation of the appropriateness of lumbar spine MRI referrals, according to the modified ACR-appropriateness criteria, was found to range from substantial to almost perfect and can be used for research and quality assurance purposes.
Authors: Hazel J Jenkins; Aron S Downie; Chris G Maher; Niamh A Moloney; John S Magnussen; Mark J Hancock Journal: Spine J Date: 2018-05-03 Impact factor: 4.166
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Steven J Kamper; Gabrielle Logan; Bethan Copsey; Jacqueline Thompson; Gustavo C Machado; Christina Abdel-Shaheed; Christopher M Williams; Christopher G Maher; Amanda M Hall Journal: Pain Date: 2020-04 Impact factor: 6.961
Authors: Peter Kent; Andrew M Briggs; Hanne B Albert; Andreas Byrhagen; Christian Hansen; Karina Kjaergaard; Tue S Jensen Journal: Chiropr Man Therap Date: 2011-07-21
Authors: Kevin Yuqi Wang; Christopher James Yen; Melissa Chen; Darshan Variyam; Tomas Uribe Acosta; Brian Reed; Max Wintermark; Christie Mary Lincoln Journal: J Am Coll Radiol Date: 2017-09-29 Impact factor: 5.532
Authors: Aron Downie; Mark Hancock; Hazel Jenkins; Rachelle Buchbinder; Ian Harris; Martin Underwood; Stacy Goergen; Chris G Maher Journal: Br J Sports Med Date: 2019-02-13 Impact factor: 13.800