Mark A Kaizik1, Alessandra N Garcia2, Mark J Hancock3, Robert D Herbert4. 1. School of Medical Sciences, Faculty of Medicine, University of New South Wales, Sydney, Australia. Electronic address: m.kaizik@student.unsw.edu.au. 2. Duke University Division of Physical Therapy, Duke Clinical Research Institute, North Carolina, USA. 3. Faculty of Medicine and Health Sciences, Macquarie University, Sydney, Australia. 4. School of Medical Sciences, Faculty of Medicine, University of New South Wales, Sydney, Australia; Neuroscience Research Australia (NeuRA), Sydney, Australia.
Abstract
OBJECTIVES: To determine the reliability, internal consistency, measurement error, convergent validity, and floor and ceiling effects of three quality assessment tools commonly used to evaluate the quality of diagnostic test accuracy studies in physical therapy. A secondary aim was to describe the quality of a sample of diagnostic accuracy studies. STUDY DESIGN AND SETTING: 50 studies were randomly selected from a comprehensive database of physical therapy-relevant diagnostic accuracy studies. Two reviewers independently rated each study with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS), Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) and Diagnostic Accuracy Quality Scale (DAQS) tools in random sequence. RESULTS: Only 7% of QUADAS items, 14% of QUADAS-2 items, and 33% of DAQS items had at least moderate inter-rater reliability (kappa>0.40). Internal consistency and convergent validity measures were acceptable (>0.70) in 33% and 50% of cases respectively. Floor or ceiling effects were not present in any tool. The quality of studies was mixed: most avoided case-control sampling strategies and used the same reference standard on all subjects, but many failed to enroll a consecutive or random sample of subjects or provide confidence intervals about estimates of diagnostic accuracy. CONCLUSION: The QUADAS, QUADAS-2 and DAQS tools provide unreliable estimates of the quality of studies of diagnostic accuracy in physical therapy.
OBJECTIVES: To determine the reliability, internal consistency, measurement error, convergent validity, and floor and ceiling effects of three quality assessment tools commonly used to evaluate the quality of diagnostic test accuracy studies in physical therapy. A secondary aim was to describe the quality of a sample of diagnostic accuracy studies. STUDY DESIGN AND SETTING: 50 studies were randomly selected from a comprehensive database of physical therapy-relevant diagnostic accuracy studies. Two reviewers independently rated each study with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS), Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) and Diagnostic Accuracy Quality Scale (DAQS) tools in random sequence. RESULTS: Only 7% of QUADAS items, 14% of QUADAS-2 items, and 33% of DAQS items had at least moderate inter-rater reliability (kappa>0.40). Internal consistency and convergent validity measures were acceptable (>0.70) in 33% and 50% of cases respectively. Floor or ceiling effects were not present in any tool. The quality of studies was mixed: most avoided case-control sampling strategies and used the same reference standard on all subjects, but many failed to enroll a consecutive or random sample of subjects or provide confidence intervals about estimates of diagnostic accuracy. CONCLUSION: The QUADAS, QUADAS-2 and DAQS tools provide unreliable estimates of the quality of studies of diagnostic accuracy in physical therapy.
Authors: William Hollingworth; L Santiago Medina; Robert E Lenkinski; Dean K Shibata; Byron Bernal; David Zurakowski; Bryan Comstock; Jeffrey G Jarvik Journal: Acad Radiol Date: 2006-07 Impact factor: 3.173
Authors: Anthony G Schneiders; S John Sullivan; Paul A Hendrick; Benjamin D G M Hones; Andrew R McMaster; Bridget A Sugden; Celia Tomlinson Journal: J Orthop Sports Phys Ther Date: 2012-07-19 Impact factor: 4.751
Authors: Anne W S Rutjes; Johannes B Reitsma; Marcello Di Nisio; Nynke Smidt; Jeroen C van Rijn; Patrick M M Bossuyt Journal: CMAJ Date: 2006-02-14 Impact factor: 8.262
Authors: Penny Whiting; Anne W S Rutjes; Johannes B Reitsma; Patrick M M Bossuyt; Jos Kleijnen Journal: BMC Med Res Methodol Date: 2003-11-10 Impact factor: 4.615