Henrica C W de Vet1, Margriet G Mullender2, Iris Eekhout3. 1. Department of Epidemiology and Biostatistics, Amsterdam Public Health Research Institute, VU University Medical Center, De Boelelaan 1089, Amsterdam 1081 HV, The Netherlands. Electronic address: hcw.devet@vumc.nl. 2. Department of Plastic Surgery, VU University Medical Center, De Boelelaan 1117, Amsterdam 1089 HV, The Netherlands. 3. Department of Epidemiology and Biostatistics, Amsterdam Public Health Research Institute, VU University Medical Center, De Boelelaan 1089, Amsterdam 1081 HV, The Netherlands; Netherlands Organization for Applied Scientific Research (TNO), Schipholweg 77-89, Leiden 2316 ZL, The Netherlands.
Abstract
OBJECTIVE: The concept of specific agreement (SA) has been proposed for dichotomous outcomes for two or more raters. We aim to extend this concept for variables with more than two ordinal or nominal categories and more than two raters. STUDY DESIGN AND SETTING: We used two data sets: four plastic surgeons classifying photographs after breast reconstruction on a 5-point ordinal scale and six raters classifying psychiatric patients into five diagnostic categories. For m raters, all (i.e., m(m-1)/2) pairwise agreement tables were summed to calculate the observed agreement (OA), SA and conditional probabilities. The 95% confidence intervals were obtained by bootstrapping. RESULTS: SA was calculated for each ordinal or nominal category to examine when one of the raters scored in a specific category, what is the probability that the other raters scored in that same category. And suppose one of the raters scored X1, what is the probability that the other raters scored X1 or any of the other categories (conditional probability). It appeared, for example, that among the psychiatric disorders, depression and personality disorders were often mixed up, whereas neurosis was rarely mixed up with schizophrenia. CONCLUSION: The concept of SA for variables with ordinal and multiple nominal categories provides relevant clinical information. The extension to conditional probabilities of alternative categories broadens the clinical application with examining which categories are most often mixed up.
OBJECTIVE: The concept of specific agreement (SA) has been proposed for dichotomous outcomes for two or more raters. We aim to extend this concept for variables with more than two ordinal or nominal categories and more than two raters. STUDY DESIGN AND SETTING: We used two data sets: four plastic surgeons classifying photographs after breast reconstruction on a 5-point ordinal scale and six raters classifying psychiatricpatients into five diagnostic categories. For m raters, all (i.e., m(m-1)/2) pairwise agreement tables were summed to calculate the observed agreement (OA), SA and conditional probabilities. The 95% confidence intervals were obtained by bootstrapping. RESULTS:SA was calculated for each ordinal or nominal category to examine when one of the raters scored in a specific category, what is the probability that the other raters scored in that same category. And suppose one of the raters scored X1, what is the probability that the other raters scored X1 or any of the other categories (conditional probability). It appeared, for example, that among the psychiatric disorders, depression and personality disorders were often mixed up, whereas neurosis was rarely mixed up with schizophrenia. CONCLUSION: The concept of SA for variables with ordinal and multiple nominal categories provides relevant clinical information. The extension to conditional probabilities of alternative categories broadens the clinical application with examining which categories are most often mixed up.
Authors: Beatrijs Mertens; Julie Hias; Laura Hellemans; Karolien Walgraeve; Isabel Spriet; Jos Tournoy; Lorenz Roger Van der Linden Journal: Eur Geriatr Med Date: 2022-03-21 Impact factor: 1.710
Authors: Karen M Oude Hengel; Alex Burdorf; Anjoeka Pronk; Vivi Schlünssen; Zara A Stokholm; Henrik A Kolstad; Karin van Veldhoven; Ioannis Basinas; Martie van Tongeren; Susan Peters Journal: Scand J Work Environ Health Date: 2021-11-17 Impact factor: 5.024
Authors: E Nobels-Janssen; E N Postma; I L Abma; J M C van Dijk; R Haeren; H Schenck; W A Moojen; M H den Hertog; D Nanda; A R E Potgieser; B A Coert; W I M Verhagen; R H M A Bartels; P J van der Wees; D Verbaan; H D Boogaarts Journal: J Neurol Date: 2021-11-08 Impact factor: 6.682
Authors: Gerben J C Zwezerijnen; Jakoba J Eertink; Coreline N Burggraaff; Sanne E Wiegers; Ekhlas A I N Shaban; Simone Pieplenbosch; Daniela E Oprea-Lager; Pieternella J Lugtenburg; Otto S Hoekstra; Henrica C W de Vet; Josee M Zijlstra; Ronald Boellaard Journal: J Nucl Med Date: 2021-03-05 Impact factor: 11.082
Authors: Rogier F de Best; Michel W Coppieters; Emie van Trijffel; Annette Compter; Maarten Uyttenboogaart; Joost C Bot; Rene Castien; Jan J M Pool; Barbara Cagnie; Gwendolyne G M Scholten-Peeters Journal: Phys Ther Date: 2021-10-01