Claudia Fischer1,2, Hester F Lingsma3, Helen A Anema4, Job Kievit5, Ewout W Steyerberg3, Niek Klazinga4. 1. Department of Public Health, Centre for Medical Decision Making, Erasmus MC, Dr. Molewaterplein 50, NA-2217, 3000 CA, Rotterdam, The Netherlands. c.fischer@erasmusmc.nl. 2. Department of Public Health, Amsterdam Medical Centre, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. c.fischer@erasmusmc.nl. 3. Department of Public Health, Centre for Medical Decision Making, Erasmus MC, Dr. Molewaterplein 50, NA-2217, 3000 CA, Rotterdam, The Netherlands. 4. Department of Public Health, Amsterdam Medical Centre, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. 5. Department of Medical Decision Making, Leiden University Medical Centre, LUMC-J10-S, PO Box 9600, 2300 RC, Leiden, The Netherlands.
Abstract
BACKGROUND: Quality indicators are increasingly used to measure the quality of care and compare quality across hospitals. In the Netherlands over the past few years numerous hospital quality indicators have been developed and reported. Dutch indicators are mainly based on expert consensus and face validity and little is known about their construct validity. Therefore, we aim to study the construct validity of a set of national hospital quality indicators for hip replacements. METHODS: We used the scores of 100 Dutch hospitals on national hospital quality indicators looking at care delivered over a two year period. We assessed construct validity by relating structure, process and outcome indicators using chi-square statistics, bootstrapped Spearman correlations, and independent sample t-tests. We studied indicators that are expected to associate as they measure the same clinical construct. RESULT: Among the 28 hypothesized correlations, three associations were significant in the direction hypothesized. Hospitals with low scores on wound infections had high scores on scheduling postoperative appointments (p-value = 0.001) and high scores on not transfusing homologous blood (correlation coefficient = -0.28; p-value = 0.05). Hospitals with high scores on scheduling complication meetings, also had high scores on providing thrombosis prophylaxis (correlation coefficient = 0.21; p-value = 0.04). CONCLUSION: Despite the face validity of hospital quality indicators for hip replacement, construct validity seems to be limited. Although the individual indicators might be valid and actionable, drawing overall conclusions based on the whole indicator set should be done carefully, as construct validity could not be established. The factors that may explain the lack of construct validity are poor data quality, no adjustment for case-mix and statistical uncertainty.
BACKGROUND: Quality indicators are increasingly used to measure the quality of care and compare quality across hospitals. In the Netherlands over the past few years numerous hospital quality indicators have been developed and reported. Dutch indicators are mainly based on expert consensus and face validity and little is known about their construct validity. Therefore, we aim to study the construct validity of a set of national hospital quality indicators for hip replacements. METHODS: We used the scores of 100 Dutch hospitals on national hospital quality indicators looking at care delivered over a two year period. We assessed construct validity by relating structure, process and outcome indicators using chi-square statistics, bootstrapped Spearman correlations, and independent sample t-tests. We studied indicators that are expected to associate as they measure the same clinical construct. RESULT: Among the 28 hypothesized correlations, three associations were significant in the direction hypothesized. Hospitals with low scores on wound infections had high scores on scheduling postoperative appointments (p-value = 0.001) and high scores on not transfusing homologous blood (correlation coefficient = -0.28; p-value = 0.05). Hospitals with high scores on scheduling complication meetings, also had high scores on providing thrombosis prophylaxis (correlation coefficient = 0.21; p-value = 0.04). CONCLUSION: Despite the face validity of hospital quality indicators for hip replacement, construct validity seems to be limited. Although the individual indicators might be valid and actionable, drawing overall conclusions based on the whole indicator set should be done carefully, as construct validity could not be established. The factors that may explain the lack of construct validity are poor data quality, no adjustment for case-mix and statistical uncertainty.
Entities:
Keywords:
Database; Health care quality; Hip replacement; Quality indicators; Validity
Authors: James P Southwell-Keely; Robert R Russo; Lyn March; Robert Cumming; Ian Cameron; Alan J M Brnabic Journal: Clin Orthop Relat Res Date: 2004-02 Impact factor: 4.176
Authors: Elizabeth H Bradley; Jeph Herrin; Brian Elbel; Robert L McNamara; David J Magid; Brahmajee K Nallamothu; Yongfei Wang; Sharon-Lise T Normand; John A Spertus; Harlan M Krumholz Journal: JAMA Date: 2006-07-05 Impact factor: 56.272
Authors: Margriet van der Heiden-van der Loo; Vincent K Y Ho; Ronald A M Damhuis; Sabine Siesling; Marian B E Menke; Petra H M Peeters; Emiel J Th Rutgers Journal: Ned Tijdschr Geneeskd Date: 2010
Authors: Lars B Engesaeter; Stein Atle Lie; Birgitte Espehaug; Ove Furnes; Stein Emil Vollset; Leif Ivar Havelin Journal: Acta Orthop Scand Date: 2003-12
Authors: Sheleika L Hervey; Harriett R Purves; Ulrich Guller; Alison P Toth; Thomas P Vail; Ricardo Pietrobon Journal: J Bone Joint Surg Am Date: 2003-09 Impact factor: 5.284
Authors: Jilske A Huijben; Eveline J A Wiegers; Nicolette F de Keizer; Andrew I R Maas; David Menon; Ari Ercole; Giuseppe Citerio; Fiona Lecky; Lindsay Wilson; Maryse C Cnossen; Suzanne Polinder; Ewout W Steyerberg; Mathieu van der Jagt; Hester F Lingsma Journal: Crit Care Date: 2019-03-22 Impact factor: 9.097