Association between quality management and performance indicators in Dutch diabetes care groups: a cross-sectional study.

OBJECTIVES: To enhance the quality of diabetes care in the Netherlands, so-called care groups with three to 250 general practitioners emerged to organise and coordinate diabetes care. This introduced a new quality management level in addition to the quality management of separate general practices. We hypothesised that this new level of quality management might be associated with the aggregate performance indicators on the patient level. Therefore, we aimed to explore the association between quality management at the care group level and its aggregate performance indicators.
DESIGN: A cross-sectional study.
SETTING: All Dutch care groups (n=97). PARTICIPANTS: 23 care groups provided aggregate register-based performance indicators of all their practices as well as data on quality management measured with a questionnaire filled out by 1 or 2 of their quality managers. PRIMARY OUTCOMES: The association between quality management, overall and in 6 domains ('organisation of care', 'multidisciplinary teamwork', 'patient centredness', 'performance management', 'quality improvement policy' and 'management strategies') on the one hand and 3 process indicators (the percentages of patients with at least 1 measurement of glycated haemoglobin, lipid profile and systolic blood pressure), and 3 intermediate outcome indicators (the percentages of patients with glycated haemoglobin below 53 mmol/mol (7%); low-density lipoprotein cholesterol below 2.5 mmol/L; and systolic blood pressure below 140 mm Hg) by weighted univariable linear regression.
RESULTS: The domain 'management strategies' was significantly associated with the percentage of patients with a glycated haemoglobin <53 mmol/mol (β 0.28 (0.09; 0.46) p=0.01) after correction for multiple testing. The other domains as well as overall quality management were not associated with aggregate process or outcome indicators.
CONCLUSIONS: This first exploratory study on quality management showed weak or no associations between quality management of diabetes care groups and their performance. It remains uncertain whether this second layer on quality management adds to better quality of care. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

For the first time, quality management and performance indicators at an aggregate level were measured at about the same time, and put together.

Data represent 24% of all Dutch care groups, spread over the country, and accountable for the treatment of 189 000 diabetes patients.

Selection bias could not be ruled out as participating care groups with better quality management seemed more willing to participate in both studies.

The cross-sectional design does not allow the drawing of causal relationships.

For further research, confirmatory factor analyses of the questionnaire should be performed and criterion validity should be tested if comparable instruments become available.

Introduction

The delivery of optimal diabetes care is becoming increasingly complex.1 Fast-growing medical knowledge and increasing attention to quality of care has changed patient care into multiprofessional teamwork involving many healthcare providers.2 Consequently, optimal collaboration and coordination among these professionals and the patient has become essential for delivering high quality of care.1 To enhance the quality of care in the Netherlands, so-called care groups emerged to organise and coordinate multidisciplinary diabetes care.3 They are similar to Accountable Care Organizations in the USA3 4 and Clinical Commission Groups in the UK.5

In the Netherlands, with a diabetes prevalence of 5%, about 85% of people with type 2 diabetes are treated by general practitioners and practice nurses in general practices, almost all within the frame of a care group.6 In 2012, there were about 100 diabetes care groups, with anywhere from 3 to 250 general practitioners involved in each; the care groups treat between 400 and 22 500 diabetes patients.7 These care groups are the main contractor of a diabetes care programme, and are responsible for the organisation, coordination and delivery of diabetes care. Their diabetes care programme is based on the Dutch Diabetes Federation healthcare standard for type 2 diabetes, which describes good quality diabetes care and corresponding performance indicators.8 To get the quality of their contracted diabetes care checked, care groups are obliged by health insurance companies (that pay for the diabetes care) to provide aggregate performance indicators on an annual basis. Based on recent data from 48 care groups, glycated haemoglobin (HbA1c) <53 mmol/mol (7%) has been achieved in two of three patients, 52% of the patients have systolic blood pressure (SBP) <140 mm Hg and 53% of the type 2 diabetes patients have low-density lipoprotein (LDL) cholesterol <2.5 mmol/L.9 Care groups are continuously working on improving the registration of these indicators.9

Quality of care is at least partially the outcome of a controlled process: quality management. Quality management comprises of procedures to monitor, assess and improve the quality of care.10 Individual general practices have different levels of quality management. The introduction of care groups created a second level of quality management, in addition to the level of general practice. The level of quality management in care groups varies, for example, in the way they organise postgraduate courses, support self-management or handle electronic patient records and benchmarks.11 Using a benchmark, a care group can address poorly performing general practices. We could demonstrate that the level of quality management in care groups may be improved by providing feedback and a benchmark.12 In our opinion, focusing on quality management is only justified if better quality management leads to better patient outcomes. A meta-analysis of quality improvement strategies showed that interventions across the entire system of chronic disease management, such as team changes, case management, promotion of self-management, education of patients as well as clinicians, and electronic patient registry diabetes management, were associated with improvements on HbA1c, LDL cholesterol, and systolic and diastolic blood pressure in diabetes care.13 However, the studies in this review were targeted at practice level. On the other hand, a systematic review found that the structure of diabetes care, for example, the adequacy of facilities, equipment, logistics, or registration, were not related to (surrogate) patient outcomes.14 Hence, it remains unclear whether a good quality management system on care group level in addition to the quality management in individual general practices might result in better process and outcome measures for patients. Therefore, we aim to explore the association between quality management at the care group level and aggregate performance indicators of care groups.

Methods

Study population and design

This cross-sectional study is based on data of 23 care groups participating in two studies (figure 1):

Figure 1

Flowchart inclusion care groups.

Study I: measuring quality management at care group level (60 care groups) (see Level of quality management section)11 15

Study II: measuring aggregated performance indicators of care groups (66 care groups) (see Diabetes performance indicators section).9

Forty-six of the 66 care groups participating in study II gave consent for linking their scores on the performance indicators to their scores on quality management. However, only 23 of these care groups also participated in study I. The number of care groups participating in the current study represents 24% of all Dutch care groups, spread all over the country and accountable for the treatment of 189 000 diabetes patients.

No ethical approval was needed.

Measurements

Level of quality management

Between January and March 2012, those responsible for diabetes quality management of all care groups (n=97) were invited to fill out an online questionnaire measuring quality management in their care group (study I). Quality management was assessed in six domains: ‘organisation of care’, ‘multidisciplinary teamwork’, ‘patient centredness’, ‘performance management’, ‘quality improvement policy’ and ‘management strategies’ (score range 0–100%). Details of the questionnaire have been described elsewhere.11 15 The mean score of these six domains is the total level of quality management.11 15 Within the domains, 27 subdomains were addressed, each subdomain containing 1–6 questions. Each question was given equal weight; all questions within a subdomain contributed X% to the score of a domain, where X was the mean weight per subdomain given by an expert panel. The score of a subdomain could vary between 0% and 100%.

The whole questionnaire contained 59 questions; each question had a maximum score of 1 point.16 Since there were different types of questions, different types of scoring were used. Some questions had X subquestions; each subquestion could count for a maximum score of 1/X. The score was higher when the developmental stage on an item, for instance, benchmarking, was higher. For example, on the question “Is benchmarking being used in your care group to make improvements?”, organisations scored 0 point if they had no policy on benchmarking. If benchmarking was occasionally used, the score was 0.33 points; if the policy to structurally use benchmarking for quality improvement was still under development, they scored 0.66 points; and if benchmarking was structurally used for quality improvement, the score was 1 point. In questions in which we assessed the number of care providers involved in a particular item, each involved care provider scored 1/Y to the maximum score of 1 point. In the latter type of question, for example, a question such as “For which care providers has the care group organised postgraduate education in the past year?”, the maximum score could be reached when a defined number (Y) of care providers was involved. A copy of the questionnaire and the scoring of the questionnaire is included (see supplementary additional file).

The questionnaire's construct validity was based on the literature and a review of seven models for quality management, resulting in the six domains for diabetes quality management.17 Its face and content validity were warranted by involving experts in a pilot study, and by the weighing of the subdomains within a domain by an expert panel.11 15 Furthermore, the Cohen's κ of the questionnaires, which was tested by allowing two respondents of the same organisation to fill out the same questionnaire independently,16 seemed to be acceptable. Criterion-related validity could not be tested since there were no comparable instruments available.

Diabetes performance indicators

The Integrated Care Organisation, a national advisory group to support care groups, collected results on performance indicators concerning the year 2011 from care groups (Study II).9 These indicators were based on the Dutch diabetes care standard,8 and consisted of process as well as outcome indicators.

Process indicators show whether tests or assessments have been completed, such as the percentage of patients whose level of HbA1c has been measured in the preceding 12 months. The nominator per care group is the number of patients with a measurement in 2011; the denominator is the total number of patients participating in a particular care group.

Outcome indicators reflect the results of an assessment. The nominator is the number of patients with a measurement and achieving a specific target (such as the number of patients with an HbA1c below 53 mmol/mol (7%)); the denominator is the total number of patients with such a measurement in a particular care group. The process and indicators used by the Integrated Care Organisation are shown in table 1.

Table 1

Overview of the process and outcome measures in diabetes care groups

Parameter	Process measure	Outcome measure
HbA1c	Percentage of patients with HbA1c measured in 2011	Percentage of patients with HbA1c<53 mmol/mol (7%)
		Percentage of patients with HbA1c>69 mmol/mol (8.5%)
Cholesterol	Percentage of patients with lipid profile measured in 2011	Percentage of patients with LDL<2.5 mmol/L
Lipid-lowering drugs		Percentage of patients using lipid lowering drugs
Renal function	Percentage of patients with at least one creatinine measurement in 2011
	Percentage of patients with at least one urine albumin measurement in 2011
Blood pressure	Percentage of patients with at least one blood pressure measurement in 2011	Percentage of patients with SBP<140 mm Hg
BMI	Percentage of patients with BMI measured in 2011	Percentage of patients with BMI<25 (kg/m2)
Smoking status	Percentage of patients with their smoking behaviour recorded in 2011	Percentage of patients that smoked
		Percentage of patients that received smoking cessation advice
Eye examination	Percentage of patients with their eyes examined in 2011	Percentage of patients with retinopathy
Foot examination	Percentage of patients with their feet examined in 2011
Overall indicator	Percentage of patients fulfilling all aforementioned measurements

Text in italic represents measures used in this study.

BMI, body mass index; HbA1c, glycated haemoglobin; LDL, low-density lipoprotein; SBP, systolic blood pressure.

We used three process indicators, the percentages of patients with at least one recording of HbA1c, lipid profile and blood pressure, in 2011. Besides, we used three outcome indicators, the percentages of patients with: HbA1c <53 mmol/mol (7%); LDL cholesterol <2.5 mmol/L; and SBP <140 mm Hg.

Indicators not used in the analyses were performance indicators on smoking status, because of inappropriate registration mainly in non-smokers; indicators on eye and foot examination, because there was no information of patients treated by ophthalmologists and chiropodists, respectively; the measurements on kidney function because of different registration methods; and the percentage of people using lipid lowering drugs and both indicators with regard to body mass index because of incomplete registration. Owing to the problems with these indicators we also decided not to use the overall indicator.

Next to the individual care group information on performance indicators of the 23 care groups participating in both study I and II, we also received the total mean (SD) on the performance indicators of all care groups in study II (N=66) together.

Statistical analysis

Descriptive data analyses were used for the baseline results. One-way analysis of variance (ANOVA) was used to check the difference on total quality management score between the 23 participating care groups in this study and the 37 non-participating care groups from study II (figure 1). Further, we compared the difference on performance indicators between the 23 participating organisations and the total group of all participants from study II (N=66).

To explore the association between quality management level and performance indicators, first, the associations between the quality management domain scores were analysed. If the quality domains were associated with each other a multiple regression analysis would be preferred. Weighted univariable linear regression analyses were performed between, on the one hand, the total level of quality management of a care group and all six quality management domains, separately; and the aforementioned process and outcome indicators on the other. These linear regressions were weighted for the square root of the number of patients with diabetes treated in a care group.

Further, we checked whether care group size confounded these associations, by checking associations between care group size and the quality domains, or between care group size and the process and outcome indicators. Since each weighted linear regression was performed with seven separate variables, we considered a p value of 0.01 significant in this explorative study. For all other tests, a p value of 0.05 was considered significant. The assumptions for linear regression were checked. Analyses were performed using the SPSS 20.0 statistic software package.

Results

Table 2 shows the baseline characteristics of the participating care groups (N=23). Their mean overall level of quality management was 63.9% (CI 60.5% to 67.2%); the highest score regarded the domain of ‘organisation of care’ (72.6%, CI 68% to 77.1%). For the performance indicators, 92.3% (CI 89.8% to 94.8%) of the patients had at least one yearly recording of blood pressure and two of three had an HbA1c <53 mmol/mol (67.7%, CI 65.5% to 69.8%).

Table 2

Baseline characteristics of the care groups (N=23) (percentages)

Baseline characteristics	Mean (CI)
Number of patients in diabetes care programme	8586 (6025 to 11 148)
Overall level of QM	63.9 (60.5 to 67.2)
QM score in ‘organisation of care’	72.6 (68.0 to 77.1)
QM score in ‘multidisciplinary teamwork’	68.6 (61.3 to 76.0)
QM score in ‘patient centredness’	50.4 (43.5 to 57.4)
QM score in ‘performance management’	67.0 (63.8 to 70.3)
QM score in ‘quality improvement strategy’	59.1 (54.7 to 63.4)
QM score in ‘management strategies’	65.4 (60.4 to 70.5)
Patients with at least one measurement of HbA1c in 2011	91.0 (88.2 to 93.8)
Patients with at least one recording of lipid profile in 2011	86.4 (83.0 to 90.0)
Patients with at least one recording of blood pressure in 2011	92.3 (89.8 to 94.8)
Patients with HbA1c <53 mmol/mol	67.7 (65.5 to 69.8)
Patients with LDL cholesterol <2.5 mmol/L*	53.9 (51.5 to 56.3)
Patients with SBP <140 mm Hg	54.2 (52.5 to 55.9)

*In one care group there was no good extraction of LDL measurements.

HbA1c, glycated haemoglobin; LDL, low-density lipoprotein; QM, quality management; SBP, systolic blood pressure.

The 23 participating care groups had significantly better total quality management than the 37 care groups of which no data on performance indicators were available (57%; CI 53.7% to 60.2%; p=0.01). In study II, the mean percentage of patients from 66 care groups with at least one annual measurement of HbA1c was 91.7% (SD 6.7%) and the percentage of patients with an HbA1c <53 was 68.5% (SD 7.6%),9 whereas the 23 participating care groups in our study scored 92.3% (CI 89.8% to 94.8%) and 67.7% (CI 65.5% to 69.8%), respectively (table 2).

Since there were some significant univariable associations between the quality management domain scores, a multivariable linear regression would have been preferable but there were too few participating care groups to do so. In the univariable analysis, there were no significant associations between the level of quality management of care groups, and both its aggregate process and outcome indicators, with the exception of an association between the domain ‘management strategies’ and the number of patients achieving an HbA1c <53 mmol/mol (β 0.28 (0.09 to 0.46) p=0.01) (table 3). Care group size was associated with neither the total quality management score (r=−0.07; p=0.76) nor with the domains.

Table 3

Weighted univariable linear regression between overall quality management, quality management in the domains and performance indicators

Quality management	Process indicators (percentage of patients measured in 2011)
	HbA1c	LDL cholesterol	Blood pressure
Organisation of care	0.01 (−0.25 to 0.27) 0.94	0.13 (−0.19 to 0.46) 0.41	0.11 (−0.14 to 0.36) 0.38
Multidisciplinary teamwork	−0.10 (−0.26 to 0.07) 0.24	−0.12 (−0.32 to 0.09) 0.26	−0.12 (−0.28 to 0.04) 0.12
Patient centredness	−0.003 (−0.18 to 0.17) 0.97	0.03 (−0.19 to 0.25) 0.76	−0.01 (−0.18 to 0.17) 0.94
Performance management	0.13 (−0.32 to 0.57) 0.56	0.17 (−0.39 to 0.73) 0.54	0.15 (−0.29 to 0.58) 0.49
Quality improvement strategies	−0.04 (−0.37 to 0.28) 0.78	−0.09 (−0.50 to 0.33) 0.67	−0.03 (−0.36 to 0.29) 0.84
Management strategies	0.24 (−0.03 to 0.51) 0.08	0.25 (−0.10 to 0.60) 0.16	0.18 (−0.09 to 0.46) 0.18
Total mean	−0.01 (−0.40 to 0.39) 0.97	0.05 (−0.45 to 0.55) 0.84	−0.01 (−0.39 to 0.38) 0.97
	Outcome indicators (percentage of patients below target)
	HbA1c <53 mmol/mol (7%)	LDL cholesterol*< 2.5 mmol/L	SBP <140 mm Hg
Organisation of care	0.05 (−0.15 to 0.26) 0.58	0.02 (−0.17 to 0.22) 0.80	−0.02 (−0.17 to 0.13) 0.80
Multidisciplinary teamwork	0.01 (−0.12 to 0.14) 0.82	0.04 (−0.10 to 0.18) 0.53	−0.01 (−0.10 to 0.09) 0.89
Patient centredness	0.06 (−0.08 to 0.19) 0.39	0.08 (−0.05 to 0.21) 0.21	0.05 (−0.04 to 0.15) 0.28
Performance management	−0.01 (−0.35 to 0.34) 0.97	−0.08 (−0.42 to 0.25) 0.60	−0.04 (−0.30 to 0.21) 0.74
Quality improvement strategies	−0.04 (−0.30 to 0.21) 0.72	0.10 (−0.15 to 0.35) 0.41	−0.004 (−0.19 to 0.18) 0.97
Management strategies	0.28 (0.09 to 0.46) 0.01	−0.004 (−0.23 to 0.22) 0.97	−0.05 (−0.22 to 0.11) 0.52
Total mean	0.15 (−0.14 to 0.45) 0.29	0.12 (−0.18 to 0.42) 0.40	0.01 (−0.22 to 0.23) 0.93

The numbers in each cell represent, respectively, the β, (95% CI) and p value. β is the expected change in the performance indicators for 1-unit change in quality management. If, for example, the score on management strategies changes 1%, then the number of patients with an HbA1c<53 mmol/mol is expected to rise with 0.28% (CI 0.09 to 0.46) %.

*In one care group there was no good extraction of LDL measurements.

Bold typeface indicates significance at p<0.01.

HbA1c, glycated haemoglobin; LDL, low-density lipoprotein; SBP, systolic blood pressure.

Discussion

This is an explorative study on the association between quality management of care groups, in addition to the existing quality management in separate general practices, and their aggregate performance indicators. It showed a significant association between the management strategies of a care group and the percentage of patients with an HbA1c <53 mmol/mol. Other associations between quality management and performance indicators did not reach significance.

Management strategies comprised of three subdomains: (1) a structural policy (with questions on the use of the Plan-Do-Check-Act cycle), (2) the availability of quality systems (questioning whether a certified quality system was used on care group or practice level) and (3) the availability of quality documents (such as a mission document, a quality action plan or an annual quality report). The combination of these factors could create preconditions for better blood glucose control in the practices belonging to a care group. But care should be taken when drawing conclusions from the associations we found in this domain. It remains unclear why no associations between other quality management domains and lipid or blood pressure management were shown. However, a previous review found no association between adequacy of facilities, equipment, logistics, or registration and (surrogate) patient outcomes, either.14

There are several reasons that might explain the lack of significant associations between quality management and performance indicators on the level of care groups. First, we measured quality management with an instrument that was not yet completely validated, although it was developed carefully and we demonstrated its ability to measure change in quality management over a 1-year period.12 Second, individual general practices might already have such good quality management that the ‘second level’ of quality management of the care groups does not further improve their diabetes care. We had no information on the differences between individual general practices; neither on their quality management levels nor on the variance of their scores on performance indicators. Assuming that an active quality management policy at the care group level will focus on practices that perform less well, it is plausible that a higher level of quality management on care group level would result in less variance between practices and less ‘bad’ practices. However, to support this hypothesis, data from separate practices should be available, beyond the aggregate data we used in this study. If, in future study, these data on practice level become available, then a multilevel analysis would be appropriate with these multilevel data. Third, quality improvements are likely to be more effective if baseline levels of intermediate outcomes are poor. In the Netherlands, the results of diabetes care are good compared with those of other European countries.18 Close collaboration between general practitioners and practice nurses already led to significant improvements in diabetes care.19 Fourth, we measured a limited set of performance indicators, because of problems with the registration of other ones. The number of adequately registered performance indicators depends on both the registration by the healthcare providers, and on the performance of the software companies that provide the registration software and the aggregate data reports.7 Dutch diabetes care groups, which are obliged to provide annual benchmark reports, put much effort into optimising the data exchange between the practices and the care groups, but still experience difficulties.20 Owing to technical problems, performance indicators are often registered and/or extracted inappropriately.9 We cannot rule out the possibility of analysis of a larger set of indicators demonstrating significant associations with the quality management of care groups. Besides, patient satisfaction and patient safety, which are no part of the national benchmark, were not measured. The domain ‘patient centredness’ might be more likely associated with patient-reported outcome measures. Fifth, the number of participating care groups might have been too low to achieve significant results. Finally, we should not exclude the possibility that there is indeed no relationship between the level of quality management at care group level and aggregate performance indicators.

Our study could enhance the improvement of quality management. Participating organisations already used the questionnaire as a checklist to ascertain whether several quality management items were present. However, if, indeed, there would be no association between quality management in care groups and their performance outcomes, the Health Inspectorate and health insurance companies should not focus on the level of quality management in care groups. However, we cannot yet draw this conclusion. First, the quality management instrument needs further validation. Second, the variation in performance indicators between practices needs to be studied. Third, the registration of performance indicators still needs improvement, although the extraction of performance indicators becomes increasingly more reliable. Besides, differences in population (case-mix) will influence the performance indicators as well and, in presenting indicators, adjustment for case-mix is obligatory.21 Although one should realise that improved recording is not necessarily a valid indicator of good quality of care,22 the only way we can measure quality of care with performance indicators is basing it on registration data. Finally, to establish a good quality management system may take several years.

A strength of this study is that, for the first time, quality management and performance indicators at an aggregate level were measured and put together at about the same time. This gave us the unique opportunity to study the association between the quality management at care group level, above what happens at the general practice level, and performance indicators.

A limitation of this study is the availability of both datasets of only 23 care groups. Although we may assume that the care groups participating in this study were comparable with all 66 care groups in study II, with regard to their performance indicators, selection bias cannot be ruled out, as participating care groups with better quality management seemed more willing to participate, in both studies. Further limitations are the cross-sectional design and the validity of the questionnaire. A cross-sectional design is not suitable to detect a causal relationship between quality of diabetes management and quality indicators. Face and content validity were warranted, and construct validity was based on the literature and a review of seven quality management models. As stated above, we demonstrated its ability to measure change in quality management over a 1-year period.12 However, if the questionnaire were to be used for further research, confirmatory factor analyses should be performed and criterion validity tested if comparable instruments become available.

To conclude: we could not demonstrate an association between quality management of care groups and better outcomes in diabetes care.