How to prioritise in quality improvement? Targeted implementation as a key for quality improvement in Kenyan health facilities.

BACKGROUND: Data from national surveys of low- and middle income countries indicates that there is still a need to improve the quality of healthcare in resource-poor settings. This study aims to understand the benefit of an integral, facility-driven, indicator-based approach used as a decision-making tool to define effective quality improvement interventions in Kenya.
OBJECTIVE: The aim of the study is to understand whether the integral approach developed leads to effective interventions.
METHODS: Categorical data is collected from ten health facilities covered by the Integrated Quality Management System (IQMS) project in Kenya. First the information on concrete improvement interventions implemented within the facilities is collected and merged into five different intervention topics. Second, groups of facilities with similar quality improvement interventions are selected to compare between the first and second quality assessment rounds. Those IQMS indicators matching the content of the intervention topic are extracted from the software VISOTOOL. In a third step, the data is summarised using means and SD. A one sample T-test is applied on the mean changes and SD. Frequency counts and percentages were used for the presentation of categorical data.
RESULTS: All improvement interventions resulted in positive and higher change values (T2-T1). Four of five intervention topics, show statistically significant improvements including neonatal mortality (42%; p<0.0001), waiting time (39%; p=0.0490), infection prevention control (28%; p=0.0007) and with shortages of staffing and transport in remote areas (32%; p=0.0194).
CONCLUSIONS: In all facilities the interventions selected have a positive impact, some of which markedly improved. It demonstrates that this integral quality improvement approach in Kenya can serve as an effective decision-making tool for identification and prioritisation of interventions. Those targeted interventions, being performed under institutionalisation in form of coaching and tutoring, effectively contribute to improving the quality of care in resource poor settings. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background

Quality of healthcare is internationally recognised as a crucial aspect in the attainment of universal health coverage and Health System Strengthening in the 2030 Sustainable Development Agenda.1 As resources at health facilities in low-income and middle-income countries (LMICs) are often scarce, facilities have to make decisions to use them efficiently.2 Despite the present efforts of improving quality of healthcare in LMICs, essential advancements causing an expected decline of mortality and morbidity rates have yet to be achieved.3 Continuous improvement efforts are required particularly in the area of utilisation of resources, service delivery and efficiency.4 5

Kenya is an LMIC that faces several challenges with its healthcare system.

The population grows continuously. In 2016, the population was 48.46 million with an average life expectancy of 67 years at birth.6 Non-communicable and chronic diseases such as diabetes and hypertension are a growing problem, while infectious diseases like malaria, HIV and tuberculosis still cause 41% of all deaths. Country data of 2017 indicate that the mortality rate of children under 5 years was 43 per 1000 and the estimated maternal mortality ratio of 342 per 100 000 live births is still high.6 Obstetric support was not accessible for the entire population: Only 62% of births are attended by skilled healthcare staff, 9% of all deaths in Kenya are caused by conditions during the perinatal period, 57.6% of pregnant women were following WHO recommendation to attend at least four antenatal care visits and the contraceptive prevalence was only 58%.7–9 These data go along with low quality of care in most Kenyan facilities, which is crucial and substantially causal for the presented health data. It indicates a need for improvement in quality of healthcare in Kenya in order to successfully deal with infectious as well as non-communicable diseases and accessibility of healthcare as simultaneous challenges.10 11

Several initiatives aimed at improving the quality of care in health facilities in Kenya.8 12 13 A notable effort for quality improvement in Kenya was the Kenya Quality Model, developed by the Ministry of Health with support from the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) in 2001, which was replaced by the Kenya Quality Model for Health (KQMH) in 2009.11 12 Entailing specific checklists and guidelines, KQMH was nationally introduced in the whole country to support health facilities to improve health services reinforcing the structures and processes to achieve better outcomes.

Despite the efforts, the Kenya Service Provision Assessment Survey revealed a persisting low quality of care, particularly in family planning, maternal and newborn care. Shortages of staff, equipment, medications and infrastructure were among the main reasons for poor service quality.14 The KQMH guidelines were often neither available nor used at facility level.15 However, due to a multitude of problems in quality improvement, it is impossible to tackle all at once and it is difficult to prioritise improvement measures and to use scarce resources efficiently.16

Aiming to improve maternal and child health in Kenya,11 17 develop and test a participatory approach that helps prioritise and target key quality problems, the Integrated Quality Management System (IQMS) project was introduced as an indicator-based quality improvement approach.17 18 The IQMS project was developed within a GIZ-funded project by evaplan ltd at the University of Heidelberg, the Institute for Applied Quality Improvement and Research in Health Care, and the Institute of Public Health Management and Research. The IQMS Kenya was designed following its ‘mother’ approach, the European Practice Assessment (EPA) in Germany and Switzerland, which was proven to be a functioning quality management programme.16

A panel of Kenyan experts representing different levels of the Kenyan health system analysed and validated 303 IQMS indicators using the modified Research And Development/University of California, Los Angeles method.17

The developed 303 IQMS quality indicators cover five areas of intervention called ‘domains’: ‘Clinical Care’, ‘People’, ‘Management’, ‘Interface In/Out-patients’ and ‘Quality and Safety’, which consist of several subareas—called ‘dimensions’.19

The assessment tools for the indicator-based quality measurement were a manager interview guide, facilitator assessment, self-assessment and surveys for patients and staff. Giving feedback and improvement planning were integral parts of the IQMS. The assessment was conducted by the facilities themselves (self-assessment) being endorsed and combined with a quality audit conducted by a trained facilitator and a research assistant.18 The specially developed software VISOTOOL allows for the results to be benchmarked against the average result of all participating facilities by visualising them in an easily understandable way, which stimulates discussion with facility staff.18 20 21 The feedback and gap analysis were based on the gaps identified and accompanied by continuous support to facilities, that included coaching through the facilitator, facility-driven tutoring, supervision, mentoring as well as on the job training. After identifying the gaps targeted improvement interventions were conducted by the facilities and a reassessment took place after 1.5 years. The IQMS quality improvement approach was successfully tested and validated during a pilot phase in ten selected facilities, which was described in detail elsewhere.22 Although the overall IQMS approach shows striking achieved improvements in all domain,18 the aspect of prioritisation and planning as well as conducting targeted interventions —as a key for quality improvement—has not been analysed yet.

Therefore, the aim of this study is to understand whether a precise and participatory measurement and detailed gap analysis can facilitate prioritisation and thus targeted interventions that lead to measurable improvements. Therefore, the outcomes of the different improvement interventions will be compared among the participating facilities. We aim to show that the IQMS quality improvement approach successfully enables facilities to reduce the problem of prioritisation and employ limited resources the most efficient way.

Methods

The data were collected from ten Kenyan IQMS facilities participating in the pilot IQMS study—comprising two health centres and two dispensaries of the Kenyan-level tier 2, four subcounty hospitals of the Kenyan-level tier 3 and two county hospitals of the Kenyan-level tier 4. All of them followed an announcement for applications. Out of 36 applying facilities, 10 facilities were sampled via convenience sampling as a pilot group to create a selection of facilities, being representative for an important part of the Kenyan healthcare system. They were from different Tier levels according to the percentage distribution into Tier levels and from different regions in order to represent the Kenyan health system. The 10 selected facilities represented a variety of Kenyan facilities of different levels, both from urban as well as from rural settings. Table 1 gives an overview of the ten pilot facilities. Five of these facilities are situated in rural areas while five of them are from urban areas.

Table 1

Baseline differences of the ten pilot facilities

No of pilot facility	Region urban/Rural (U/R)	Health system tiers	Ownership	Annual antenatal care attendance	No of beds (total)	Annual outpatient attendance
1	U	4	Public	9062	180	43 971
2	U	2	Public	–	20	–
3	R	3	Faith Based	–	100	–
4	U	3	Public	13 398	63	25 298
5	R	3	Public	635	37	10 154
6	U	3	Public	1460	56	100
7	R	2	Faith Based	–	13	–
8	R	2	Public	1242	12	6734
9	R	2	Public	–	30	–
10	U	4	Public	2132	162	58 277

Selection criteria were used to ensure the comparability between health facilities and to reduce structural variables; some examples of selection criteria were a specified minimum level medical equipment and infrastructure as well as previous experience in the field of quality management.18 22

The 303 indicators are allocated to the previously mentioned five domains ‘clinical care’, ‘people’, ‘management’, ‘interface inpatients/outpatients’ and ‘quality and safety’. The achievements of all five domains based on the 303 indicators were visualised by a specialised software VISOTOOL-Kenya (figure 1).22

Figure 1

Example of a visual.17

A first assessment (T1) and a second assessment after 1.5 years (T2) were conducted in 10 facilities between 2013 and 2016. In between of T1 and T2, the facilities themselves supported by the facilitator used the given analysis and feedback of assessment T1 results for decision-making on what intervention should be given priority and should be implemented depending on identified gaps at the facility. Therefore, the specially developed software VISOTOOL was used for visualising the T1 achievements by benchmarking and giving feedback in form of a graphical representation of the results, which allowed the facilities to compare their achieved domain, dimension and indicator values with the mean value of the other participating facilities. Together with the facilitator and a research assistant, the facilities themselves conducted the assessment, processed feedback and planned improvement interventions. The self-assessment of the facilities included patient and staff surveys, while the external facilitator followed a checklist and conducted management interviews. Data were entered into VISOTOOL and an immediate feedback was given to the facility staff.18 The facilities then conducted targeted quality improvement interventions—both process and structure related. After 1.5 years, the second assessment was conducted. We assessed the difference between T1 and T2 results, assuming that if a significant positive effect was achieved, a ‘good decision was made’ by the health facilities supported by the facilitator.

Patient involvement in form of surveys and self-assessments for patients was part of the assessment tools for the indicator-based IQMS quality measurement. In the observational study part on the prioritisation and implementation of interventions, there was no patient involved.

For analysing, whether the chosen interventions are successful, we chose a three-step method:

In the first step, we clustered all interventions performed at the participating facilities in improvement intervention topic areas, that is, neonatal mortality, the completeness of partographs, establishment of a quality improvement team, kitchen hygiene, waiting times, staff satisfaction, use of opportunities for improvement, infection prevention control (IPC), stock of essential medicines and supplies, shortages of staffing and transportation in remote areas, patients’ satisfaction as well as standard of antenatal care services.

We then grouped facilities for each topic area according to whether or not a concrete improvement intervention was conducted.

We only considered those improvement intervention topics that were conducted at more than one facility for a comparison of their T1 and T2 results. Accordingly, a total of five improvement intervention topics were selected: neonatal mortality, the completeness of partographs, waiting times, IPC as well as shortages of staffing and transportation in remote areas (table 2). All facilities without these improvement intervention topics served as a non-intervention group to compare each of the improvement intervention topics. The n values of the intervention and non-intervention groups are shown in table 2. As every facility did an individual gap analysis on its own T1 results after the measurement, there were differences in the choice of improvement intervention topics. This diversity of intervention topics among all ten facilities lead to the existence of intervention and non-intervention groups for every intervention topic.

Table 2

Summarised inducements and intervention contents of concrete improvement intervention topics

Improvement intervention topic*	Inducem*ent	Intervention contents
Neonatal mortality (2/8)	High neonatal mortality rates	Root cause analysis, auditing of all perinatal deaths, a creation of a separate newborn unit, minor renovations and improving of IPC practices
Completeness of partograph (4/6)	Low percentages of sampled partographs correctly filled	Conduction of Continuing Medical Education for staff and the institution of monitoring
Waiting times (2/8)	Longer than promised to clients waiting times	Introduction of a customer desk, the sensitisation of all departments and units and an overall introduction of customer flow systems
IPC (5/5)	IPC not meeting the required standards	Training and implementation of 5S, training on root cause analysis, improvement in IPC practices and a carrying out of regular assessments
Shortages of staffing and transportation in remote areas (2/8)	Transferring out of staff and a poor public transport system	Improvement on the referral system, establishment of a good communication with coordination of ambulances, attendance to all emergency cases within 45 min and the possibility of referral

*In brackets: number of facilities with a particular improvement intervention (intervention group) / without that particular intervention (focusing therefore on their chosen interventions) (non-intervention group) of the total 10 facilities.

IPC, infection prevention control.

In a second step, we analysed the IQMS quality indicators related to the respective intervention. The aim was to compare the outcome of these interventions to the non-intervention facilities.

Out of the 303 indicators, we selected 134 indicators due their relation to the improvement intervention topics (72 relevant indicators for the improvement interventions on neonatal mortality, 11 for the completeness of partographs, 12 for the one on waiting times, 24 for the one on IPC and 15 for the one on shortages of staffing and transportation in remote areas). Beyond every indicator were measurement prescriptions (‘items’) that were used to acquire the data for each indicator. These were dichotomic with ‘0’ and ‘1’ for the availability of the measured variable. All indicators were weighted equally. Percentages were calculated by dividing the number of indicators with dichotome ‘1’ by the total number of indicators.

We analysed the average T1 values and then compared them with each other and with the average change values (difference: T2 minus T1 value) among themselves for the groups of the 10 facilities with (intervention group) and without (non-intervention group) concrete improvement interventions topics. We only compared those indicators with complete data and therefore an available average group value in both groups of both T1 and T2 from each facility.

In a third step, we then summarised data calculating means and SD for the average of both the intervention and non-intervention groups. Frequency counts and percentages served the representation of categorical data from the indicators. A t-test was considered a suitable method in awareness of the low data variance, after also having taken account of a Z-testing. The calculation of p values was performed by applying the one sample t-test on the mean change and the SD of all comparable indicators in Microsoft Excel 2010. A double-tailed event was chosen since positive as well as negative changes were possible. Our null hypothesis expected a change value of 0, stating that the improvement intervention had no significant impact on a change of its relying indicators. We chose a significance level of α=0.05 for a CI of 95% and considered changes (as improvement if positive or deterioration if negative) as significant if p<α. This was the condition for a refusal of the null hypothesis.

Results

Looking at the analysis of data, table 3 summarises the mean percentage changes, significances and T1 values for the intervention and non-intervention groups (the term ‘(non-)intervention group/facility‘ always concerns one concrete improvement intervention topic, as all facilities conducted some interventions, but not all conducted interventions on the same topics).

Table 3

Comparison of indicator change for intervention and non-intervention group related to the different improvement intervention topics

Improvement intervention	Mean values of the comparable indicators in % at T1, T2, % of change (T2-T1) and p value of the facilities of the intervention and non-intervention group (all significant changes p<α (α=0.05) are highlighted)
	Intervention group (T1, T2 :percentage)				Non-intervention group
	T1	T2	change	P value	T1	T2	change	P value
Neonatal mortality	46.41	88.74	42.33	<0.0001	46.07	61.64	15.57	0.0007
Completeness of partographs	64.17	84.11	19.94	0.1451	57.13	53.3	−3.83	0.7370
Waiting times	51.50	90.91	39.41	0.0490	81.34	77.69	3.65	0.6232
IPC	54.31	82.08	27.77	0.0007	48.45	66.55	18.10	0.0012
Shortages of staffing and transport in remote areas	44.46	76.46	32.00	0.0194	59.18	73.32	14.14	0.0180

Bold values in table 3 refer to all significant changes.

IPC, infection prevention control.

Interventions performed in order to reduce the high neonatal mortality rates, interventions conducted to decrease waiting times, interventions conducted to improve IPC and interventions carried out to improve shortages of staffing and transportation in remote areas had a marked positive impact on the respective indicators and domains. Those interventions targeting the accuracy of partographs also had a—although less evident—improvement tendency.

As a result, the facilities reported a reduction of neonatal mortality rates in the facility, a higher number of completed partographs, less waiting times, a better IPC performance and an aversion of patients’ deaths.

Facilities that decided on an intervention for a certain topic did not necessarily have lower T1 values than those that did not decide on an intervention. For example, the average T1 value concerning neonatal mortality for both the intervention and non-intervention group was similar, while those facilities that decided on the mentioned improvement interventions on the completeness of partographs or IPC already had higher T1 values than those that did not participate. Facilities with lower T1 values regarding waiting time or shortages of staffing in remote areas decided to opt for interventions, while those with higher scores did not. This could be explained by the previously mentioned baseline differences and characteristics of the ten facilities (table 1).

Looking at the results of interventions, the analysis showed that the mentioned improvement interventions conducted to reduce neonatal mortality achieved higher improvement rates (change) (42.33%) than the non-intervention group, where the improvement of the comparable indicators was also significant (15.57%).

The analysis further shows that those facilities that implemented concrete activities to improve their IPC, achieved higher significant improvements (28%) than those facilities that did not. Nevertheless, we also observed significant improvements (18%) for those facilities without concrete IPC interventions.

Similarly, interventions to improve shortages of staffing and transport in remote areas showed higher significant improvements (32%) in comparison to the non-intervention group. Again, we also noted significant improvements among the latter facilities (14%).

Looking at the impact of interventions to reduce waiting time, those facilities that did take concrete measures achieved a marked improvement (39%), whereas the non-intervention group also improved (4%), but not significantly. The observed improvements in the non-intervention groups are subject of the following discussion section regarding possible spill over effects and confounders.

The improvement intervention on the completeness of partographs lead to an improvement (20%)—even if not significant—, whereas we found a—not significant—decline for the same filtered indicators of the non-intervention group (−4%).

Discussion

Analysis of targeted improvement interventions conducted in the facilities shows measurable improvements. The statistics are underlined by reduced neonatal mortality rates and higher rates of antenatal care visits.23

Beside the positive intervention results, the T1 results build an important base for decision-making and their percentage rate at the first measurement creates different starting points for each improvement intervention.

Looking at IQMS as a decision-making tool and the prioritisation of interventions done by facilities, analysis shows that there are different motivations to start an improvement intervention—as both poor as well as above average performance analysed at T1 drove facilities to start an improvement intervention in this field.

On one hand, the IQMS measurement of low T1 performance led to the prioritisation of interventions. On the other hand, some interventions were also planned although T1 performance was average or above average, which might be caused by those facilities following more strictly national policies.

Based on the differences between T1 values for the intervention and non-intervention groups, conclusions of the initial situation and prioritisation for deciding on an intervention can be drawn. Additionally, improvements can be looked at within the context of a possible previous deficiency measured by IQMS. But the decision on interventions in the case of previous above average performance might be also influenced by other factors for decision making.

Similar T1 values, like in the case of neonatal mortality improvement demonstrate that the IQMS intervention is clearly able to achieve better improvements within a similar initial setting regarding high neonatal mortality rates. The higher achievements of this improvement intervention in relation to the comparison group demonstrate the effectiveness in the form of reduced neonatal mortality rates of this targeted intervention. This is a remarkable achievement since the improvement of outcome indicators is an important aspect.

Lower T1 values at the participating facilities—for example, in the improvement interventions on waiting time and on shortages of staffing and transport in remote areas—underline the validity of the integral IQMS quality improvement approach for revealing the deficiency within this area.16

Higher T1 values at the participating facilities, like in the improvement interventions on the completeness of partographs and IPC, show that the facilities decided on interventions because of their own deficits compared with a better performance in other measured aspects at the same facility. The decision making on the intervention in these cases might also be caused by other influencing factors such as facilities following national policies, for example, following the National Health Sector Strategic Plan for providing universal access for neonatal and maternal health services.13 Higher positive changes in both cases underline the effectiveness of these targeted improvement interventions or facilities following National Health Sector strategies and lead to an even bigger advancement of T2 values compared with those facilities that did not participate.

Potential confounders, bias and limitations also need to be critically discussed. Therefore, this analysis and discussion of results needs to be understood with regard to issues that could have affected positive and negative findings, limitations and bias.

As limitations of this study, the small sample size, the relatively short time period of measurement and the performance of individual improvement interventions at each facility need to be mentioned. The conducted study was not planned as a statistically representative study but as an observational intervention study. A bigger sample size of facilities with similar improvement interventions and available T1 as well as T2 values would be necessary.

T3 values do not exist yet, but monitoring over a longer time period would allow more valuable conclusions. Another limitation is that interventions were implemented differently across all ten facilities due to individual facilities’ decisions after the gap analysis. Despite the supervision by a trained facilitator, differences in the methodology and implementation of the chosen interventions are possible, and therefore, build a limitation for statistical comparison.

When taking bias into account, we need to be aware of spill over effects that occur when externalities of an economic activity are non-monetary effects on non-participants. In our case, this can, for example, possibly augment T2 values in the non-intervention group leading to also significant improvements for the non-intervention group. Although observed improvements in the non-intervention groups are assumed to be related to the IQMS assessment at T1, the attribution of the observed improvements at both the intervention group and non-intervention group facilities to only the operation of the IQMS Kenya might be difficult, as also other interventions performed by government (such as the Health Sector Strategic Plan) or non-governmental organisations within the counties might interact as confounders. The other way around, the IQMS might also support the aims of other quality improvement approaches within its operating area.22 A sampling bias also needs to be thought of due to local differences in, for example, infrastructure, staffing, transport and existing resources. Another unconscious bias is the sensitisation of staff, which is achieved by the performance of the measurement itself. In a study by Goetz et al on EPA in Switzerland, it is shown that this already leads to an improved performance at the reassessment.16

Since the evaluation of interventions is difficult, one should be cautious saying T2 is a pure effect of the intervention, although it does reflect whether the participative and precise IQMS analysis is successful to plan targeted interventions.

After reflecting on possible limitations and biases, the results still show that the IQMS interventions start at points where (even significant) improvements are achievable—as we do not observe deterioration or stagnation of the analysed indicators. This can be attributed to the precise gap analysis. Despite the limitations at the level of significance, higher improvement percentages and T2 values are achieved for all intervention groups compared with the non-intervention groups.

We, therefore, assume that a precise, detailed and participative measurement and gap analysis is a basic requirement for setting the improvement process in motion. It leads to effective targeted improvement interventions. Analysis on the effectiveness of the precursor of the IQMS Kenya, EPA in Germany and Switzerland, shows significant improvements in three of four analysed domains and demonstrated the ability of EPA as functioning quality management programme.16 In accordance to the results from EPA, our results also show considerable improvements in four of the five analysed improvement intervention topics of the IQMS Kenya (neonatal mortality; waiting times; IPC; shortages of staffing and transport in remote areas).16 24–26 They underline the success of the prioritised interventions performed under continuous survey of the IQMS Kenya and thereby also the effectiveness of detailed problem analysis and active staff participation of VISTOOL-Kenya. The coaching and engagement of facility staff in processes—such as the internal quality audit teams—furthermore leads to sustainability of the quality improvement success, by encouraging facilities to take forward the identified quality improvement interventions by themselves. By using elements of this well-focused and intensified support—such as the coaching—accorded the small sample size of ten facilities, it would be possible for future, innovative quality improvement approaches to be implemented at national level and be scaled up to over 10 000 facilities in Kenya. Thereby similar or even better results with higher levels of significance could be achieved.

According to the Pareto principle, which is used as a prediction model by economists for the imbalance between inputs and outputs, 80% of problems can be resolved by only 20% of resources.27 This endorses our hypothesis that prioritisation is the key for successful quality improvement in limited-resource settings as the existing limited resources need to be employed the most efficient way by targeted implementations.

Conclusion

In limited-resource settings such as Kenya, activities to improve the quality of healthcare continuously need to be prioritised in order to use existing resources more efficiently.28

The methodology chosen to collect and analyse IQMS project data, accompanied by providing a certain continuity of support for facilities for a prioritised planning and conducting of targeted interventions,23 proves to be highly appropriate and may be recommended to build further evidence in similar settings. According to the Pareto principle, only a few improvement efforts already achieve most of the objectives, whereas the few remaining goals require the most efforts.27 This underlines that further optimisation of the IQMS quality improvement processes is able to achieve even higher improvements.