Quality of healthcare for children with severe acute malnutrition in a refugee setting: cross-sectional study in West Nile Region, Uganda.

OBJECTIVES: 5.0 million annual deaths in low-income and middle-income countries are due to poor quality of care (QOC). We evaluated the QOC provided to malnourished children in West Nile Region in Uganda.
DESIGN: Cross-sectional study.
SETTING: West Nile Region, an area hosting over one million refugees. PARTICIPANTS: Among 148 facilities providing nutritional services, 30 randomly selected facilities (20%) and the records of 1467 children with severe acute malnutrition (100% of those attending the 30 facilities during last year) were assessed. OUTCOMES: The national Nutrition Service Delivery Assessment (NSDA) tool was used to assess capacity areas related to QOC. Case management, data quality and health outcomes were assessed from official health records. Multivariate analysis was performed to explore factors significantly associated with better cure rates.
RESULTS: Of 305 NSDA scores allocated to 30 participating centres, 201 (65.9%) were 'good' or 'excellent'. However, 20 (66.7%) facilities had 'poor' 'quality improvement mechanisms' and 13 (43.3%) had 'poor' 'human resources'. Overall data quality in official records was poor, while recorded quality of case management was overall fair. Average cure rate was significantly lower than international Sphere standards (50.4% vs 75% p<0.001) with a higher default rate (23.2% vs 15% p<0.001). Large heterogeneity among facilities was detected for all indicators. Refugee-hosting and non-refugee-hosting facilities had a similar cure rate (47.1% vs 52.1%) though transfer rates were higher for those hosting refugees (21.5% vs 1.9%, p<0.001) despite better 'equipment and supplies'. 'Good/excellent' 'equipment' and 'store management' were significantly associated with better cure rates in outpatient therapeutic centres (+55.9, p<0.001; +65.4, p=0.041, respectively) in multivariate analysis.
CONCLUSIONS: Though most NSDA capacity areas were rated good or excellent, health outcomes of malnourished children in West Nile Region, both in refugee-hosting and non-refugee-hosting facilities, are significantly below international standards. Effective and sustainable approaches to improve malnourished child health outcomes are needed. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study explored the quality of care delivered to malnourished children in a refugee-hosting setting, an area which has been very poorly investigated in previous literature.

The study assessed multiple indicators in each facility—including indicators related to the overall service capacity to deliver high quality care, process indicators on quality of case management, quality of data and health outcomes—thus exploring several different dimensions of quality of care.

Limitations of the study mostly relate to poor data quality in official administrative records, from which most data were extracted.

However, documenting poor data quality in official administrative records is a critical finding in its own right, and advocates for more investment to strengthen existing routine systems for data collection and analysis, in line with the WHO Global Strategy for Mothers and Children recommendations.

Introduction

Poor quality of care (QOC) has been implicated as a major risk factor for excess mortality across conditions, especially in low-income and middle-income countries (LMICs). Globally, an estimated 5.0 million deaths in LMICs are directly linked to poor QOC, while an additional 3.6 million excess deaths are due to non-utilisation of health services.1

Achieving high QOC may be even more challenging in humanitarian settings, such as West Nile Region in Uganda. According to a recent report from the United Nations High Commissioner for Refugees, since 2017 over one million refugees have fled to Uganda, currently is the third largest refugee-hosting country in the world with 1.36 million refugees.2 Wars, violence and persecution in the Horn of Africa and Great Lakes region were the main drivers of forced displacement into Uganda, specifically South Sudan’s conflict, insecurity and ethnic violence in the Democratic Republic of the Congo, and political instability and human rights violations in Burundi.2 About 92% of refugees live in settlements alongside local communities, and the largest number of refugees (over one million) are located in the West Nile Region in Northern Uganda.2

More than 60% of Uganda’s refugees are children, with implications for protection and prevention services.2 A survey conducted in October 2017 in refugee-hosting areas in Uganda revealed that the prevalence of global acute malnutrition in children ranged from 4% to 12% among refugee communities, and from 5% to 11% in hosting communities.3 These data are in line with other studies conducted in similar settings which report high rates of malnutrition among children both in refugee camps4–6 and in hosting communities, with large variations among districts.2 4–7

While previous studies in LMICs such as Kenya, Benin and Brazil reported substandard QOC for malnourished children,8–10 very little is known about QOC delivered to children with malnutrition in refugee-hosting areas. A recent survey conducted in the district of Arua in West Nile Region reported an average cure rate for children with malnutrition of 52.9%.11 However, the study included only a small sample of facilities, none specifically dedicated to refugees and none providing inpatient care.

In order to provide a more comprehensive assessment of QOC offered to malnourished children in West Nile Region, the present study was designed to include a larger, more geographically representative sample of facilities, including facilities of different levels providing both inpatient and outpatient care and facilities dedicated to both refugees and the local community. The study aimed to assess the overall service capacity to deliver high-quality care, the quality of case management, the quality of data and health outcomes at each facility. It also aimed to explore health facility-level factors associated with better cure rates. This study contributes to evidence on current QOC for children with severe acute malnutrition in LMICs and particularly in refugee settings, and may be used by researchers and policy-makers to design tailored interventions to improve the quality of nutritional services in Uganda and other similar settings.

Methods

Study design and setting

This was a cross-sectional study, and the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for reporting on cross-sectional studies were applied.12 The study was conducted between 20 August and 7 September 2018 in the West Nile Region of Uganda. The region has a population of approximately 2 180 947,13 with an estimated number of refugees at the time of the study between 1 074 000 and 1 200 000.2 14

Levels II, III and IV health centres (HCs) and hospitals provide nutritional care in Uganda. Based on the current national regulations, level II HCs should serve about 5000 people and be headed by a nurse working with other nurses and midwives to provide care for common conditions (eg, malaria, antenatal care). Level III HCs should be led by a senior clinical officer, serve about 20 000 people and have a functioning laboratory. Level IV HCs should have a senior medical officer and at least another doctor, be able to admit patients and carry out emergency operations and serve about 100 000 people.

In Uganda, outpatient care to children with severe malnutrition is provided in outpatient therapeutic centres (OTCs), which are usually located at HCs and, in rare cases, in hospitals.15 Inpatient therapeutic centres (ITCs) are most often located in hospitals, with some in higher level HCs (level III or IV). Given the large number of refugees, some facilities for outpatient care are specifically located in refugee camps and are designated as refugee-hosting OTCs. Inpatient care for refugees is provided in the same facilities used by local community members (eg, ‘non-refugees’).

Health facility sample size and selection

Five refugee-hosting districts in West Nile region—namely Arua, Koboko, Yumbe, Moyo and Adjumani (see map in online supplementary appendix 1)—were selected for the study.

Out of the 148 facilities providing nutritional services in the five districts under study, a sample of 30 (20%) facilities was determined to be adequate to detect a 52.9% cure rate (identified as primary outcome), with a type I error of 5%, a 95% CI from 36.9% to 68.9%. The anticipated cure rate and its 95% CI were estimated based on the most recent relevant literature.11

Before randomisation, facilities were stratified in order to ensure geographical coverage, including a representative sample of about 20% of the total facilities in each district, with at least one centre providing inpatient care in each district, to include facilities from lowest level II HCs, to level III and IV HCs and hospitals, and to include at least 20% of all refugee OTCs (online supplementary appendix 2). Facilities offering either ITC or OTC services where health facility staff agreed to participate were eligible for inclusion in the study, while facilities with emerging security concerns impeding participation and those without staff assigned to be responsible for nutrition service delivery were excluded. Three districts had only one facility providing inpatient care, which were automatically included in the sample.

Study variables, data collection tools and procedures

Service capacity to deliver high-quality care

The evaluation of the quality of nutritional services was performed at each participating facility using the Nutrition Service Delivery Assessment (NSDA) Tool,16 the official national instrument for assessing performance of nutritional services in Uganda. The tool assesses the following 10 capacity areas for OTC (see online supplementary appendix 3 for details): (1) general information on service implementation, assessing the existence a person in charge of the nutrition service and a quality improvement team; (2) adequate human resources, evaluating the number, type and percentage of staff trained in six key courses; (3) provision of nutritional services, assessing all steps related to nutrition assessment of the patient, nutrition counselling and micronutrient supplementation; (4) community linkage, assessing links with community-based health workers and community groups, and effective referral; (5) quality improvement activities, investigating availability of a functional quality improvement team; (6) materials and supplies, assessing availability of guidelines, counselling cards and many other job aids; (7) facility nutrition equipment, assessing a list of 17 key items, including weighting scales, glucometers, and so on; (8) store management, evaluating conditions of the storage room and its management; (9) logistics management for commodities, assessing specifically logistical aspects, such as correct and regular use of order forms; (10) monitoring and evaluation (M&E), assessing five items, including the availability of a designated person for health management information system data, data collection and data use. The same 10 capacity areas were assessed for ITCs, plus requirements for inpatient care such as kitchen equipment and ingredients for therapeutic foods.16

Data were collected using predefined checklists and data sources including direct observation, document review and interviews with health workers, as indicated by the NSDA tool. Each area was scored based on the results of the checklists and according to predefined criteria, with four possible score categories: ‘poor’, ‘fair’, ‘good’ and ‘excellent’. For example, to achieve a score of good under ‘facility nutrition equipment’, 12 out of 17 items need to be available. Similarly, to achieve a score of good under ‘Store management’, at least 13 out of 17 items in the checklist need to be available.16

Quality of case management, quality of data and health outcomes

Data on quality of case management, quality of data and health outcomes of children were extracted at each facility from the official ‘Integrated Nutrition Register’. Each case of a child diagnosed with severe acute malnutrition and discharged in the period covering the financial year 2017/2018, which in Uganda covers the period July 2017 to June 2018, was reviewed. Integrated Nutrition Registers are the official documents at health facility level where all information on each malnourished child is recorded according to a standard format. Aggregate level data were collected using field-tested data collection tools (online supplementary appendix 4-6) developed based on national guidelines15 and previous experience from similar evaluations.11 17 The case definition of severe acute malnutrition was based on national guidelines, as follows: weight-for-height zeta-score < −3 SD, mid-upper-arm circumference (MUAC) <11.5 cm (6–59 months), or bilateral pitting oedema.15

Quality of case management was measured with the following five process indicators, using national guidelines as reference standards15: (1) correct diagnosis of malnutrition based on weight-for-height Z-score or MUAC; (2) correct treatment of malnutrition evaluated as correct ready to use therapeutic food dosage according to national guidelines15; (3) correct evaluation of HIV as per the national guideline15; (4) correct counselling of care givers on nutrition, ready-to-use therapeutic foods (RUFT) administration, hygiene and HIV; (5) correct exit outcome assignment as per national guidelines15; see online supplementary appendix 7 for more details.

Data quality was assessed using the following two predefined indicators: (1) data completeness (presence of 15 key items in Integrated Nutrition Registers); and (2) internal consistency (consistent visit dates and child height measurements over time (see online supplementary appendix 7 for more details).

Findings on case management and data quality were compared against predefined targets of at least 75% correctness, data completeness and internal consistency.

Health outcomes were determined for each child with severe acute malnutrition admitted during the study period following predefined categories indicated by the national guidelines15: (1) died (while in the programme); (2) cured (weight for height ≥ −2 SD or MUAC of ≥12.5 cm with no bilateral pitting oedema for 2 weeks and clinically well); (3) non-responders (not reaching discharge criteria after 3 months, or 4 months for patients with HIV/tuberculosis); (4) defaulters (absent for two consecutive follow-up visits); (5) transferred to inpatient care (ITC); (6) transferred to another outpatient care facility (OTC; see online supplementary appendix 7 for more details). Health outcomes were compared with SPHERE indicators, which represent the internationally recognised minimum standards for management of severe acute malnutrition in humanitarian response.18 According to SPHERE standards, the acceptable cure rate should be at least 75%, default rate should always be below 15% and the death rate should be below 10% at inpatient level and below 3% at outpatient level.18 Cases not classifiable due to missing information or to misclassification were labelled as ‘missing outcome information’ and, in line with SPHERE standards,18 were not considered for the calculation of health outcomes.

Data quality assurance procedures

The study team included a senior paediatrician, a nutritionist, a public health expert and six data collectors, all experienced in using the national guidelines for the Integrated Management of Acute Malnutrition (IMAM),15 and the NSDA tool.16 Before data collection, data collectors were provided refresher training on the key concepts of the IMAM guidelines, as well as training on data collection using predefined study tools and standard operating procedures (SOPs). Training included tests in the field, where data collection performance was evaluated and improved until considered sufficient.

All data collection tools included explicit case definitions on each page of the tool based on national IMAM guidelines.15 All tools were predefined and field tested, and staff was trained in the SOPs, and routinely supervised. Data collection was routinely monitored by two senior supervisors on site. Hard copies of data collection forms were checked daily for completeness and accuracy with inconsistencies discussed and corrected as needed prior to data entry. Data were cleaned, coded and double entered into an Excel database. Range, consistency and validity checks were built into the entry programme to minimise errors. Entered data were double checked for completeness and internal consistency and any problems (such as missing data) were discussed and solved in real time. The dataset was monitored remotely at regular intervals by a senior paediatrician.

Data analysis

First, we conducted a descriptive analysis of findings of the assessment. Data were presented as frequencies with respective proportions for categorical parameters and as median and range for continuous non-normally distributed variables.

We looked at variation of outcomes across single facilities by conducting exploratory subgroup analyses comparing outcomes between OTCs and ITCs and between refugee and non-refugee OTCs, as ITCs provide care both to refugees and to residents. We tested for differences in outcome distribution between groups using a χ2 test or Fisher’s exact test as appropriate.

Lastly, we performed a multivariate analysis with a general linear model using Gaussian family with identity link function to assess the association between the cure rate of children in OTCs (analysed as a continuous variable) and the following independent variables: health facility level; district; refugee facility status; number of admissions; the areas assessed by the NSDA tool. Findings were presented with β coefficients and 95% CIs, representing change in cure rate. We could not fit a multivariate model of cure rate in ITCs, given the low number of this type of facility (n=5). In all analyses, a p value of <0.05 was taken as statistically significant. Stata 14 was used for data analysis.

Patient and public involvement

In each facility, a formal written informed consent (online supplementary appendix 8) was sought before data collection from health workers. All communication was conducted in English and Lugbara, Madi or Kakwa local languages. The study did not involve patient interaction and therefore patient consent was not needed. The study did not include direct patient participation. However, selection of the study outcomes carefully took into consideration outcomes that are of primary importance for patients, such as child health outcomes.

Results

Characteristics of the sample

Out of all 148 facilities in the five selected districts in West Nile Region, 30 (20%) were identified by random selection; none presented exclusion criteria. Among the 30 selected facilities, 8 (32% of total OTCs) were refugee-hosting OTCs. All facilities were evaluated for quality of nutritional services with the NSDA tool.

A total of 1467 children with severe malnutrition were admitted to the 30 selected facilities during the financial year 2017/2018. All cases were reviewed to asses quality of case management, quality of data and child health outcomes (figure 1).

Figure 1

Study flow diagram. ITCs, inpatient therapeutic centres; NA, not applicable; OTCs, outpatient therapeutic centres.

Characteristics of health facilities are reported in table 1. A high percentage (44.6%) of children was enrolled in Arua district, which also is the most populated district, and the one with the highest number of facilities (online supplementary appendix 2). Most of the included OTCs were level III HCs, which is the facility type most often providing outpatient nutritional care in the region.

Table 1

Health facility characteristics

Facility characteristics	Health facility type			OTC facilities
	Overall(n=30)	OTC(n=25)	ITC(n=5)	Non-refugee (n=17)	Refugee (n=8)
Health facility level, n (%)
II	4 (13.3)	4 (16.0)	0	2 (11.7)	2 (25.0)
III	18 (60.0)	18 (72.0)	0	13 (76.5)	5 (62.5)
IV	3 (10.0)	2 (8.0)	1 (20.0)	1 (5.9)	1 (12.5)
Hospital	5 (16.7)	1 (4.0)	4 (80.0)	1 (5.9)	0
Children admitted, n (%)*	1467 (100)	909 (62.0)	558 (38.0)	533 (58.6)	376 (41.4)
Children admitted by district, n (%)*
Adjumani (# facilities=4, population=113 404)	141 (9.6)	117 (12.9)	24 (4.3)	0	117 (31.1)
Arua (# facilities=14, population=429 185)	654 (44.6)	366 (40.3)	288 (51.6)	282 (52.9)	84 (22.3)
Koboko (# facilities=3, population=118 868)	208 (14.2)	127 (14.0)	81 (14.5)	77 (14.5)	50 (14.3)
Moyo (# facilities=4, population=76 388)	100 (6.8)	79 (8.7)	21 (3.8)	32 (6.0)	47 (12.5)
Yumbe (# facilities=7, population=310 288)	364 (24.8)	220 (24.2)	144 (25.8)	142 (26.6)	78 (20.8)
Admissions per facility, median (range)	33.5 (6–288)	32 (6–77)	81 (21–288)	30 (6–77)	48.5 (20–71)

Data on population in each district derive from the National Population and Housing Census 2014.

*Percentage calculated on the total number of children admitted in the 30 selected facilities (n=1467).

ITC, inpatient therapeutic care; OTC, outpatient therapeutic care.

The median (range) number of yearly admissions was quite variable, with 81 (21–288) in ITCs and 32 in (6–77) OTCs. There was no significant difference between the average number of children admitted to OTCs when comparing refugee to non-refugee OTCs (p=0.3), or when looking at the HC level (p=0.1). Districts of Adjumani and Moyo admitted more children in refugee-hosting OTCs than non-refugee (p≤0.001 and p=0.01, respectively) while the number of children admission in Arua and Yumbe was higher in non-refugee OTCs (p=<0.001 and p=0.041, respectively). Other characteristics of refugee-hosting versus non-refugee-hosting OTCs were not significantly different (table 1).

Quality of nutritional services

The frequency of NSDA scores in each of the 11 capacity areas assessed in all 30 participating facilities is reported in table 2. Looking at the overall distribution of the 305 assessed scores, 107 (35.1%) were excellent, 94 (30.8%) good, 63 (20.7%) fair and only 41 (13.4%) were poor.

Table 2

Nutrition Service Delivery Assessment (NSDA) scores

	Score assessed with the NSDA tool (30 facilities)N (%)
Capacity area	Poor	Fair	Good	Excellent
1. General information*	0	6 (20.0)	3 (10.0)	21 (70.0)
2. Human resources	13 (43.3)	9 (30.0)	6 (20.0)	2 (6.7)
3. Provision of services	0	1 (3.3)	11 (36.7)	18 (60.0)
4. Community linkage	2 (6.7)	12 (40.0)	0	16 (53.3)
5. Quality improvement	20 (66.7)	4 (13.3)	4 (13.3)	2 (6.7)
6. Materials and supplies	4 (13.3)	17 (56.7)	6 (20.0)	3 (10.0)
7. Nutrition unit requirements†	0	2 (40.0)	0	3 (60.0)
8. Facility nutrition equipment	0	7 (23.3)	21 (70.0)	2 (6.7)
9. Store management	0	4 (13.3)	19 (63.3)	7 (23.3)
10. Logistics management for nutrition commodities	0	0	22 (73.3)	8 (26.7)
11. Monitoring and evaluation	2 (6.7)	1 (3.3)	2 (6.7)	25 (83.3)

*This area includesdifferent key items, from human resources to key aspects of quality improvement mechanisms and supportive supervision.

†Domain to be assessed only for inpatient care (n=5).

Overall, 25 (83.3%) and 21 (70%) facilities had excellent ‘monitoring and evaluation’ and ‘general information’ scores, respectively. ‘Provision of service’ was scored as excellent in 18 (60%) facilities and as good in 11 (36.7%). ‘Nutritional equipment’, and ‘logistics management’ were most often scored as good (70%, and 73.3% of total facilities, respectively). ‘Materials and supplies’ was scored as either fair, good or excellent in 26 (86.6%) facilities. Six capacity areas were never scored as poor, namely: general information; provision of services; ‘nutrition unit requirements’ (evaluated only in ITCs); ‘facility nutrition equipment’; store management and ‘logistics management for nutrition commodities’.

On the other hand, two capacity areas were frequently scored as poor: ‘quality improvement mechanism’ in 20 (66.7%) facilities and ‘human resources’ in 13 (43.3%). Scores for ‘community linkage’ were variable, excellent in 16 (53.3%) facilities but only fair in 12 (40.0%).

When comparing NSDA scores between ITCs and OTCs (online supplementary appendix 9), significant differences were observed in materials and supplies (significantly better in ITCs, exact p=0.029), and in community linkage (significantly better in OTCs, exact p=0.005). Refugee-hosting compared with non-refugee-hosting OTCs received overall higher scores in materials and supplies (exact p ≤0.001; online supplementary appendix 9).

When looking at data by single facility, among the 30 facilities assessed, about one-third (26.7%, two of which were ITCs) did not score poor in any of the 11 capacity areas, while a maximum of four capacity areas were scored as poor in any single facility (observed in two facilities).

Quality of data

Overall, data quality was poor (table 3): only 43.1% of admitted children had all essential data present in registers, and only 62.6% of cases had consistent height and date data over time (most of these being related to inconsistent height measurement in subsequent visits).

Table 3

Quality of data and case management

Variables	Health facility type				OTC facilities
	Overall	OTC	ITC	P value	Non-refugeeN=533	RefugeeN=376	P value
	N=1467	N=909	N=558
	n (%)	n (%)	n (%)		n (%)	n (%)
Data quality
Data completeness
Yes	632 (43.1)	362 (39.8)	270 (48.4)	0.044	210 (39.4)	152 (40.4)	0.756
No	835 (56.9)	547 (60.2)	288 (51.6)		323 (60.6)	224 (59.6)
Internal consistency
Yes	918 (62.6)	471 (51.8)	447 (80.1)	0.001	267 (50.0)	204 (54.3)	0.421
No	359 (24.5)	268 (29.5)	91 (16.3)		165 (31.0)	103 (27.4)
Not classifiable*	190 (13.0)	170 (18.7)	20 (3.6)		101 (19.0)	69 (18.3)
Quality of case management
Correct diagnosis†
Yes	1212 (82.6)	723 (79.5)	489 (87.6)	0.223	430 (80.7)	293 (78.0)	0.139
No	193 (13.2)	159 (17.5)	34 (6.1)		84 (15.8)	75 (20.0)
Missing	62 (4.2)	27 (3.0)	35 (6.3)		19 (3.6)	8 (2.0)
Correct treatment
Yes	1004 (68.4)	605 (66.6)	399 (71.5)	0.393	334 (62.7)	271 (72.1)	0.012
No	250 (17.0)	213 (23.4)	37 (6.6)		140 (26.3)	73 (19.4)
Missing	212 (14.5)	91 (10.0)	122 (21.9)		59 (11.0)	32 (8.5)
Evaluation of HIV status
Yes	1181 (80.5)	722 (79.4)	459 (82.3)	0.663	461 (86.5)	261 (69.4)	<0.001
No	56 (3.8)	56 (6.2)	0	8 (1.5)	48 (12.8)
Missing	230 (15.7)	131 (14.4)	99 (17.7)		64 (12.0)	67 (17.8)
Caregiver counselling
Yes	1130 (77.0)	807 (88.8)	323 (57.9)	0.001	446 (83.7)	361 (96.0)	<0.001
No	15 (1.0)	15 (1.7)	0		15 (2.8)	0
Missing	322 (21.9)	87 (9.6)	235 (42.1)		72 (13.5)	15 (4.0)
Correct assignment of exit outcome
Yes	1035 (70.6)	725 (79.8)	310 (55.5)	<0.001	413 (77.5)	312 (83.0)	0.001
No	307 (20.9)	113 (12.4)	194 (34.8)	58 (10.9)	55 (14.6)	0.001
Missing	125 (8.5)	71 (7.8)	54 (9.7)		62 (11.6)	9 (2.4)

*Cases of children presenting only with one visit could not be assessed for consistency.

†At enrolment.

ITC, inpatient therapeutic centres; OTC, outpatient therapeutic centres.

Both data completeness and consistency were significantly better in ITCs compared with OTCs (48.4% vs 39.8%, p=0.044 for completeness and 80.1% vs 51.8% p=0.001 for consistency). Data quality was not significantly different between refugee-hosting and non-refugee-hosting OTCs. Overall, only 4 (13.3%) facilities (1 ITC) had more than 75% record completeness, and 7 (23.3%) facilities (three ITCs) had more than 75% internally consistency of records.

Process indicators of case management

Four out of five process indicators of case management—specifically diagnosis, evaluation of HIV status, caregiver counselling and evaluation of exit outcome—were recorded as correctly performed in ≥75% of children (table 3). Treatment was correctly performed in 68.4% of total cases.

Significant differences were observed by facility type: according to records, caregiver counselling was less well performed in ITCs compared with OTCs (57.9% vs 88.8%, p=0.001); moreover, fewer ITC cases were correctly assigned an exit outcome (55.9% vs 74.9%, p=0.001).

Process indicators of case management also significantly varied between refugee-hosting and non-refugee-hosting OTCs, although different indicators varied in different directions: for example, caregiver counselling and correct treatment were better performed in refugee-hosting compared with non-refugee-hosting OTCs (p=0.001), while correct evaluation of HIV status was less frequent in refugee-hosting OTCs (p=0.001).

Large heterogeneity in outcomes was observed across facilities. Correct diagnosis rates ranged from 33.3% to 100% in OTCs and from 82.3% to 98.6% in ITCs, with 15 (60%) OTCs having correct diagnosis rates greater than 75%. Correct treatment rates ranged from 14.3% to 100% in OTCs and from 45.7% to 97.2% in ITCs. Overall, 10 (33%) facilities had correct treatment rates of greater than 75%.

Health outcomes

Overall health outcomes could be evaluated in 1035 (70.6%) children; 432 (24.4%) cases were excluded due to missing information on final health outcome (table 3). Based on data recorded in official forms, the average cure rate was 50.4%, significantly lower than SPHERE standards of ≥75% (p<0.001), without significant differences between OTCs and ITCs (table 4). The average death rate was 10.5% (table 4), with a death rate of 2.2% in OTCs and 30.0% in ITCs, the latter being significantly higher than SPHERE standards of <10% (p<0.001). The overall default rate was 23.2%, significantly higher than SPHERE standards of <15% (p<0.001), and with significant more defaulters at OTCs (30.5%) compared with ITCs (6.5%, p<0.001). The prevalence of other health outcomes assessed was overall low.

Table 4

Health outcomes

Variables	Overall	Health facility type			OTC facilities
		OTC	ITC	P value	Non-refugee	Refugee	P value
Available data	N=1035	N=725	N=310	<0.001	N=413	N=312	0.042
Dead, n (%)*	109 (10.5)	16 (2.2)	93 (30.0)	<0.001	7 (1.7)	9 (2.9)	0.28
Cured, n (%)*	522 (50.4)	362 (49.9)	160 (51.6)	0.62	215 (52.1)	147 (47.1)	0.188
Non-responders, n (%)*	18 (1.7)	18 (2.5)	0	0.005	6 (1.4)	12 (3.9)	0.04
Defaulters, n (%)*	240 (23.2)	221 (30.5)	19 (6.1)	<0.001	151 (36.6)	70 (22.4)	<0.001
Transferred to OTC, n (%)*	106 (10.2)	75 (10.3)	31 (10.0)	0.867	8 (1.9)	67 (21.5)	<0.001
Transferred to ITC, n (%)*	40 (3.9)	33 (4.6)	7 (2.3)	0.08	26 (6.3)	7 (2.2)	0.01

*Percentage calculated on the total number of available data.

ITC, inpatient therapeutic centre; OTC, outpatient therapeutic centre.

Refugee-hosting compared with non-refugee-hosting OTCs (table 4) had a similar cure rate (47.1% vs 52.1%, p=0.188) with a significantly higher rate of children transferred to OTCs (21.5% vs 1.9%, p<0.001) and a lower rate of default (22.4% vs 36.6%, p<0.001) and transfer to ITC (2.2% vs 6.3%, p=0.010). Non-response rate was also higher in refugee-hosting compared with non-refugee-hosting OTCs (3.9% vs 1.4%, p=0.040), however with low absolute numbers.

Large heterogeneity in health outcomes was detected between single facilities. Death rates ranged from 0% to 11.1% in OTCs and from 0% to 59.1% in ITCs, with 7 (23.3%) facilities having a rate above the SPHERE standard, specifically 3 OTCs exceeded the OTC target level of 5% and 4 ITCs the ITC target level 10%. Cure rates ranged from 28.3% to 85.7% with only 3 (10%) facilities within SPHERE standards of ≥75%. Default rates in OTCs ranged from 0% to 100%, with 21 (84%) of OTCs recording a rate within the acceptable SPHERE standards of 15%.

Multivariate analysis

In multivariate analysis (table 5), a score of ‘good/excellent’ under the NSDA capacity areas of ‘availability of facility nutrition equipment’ (+55.9, 95% CI 25.2 to 86.6, p=<0.001), and ‘store management’ (+65.4, 95% CI 2.7 to 128.1, p=0.041) were significantly associated with a better cure rate at the OTC level. ‘Good/excellent’ scores on ‘availability of materials and supplies’ were significantly associated with lower cure rates (−74.7), although with very large CIs (95% CI −130.1 to −19.2, p=0.008).

Table 5

Factors independently associated with a better cure rate at outpatient therapeutic centres level (N=25)

Variable	Beta coefficient (95% CI)(difference in cure rate)	P value
Health facility level
HCII	Reference
HC III	−40.5 (−81.4 to 0.4)	0.052
HC IV	−18.2 (−40.4 to 76.7)	0.543
Hospital	68.4 (−21.5 to 158.3)	0.136
District
Arua	Reference
Adjumani	57.9 (−0.8 to 116.7)	0.053
Koboko	−10.7 (−75.7 to 54.2)	0.746
Moyo	−33.4 (−68.8 to 2.0)	0.065
Yumbe	−22.3 (−59.6 to 15.0)	0.241
Refugee facilities
No	Reference
Yes	18.8 (−15.9 to 53.5)	0.289
Total number of admissions
≤50	Reference
>50	18.0 (−33.8 to 70.0)	0.494
Area assessed by NSDA tool*
1. General information†
Poor or fair	Reference
Good or excellent	23.7 (−6.3 to 53.7)	0.121
2. Human resources
Poor or fair	Reference
Good or excellent	−40.3 (−73.4 to 7.3)	0.117
3. Provision of services
Poor or fair	Reference
Good or excellent	34.8 (−21.7 to 91.3)	0.228
4. Community linkage
Poor or fair	Reference
Good or excellent	34.9 (−0.8 to 70.6)	0.056
5. Quality improvement mechanism
Poor or fair	Reference
Good or excellent	−6.8 (−37.5 to 23.9)	0.662
6. Materials and supplies
Poor or fair	Reference
Good or excellent	−74.7 (−130.1 to −19.2)	0.008
8. Facility nutrition equipment
Poor or fair	Reference
Good or excellent	55.9 (25.2 to 86.6)	<0.001
9. Store management
Poor or fair	Reference
Good or excellent	65.4 (2.7 to 128.1)	0.041
11. Monitoring and evaluation
Poor or fair	Reference
Good or excellent	10.4 (−33.2 to 54.1)	0.64

*Two Nutrition Service Delivery Assessment (NSDA) domains were not included: the domain # 7 Nutrition unit requirements, which is pertinent only to inpatient therapeutic centres, and the domain # 10 Logistics Management for nutrition commodities, where no facility scored as either poor or fair.

†This area includes including different key items, from human resources to key aspects of quality improvement mechanisms and supportive supervision.

Discussion

This study showed that in the West Nile Region, an area hosting over 1 million refugees in Northern Uganda, average cure rates for severe acute malnutrition in children were significantly below the minimum SPHERE standards (50.4% vs 75%, p<0.001). This was despite overall good scores on the capacity to provide nutrition services as assessed by the national NSDA tool, and good process indicators as recorded in official patient charts. Previous studies from other LMICs have reported low QOC for malnourished children, highlighting that poor health outcomes and problems in data quality are frequent findings in these settings.2–4 11 The current study adds to this previous knowledge as it is the first to our knowledge to assess QOC and health outcomes of malnourished children in refugee settings. Acute malnutrition is considered a condition of public health importance in Uganda,19 and as such, the findings of this study also strongly advocate for further actions to monitor and improve health outcomes of malnourished children in West Nile region across all facility types.

Some findings of this assessment deserve further discussion. First, this study highlighted that good health service ‘capacity’ as assessed by the NSDA tool and good process indicators as recorded in patient files may not directly translate into good health outcomes. This has been observed in other studies,11 16 and underscores the importance of including ‘hard outcomes’ such as child health indicators in any evaluation, in order to analyse the actual output of nutritional services in addition to inputs and processes.

Second, some of the apparent discrepancies between NSDA tool findings and data quality and internal consistency and final health outcomes may be explained by the nature of the NSDA tool. For example, the finding that most (83.3%) facilities were rated as excellent in ‘M&E’ seems incongruous with the fact that both data completeness and internal consistency were found to be below SPHERE standards. This may be because the current version of the NSDA tool allocates an excellent score for M&E when a person is designated to record data and records them, with no consideration of the quality of resulting data available for analysis. As a consequence, the NSDA tool may overestimate the real quality of the M&E. In general, low effectiveness of routine M&E systems is a well-documented problem in LMICs.20 21 According to the current system of M&E in Uganda, supportive supervision should be performed quarterly.15 In practice, however, supervision is often weak due to multiple barriers including lack of funds and physical resources (internet access, vehicles and fuel for conducting supervision visit), lack of leadership and coordination, and low supervisory capacity.11 Existing literature,22–24 including randomised controlled trials,17 indicates that supportive supervision delivered by trained staff at fixed intervals can significantly increase cure rate and QOC for children.

Similarly, materials and supplies availability may not be properly estimated by the NSDA tool, as several items like the availability of ready-to-use therapeutic foods are only assessed as ‘present at the moment of the assessment’, while availability in the previous 2 years is only assessed indirectly. Existing literature11 17 showed that frequent stock-outs are related to higher default rates, and people aware of stock-outs tend not to come back for follow-up visits, as they assume they will not receive therapeutic foods. This may at least partly explain a high default rate in areas with frequent stock-outs.11 17

Findings of this study are in line with previous studies,11 but differ from those of routine monitoring data.25 This may be due to multiple factors, including different time periods and samples of facilities and children, and different data sources, for example, the use of dispensing logs rather than nutrition registers. In addition, differences in case definitions and discharge criteria likely led to different findings. Once again, this discrepancy highlights the need to strengthen and standardise routine data collection systems, data validation procedures and mechanisms of effective M&E.

High heterogeneity in outcomes among facilities has been reported in other quality assessment studies.11 24 26 27 Research has indicated that applying quality improvement interventions can significantly reduce this heterogeneity among facilities.17 24

Other findings such as the observation of poor availability of human resources are not surprising.

The latest WHO strategy on health workforces recognises that global investment in the health workforce is lower than is often assumed,28 despite literature showing that the density of human resources is significantly related to maternal, infant and under-five mortality rates.29

The exploratory finding that cure rates were similar in refugee-hosting and non-refugee-hosting OTCs (47.1% vs 52.1%) is plausible, given the low-resourced context and the fact that more resources are often available in LMICs for managing emergencies than for maintaining the national health system.30 Interestingly, refugee-hosting facilities had a high rate of children transferred to other OTCs (21.5%), despite the finding that overall they are better equipped with materials and supplies when evaluated with the NSDA tool. This high transfer rate may potentially overload other facilities and indicates the need for better coordination and a systematic approach to caring for malnutrition among children in these settings.

While findings of the multivariate analysis require further confirmation in other studies, it is plausible that capacity areas identified as significantly associated with better cure rates (ie, ‘facility nutritional equipment’ and store management) may indeed contribute to better health outcomes. However, it is also plausible that many other aspects may contribute to better health outcomes of children, including, as already discussed, regular availability of therapeutic foods. Although measures of these items were included in our multivariate models, the accuracy and validity of these measures could be improved, as discussed previously.

Limitations of this study include the retrospective nature of the data collected. Poor quality of official health records may have affected the reported cure rate, as well as other indicators, such as correct diagnosis, although we cannot know in which direction. Poor quality of data is per se an interesting finding, certainly worth documenting for advocating for better quality data, as pointed out in previous studies.11 20 21 Similarly, process indicators relied on recorded information, and may actually overestimate the quality of case management (eg, caregiver counselling was almost always recorded as ‘performed’, but we cannot be sure whether this was actually performed in reality, and with what quality).

Another limitation of the study is that we did not collect individual patient information in each facility (such as HIV status) so we cannot exclude that differences in cure rate among facilities may be due to some extent to difference in the case mix of children. However, following major investments in prevention and control, HIV prevalence among children in Uganda is currently estimated to be relatively low (0.5%),31 with an observed prevalence in malnourished children in West Nile Region of around 2.4%,17 significantly lower than that reported in other nearby countries.32 Additionally, studies have reported that the health status of refugees can be quite variable even within nearby areas, and volatile within a short time frame, due to multiple factors including population type and place of origin, local conditions like water and hygiene sanitation, existing protection services like distribution of food rations, and treatment services such as access to healthcare and availability of drugs and therapeutic foods.2 4–7 The influence of all these factors on the final outcomes of refugee children with malnutrition should be better evaluated in future prospective studies.

Lastly, according current regulations, refugees with registered status have the right to access any existing health facility.2 As such, it is possible that some of the children admitted to non-refugee-hosting facilities were actually refugees. We believe this does not undermine the general message of the study, which indicates that improving QOC for malnourished children in West Nile Region is a priority, regardless of their refugee status.

Findings of this study cannot be directly generalised to other refugee settings. However, they do suggest that more evaluation of the QOC delivered to children in these settings is needed.

Conclusions

This study shows that health outcomes of malnourished children admitted to both refugee-hosting and non-refugee-hosting facilities in West Nile Region were significantly below international Sphere standards, despite the fact that NSDA ratings of the majority of the service capacity areas were good or excellent. Effective and sustainable approaches to improve child health outcomes should be promoted, with more investment clearly needed to strengthen existing routine systems of data collection and analysis, in line with the WHO Global Strategy for Mothers and Children recommendations.33