Development and an initial validation of the Responses to Illness Severity Quantification (RISQ) score for severely malnourished children.

AIM: To develop and perform an initial validation of a score to measure the severity of illness in hospitalised children with severe acute malnutrition (SAM).
METHODS: A prospective study enrolled SAM children aged 6-59 months hospitalised in Borno State, Nigeria. Candidate items associated with inpatient mortality were combined and evaluated as candidate scores. Clinical and statistical methods were used to identify a preferred score.
RESULTS: The 513 children enrolled had a mean age of 15.6 months of whom 48 (9%) died. Seven of the 10 evaluated items were significantly associated with mortality. Five different candidate scores were tested. The final score, Responses to Illness Severity Quantification (RISQ), included seven items: heart rate, respiratory rate, respiratory effort, oxygen saturation, oxygen delivery, temperature and level of consciousness. The mean RISQ score on admission was 2.6 in hospital survivors and 7.3 for children dying <48 h. RISQ scores <24 h before death had an area under the receiver operating characteristic curve (AUROC) of 0.93. The RISQ score performed similarly across differing clinical conditions with AUROCs 0.77-0.98 for all conditions except oedema.
CONCLUSION: The RISQ score can identify high-risk malnourished children at and during hospital admission. Clinical application may help prioritise care and potentially improve survival.

area under the receiver operating characteristic curve

PEWS

Bedside Paediatric Early Warning System

Capillary refill time

Heart rate

Integrated Management of Childhood Illness

Level of consciousness

Mid‐upper arm circumference

Oxygen

Oxygen saturation

Oxygen therapy

Respiratory Effort

Responses to Illness Severity Quantification

Respiratory rate

Severe acute malnutrition

systolic blood pressure

World Health Organisation

The Responses to Illness Severity Quantification (RISQ) score is a seven‐item severity of illness score that can differentiate malnourished children at high risk of death.

Score performance is similar in the presence or absence of conditions commonly associated with malnutrition, and is sensitive to changes in clinical condition.

Potential implications of use are rapid identification and effective prioritisation of children with a range of illnesses and with evolving critical illnesses.

INTRODUCTION

Children with severe acute malnutrition (SAM) in low‐income countries have an increased risk of death often related to infectious illnesses such as diarrheal disease, malaria, HIV and respiratory tract infections. Children with SAM who have medical complications are admitted to hospital with decisions guided by the Integrated Management of Childhood Illness guidelines. These guidelines consist of algorithms and clinical danger signs reflective of disease severity to help health care workers in outpatient clinics with the triage of children who need hospitalisation. Once in hospital, the World Health Organisation (WHO) has specific guidelines for inpatient care for children with SAM, including nutritional and medical guidance for treatment in the first 7 days of the stabilisation phase (i.e. hypoglycaemia, hypothermia, de‐hydration, infection and initial feeding) and the following 2–6 weeks in the transition phase (catch up feeding, sensory stimulation and preparation for follow‐up). The guidelines do not provide specific advice concerning the management of vital signs, in particular tachycardia, in clinical decision making. , Despite this guidance, mortality remains high in hospitalised children. Timely identification of high‐risk children could help prioritise care and potentially improve survival.

The Responses to Illness Severity Quantification (RISQ) score is based upon a previously developed and evidence‐based clinical decision‐support tool that objectively identifies hospitalised children at risk for clinical deterioration – The Bedside Paediatric Early Warning System (BedsidePEWS). BedsidePEWS includes a validated seven‐item severity of illness score to quantify the severity of illness, a documentation record, and recommendations for care escalation and de‐escalation among hospitalised patients. The seven items used to calculate the score are: heart rate, systolic blood pressure, capillary refill time, respiratory rate, respiratory effort, transcutaneous oxygen saturation and oxygen therapy. The possible range of scores is 0–26 with the highest scores in the severely ill patients. Use of the BedsidePEWS improves the timeliness of escalation and de‐escalation of care.

Based on experience with the BedsidePEWS in high‐resource settings and interest in the tool from practitioners caring for children in low‐resource settings, we sought to evaluate and adapt this validated score in resource‐limited contexts to assist in clinical evaluation and in decision making for potential care escalation and de‐escalation. Thus, our objective was to evaluate the validity of the BedsidePEWS score as a measure of severity of illness in children with SAM and explore the impact of simple modifications to scoring on quality of prediction in hospitalised children.

METHODS

Study design and patients

A prospective observational study was performed. There was no study intervention. The study enrolled a cohort of children admitted to the Alliance for International Medical Action (ALIMA) inpatient therapeutic feeding centre (ITFC) at the University of Maiduguri Teaching Hospital (UMTH) in Borno State, Nigeria (NCT04582773). The study was approved by the Research Ethics Board of Hospital for Sick Children and UMTH.

Eligible patients were aged 6–59 months with SAM (defined as one or more of Weight‐ for‐Height Z‐score ≤ −3, and/or mid‐upper arm circumference [MUAC] <115 mm, and/or bilateral pitting oedema) who received inpatient care. Children with congenital anomalies that interfered with feeding (i.e. cleft lip or palate) and those previously enrolled in the study (and re‐admitted to ITFC) were ineligible.

The primary outcome was inpatient mortality and the secondary outcomes were mortality within or after 48 h of admission for inpatient care.

Clinical parameters recorded were comprised of seven items from the BedsidePEWS [heart rate (HR), respiratory rate (RR), systolic blood pressure (SBP), capillary refill time (CRT), oxygen saturation (O2 sat), oxygen therapy (O2) and respiratory effort (RE)], two items from the WHO IMCI guidelines [level of consciousness (LOC) and temperature (Temp)] and MUAC as measure of malnutrition predictive of mortality. These 10 candidate items were evaluated and then combinations of promising items were considered for candidate scores.

Items belonging to the BedsidePEWS were scored using the established BedsidePEWS categories. Level of consciousness was assessed and categorised using the ‘alert, voice, pain, unresponsive’ (AVPU) system and scored accordingly. Temperature was categorised, recognising the impact of hypothermia in malnourished children and the relevance of fever and hypothermia as a marker of sepsis. MUAC was categorised into four measurement groupings with the lowest measurements receiving the highest scores (Table 1).

TABLE 1

Candidate item scores

Individual item	Age group (in months)	Sub‐score
		0	1	2	4
Heart rate (bpm)	6 to <12	>100 and <150	≥150 or ≤ 100	≥170 or ≤80	≥180 or ≤ 70
	12–60	>90 and <120	≥120 or ≤90	≥150 or ≤70	≥170 or ≤ 60
Respiratory rate (breaths per minute)	6 to <12	>24 or < 51	≥51 or ≤ 24	≥71 or ≤19	≥81 or ≤ 15
	12–60	>19 or < 41	≥41 or ≤ 19	≥61 or ≤15	≥71 or ≤ 12
Systolic blood pressure (mmHg)	6 to <12	>80 or < 100	≥100 or ≤ 80	≥120 or ≤70	≥150 or ≤ 60
	12–60	>90 or < 120	≥110 or ≤ 90	≥125 or ≤ 70	≥160 or ≤ 65
Capillary refill	6–60	<3 s			≥3 s
Oxygen saturation (%)	6–60	>94	91–94	≤ 90
Oxygen therapy	6–60	Room air		Any to <4 L/min or < 50%	≥4 L/min or ≥50%
Respiratory effort	6–60	Normal	Mild	Moderate	Severe
Additional items
Mid‐upper arm Circumference (mm)	6–60	>125	115–125	100–114	<100
Level of consciousness		Awake and alert	Response to voice	Response to pain	Unresponsive
Temperature (°C) (axilla)	6–60	≥36–38.5		<36	>38.5

Study measurements

After admission to the inpatient unit the guardians of eligible children were approached for consent by a study nurse fluent in the guardian's language. Children were enrolled after consent was obtained. Admission diagnoses were classified using the admitting doctor's clinical assessment. Severe anaemia was defined as a haemoglobin level <7.0 g/dl (HemoCue® Hb 301), hypoglycaemia as glucose level < 3.0 mmoL/L (Nova StatStrip Xpress®), malaria as a positive result from either microscopy or rapid test (SD Bioline Malaria Antigen Pf ® (HRP2), diarrhoea (3 or more liquid stools per day) and vomiting by the admission history. Clinical signs of oedema were recorded as bilateral pitting oedema in feet, lower limbs, and generalised (levels 1, 2, and 3+ respectively), and current breastfeeding status was recorded as yes or no. Pneumonia diagnosis was made by the physician through respiratory assessment including auscultation of lungs.

Candidate items were measured as follows: MUAC was measured to the nearest millimetre, using a standard non‐elastic MUAC tape, daily in the morning, and the remaining items were collected using a bedside assessment. The assessment included: manual 1‐min heart rate and respiratory rate counts, automated systolic blood pressure (CARESCAPE™ V100), oxygen saturation (Masimo Rad‐G™ pulse oximeter) and axillary temperature. Capillary refill time was measured in seconds, level of consciousness was measured using the AVPU system, respiratory effort was measured as normal, mild, moderate or severe as per the training protocol, and oxygen therapy was measured in litres/min by nasal prongs. Definitions were provided on case report forms. The bedside assessment was conducted at enrolment and twice daily throughout the hospital stay for each participant, once in the morning and once in late afternoon, with a minimum of 8 h in between assessments. The date of hospital discharge and vital status at hospital discharge were also collected. Clinicians in the ITFC applied the WHO guidelines for the management of SAM, including detection of hypothermia, severe pallor, hypoglycaemia and hypoxaemia. The data collection was done by the study team and did not replace the routine assessments established as per UMTH and ALIMA protocols. The ITFC doctors and bedside nurses caring for the children had access to the study measurements at any time upon their request. Neither the study team nor hospital staff were informed about the BesidePEWS system beyond that it was a system to potentially guide decision making for escalation and de‐escalation of care. Items were stored in raw form and were not scored. We used the first set of measurements taken at admission and the last set of measurements taken within 24 h of hospital discharge or death to compute scores for analyses.

The study team consisted of four study nurses (RN), a study coordinator (MD), a local principal investigator (MD) and a data archivist. The study team received a 1‐week on‐site theoretical and practical training session on measurement of candidate items and study procedures by the international study team from SickKids and ALIMA and had ongoing support by the study coordinator who reviewed the study nurses' bedside assessments routinely throughout the enrolment period.

Data handling and statistical analysis

Case report forms were scanned and uploaded into a web‐portal where they were reviewed for completeness. Approved data were entered into a bespoke database (Oracle, Redmond, CA). The accuracy of data entry was verified by secondary review of 20% of entered data. Data were reviewed for completeness, outliers and implausible values and reconciled against the scanned file or the child's original hospital chart. The WHO Growth Standards Z‐scores were calculated by using the WHO anthro software (https://www.who.int/childgrowth/software/en/).

The sample size was one of convenience as the analysis involved validation of existing scores rather than building of a new multivariate model. The study cohort is described using counts and percentages, and characteristics of survivors and non‐survivors are compared using chi‐squared or Fisher's exact tests.

Score development

Candidate items

Each candidate item was evaluated separately using the first set of measurements taken at admission. The mean and standard deviation of the sub‐score for each individual item was calculated and the distributions of sub‐scores were compared between three groups: those who survived their hospitalisation, those who died after 48 h of hospital admission and those who died within 48 h of hospital admission, using the Kruskal–Wallis test. Next, the same analysis was done for the final measurements taken within 24 h of death or discharge. Third, logistic regression was used to compute the odds ratio for any death (and its 95% confidence interval [CI]) per point of individual items. Finally, the nonparametric area under the receiver operating characteristic curve (AUROC) and its CI were calculated for each item for any death.

Candidate scores

Several candidate scores were constructed. First evaluated was the seven‐item BedsidePEWS score. Items from the BedsidePEWS score that were significantly associated with mortality in the item‐by‐item analyses were used as a base score. To this were added, individually and together, items from the remaining candidate items that were statistically significantly associated with mortality. The resulting five candidate scores were assessed using these criteria: (i) discrimination of any death from survival (using AUROC) using admission scores, (ii) discrimination of any death from survival (using AUROC) using discharge scores and (iii) number of items. Selection of final score combined judgements on AUROC and number of items, with higher AUROC, fewer items and greater numerical separation between groups being preferred.

Once the final score was identified, we evaluated its performance at admission and discharge as a predictor of (i) any death; (ii) death within 48 h of admission and (iii) death >48 h from admission among those alive at 48 h. Initial validation was done by evaluating the performance of the score in the entire cohort as above, comparing the admission scores with the final scores in patients who (i) died within 48 h of admission; (ii) died after 48 h of admission; and (iii) survived to hospital discharge and finally performance of the score in groups based on clinical diagnoses at admission: pneumonia, malaria, anaemia, hypoglycaemia, oedema, diarrhoea, vomiting, and, in addition, breastfeeding status.

R version 4.1.1 was used for score calculation and statistical analyses.

RESULTS

In the 11 months ending June 2020, we identified 579 eligible patients and enrolled 513. The children eligible but not enrolled arrived at the inpatient therapeutic feeding centre (ITFC) late in the evening, outside of the security‐restricted working hours of the study team. Candidate items and scores were evaluated in clinical data from 513 children. Forty‐eight (9%) children died. Fifteen children left the hospital against medical advice more than 48 h after admission and thus were included in the analysis as they survived past 48 h. The mean age (SD) was 15.6 (0.4) months, 247 (48%) were male, 79 (15%) had oedema and 42 (8%) had pre‐existing medical conditions, including 9 (2%) who were HIV positive. Common clinical conditions diagnosed at admission were diarrhoea 387 (75%); pneumonia 179 (35%) and severe anaemia 78 (15%). Malaria was present in 63 (12%) and was statistically significantly associated with inpatient mortality (20% prevalence in those who died and 11% prevalence in survivors, p = 0.02), as was hypoglycaemia (9% vs. 3%; p = 0.05), although there were differences of at least a similar size for anaemia (29% vs. 14%; p = 0.07). Admissions were from ambulatory feeding programmes 343 (67%), inpatient transfers 31 (6%) and directly from the hospital emergency department 139 (27%) (Table 2).

TABLE 2

Baseline characteristics of children admitted to the study

Variable	Total N (%)	Survivors N a (%)	Non‐survivors N (%)	p‐value
Total enrolment	513 * (100)	465 (91)	48 (9)
Male	247 (48)	230 (49)	17 (35)	0.09
Mean age in months (SD)	15.6 (8.8)	15.6 (8.7)	15.5 (9.3)	0.96
Age 6–11 months	202 (39)	180 (39)	22 (46)	0.42
Age 12–59 months	311 (61)	285 (61)	26 (54)
Length of stay (days) (median, IQR)	5 (4, 8)	5 (4, 8)	3 (1,8)	0.001
Oedema	79 (15)	73 (16)	6 (13)	0.71
Oedema and MUAC <115 mm	40 (8)	39 (8)	1 (2)	0.21
MUAC (mm) at admission (no oedema) (median, IQR)	108 (101,113) N = 434	108 (102, 113) N = 392	106 (99, 113) N = 42	0.38
MUAC <100 mm	84 (16)	73 (16)	11 (23)	0.28
WAZ < 3Z	397 (77)	360 (77)	37 (77)	0.59
HAZ < 3Z	191 (37)	173 (37)	18 (38)	1.00
Admission directly from hospital emergency room	139 (27)	129 (28)	10 (21)	0.26
Transfer from ATFC b	343 (67)	310 (67)	33 (69)
Transfer from ITFC c	31 (6)	26 (6)	5 (10)
Appetite test positive	5 (1)	5 (1)	0	1.00
Chronic medical condition at admission
Any	42 (8)	36 (8)	6 (13)	0.27
HIV positive	9 (2)	7 (2)	2 (4)	0.20
Tuberculosis	7 (1)	7 (2)	0 (0)	1.00
Sickle cell	6 (1)	6 (1)	0 (0)	1.00
Cerebral palsy	1 (<1)	4 (<1)	0 (0)	1.00
Cardiac disease	4 (<1)	1 (<1)	0 (0)	1.00
Trisomy	2 (<1)	2 (<1)	0 (0)	1.00
Sickle cell + tuberculosis	2 (<1)	1 (<1)	1 (2)	0.18
Other d	11 (2)	8 (2)	3 (6)	0.07
Admission history or clinical assessment
Pneumonia diagnosis	179 (35)	159 (34)	20 (42)	0.38
Malaria positive (rapid test or microscopy)	63 (12)	53 (11)	10 (20)	0.02
Severe anaemia (haemoglobin <7 g/dL)	78 (15)	64 (14)	14 (29)	0.07
Hypoglycaemia (glucose <3 mmol/L)	16 (3)	12 (3)	4 (8)	0.05
Diarrhoea	387 (75)	356 (77)	31 (65)	0.10
Vomiting	277 (54)	251 (54)	26 (54)	1.00
No breastfeeding e 6–59 months	220 (43)	200 (43)	20 (42)	0.92
Breastfeeding <12 months	184 (91)	165 (92)	19 (86)	0.41
No breast feeding <12 months	17 (9)	14 (8)	3 (14)

Includes 15 children who left against medical advice, all >48 h of admission.

Ambulatory therapeutic feeding centre.

Inpatient therapeutic feeding centre.

Measles; pertussis; TB + pertussis; meningitis; acute glomerulonephritis.

1 missing; IQR, interquartile range.

Candidate Items for the RISQ score

There were 508 (99%) children with all 10 individual items available at the initial assessment. Two children had missing measurements for capillary refill, heart rate, oxygen therapy and level of consciousness, and three had missing measurements of oxygen saturation, respiratory effort and temperature at admission. All 513 (100%) children had the 10 items measured within 24 h of hospital discharge or death.

Four of the seven items of the BedsidePEWS score were significantly associated with inpatient mortality: heart rate, respiratory effort, oxygen saturation and oxygen therapy. Respiratory rate on admission was associated with inpatient mortality after 48 h. Systolic blood pressure at admission (p = 0.181) or at exit (p = 0.100) was not significantly associated with mortality. Capillary refill time was categorised as <3 s in all but three measurements so all scores were 0 and significance was not available. Level of consciousness and temperature were significantly associated with mortality at all times and MUAC was not associated with mortality when assessed at admission (p = 0.31) or at exit (p = 0.46). The items assessed at admission with the strongest associations with mortality were respiratory effort, oxygen use and level of consciousness (Table 3). Predicting death was associated with point estimates for the AUROC from 0.51 to 0.73 for all items at admission, and 0.56–0.81 for the same items derived taken within 24 h of death (Table 4), with the exception of capillary refill time, where scores were all 0.

TABLE 3

Mean individual item and candidate scores at admission and exit

Score	Survivors (n = 465)			Died >48 h (n = 24)			Died <48 h (n = 24)			Comparisons between groups
	Admission	Exit	Change	Admission	Exit	Change	Admission	Exit	Change	Admission	At exit
HR	1.0 (1.1)	0.7 (0.7)	−0.4 (1.1)	2.3 (1.8)	1.7 (1.1)	−0.6 (1.7)	1.8 (1.7)	1.8 (1.8)	−0.1 (1.1)	0.001	<0.001
RE	0.2 (0.5)	0.0 (0.1)	−0.2 (0.5)	0.5 (0.6)	0.5 (0.7)	0.0 (0.9)	1.2 (1.0)	1.4 (1.0)	0.2 (0.6)	<0.001	<0.001
O2 Sat	0.1 (0.4)	0.1 (0.3)	−0.1 (0.5)	0.2 (0.4)	0.1 (0.3)	−0.1 (0.5)	0.6 (0.9)	0.5 (0.8)	−0.1 (0.4)	<0.001	<0.001
RR	0.5 (0.7)	0.3 (0.4)	−0.2 (0.7)	0.8 (0.9)	0.8 (0.7)	0.0 (1.2)	0.8 (0.8)	0.6 (0.5)	−0.1 (0.7)	0.026	<0.001
O2	0.1 (0.5)	0.0 (0.0)	−0.1 (0.5)	0.3 (0.8)	0.7 (1.0)	0.3 (1.0)	1.2 (1.2)	1.6 (1.0)	0.4 (0.8)	<0.001	<0.001
SBP	0.7 (0.8)	0.6 (0.7)	−0.1 (1.0)	1.0 (0.9)	0.9 (0.7)	−0.2 (1.0)	0.9 (1.1)	0.9 (1.2)	0.0 (0.4)	0.181	0.100
Temp	0.6 (1.3)	0.4 (0.8)	−0.2 (1.5)	1.1 (1.8)	0.6 (1.3)	−0.5 (1.5)	1.5 (1.9)	1.2 (1.7)	−0.3 (1.7)	0.007	0.061
LOC	0.0 (0.3)	0.0 (0.0)	0.0 (0.3)	0.2 (0.6)	0.5 (1.1)	0.4 (1.0)	0.2 (0.7)	0.4 (0.8)	0.2 (0.6)	<0.001	<0.001
MUAC	2.0 (1.0)	2.1 (1.0)	0.0 (0.7)	1.8 (1.1)	2.4 (1.2)	0.6 (1.1)	2.4 (1.2)	2.3 (1.2)	0.0 (0.8)	0.308	0.460
Candidate RISQ scores
A	2.7 (2.2)	1.6 (1.3)	−1.0 (2.3)	5.0 (3.0)	4.3 (2.3)	−0.7 (2.9)	6.4 (4.2)	6.5 (3.4)	0.1 (2.5)	<0.001	<0.001
B	2.0 (2.1)	1.0 (1.1)	−0.9 (2.1)	4.1 (3.0)	3.5 (2.1)	−0.5 (3.0)	5.5 (3.8)	5.8 (3.0)	0.2 (2.4)	<0.001	<0.001
C	2.5 (2.6)	1.5 (1.3)	−1.1 (2.7)	5.2 (4.2)	4.1 (2.4)	−1.0 (3.3)	7.0 (4.7)	6.9 (4.0)	−0.1 (2.9)	<0.001	<0.001
D	2.0 (2.2)	1.0 (1.1)	−0.9 (2.2)	4.2 (3.0)	4.0 (2.4)	−0.2 (3.0)	5.8 (3.8)	6.2 (2.9)	0.4 (2.6)	<0.001	<0.001
E	2.6 (2.7)	1.5 (1.3)	−1.1 (2.8)	5.3 (4.2)	4.6 (2.9)	−0.7 (3.4)	7.2 (4.6)	7.3 (3.8)	0.1 (3.0)	<0.001	<0.001

Note: Table shows mean (SD). p‐values comparing scores across the three groups were calculated using Kruskal–Wallis tests; capillary refill time values were 0.

Candidate RISQ scores: All scores include the five items: HR, RR, O2 Sat, O2 and RE of the core score.

Additional items in each score are as follows: (A) Core + SBP + capillary refill time; (B) Core alone; (C) Core + Temp; (D) Core + LOC; (E) Core + LOC + Temp.

Abbreviations: HR, heart rate; RR, respiratory rate; O2 sat, Oxygen saturation; O2, Oxygen therapy; RE, respiratory effort; SBP, systolic blood pressure.

TABLE 4

Predicting any death by admission and exit scores for individual items and candidate scores

	Predicting any death by admission scores		Predicting any death by exit RISQ scores
	OR (per point)	AUC	OR (per point)	AUC
HR	1.78 (1.44, 2.21)	0.65 (0.56, 0.74)	2.67 (1.99, 3.66)	0.70 (0.61, 0.79)
RE	3.46 (2.27, 5.52)	0.73 (0.66, 0.81)	71 (28.1, 202)	0.81 (0.74, 0.88)
Sat	2.33 (1.43, 3.67)	0.59 (0.52, 0.65)	2.76 (1.58, 4.69)	0.57 (0.51, 0.63)
RR	1.66 (1.15, 2.35)	0.60 (0.53, 0.68)	5.09 (2.83, 9.49)	0.69 (0.61, 0.76)
O2	2.69 (1.92, 3.79)	0.65 (0.58, 0.72)	N/A*	0.78 (0.70, 0.85)
SBP	1.4 (1, 1.92)	0.57 (0.49, 0.64)	1.51 (1.05, 2.13)	0.58 (0.49, 0.66)
Temp	1.34 (1.12, 1.6)	0.59 (0.52, 0.67)	1.56 (1.18, 2.04)	0.56 (0.48, 0.63)
LOC	2.33 (1.29, 4.2)	0.54 (0.50, 0.59)	N/A*	0.61 (0.55, 0.67)
MUAC	1.09 (0.8, 1.44)	0.51 (0.43, 0.60)	1.29 (0.93, 1.78)	0.56 (0.46, 0.66)
Candidate RISQ scores
A	1.43 (1.29, 1.59)	0.77 (0.70, 0.84)	2.83 (2.23, 3.74)	0.89 (0.83, 0.95)
B	1.43 (1.28, 1.59)	0.77 (0.70, 0.84)	3.36 (2.56, 4.63)	0.90 (0.85, 0.96)
C	1.33 (1.22, 1.45)	0.75 (0.68, 0.83)	2.57 (2.06, 3.33)	0.88 (0.82, 0.94)
D	1.43 (1.29, 1.59)	0.78 (0.72, 0.85)	3.48 (2.66, 4.78)	0.91 (0.86, 0.97)
E	1.33 (1.23, 1.45)	0.77 (0.69, 0.84)	2.72 (2.16, 3.56)	0.89 (0.83, 0.96)

Note: Capillary refill time odds ratio not calculated; N/A* All deaths occur in children with scores >0, and the estimated odds ratios are infinite;

Candidate RISQ scores: All scores include these five core items HR, RR, O2 Sat, O2 and RE.

Additional items in each score are as follows: (A) Core + SBP + capillary refill time; (B) Core alone; (C) Core + Temp; (D) Core + LOC; (E) Core + LOC + Temp.

Abbreviations: HR, heart rate; RR, respiratory rate; O2 sat, Oxygen saturation; O2, Oxygen therapy; RE, respiratory effort; SBP, systolic blood pressure.

Candidate scores

Five combinations of candidate scores were evaluated: First we evaluated the original seven‐item BedsidePEWS score. Next, we removed CRT (values were all 0) and SBP (not significantly associated with mortality), and then to these five items were added level of consciousness and temperature individually and then together (Table 3). MUAC was not included in any scores as it was not significantly associated with mortality. Predicting any death during hospitalisation for candidate scores was associated with AUROCs ranging from 0.75 to 0.78 at admission and from 0.88 to 0.91 at exit (Table 4).

The final score was named the ‘Responses to Illness Severity Quantification’ (RISQ) score. The RISQ score (Group E) included seven items: heart rate, respiratory rate, oxygen saturation, oxygen use, respiratory effort, level of consciousness and temperature (Tables 3 and 4). Group E was chosen as it demonstrated the greatest difference between mean values of the groups (Table 3) and had favourable AUROC to predict mortality. The observed RISQ scores ranged from 0 to 17 out of a maximum possible value of 26. Admission scores had a mean of 2.6 in those who survived, 5.3 in those who died after 48 h and 7.3 in those who died within 48 h (Figure 1). The RISQ score remained high when children became sicker and closer to death and decreased when the children became well and ready for discharge/exit from hospital. Mean (SD) scores at admission and discharge for those dying <48 h were 7.2 (4.6) and 7.3 (3.8), whereas mean scores in those who survived decreased from 2.6 (2.7) to 1.5 (1.3) between admission and discharge (Table 3). Each point increase in the admission score increased the odds of death within 48 h by 1.36 and the AUROC was 0.81. As expected, discharge scores were even more strongly associated with mortality with AUROC of 0.93 for death within 48 h and 0.85 for death after 48 h (Table S1).

FIGURE 1

Box plot showing the distributions of candidate scores at admission by final outcome. The box extends between the upper and lower quartiles, the line in the box marks the position of the median and the lines on either sides of the box show the minimum and maximum values. HR, heart rate; RR, respiratory rate; O2 sat, oxygen saturation; O2, oxygen therapy; RE, respiratory effort; SBP, systolic blood pressure; CRT, capillary refill time. Candidate RISQ scores: All scores include these five core items: HR, RR, O2 Sat, O2 and RE. Additional items in each score are as follows: (A) Core + SBP + CRT; (B) Core alone; (C) Core + Temp; (D) Core + LOC; (E) Core + LOC + Temp

Performance of score associated with specific clinical conditions

Performance of the seven‐item RISQ score as a predictor of mortality using the discharge score was similar in patients with pneumonia, malaria, anaemia, hypoglycaemia, diarrhoea, vomiting, other medical conditions and children less than 12 months who did not breastfeed. Odds ratios for death ranged from 2.1 to 8.8 and the AUROC ranged from 0.77 to 0.98. The score was not significantly associated with mortality in the 6/78 (8%) children presenting with oedema who died (Table 5). Odds ratios for death using admission scores were almost identical for all clinical conditions (rage 1.2–1.6) with the exception of oedema (OR 0.9) (Table S2).

TABLE 5

Performance of discharge RISQ score as predictor of mortality with different clinical conditions

Clinical condition	Status	Discharge RISQ a scores by inpatient mortality				Predicting death by RISQ
		Survival		Death		OR (95% CI)	AUROC (95% CI)
		N	Mean (SD)	N	Mean (SD)
Pneumonia	Yes	159	1.4 (1.2)	20	7.1 (4.0)	3.4 (2.2, 6.2)	0.93 (0.84, 1.00)
	No	306	1.5 (1.3)	28	5.2 (3.2)	2.5 (1.9, 3.4)	0.87 (0.78, 0.95)
Malaria	Yes	53	1.8 (1.4)	10	6.2 (2.1)	8.8 (2.9, 72.3)	0.98 (0.95, 1.00)
	No	412	1.4 (1.3)	38	5.9 (3.9)	2.4 (1.9, 3.2)	0.87 (0.79, 0.95)
Severe Anaemia (Hgb <7.0 g/dl)	Yes	64	1.8 (1.4)	14	5.6 (3.5)	2.1 (1.5, 3.2)	0.84 (0.70, 0.98)
	No	401	1.4 (1.2)	34	6.1 (3.7)	3.1 (2.3, 4.4)	0.91 (0.84, 0.98)
Hypoglycaemia (glucose <3.0 mmoL/L)	Yes	12	2.4 (1.6)	4	4.8 (4.6)	1.4 (0.9, 2.9)	0.65 (0.20, 1.00)
	No	453	1.5 (1.3)	44	6.1 (3.5)	2.9 (2.3, 3.9)	0.91 (0.85, 0.97)
Oedema	Yes	73	2.0 (1.2)	6	2.3 (1.5)	1.3 (0.6, 2.4)	0.59 (0.32, 0.86)
	No	392	1.4 (1.3)	42	6.5 (3.5)	3.1 (2.4, 4.3)	0.93 (0.87, 0.99)
Medical condition b	Yes	36	1.8 (1.1)	6	4.8 (3.6)	2.2 (1.3, 4.8)	0.77 (0.46, 1.00)
	No	429	1.4 (1.3)	42	6.1 (3.6)	2.8 (2.2, 3.8)	0.91 (0.85, 0.97)
History of diarrhoea	Yes	356	1.4 (1.3)	31	5.7 (3.4)	2.6 (2, 3.7)	0.90 (0.83, 0.96)
	No	109	1.6 (1.2)	17	6.4 (4.0)	2.8 (1.9, 4.8)	0.88 (0.74, 1.00)
History of vomiting	Yes	251	1.4 (1.3)	26	6.2 (3.2)	2.9 (2.1, 4.4)	0.93 (0.86, 1.00)
	No	214	1.6 (1.2)	22	5.7 (4.1)	2.5 (1.8, 3.7)	0.85 (0.73, 0.96)
Breastfeeding (6–11 months)	Yes	165	0.8 (1.0)	19	6.1 (3.6)	4.0 (2.4, 7.9)	0.92 (0.82, 1.00)
	No	14	1.3 (1.1)	3	4.0 (2.0)	3.6 (1.3, 20.9)	0.91 (0.75, 1.00)

RISQ score (score E) includes heart rate, respiratory rate, oxygen saturation, oxygen therapy, respiratory effort, level of consciousness; and temperature.

HIV positive, tuberculosis, sickle cell disease, cerebral palsy, cardiac disease, trisomy, sickle cell +tuberculosis, measles; pertussis; TB + pertussis; meningitis; acute glomerulonephritis.

DISCUSSION

We developed the RISQ score to measure severity of illness in hospitalised children with SAM using mortality as a measure of severity of illness. Ten items were reviewed, three were removed and the remaining seven items – heart rate, respiratory rate, respiratory effort, oxygen saturation, oxygen use, temperature and level of consciousness make up the RISQ score. The score is simple to measure and was highest for those children who died during hospitalisation.

The four main findings relate to: score development; performance in different clinical conditions; responsiveness to changed clinical conditions; and application of the score in practice. First, using admission measurements, the RISQ score and the other candidate scores identified children at high risk of inpatient mortality. The removal of CRT and SBP and the addition of temperature and level of consciousness to the BedsidePEWS score improved score performance. We found that CRT was >3 s in three of 511 children with admission CRT measurements. Two of these children died within 48 h of admission, suggesting the clinical relevance of this assessment when prolonged; however, it did not add significantly to overall score performance and thus was excluded from the score. Children with low MUAC are known to be at higher risk of death and MUAC has been included in other predictive models for mortality in SAM. Post‐hoc exploratory analysis of MUAC as a continuous measure did not show significant association with death, despite our highly wasted study population. It is possible that MUAC is reflective of mortality risk over a longer time period than we studied or that its value is greater for identifying high‐risk children in the community that would benefit from therapeutic care. The RISQ score < 24 h before death for children who died <48 or > 48 h of admission had an AUROC of 0.93 and 0.85 respectively, without inclusion of MUAC.

Second, the RISQ score performance was similar among children with common clinical conditions associated with malnutrition. Thus, a potential advantage associated with the use of the RISQ score is that it is a diagnosis‐independent measure of severity of illness. Oedema has been highlighted as a risk for mortality in hospitalised children with malnutrition and constitutes an independent criteria for admission into CMAM programme due to the unique treatment regime required. Oedema was present in six of the 48 children who died (12.5%) – all but one of whom died more than 48 hr after admission – which differs considerably to other studies having children with oedema comprising up to 30% of mortality. Further study in larger populations of children with oedema in different contexts, with scores measured throughout their hospital stay, are needed to more accurately reflect score performance.

Third, the RISQ score was sensitive to the evolving clinical status of the hospitalised children. The scores of children ready for discharge to ambulatory care were lower than their admission scores and considerably lower than the scores of children whose conditions deteriorated throughout their admission. As many deaths occur throughout hospitalisation, the ability of the score to differentiate clinical status highlights the potential use of the score to guide care throughout hospitalisation beyond the point of admission, as we found with the BedsidePEWS. ,

Fourth, the seven items of the RISQ score were obtained using both manual and point‐of‐use portable technology. Laboratory testing of biological samples were not part of the score. The use of point‐of‐care portable technology is more often found in high‐resource settings. It is important to note that the transfer of clinical tools and technology from high‐ to limited‐resource settings is often not straightforward. Irrespective of the setting, it is essential that the chosen devices perform accurately and can withstand environmental and logistic constraints. The potential benefit of newer multi‐modal devices, becoming increasingly more available in inpatient units in low‐resource settings, is their capacity for obtaining measurements, such as pulse rate and respiratory rates, that often prove challenging to obtain manually. , Our study, conducted in a highly volatile region of Africa known for humanitarian crises, as well as other studies in LMICs, illustrates that with training, measurements and implementation of scores are feasible. ,

The WHO has revised and improved IMCI guidelines and paediatric tools suitable for resource‐limited settings. , , The recent revision of the IMCI pneumonia management guidance reflects the value of multi‐item assessments including the addition of oxygen saturation. The restrictiveness of the WHO‐defined thresholds for determining hospital admission based on specific items (i.e. respiratory rate, oxygen saturation) has been highlighted as it may lead to misclassification of children in need of inpatient care. The multi‐item RISQ score, with sub‐scores for each item, addresses some of the misclassification issues identified with current practices. Other paediatric scoring tools for low‐resource settings predating RISQ include partial or different items in their scores and have lower overall performance for predicting mortality (AUROC 0.76–0.86) than RISQ. , The score with the most similar selection of items, having a sensitivity/specificity of 96.2%/87.3% for a specific score threshold in identifying patients at risk for clinical deterioration, excludes the measurement of oxygen saturation. Inclusion of oxygen saturation in the score is important for wider‐scale clinical application as well as for identifying children in need of oxygen delivery. Lastly, a recently published study that used secondary analysis of clinical trial data of children with SAM demonstrated that clinical warning signs measured on a daily basis improved the accuracy of predicting mortality (C‐index 0.81) in complicated SAM, however the prognostic value of the different clinical signs declined over time during hospitalisation.

Strengths and limitations

The strengths of this study include the individual evaluation of clinically relevant and previously validated markers of risk and their pragmatic combination into candidate scores, applying a method we have used previously. Second, the study population and the location of the hospital were reflective of children and an environment that would most benefit from a score predictive of inpatient mortality. Third, the generalisability of the score is suggested by the findings of similar score performance in children with different clinical conditions. Fourth, preliminary evidence of responsiveness is suggested by the gradation of scores between children dying within 48 h, after 48 h and those surviving to discharge.

The main limitation of this study is that the data collection was taken from a single study site. Our findings may not generalise to other settings. Second, the data were collected from children with medical complications deemed to require hospitalisation. Including children who were not admitted to the inpatient programme may provide additional validity evidence to support prioritisation decisions for children assessed in outpatient programmes in resource limited settings. Third, as per study protocol, measuring the first set of vital signs was not to interfere in any resuscitative procedure required at admission and thus some children were provided with supplementary oxygen by the time they met the study team. Potentially lower O2 saturation or work of breathing scores would be compensated by a higher score for oxygen delivery. Conversely, the clinical data were very complete, supporting robust conclusions about the items and scores. Evaluation of interval scores may have provided a greater sense of the child's clinical trajectory.

Implications for use in CMAM programmes

The development of RISQ represents an effort to assist clinicians with timely triage and more effective prioritisation of those at risk of deterioration. Potential implications of use include rapid identification and prioritisation of malnourished children with a range of illnesses seen in hospital triage and for potentially identifying children with evolving critical illness or improving conditions in inpatient settings. The RISQ score is intended to be a foundational component of the RISQ system that will serve as an adjunct to existing processes, to help prioritise expertise to the children who may benefit most. The development of the RISQ score represents one facet of the RISQ system. The RISQ system, like the Bedside Paediatric Early Warning System, will include recommendations for care matched to local resources and integrated throughout the spectrum of care from community to inpatient.

CONCLUSION

The RISQ score is a seven‐item severity of illness score comprised of commonly measured clinical signs that, with training, can be obtained in a few minutes and requires only the availability of a pulse oximeter. The RISQ score can differentiate children at high risk of death and performs similarly well in children with different clinical conditions. Confirmatory prospective validation of the RISQ score in different levels of care within the CMAM model as well as clinical trials to evaluate the overall implementation of the score will be important next steps prior to wide‐spread implementation.

CONFLICT OF INTEREST

C. Parshuram is the named inventor of the Bedside Paediatric Early Warning System and has shares in a Decision Support Company in part owned by the Hospital for Sick Children. Other authors declare no conflicts.

Table S1

Table S2

Click here for additional data file.