Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: A data linkage study.

BACKGROUND: The mortality impact of pulse oximetry use during infant and childhood pneumonia management at the primary healthcare level in low-income countries is unknown. We sought to determine mortality outcomes of infants and children diagnosed and referred using clinical guidelines with or without pulse oximetry in Malawi. METHODS AND
FINDINGS: We conducted a data linkage study of prospective health facility and community case and mortality data. We matched prospectively collected community health worker (CHW) and health centre (HC) outpatient data to prospectively collected hospital and community-based mortality surveillance outcome data, including episodes followed up to and deaths within 30 days of pneumonia diagnosis amongst children 0-59 months old. All data were collected in Lilongwe and Mchinji districts, Malawi, from January 2012 to June 2014. We determined differences in mortality rates using <90% and <93% oxygen saturation (SpO2) thresholds and World Health Organization (WHO) and Malawi clinical guidelines for referral. We used unadjusted and adjusted (for age, sex, respiratory rate, and, in analyses of HC data only, Weight for Age Z-score [WAZ]) regression to account for interaction between SpO2 threshold (pulse oximetry) and clinical guidelines, clustering by child, and CHW or HC catchment area. We matched CHW and HC outpatient data to hospital inpatient records to explore roles of pulse oximetry and clinical guidelines on hospital attendance after referral. From 7,358 CHW and 6,546 HC pneumonia episodes, we linked 417 CHW and 695 HC pneumonia episodes to 30-day mortality outcomes: 16 (3.8%) CHW and 13 (1.9%) HC patients died. SpO2 thresholds of <90% and <93% identified 1 (6%) of the 16 CHW deaths that were unidentified by integrated community case management (iCCM) WHO referral protocol and 3 (23%) and 4 (31%) of the 13 HC deaths, respectively, that were unidentified by the integrated management of childhood illness (IMCI) WHO protocol. Malawi IMCI referral protocol, which differs from WHO protocol at the HC level and includes chest indrawing, identified all but one of these deaths. SpO2 < 90% predicted death independently of WHO danger signs compared with SpO2 ≥ 90%: HC Risk Ratio (RR), 9.37 (95% CI: 2.17-40.4, p = 0.003); CHW RR, 6.85 (1.15-40.9, p = 0.035). SpO2 < 93% was also predictive versus SpO2 ≥ 93% at HC level: RR, 6.68 (1.52-29.4, p = 0.012). Hospital referrals and outpatient episodes with referral decision indications were associated with mortality. A substantial proportion of those referred were not found admitted in the inpatients within 7 days of referral advice. All 12 deaths in 73 hospitalised children occurred within 24 hours of arrival in the hospital, which highlights delay in appropriate care seeking. The main limitation of our study was our ability to only match 6% of CHW episodes and 11% of HC episodes to mortality outcome data.
CONCLUSIONS: Pulse oximetry identified fatal pneumonia episodes at HCs in Malawi that would otherwise have been missed by WHO referral guidelines alone. Our findings suggest that pulse oximetry could be beneficial in supplementing clinical signs to identify children with pneumonia at high risk of mortality in the outpatient setting in health centres for referral to a hospital for appropriate management.

Introduction

Pneumonia remains a leading cause of death in children under 5, especially in low-income and middle-income countries (LMICs), with around 800,000 pneumonia-related deaths a year globally [1]. Incidence of clinical pneumonia is estimated to be as high as 500 episodes per 1,000 child-years in some regions, with an average of 122 episodes per 1,000 child-years in Africa in 2017 [2]. Although hospitalisation rates are increasing and hospital case fatality rates are decreasing, case fatality rates are still typically around 3%–5% in LMICs [3]. Many deaths still occur at home after care seeking [4].

Early identification and action is required to prevent more pneumonia-related deaths that currently occur in hospital, often because of late presentation [5], and at home, often because their illness severity is unrecognised by primary care healthcare providers or there are barriers to accessing secondary care.

As an objective measurement of physiological illness severity, noninvasive peripheral oxygen saturation (SpO2) measurement by pulse oximeters at outpatient primary care and first level health facilities has a potential role to aid early recognition and referral of severe pneumonia episodes for oxygen and injectable antibiotic treatment [6, 7]. The current 2014 World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) chart booklet includes pulse oximetry as optional rather than mandatory and stipulates a threshold of <90% to indicate hypoxaemia requiring immediate referral to hospital [8]. However, SpO2 < 93% has also been shown to predict fatality in high-mortality settings [9, 10].

Evidence linking outpatient primary care SpO2 measurement to hospital referral and fatality is lacking [11, 12]. We aim to address this gap by linking data from previously described community and hospital based morbidity and mortality surveillance studies [9, 13–16] with a concurrent study in the same district involving outpatient pulse oximeter use at the community health worker (CHW) and health centre (HC) levels [6]. We assess the potential added value of outpatient pulse oximetry to hospital referrals and mortality outcomes using the current <90% SpO2 threshold and a <93% threshold, in conjunction with pre-2018 Malawi guidelines in use at the time of data collection (hereafter ‘Malawi guidelines’) [17]. Additionally, we explored the theory that the inability to obtain a pulse oximetry measurement may be associated with mortality [18]. Malawi guidelines for HC cases, unlike the WHO IMCI 2014 chart booklet, included mandatory hospital referral for 2- to 59-month–old children with chest indrawing (i.e., bilateral inward pulling of the lower anterior subcostal tissue during inspiration). For CHW cases, both WHO integrated community case management (iCCM) [19] and Malawi guidelines [17] indicate referral of infants and children with chest indrawing or danger signs (Table 1). This study was agreed on after an exploratory meeting by WHO on IMCI danger signs for pneumonia [12].

Table 1

Matching of outpatient child pneumonia episodes to mortality outcome data.

		CHW Episodes with Outcome Data (N = 417), n (%)	CHW Episodes without Outcome Data (N = 6,941), n (%)	p-Valuea	HC Episodes with Outcome Data (N = 695), n (%)	HC Episodes without Outcome Data (N = 5,761), n (%)	p-Valuea
Outcome:	Deathi	16 (3.8%)	no data		13 (1.9%)	no data
	Survival	401 (96.2%)	no data		682 (98.1%)	no data
SpO2	<90%	7 (1.9%)	79 (1.2%)	0.240	65 (10.1%)	543 (10.2%)	0.971
	≥90%	362 (98.1%)	6,496 (98.8%)		578 (89.9%)	4,804 (89.8%)
	<93%	14 (3.8%)	520 (7.9%)	0.004	128 (19.9%)	1,056 (19.8%)	0.925
	≥93%	355 (96.2%)	6,055 (92.1%)		515 (80.1%)	4,291 (80.2%)
	Failed measurement	48 (11.5%)	366 (5.3%)	<0.001	52 (7.5%)	414 (7.2%)	0.776
	Measured	369 (88.5%)	6,575 (94.7%)		643 (92.5%)	5,347 (92.8%)
Chest indrawing		16 (3.8%)	105 (1.5%)	<0.001	241 (34.7%)	1,457 (25.3%)	<0.001
Danger signsii (WHO guidelines clinically eligible for referral = same as Malawi guidelines at CHW level)		41 (9.8%)	962 (13.9%)	0.020	110 (15.8%)	612 (10.6%)	<0.001
Abnormally sleepy		2 (0.5%)	11 (0.2%)	0.129	6 (0.9%)	74 (1.3%)	0.343
Baby apnoeic		no data	no data		9 (1.3%)	18 (0.3%)	<0.001
Had convulsions		1 (0.2%)	266 (3.8%)	<0.001	5 (0.7%)	58 (1.0%)	0.467
Not breastfeeding or drinking		3 (0.7%)	73 (1.1%)	0.514	20 (2.9%)	143 (2.5%)	0.530
Vomiting everything		17 (4.1%)	554 (8.0%)	0.004	no data	no data
Stridor when calm		no data	no data		17 (2.5%)	165 (2.9%)	0.529
HIV exposure/infection		no data	no data		20 (3.0%)	93 (1.7%)	0.018
Swelling of both feet		6 (1.4%)	77 (1.1%)	0.536	no data	no data
Malnutrition (clinical)		no data	no data		7 (1.0%)	40 (0.7%)	0.359
SAM (MUAC < 11.5 cm, ≥6 months old)		6 (2.1%)	31 (0.5%)	0.001	10 (2.1%)	49 (1.1%)	0.063
Malawi guidelines clinically eligible for referraliii		41 (9.8%)	962 (13.9%)	0.020	275 (39.6%)	1,690 (29.3%)	<0.001
Sex	Male (% not missing)	218 (53.2%)	3,325 (48.4%)	0.062	363 (56.2%)	2,751 (51.6%)	0.027
	Female (% not missing)	192 (46.8%)	3,541 (51.6%)		283 (43.8%)	2,581 (48.4%)
	Missing data (% total)	7 (1.7%)	74 (1.1%)		49 (7.1%)	429 (7.5%)
Age [months] mean (SD, min–max)		10 (7, 2–58)	24 (15, 0–59)	<0.001	8 (7, 1–48)	16 (12, 0–59)	<0.001
Missing data (n, %)		16 (3.8%)	682 (9.8%)		0 (0%)	0 (0%)
Respiratory Rateiv:	Normal (% not missing)	3 (0.8%)	84 (1.4%)	0.380	27 (4.1%)	164 (3.0%)	0.125
	Fast (% not missing)	359 (93.0%)	5,716 (93.6%)		571 (85.7%)	4,665 (84.9%)
	Very fast (% not missing)	24 (6.2%)	309 (5.1%)		68 (10.2%)	669 (12.2%)
	Missing data (% total)	31 (7.4%)	832 (12.0%)		29 (4.2%)	262 (4.6%)
WAZv:	Normal (>−2 z-scores)	no data	no data		561 (88.6%)	4,395 (84.3%)	0.011
	Low (−3 to −2 z-scores)				44 (7.0%)	545 (10.5%)
	Severely low (<−3 z-scores)				28 (4.4%)	276 (5.3%)
	Missing data (n, %)				62 (8.9%)	544 (9.4%)

iWithin 30 days of being seen at CHW or HC level (all were 0–7 days).

iiA composite indicator variable coded as yes (1) if any of the danger signs in the 10 rows below are present (6 for CHWs and 8 for HCs)—this is equivalent to WHO 2014 iCCM guidelines clinically eligible for referral for community (CHW episodes) and to IMCI guidelines clinically eligible for referral for HC episodes. The 4 danger signs with no data for the CHW episodes were not assessed by the CHWs because they are not part of the iCCM guidelines: baby apnoeic and stridor because they require clinical training beyond CHW level to assess, HIV because testing is not available at community level, and ‘malnutrition (clinical)’ is covered by ‘swelling of both feet’ above. The 2 danger signs with no data for the HCs were not assessed by the HC workers because they are not part of the IMCI guidelines: ‘vomiting everything’ because it is covered under a full assessment of ‘not breastfeeding and drinking’ above and ‘swelling of both feet’ because it is covered by full assessment of ‘malnutrition (clinical)’ below. Please note danger signs denoting referral are different for 0- to 2-month–old infants, and the danger signs variables were coded for these very young infants accordingly, both for CHW and HC episodes; in particular, it is important to note that chest indrawing in 0- to 2-month–old infants is a danger sign requiring referral in IMCI (HC episodes). In iCCM guidelines, chest indrawing is a danger sign requiring referral for all children aged 0–59 months; therefore, WHO guidelines (iCCM) are the same as Malawi guidelines for CHW episodes.

iiiA composite variable coded as yes (1) if chest indrawing or any of the danger signs in the 10 rows above are present (6 for CHWs and 8 for HCs).

ivFast breathing: ≥60 and ≤79, ≥50 and ≤69, ≥40 and ≤59 breaths per minute for <2, 2–11, and 12–59 months of age categories; very fast breathing: ≥80, ≥70, and ≥60 breaths per minute for <2, 2–11, and 12–59 months of age categories.

vWAZ calculated from age in months and weight in kilograms, using WHO growth curves for males and females separately via the zanthro user-written add-on function in Stata. Percentages without missing data shown.

aChi-squared test for categorical variables (sex) missing data category excluded, t test for continuous variables (age, weight).

Abbreviations: CHW, community health worker; HC, health centre; iCCM, integrated community case management; IMCI, integrated management of childhood illness; MUAC, mid-upper arm circumference; SAM, Severe Acute Malnutrition; SpO2, oxygen saturation; WAZ, Weight for Age Z-score; WHO, World Health Organization.

We hypothesised that pulse oximetry use for outpatients could identify children at risk of dying who would not be identified by clinical signs alone.

Methods

Our objectives were to determine whether pulse oximetry at outpatient CHW and HC primary care levels identifies infant and child pneumonia patients for referral to hospital independently of clinical signs included in the Malawi and WHO guidelines for CHW and HC patients (Table 1) and the fatality outcomes at 30 days postdiagnosis. To determine fatality, we linked CHW and HC outpatient pneumonia data sets of 0- to 59-month–olds in Lilongwe and Mchinji districts, Malawi (Fig 1) from 1st Jan 2012 to 30th June 2014 [6, 13] to hospital [13] data for the same time period. We also linked the outpatient data to community-surveillance mortality data available for the Mchinji district only [14]. We developed and then followed our prespecified analysis plan (S1 Appendix) as far as we were able given the limitations of the final matched data set. We added the sensitivity and specificity analyses at the request of the statistical reviewer. This study is reported as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline (S1 Checklist).

Fig 1

Map of the study area.

CHW, community health worker.

Setting

In the Malawi healthcare system, children are intended to access care at either village clinics or HCs. At the village clinic level, if the child is found to be referral eligible, then the child is expected to be referred by the CHW to either the HC or hospital. At the HC level, children meeting referral criteria are referred to hospital.

Sample characteristics

Children were aged 0–59 months with a clinical pneumonia diagnosis according to routine data prospectively collected by 38 CHWs and providers from 18 HCs in rural Lilongwe and Mchinji districts, Malawi (Table 1) [6].

Pulse oximetry measurements

Healthcare providers underwent a 1-day training in pulse oximetry, medical record keeping, and the definition of pneumonia at the start of the study period; they had continued support through monthly mentorship visits. SpO2 measurements were taken using the Lifebox device (Acare Technology, Xinzhuang, Taiwan, China), with a universal adult clip probe applied to the child’s big toe if less than 2 years of age or below 10 kg. Otherwise, for older or heavier children, providers were instructed to use either the big toe or an appropriately sized finger. A paediatric probe was not available during this time period. CHW and HC workers were trained and retrained in the use of pulse oximetry by a paediatric pulmonologist (EDM) as described by McCollum and colleagues [6]. Providers were trained to record measurements that demonstrated consistent plethysmography waveforms along with a stable, nondrifting SpO2 and age-appropriate pulse rate. Given this work was conducted within a routine clinical context, providers were not required to repeat measurements meeting these quality criteria. This study showed moderate agreement in sequentially obtained SpO2 readings between EDM and each cadre of health workers [6].

Matching

Matching of the CHW and HC data sets with the hospital and population mortality surveillance outcome data was done using the following parameters: child name, caregiver or parent name, age at known date or date of birth, address, and PCV13 vaccination dates, using a probabilistic algorithm (S2 Appendix, pp. 1–3). We compared outpatient pneumonia episodes successfully matched to 30-day fatality data to those remaining unmatched to assess the representativeness of the matched sample.

Analysis

We constructed sets of 6 mutually exclusive and complete groupings of episodes according to whether they met SpO2 thresholds for referral or failed attempted SpO2 measurement (defined as no stable reading after 5 minutes of measurement [6]), clinical referral criteria, both, or neither (Table 2).

Table 2

Mortality outcomes by SpO2 and danger sign exposure group sets.

CHW Data
N = 417, n (col %)	Malawi guidelines (= WHO guidelines), <90% SpO2 threshold (this was used by the healthcare workers)	Died within 30 days, n (row %)	N = 417, n (col %)	Malawi guidelines (= WHO guidelines), <93% SpO2 threshold	Died within 30 days, n (row %)
329 (78.9%)	NOT Malawi clinically eligible and SpO2 > 90%	12 (3.6%)	324 (77.7%)	NOT Malawi clinically eligible and SpO2 ≥ 93%	12 (3.7%)
33 (7.9%)	Malawi clinically eligible only and SpO2>90%	2 (6.1%)	31 (7.4%)	Malawi clinically eligible only and SpO2 ≥ 93%	1 (3.2%)
4 (1.0%)a	SpO2 <90% only and not Malawi clinically eligiblea	1 (25.0%)a	9 (2.2%)a	SpO2 < 93% only and not Malawi clinically eligiblea	1 (11.1%)a
3 (0.7%)	SpO2 <90% and Malawi clinically eligible	0 (0.0%)	5 (1.2%)	SpO2 <93% and Malawi clinically eligible	1 (20.0%)
5 (1.2%)	failed SpO2 measurement but Malawi clinically eligible	0 (0.0%)	5 (1.2%)	failed SpO2 measurement but Malawi clinically eligible	0 (0.0%)
43 (10.3%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (2.3%)b	43 (10.3%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (2.3%)b
HC Data
N = 695, n (col %)	Malawi guidelines, <90% SpO2 threshold (this was used by the healthcare workers)	Died within 30 days, n (row %)	N = 695, n (col %)	Malawi guidelines, <93% SpO2 threshold	Died within 30 days, n (row %)
387 (55.7%)	NOT Malawi clinically eligible and SpO2 > 90%	2 (0.5%)	363 (52.2%)	NOT Malawi clinically eligible and SpO2 ≥ 93%	2 (0.6%)
191 (27.5%)	Malawi clinically eligible only and SpO2 > 90%	4 (2.1%)	152 (21.9%)	Malawi clinically eligible only and SpO2 ≥ 93%	3 (2.0%)
15 (2.2%)a	SpO2 <90% only and not Malawi clinically eligiblea	0 (0.0%)a	39 (5.6%)a	SpO2 < 93% only and not Malawi clinically eligiblea	0 (0.0%)a
50 (7.2%)	SpO2 < 90% and Malawi clinically eligible	6 (12.0%)	89 (12.8%)	SpO2 < 93% and Malawi clinically eligible	7 (7.9%)
34 (4.9%)	failed SpO2 measurement but Malawi clinically eligible	0 (0.0%)	34 (4.9%)	failed SpO2 measurement but Malawi clinically eligible	0 (0.0%)
18 (2.6%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (5.6%)b	18 (2.6%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (5.6%)b
N = 695, n (col %)	WHO guidelines, <90% SpO2 threshold	Died within 30 days, n (row %)	N = 695, n (col %)	WHO guidelines, <93% SpO2 threshold	Died within 30 days, n (row %)
512 (73.7%)	NOT WHO clinically eligible and SpO2 ≥ 90%	4 (0.8%)	461 (66.3%)	NOT WHO clinically eligible and SpO2 ≥ 93%	3 (0.7%)
66 (9.5%)	WHO clinically eligible only and SpO2 ≥ 90%	2 (3.0%)	54 (7.8%)	WHO clinically eligible only and SpO2 ≥ 93%	2 (3.7%)
41 (5.9%)a	SpO2 < 90% only and not WHO clinically eligiblea	3 (7.3%)a	92 (13.2%)a	SpO2 < 93% only and not WHO clinically eligiblea	4 (4.3%)a
24 (3.5%)	SpO2 < 90% and WHO clinically eligible	3 (12.5%)	36 (5.2%)	SpO2 < 93% and WHO clinically eligible	3 (8.3%)
20 (2.9%)	failed SpO2 measurement but WHO clinically eligible	0 (0.0%)	20 (2.9%)	failed SpO2 measurement but WHO clinically eligible	0 (0.0%)
32 (4.6%)b	failed SpO2 measurement and not WHO clinically eligibleb	1 (3.1%)b	32 (4.6%)b	failed SpO2 measurement and not WHO clinically eligibleb	1 (3.1%)b

aHypoxaemic cases and deaths identified with pulse oximetry that would not have been identified using clinical guidelines alone.

bCases and deaths identified by failure of attempted pulse oximetry that would not have been identified using clinical guidelines alone.

Abbreviations: CHW, community health worker; HC, health centre; SpO2, oxygen saturation; WHO, World Health Organization.

We described the distribution of deaths in the matched data set and crude differences in fatality for each of the 6 groupings in the SpO2 and clinical guidelines exposure sets (Table 2). We determined the independent associations of SpO2 and danger sign exposures on fatality, using generalised linear models (GLMs) of the binomially distributed binary outcome, with a log link (Eq 1); these are analogous to logistic regression but produce Risk Ratios (RRs), which are easier to interpret than the odds ratios produced by logistic regression [20]. We ran GLMs for the matched CHW and HC data separately, i.e., for each of the 6 exposure sets. The base-case unadjusted model using <90% SpO2 and Malawi guidelines clinical referral criteria thresholds (Model M90, see Table 3), is where μ is the probability of death for individual i, Y1 is the outcome, death, for individual i, and the exponent of β1 is the modelled parameter of interest: the relative risk of death when SpO2 is measured at <90% (X1_1 = 1) compared to when it is measured at ≥90% (X1_1 = 0). Children whose SpO2 reading failed are separately categorised (X1_1 = 2), not shown for simplicity), controlling for presence of Malawi guidelines clinical signs (X2_1). We constructed separate models for SpO2 < 93% (X1_2) (Model M93) and, for HC data for which WHO guidelines are different from Malawi guidelines, for WHO (X2_2) guidelines (Models W90 and W93; see Table 3).

Table 3

Independent associations of SpO2 and danger sign exposures on mortality, unadjusted GLM regression results.

CHW Data
Model	Coefficient	RR	95% CI	p-value
M90. Malawi (= WHO) guidelines, <90% SpO2 threshold (this was used by the healthcare workers) (N = 409)	SpO2 ≥ 90%	1 (ref)
	<90%	6.85	(1.15–40.9)	0.035
	failed	0.64	(0.08–4.78)	0.662
	Malawi danger signs: absent	1 (ref)
	present	1.66	(0.39–7.11)	0.494
	SpO2 < 90% × danger signs	(empty)a
	failed SpO2 × danger signs	(empty)a
	constant (baseline risk)	0.036	(0.021–0.064)	<0.001
M93. Malawi (= WHO) guidelines, <93% SpO2 threshold (N = 412)	SpO2 ≥ 93%	1 (ref)
	<93%	3.00	(0.44–20.7)	0.264
	failed	0.63	(0.84–4.71)	0.651
	Malawi danger signs: absent	1 (ref)
	present	0.87	(0.12–6.48)	0.893
	SpO2 <90% × danger signs	2.07	(0.81–52.9)	0.661
	failed SpO2 × danger signs	(empty)b
	constant (baseline risk)	0.037	(0.021–0.065)	<0.001
HC Data
Model	Coefficient	RR	95% CI	p-value
M90. Malawi guidelines, <90% SpO2 threshold (this was used by the healthcare workers) (N = 646)	SpO2 ≥ 90%	1 (ref)
	<90%	5.73	(1.68–19.5)	0.005
	failed	10.8	(1.02–113.1)	0.048
	Malawi danger signs: absent	1 (ref)
	present	4.05	(0.75–21.9)	0.104
	SpO2 <90% × danger signs	(empty)c
	failed SpO2 × danger signs	(empty)c
	constant (baseline risk)	0.005	(0.001–0.021)	<0.001
M93. Malawi guidelines, <93% SpO2 threshold (N = 622)	SpO2 ≥ 93%	1 (ref)
	<93%	3.99	(1.06–15.0)	0.041
	failed	10.1	(0.96–106.1)	0.054
	Malawi danger signs: absent	1 (ref)
	present	3.58	(0.60–21.2)	0.160
	SpO2 < 90% × danger signs	(empty)c
	failed SpO2 × danger signs	(empty)c
	constant (baseline risk)	0.006	(0.001–0.022)	<0.001
W90. WHO guidelines, <90% SpO2 threshold (N = 675)	SpO2 ≥ 90%	1 (ref)
	<90%	9.37	(2.17–40.4)	0.003
	failed	4.00	(0.46–34.8)	0.209
	WHO danger signs: absent	1 (ref)
	present	3.88	(0.72–20.8)	0.113
	SpO2 < 90% × danger signs	0.44	(0.04–4.23)	0.478
	failed SpO2 × danger signs	(empty)d
	constant (baseline risk)	0.008	(0.003–0.021)	<0.001
W93. WHO guidelines, <93% SpO2 threshold (N = 675)	SpO2 ≥ 93%	1 (ref)
	<93%	6.68	(1.52–29.4)	0.012
	failed	4.80	(0.51–44.9)	0.169
	WHO danger signs: absent	1 (ref)
	present	5.69	(0.97–33.3)	0.054
	SpO2 < 93% × danger signs	0.34	(0.03–3.30)	0.350
	failed SpO2 × danger signs	(empty)d
	constant (baseline risk)	0.007	(0.002–0.020)	<0.001

× = interaction term. Please note that we know these models are correctly specified because they predict the observed mortality rates for each category shown in Table 2.

(empty) = no deaths in this group, so coefficient was not possible to estimate.

aSee Table 2, CHW data, left orange panel, n = 3 and 0 deaths in group ‘SpO2 < 90% but Malawi clinically eligible’ and n = 5 and 0 deaths in group ‘failed SpO2 measurement but Malawi clinically eligible’.

bSee Table 2, CHW data, right yellow panel, n = 5 and 0 deaths in group ‘failed SpO2 measurement but Malawi clinically eligible’.

cSee Table 2, HC data, top left orange panel and top right yellow panel, n = 34 and 0 deaths in group ‘failed SpO2 measurement but Malawi clinically eligible’ and n = 15 (SpO2 < 90%) or n = 39 (SpO2 < 93%) and 0 deaths in group ‘SpO2 < 90% (<93%) only and not Malawi clinically eligible’.

dSee Table 2, HC data, bottom left green panel and bottom right blue panel, n = 20 and 0 deaths in group ‘failed SpO2 measurement but WHO clinically eligible’.

Abbreviations: CHW, community health worker; GLM, generalised linear model; HC, health centre; ref, reference (baseline) category; RR, risk ratio; SpO2, oxygen saturation; WHO, World Health Organization.

We adjusted for confounding by age, sex, respiratory rate, and, in analyses of HC data only, Weight for Age Z-score (WAZ). Missing data prevented us from including maternal age, education, marital status, and wealth quintile as potential confounders. Too few deaths in each exposure group prevented assessment of effect modification by age group, sex, or CHW or HC level. Although we adjusted for clustering of illness episodes by child and CHW and HC catchment area, these models were unstable and not presented. Our unadjusted models were robust to clustering, with similar headline results following 100 iterations for models that did not converge.

The extent of missing data on outcomes due to the majority of outpatient episodes remaining unmatched to 30-day postdiagnosis survival (mortality) outcomes (Table 1) meant that multiple imputation of the missing outcome data was not feasible. The small numbers of deaths and episodes with low SpO2 also precluded our planned regression discontinuity analyses of the effect of changing the SpO2 threshold on mortality and referral outcomes.

Separately to the fatality outcome, we determined the association between our SpO2/clinical sign exposures and hospital referral as the outcome (Y2) using the same logistic regression Eq (1) except substituting the fatality outcome (Y1) with Y2. Because not all severely ill children referred to hospital actually arrive, as a sensitivity analysis, we repeated this analysis with referral decision from the outpatient exposure data set regardless of actual hospital arrival as the outcome (Y3).

We calculated the sensitivity, specificity, and diagnostic odds ratio (DOR, with 95%CI) [21] of clinical and SpO2 eligibility on the mortality outcome and compared it to the sensitivity, specificity, and DOR of clinical eligibility only for SpO2 eligibility thresholds of <90% and <93% SpO2 (WHO and Malawi eligibilities) at CHW and HC levels. We included SpO2 failed measurements as well as SpO2 below threshold as SpO2 eligible in these analyses, given failed SpO2 measurements are associated with mortality.

Ethics statement

Because data were deidentified and analysed anonymously, no authorisation or waiver of authorisation by patients for the release of individually identifiable protected health information was required. This study is a data linkage study, and the data it links together are from studies approved by the ethics boards of University College London (protocol 2006/002), the Malawi National Health Sciences Research Committee (protocols 941 and 837), and the London School of Hygiene & Tropical Medicine (protocol 6047), as detailed in the published research articles from the original studies [6, 9, 13–16].

Results

We successfully matched 417/7,357 (5.7%) CHW diagnosed pneumonia episodes and 695/6,456 (10.8%) HC pneumonia episodes to outcome data (Fig 2). Sixteen (3.8%) of the 417 CHW patients and 13 (1.9%) of the 695 HC patients died within 7 days of diagnosis. There were no deaths between 8 and 29 days after the initial assessment. A further 9 CHW and 17 HC children were recorded as dying between 30 and 1,537 days after outpatient diagnosis and are included in the survival at 30 days groups. There were up to 7 pneumonia episodes in the same child in both CHW and HC data sets combined (mean: 1.2, SD: 0.6), with 1,112 episodes amongst 931 children, though only 36 children had multiple care-seeking episodes, and 32 of these had multiple episodes within 30 days of each other (S2 Appendix Table A). These episodes are all retained in the analysis given the relevance of each opportunity for case management. SpO2 < 90% hypoxemia was more common in HC pneumonia episodes (608/6,459 = 9.4%) than CHW episodes (88/7,358 = 1.2%, Table 1).

Fig 2

Pneumonia episodes matched to outcome data.

CHW, community health worker; SpO2, oxygen saturation.

There were some differences in exposures (SpO2 measurements, chest indrawing, danger signs) and potential confounders (age, sex, respiratory rate, WAZ) between matched and unmatched episodes (Table 1). CHW patients with matched outcome data were less likely to have SpO2 recorded and less likely to have saturations below 93% when recorded (though there was no significant difference in the proportions with saturations below 90%; Table 1). CHW patients with outcome data were less likely to have been recorded as vomiting everything and having convulsions, and consequently as having danger signs eligible for referral according to WHO and Malawi guidelines (which are the same at CHW level), than CHW patients not matched to outcome data (Table 1), though they were more likely to have been recorded as having severe acute malnutrition. CHW patients with matched outcome data were also more likely to be male (53%) and younger (mean age 10 months old) than those not matched to outcome data (48% male, mean age 24 months old).

HC patients with matched outcome data were not found to have any significant differences in SpO2 categories compared with those without matched outcome data (Table 1). HC patients with outcome data were more likely to have been recorded as HIV exposed/infected and of being apnoeic, and consequently (though there were no other significant differences across the danger signs) as having danger signs eligible for referral according to WHO guidelines (16%), than HC patients not matched to outcome data (11%; Table 1). HC patients with matched outcome data were more likely to have chest indrawing recorded (35%) than those unmatched to outcome data (25%) and consequently more likely to have Malawi guideline danger signs eligible for referral (40%) than those unmatched to outcome data (29%; Table 1). HC patients with matched outcome data were also more likely to be male (56%), were younger (mean age 8 months old) than those not matched to outcome data (52% male, mean age 16 months old), and were less likely to have low WAZs, on average (Table 1).

Table 2 CHW data show the distribution of the 16 deaths at CHW level and risk of death for each of the 2 SpO2 and danger sign exposure group sets. Those who were not clinically or SpO2 eligible for referral had lower mortality rates than those SpO2-eligible only (3.6%–3.7% versus 11%–25%).

Table 2 HC data show the distribution of the 13 deaths at HC level and risk of death for each of the 4 SpO2 and danger sign exposure group sets. In all 4 scenarios, those who are not clinically or SpO2 eligible for referral had lower mortality rates than those both clinically and SpO2 eligible (0.5%–0.8% versus 7.9%–12.5%). SpO2-eligible only episodes had higher mortality than clinically eligible only episodes when applying WHO IMCI chart booklet criteria, especially using the <90% SpO2 threshold (7.3% versus 3.0%).

For HC episodes, SpO2 < 90% and <93%, respectively, identified 3 (23%) and 4 (31%) of the 13 deaths that would not have been identified using WHO IMCI guidelines alone (Table 2). Malawi guidelines, which include chest indrawing, identified all of these deaths, and 10 (77%) of the 13 HC deaths in total compared with 5 (38%) of the 13 HC deaths identified by WHO IMCI guidelines (Table 2).

For CHW episodes, SpO2 < 90% and <93% both identified 1 (6%) of the 16 deaths that would not have been identified using iCCM/Malawi guidelines alone (Table 2). Twelve (75%) of the 16 deaths amongst CHW episodes were not identified by iCCM guidelines, which only identified 2 (12.5%) of the 16 deaths (Table 2). These 12 deaths also had SpO2 ≥ 93%. Failed SpO2 measurement independently identified 1 (6%) of the 16 CHW deaths and 1 (8%) of the 13 HC deaths (Table 2).

The unadjusted GLM analyses of the associations of SpO2 and danger sign exposures with death predict the observed mortality rates for each exposure group. In CHW episodes (Table 3), there is an increased risk of death for those with SpO2 < 90% than those with SpO2 ≥ 90%, independent of the presence of danger signs (RR: 6.85, 95% CI: 1.15–40.9, p = 0.035). The SpO2 < 93% threshold was not associated with an increased risk of death compared to those with SpO2 ≥ 93% (RR: 3.00, 95% CI: 0.44–20.7, p = 0.264)

In HC episodes, only 2–4 of 13 deaths occurred in the ‘reference’ SpO2 ≥ 90%/93% no-danger–signs groups (Table 2), and both the SpO2 < 90% (RR: 9.37 [95% CI: 2.17–40.4]) and <93% (RR: 6.68 [95% CI: 1.52–29.4]) thresholds are significantly associated with mortality in both the Malawi and WHO IMCI guideline models (Table 3).

Adjusting the models for age in months, sex, respiratory rate, and, for HC data, WAZ produced broadly similar results to the unadjusted models in Table 3 (S2 Appendix Table B and associated text).

Table 4 shows the outpatient referral decision indication by healthcare providers and linked hospital inpatient records for each of the SpO2 and danger sign exposure sets. For CHWs, only 9.3% (39) of the 417 episodes had outpatient referral decisions, and these were more common in patients who were clinically or SpO2 eligible (33.3%–100%) than those who were not (3.6%–7.0%). Only 0.7% (3) of the CHW episodes were found to be hospital inpatients within 7 days of outpatient diagnosis (Table 4), precluding regression analysis of this outcome in CHW episodes. In the HC data (Table 4), 30.4% (211) of the 695 patients had an outpatient referral decision. Notably, for Malawi guidelines, including <90% SpO2 threshold (i.e., what was used by the healthcare workers in practice), healthcare workers referred 90.6% who were eligible by both criteria, 62.3% who were clinically eligible only, and only 20.0% of those eligible because of SpO2 < 90% only. Seventy (10.1%) of the 695 HC episodes were hospital inpatients within 7 days; 60 of these followed an outpatient referral decision (28% of the 211 referral decisions were admitted as inpatients within 7 days).

Table 4

Outpatient referral decision indication and hospital inpatients within 7 days by SpO2 and danger sign exposure group sets.

CHW Data
N = 417, n (%)	Malawi guidelines (= WHO guidelines), <90% SpO2 threshold (this was used by the healthcare workers)	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)	N = 417, n (%)	Malawi guidelines (= WHO guidelines), <93% SpO2 threshold	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)
329 (78.9%)	NOT Malawi clinically eligible and SpO2 > 90%	12 (3.6%)	1 (0.3%)	324 (77.7%)	NOT Malawi clinically eligible and SpO2 ≥ 93%	12 (3.7%)	1 (0.3%)
33 (7.9%)	Malawi clinically eligible only and SpO2 > 90%	15 (45.5%)	1 (3.0%)	31 (7.4%)	Malawi clinically eligible only and SpO2 ≥ 93%	13 (41.9%)	1 (3.2%)
4 (1.0%)a	SpO2 < 90% only and not Malawi clinically eligiblea	3 (75.0%)a	0 (0.0%)a	9 (2.2%)a	SpO2 <93% only and not Malawi clinically eligiblea	3 (33.3%)a	0 (0.0%)a
3 (0.7%)	SpO2 < 90% and Malawi clinically eligible	3 (100%)	1 (33.3%)	5 (1.2%)	SpO2 < 93% and Malawi clinically eligible	5 (100%)	1 (20.0%)
5 (1.2%)	failed SpO2 measurement but Malawi clinically eligible	3 (60.0%)	0 (0.0%)	5 (1.2%)	failed SpO2 measurement but Malawi clinically eligible	3 (60.0%)	0 (0.0%)
43 (10.3%)b	failed SpO2 measurement and not Malawi clinically eligibleb	3 (7.0%)b	0 (0.0%)b	43 (10.3%)b	failed SpO2 measurement and not Malawi clinically eligibleb	3 (7.0%)b	0 (0.0%)b
HC Data
N = 695, n (%)	Malawi guidelines, <90% SpO2 threshold (this was used by the healthcare workers)	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)	N = 695, n (%)	Malawi guidelines, <93% SpO2 threshold	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)
387 (55.7%)	NOT Malawi clinically eligible and SpO2 > 90%	12 (3.1%)	7 (1.8%)	363 (52.2%)	NOT Malawi clinically eligible and SpO2 ≥ 93%	9 (2.5%)	5 (1.4%)
191 (27.5%)	Malawi clinically eligible only and SpO2 > 90%	119 (62.3%)	39 (20.4%)	152 (21.9%)	Malawi clinically eligible only and SpO2 ≥ 93%	86 (56.6%)	27 (17.8%)
15 (2.2%)a	SpO2 < 90% only and not Malawi clinically eligiblea	3 (20.0%)a	0 (0.0%)a	39 (5.6%)a	SpO2 < 93% only and not Malawi clinically eligiblea	6 (15.4%)a	2 (5.1%)a
50 (7.2%)	SpO2 < 90% and Malawi clinically eligible	45 (90.0%)	16 (32.0%)	89 (12.8%)	SpO2 < 93% and Malawi clinically eligible	78 (87.6%)	28 (31.5%)
34 (4.9%)	failed SpO2 measurement but Malawi clinically eligible	31 (91.2%)	6 (17.6%)	34 (4.9%)	failed SpO2 measurement but Malawi clinically eligible	31 (91.2%)	6 (17.6%)
18 (2.6%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (5.6%)b	2 (11.1%)b	18 (2.6%)b	failed SpO2 measurement and not Malawi clinically eligibleb	1 (5.6%)b	2 (11.1%)b
N = 695, n (%)	WHO guidelines, <90% SpO2 threshold	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)	N = 695, n (%)	WHO guidelines, <93% SpO2 threshold	Outpatient referral decision, n (%)	Hospital inpatients within 7 days, n (%)
512 (73.7%)	Not WHO clinically eligible and SpO2 ≥ 90%	101 (19.7%)	33 (6.4%)	461 (66.3%)	Not WHO clinically eligible and SpO2 ≥ 93%	74 (16.1%)	23 (5.0%)
66 (9.5%)	WHO clinically eligible only and SpO2 ≥ 90%	30 (45.5%)	13 (19.7%)	54 (7.8%)	WHO clinically eligible only and SpO2 ≥ 93%	21 (38.9%)	9 (16.7%)
41 (5.9%)a	SpO2 < 90% only and not WHO clinically eligiblea	27 (65.9%)a	9 (22.0%)a	92 (13.2%)a	SpO2 < 93% only and not WHO clinically eligiblea	54 (58.7%)a	19 (20.7%)a
24 (3.5%)	SpO2 < 90% and WHO clinically eligible	21 (87.5%)	7 (29.2%)	36 (5.2%)	SpO2 < 93% and WHO clinically eligible	30 (83.3%)	11 (30.6%)
20 (2.9%)	failed SpO2 measurement but WHO clinically eligible	18 (90.0%)	4 (20.0%)	20 (2.9%)	failed SpO2 measurement but WHO clinically eligible	18 (90.0%)	4 (20.0%)
32 (4.6%)b	failed SpO2 measurement and not WHO clinically eligibleb	14 (43.8%)b	4 (12.5%)b	32 (4.6%)b	failed SpO2 measurement and not WHO clinically eligibleb	14 (43.8%)b	4 (12.5%)b

aHypoxaemic episodes identified with pulse oximetry that would not have been identified using clinical guidelines alone.

bEpisodes identified by failure of attempted pulse oximetry that would not have been identified using clinical guidelines alone.

Abbreviations: CHW, community health worker; HC, health centre; SpO2, oxygen saturation; WHO, World Health Organization.

Table 5 CHW data show both low SpO2 and danger signs are associated with outpatient referral decisions for CHWs. Table 5 HC data show the same for HC episodes and that failed SpO2 readings were also associated with referral decisions in WHO guideline models. Referrals were also associated with low SpO2 under SpO2 < 90% and <93% thresholds and danger signs according to both Malawi and WHO IMCI protocols in HC episodes (Table 5). The interaction terms between the SpO2 and danger sign parameters were below 1 and statistically significant for the following HC referral models (Table 5): M90 and M93, outpatient referral decision indication outcome, SpO2 < 90% × danger signs; hospitalisation outcome, failed SpO2 × danger signs; and W90, outpatient referral decision indication outcome, SpO2 < 90% × danger signs. These results indicate that the association between the SpO2 parameter and the outcome was attenuated by the presence of danger signs. Hospital referrals and outpatient episodes with referral decision indications were associated with mortality (see S2 Appendix, p. 11 for further details).

Table 5

Independent associations of SpO2 and danger sign exposures on referrals, unadjusted GLM regression results.

CHW Data
		Outcome = outpatient referral decision indication
Model	Coefficient	RR	(95% CI)	p-value
M90. Malawi (= WHO) guidelines, <90% SpO2 threshold (this was used by the healthcare workers) (N = 409)†	SpO2 ≥ 90%	1 (ref)
	<90%	79.3	(7.67–819.0)	0.002
	failed	1.98	(0.54–7.3)	0.305
	Malawi danger signs: absent	1 (ref)
	present	22.0	(8.99–53.9)	<0.001
	SpO2 < 90% × danger signs	(empty)a
	failed SpO2 × danger signs	0.91	(0.09–9.24)	0.935
	constant (baseline risk)	0.038	(0.021–0.067)	<0.001
M93. Malawi (= WHO) guidelines, <93% SpO2 threshold (N = 412)	SpO2 ≥ 93%	1 (ref)
	<93%	9.00	(3.06–26.5)	<0.001
	failed	1.88	(0.55–6.41)	0.311
	Malawi danger signs: absent	1 (ref)
	present	11.3	(5.66–22.6)	<0.001
	SpO2 < 93% × danger signs	(empty)b
	failed SpO2 × danger signs	0.76	(0.17–3.32)	0.715
	constant (baseline risk)	0.037	(0.021–0.064)	<0.001
HC Data
		Outcome = outpatient referral decision indication			Outcome = hospitalisation (within 7 days)
Model	Coefficient	RR	(95% CI)	p-value	RR	(95% CI)	p-value
M90. Malawi guidelines, <90% SpO2 threshold (this was used by the health workers) (left: N = 695; right: N = 680)	SpO2 ≥ 90%	1 (ref)			1 (ref)
	<90%	6.45	(2.03–20.5)	0.002	1.56	(0.96–2.56)	0.073
	failed	1.79	(0.25–13.0)	0.565	6.14	(1.37–27.5)	0.018
	Malawi danger signs: absent	1 (ref)			1 (ref)
	present	20.09	(11.4–35.5)	<0.001	11.3	(5.15–24.8)	<0.001
	SpO2 < 90% × danger signs	0.22	(0.07–0.72)	0.012	(empty) c
	failed SpO2 × danger signs	0.82	(0.11–5.98)	0.842	0.14	(0.03–0.76)	0.023
	constant (baseline risk)	0.031	(0.018–0.054)	0.000	0.018	(0.009–0.038)	<0.001
M93. Malawi guidelines, <93% SpO2 threshold (N = 695)	SpO2 ≥ 93%	1 (ref)			1 (ref)
	<93%	6.21	(2.33–16.5)	<0.001	3.72	(0.75–18.6)	0.109
	failed	2.24	(0.30–16.7)	0.432	8.07	(1.68–38.8)	0.009
	Malawi danger signs: absent	1 (ref)			1 (ref)
	present	22.8	(11.8–44.2)	<0.001	12.9	(5.06–32.9)	<0.001
	SpO2 < 93% × danger signs	0.25	(0.09–0.67)	0.006	0.48	(0.09–2.53)	0.383
	failed SpO2 × danger signs	0.72	(0.10–5.41)	0.749	0.12	(0.02–0.72)	0.020
	constant (baseline risk)	0.025	(0.013–0.047)	<0.001	0.014	(0.006–0.033)	<0.001
W90. WHO guidelines, <90% SpO2 threshold (N = 695)	SpO2 ≥ 90%	1 (ref)			1 (ref)
	<90%	3.34	(2.52–4.42)	<0.001	3.41	(1.75–6.62)	<0.001
	failed	2.22	(1.44–3.41)	<0.001	1.94	(0.73–5.14)	0.183
	WHO danger signs: absent	1 (ref)			1 (ref)
	present	2.30	(1.68–3.16)	<0.001	3.06	(1.70–5.50)	<0.001
	SpO2 < 90% × danger signs	0.58	(0.38–0.87)	0.009	0.43	(0.15–1.22)	0.114
	failed SpO2 × danger signs	0.89	(0.53–1.51)	0.672	0.52	(0.13–2.12)	0.364
	constant (baseline risk)	0.197	(0.166–0.235)	<0.001	0.064	(0.046–0.090)	<0.001
W93. WHO guidelines, <93% SpO2 threshold (N = 695)†	SpO2 ≥ 93%	1 (ref)			1 (ref)
	<93%	7.43	(4.58–12.1)	<0.001	4.14	(2.35–7.28)	<0.001
	failed	4.07	(1.94–8.54)	<0.001	2.51	(0.92–6.81)	0.072
	WHO danger signs: absent	1 (ref)			1 (ref)
	present	3.33	(1.82–6.07)	<0.001	3.34	(1.63–6.84)	0.001
	SpO2 < 93% × danger signs	1.06	(0.34–3.31)	0.924	0.44	(0.17–1.15)	0.096
	failed SpO2 × danger signs	3.47	(0.62–19.6)	0.157	0.48	(0.11–2.06)	0.322
	constant (baseline risk)	0.191	(0.149–0.245)	<0.001	0.050	(0.033–0.074)	<0.001

(empty) = no referrals in this group, so coefficient was not possible to estimate

× = interaction term. Please note that we know these models are correctly specified because they predict the observed referral rates for each category shown in Table 4.

†The CHW M90 and HC W93 outpatient referral decision GLMs with binomial family and log link did not converge; therefore, we report the analogous logistic regression models for these 2 analyses. These models report results in ORs rather than RRs. The ORs are more extreme than the RRs, especially for the HC outpatient referral decision outcome, which is relatively common (30%: 211 of 695 episodes; the outpatient referral decision is less common for CHW episodes: 9%: 39 out of 417 episodes).

aSee Table 4 CHW data, top left orange panel, n = 4 and 0 referrals in group ‘SpO2 < 90% only and not Malawi clinically eligible’.

bSee Table 4, CHW data, top right yellow panel, n = 9 and 0 referrals in group ‘SpO2 < 93% only and not Malawi clinically eligible’.

cSee Table 4, HC data, orange panel, n = 15 and 0 referrals in group ‘SpO2 < 90% only and not Malawi clinically eligible’.

Abbreviations: CHW, community health worker; GLM, generalised linear model; HC, health centre; OR, Odds Ratio; ref, reference (baseline) category; RR, Risk Ratio; SpO2, oxygen saturation; WHO, World Health Organization.

Table 6 shows the sensitivity, specificity, and DOR of pulse oximetry with clinical signs versus clinical signs only in identifying deaths. We focus on sensitivity because the objective is to identify as many of the deaths as possible. At the CHW level, using both SpO2 thresholds, the sensitivity was 25% compared with a sensitivity of 12.5% for Malawi clinical iCCM criteria only. At the HC level, both <90% and <93% SpO2 eligibility thresholds give a sensitivity of 85% compared with a sensitivity of 77% for Malawi clinical IMCI criteria only. At the HC level, in scenarios using WHO clinical criteria, 9 of the 13 deaths are identified by the <90% SpO2 eligibility threshold, and 10 of the 13 deaths are identified by the <93% SpO2 eligibility thresholds. These sensitivities of 69% and 77% compare with a sensitivity of 38% for WHO clinical IMCI criteria only. Given the small numbers of deaths, the differences in sensitivity between the scenarios are not statistically significant. The DORs show that at the CHW level, neither pulse oximetry with clinical signs (Table 6: <90% SpO2 threshold, DOR 1.26 [95% CI: 0.40–4.00] and <93% SpO2 threshold, DOR 1.17 [95% CI: 0.37–3.71]) nor iCCM clinical signs alone (DOR: 1.33 [95% CI: 0.29–6.05]) accurately identified those who died. At the HC level, pulse oximetry was able to accurately identify those who die with both Malawi (<90% SpO2 threshold, DOR: 7.13 [95% CI: 1.57–32.4]; <93% SpO2 threshold, DOR: 6.19 [95% CI: 1.36–28.1]) and WHO (<90% SpO2 threshold, DOR: 6.57 [95% CI: 2.00–21.6]; <93% SpO2 threshold, DOR: 6.82 [95% CI: 1.86–25.0]) IMCI clinical signs. Although the point estimates for these DORs are higher than those for when Malawi and WHO IMCI clinical signs alone are used without pulse oximetry (Malawi: DOR 5.25 [95% CI: 1.43–19.2), WHO: DOR 3.43 [95% CI: 1.10–10.7]), the confidence intervals are wide and overlapping, indicating these differences are not statistically significant.

Table 6

Sensitivity and specificity of pulse oximetry with clinical signs versus clinical signs only in identifying patients who die.

CHW Data
	Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity		Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity
iCCM severe (including hypoxaemia <90% & failed SpO2)	4	84	5%	96%	1.26 (0.40–4.00)	25%	79%	iCCM severe (including hypoxaemia <93%)	4	89	4%	96%	1.17 (0.37–3.71)	25%	78%
iCCM nonsevere	12	317						iCCM nonsevere	12	312
versus
iCCM severe	2	39	5%	96%	1.26 (0.40–4.00)	12.5%	90%
iCCM nonsevere	14	362
HC Data
	Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity		Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity
Malawi IMCI severe (including hypoxaemia <90%)	11	297	4%	99%	7.13 (1.57–32.4)	85%	56%	Malawi IMCI severe (including hypoxaemia <93%)	11	321	3%	99%	6.19 (1.36–28.1)	85%	53%
IMCI nonsevere	2	385						IMCI nonsevere	2	361
versus
Malawi IMCI severe	10	265	4%	99%	5.25 (1.43–19.2)	77%	61%
Malawi IMCI nonsevere	3	417
	Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity		Died	Alive	PPV	NPV	DOR (95% CI)	Sensitivity	Specificity
WHO IMCI severe (including hypoxaemia <90%)	9	174	5%	99%	6.57 (2.00–21.6)	69%	74%	WHO IMCI severe (including hypoxaemia <93%)	10	224	4%	99%	6.82 (1.86–25.0)	77%	67%
IMCI nonsevere	4	508						IMCI nonsevere	3	458
versus
WHO IMCI severe	5	105	5%	99%	3.43 (1.10–10.7)	38%	85%
WHO IMCI nonsevere	8	577

Alive denotes 30-day survival. DORs over 1 discriminate properly, i.e., those who have the feature are more likely to have the outcome. Abbreviations: CHW, community health worker; DOR, diagnostic odds ratio; HC, health centre; iCCM, integrated community case management; IMCI, integrated management of childhood illness; NPV, Negative Predictive Value; PPV, Positive Predictive Value; WHO, World Health Organization.

Discussion

We show the important role of confirmed hypoxaemia in identifying otherwise unrecognised fatal childhood pneumonia episodes accessing primary healthcare centres by applying the 2014 WHO IMCI clinical signs, which do not include chest indrawing as a referral sign, in children aged 2–59 months old. Pulse oximeter use will identify hypoxaemia in up to 30% of episodes that are not identified by clinical signs alone [22]. The 2014 WHO IMCI recommends use of pulse oximetry when available and referral to the hospital if SpO2 is less than 90%; however, pulse oximetry is not frequently available at the outpatient level. With low-cost portable pulse oximeters becoming much more available compared to 2014, especially in the context of the ongoing COVID pandemic [23], it is likely that they will be available more widely and used more often at the outpatient level than before. Even so, it will further facilitate pulse oximetry use if WHO were to recommend them without any conditions to evaluate pneumonia episodes.

Pre-2018 Malawi guidelines for community and outpatient case management [17] were consistent with the previous WHO guidelines [19, 24], i.e., chest indrawing is considered a referral criteria. This criteria was removed as a referral sign in the revised 2014 WHO IMCI chart booklet [8] following controlled low-mortality trials that indicated chest indrawing episodes could be managed at home without increased risk [25, 26], and Malawi made this change too in 2018. Since then, 3 other randomised trials—one each in India, Kenya, and Malawi—have demonstrated that chest indrawing pneumonia can be treated with oral amoxicillin on an outpatient basis [27-29]. However, for CHWs, chest indrawing remains a referral sign. The pre-2018 Malawi IMCI protocol identified 2-fold more (10/13 = 77%) of the HC patients who died within 7 days of outpatient diagnosis than the 2014 WHO IMCI protocol (5/13 = 38%). These data and recent data from hospitalised children in Kenya showed that in routine care settings, presence of chest indrawing may require referral in some high-mortality settings [30]. Because not much is known about other factors that influenced adverse outcomes in these children with chest indrawing, such as care seeking and appropriate and timely treatment, further research is required to examine the beneficial effects of hospitalisation of children with chest indrawing in routine settings.

An abnormal SpO2 measured by pulse oximetry most commonly indicates ventilation–perfusion mismatch from a pulmonary illness and is considered a more objective measurement than observing chest indrawing, which can be subjective and hard to see [31]. Therefore, although chest indrawing identified all of the deaths from HC episodes and all but one of the deaths from CHW episodes identified by pulse oximetry, pulse oximetry may lead to more reliable decision-making. However, pulse oximetry can also be done poorly, has cost implications, and still needs to be conducted in conjunction with a thorough clinical assessment. Additionally, a ‘normal’ SpO2 reading should be considered an adjunct to, not in lieu of, recognition of clinical danger signs such as inability to drink when making decisions about hospital referral. Whilst pulse oximetry in conjunction with clinical signs can identify more fatal pneumonia episodes than clinical signs alone, this increased sensitivity of diagnosis comes at the price of reduced specificity (Table 6) and therefore has potential to overwhelm weak hospital systems via identifying additional patients for referral to hospital. Detailed understanding of local hospital system capacities will be fundamental when deciding the appropriate SpO2 threshold for referring patients. Ultimately, further research is needed to examine these trade-offs, though our research indicates that both pulse oximetry and chest indrawing are important in a high-child–mortality setting.

Notably, 12 (75%) of the 16 deaths in CHW episodes that occurred within 7 days of diagnosis were neither hypoxaemic nor clinically eligible for referral highlighting the importance of follow-up and continuing case management, as well as more specific diagnostic approaches (S2 Appendix, pp. 11–12 has further detail on these episodes). CHWs may also have missed danger signs, meaning support for training, supportive supervision, and mentoring for iCCM may be important in this context [32]. The use of adult probes for pulse oximetry in our study may have contributed to missed hypoxaemic episodes. The increasing availability of paediatric probes may provide greater sensitivity to detect children who may die of pneumonia.

The CHW and HC workers in our study were instructed to follow the Malawi guidelines and <90% SpO2 threshold for identifying outpatient episodes for referral. Nevertheless, we found that SpO2 readings of <90% and <93% and, at the HC level, both WHO and Malawi clinical danger signs were associated with a healthcare provider decision to refer the child and hospitalisation within 7 days of outpatient diagnosis. Though most of the hospitalisations followed an outpatient referral decision, these represent only 2 (5%) of 39 episodes with an outpatient referral decision at the CHW level and 60 (28%) out of 211 with an outpatient referral decision at HC level. This is likely to be due to a combination of difficulties with transport, finances or family circumstances precluding travel to hospital, or hospitals not admitting referred patients [33, 34]. It could also be due to incomplete hospital inpatient records and issues with matching outpatient episodes to inpatient records, preventing us from knowing of some successful hospitalisations (e.g., referrals to hospitals outside of our study area). When referral is not feasible, prompt treatment at the CHW and HC level with oral antibiotics is required as minimum care, pending injectable antibiotics and oxygen therapy as required.

Outpatient referral decision, and hospitalisations within 7 days of outpatient diagnosis were associated with mortality. All 12 deaths amongst the 73 hospital referrals occurred on the same day of hospitalisation and were from HC episodes (only 3 of the 73 hospitalisations were from CHW episodes). Nine of the 12 hospital deaths were on the same day as outpatient diagnosis and one the following day, suggesting rapid deterioration, delayed care seeking and presentation at the HC, or a combination of all three [4, 35].

At the CHW level, 13 (81%) of the 16 deaths were in patients for whom outpatient HC referral was not recommended, generally reflecting correct guideline application; i.e., the CHW did not record any clinical signs indicating they should be referred. Of these 13 deaths, none were subsequently hospitalised, and all died within 1–5 days of diagnosis by a CHW. This suggests they died at home whilst receiving oral antibiotic treatment. At the HC level, 4 (31%) of the 13 deaths were in episodes not referred to the hospital, although only one met referral criteria. All of these 4 deaths were eventually hospitalised. These episodes all present missed opportunities for intervention at outpatient primary care.

Our findings on referral decision-making are similar to those we previously reported using the whole CHW and HC data sets [6], suggesting our matched sample is representative of the wider sample in this respect. Our finding that significantly more patients with both clinical and SpO2 eligibility were referred than those who were either only clinically or SpO2 eligible suggests healthcare workers at the outpatient level considered both pulse oximetry and clinical signs for referral decision-making rather than one or the other. More research to understand how referral decisions are determined when pulse oximeters are available is needed. Training is also required to re-emphasise the importance of following both clinical guidelines and SpO2 when making decisions to refer patients. Devices with in-built decision support or that are complemented by mHealth tools could play a role in supporting appropriate case management and should be evaluated for both clinical outcomes and changes in care seeking and referral outcomes.

Our main limitation was our ability to only match 6% of CHW episodes and 11% of HC episodes to mortality outcome data. Given the geographical coverage and age differences of the data sets, we were able to match 29% of the CHW episodes and 24% of the HC episodes for which matching was potentially feasible (S2 Appendix, page 3). Although the samples of CHW and HC episodes matched to mortality outcome data had some differences in characteristics from those unmatched to mortality outcome data, the characteristics of our sample and our findings are generalisable to similar LMIC populations likely to be found in high-mortality settings. The matched patients were younger on average than the unmatched patients; the mortality surveillance data only included children born after October 2011, and therefore, older children were only present in the clinical data set. The lower proportions with danger signs and SpO2 < 93% in the matched CHW data compared with the unmatched CHW data suggest that the matched CHW patients were less sick on average, and therefore, we may have missed some deaths. On the other hand, HC matched patients were sicker on average than the unmatched HC patients, with greater proportions having danger signs, although there were no significant differences in SpO2 categories (Table 1).

Given the similarity of the unadjusted and adjusted results, the fact the regression models already lack precision due to the small numbers of deaths, and that we were only able to use age, sex, and respiratory rate as confounders, any propensity score matching analysis to balance such potential measured confounders across exposure groups was deemed futile and abandoned. Additionally, given the low matching rates, multiple imputation of missing data was not included because it produced imputations, and consequent regression coefficient estimates, that were too unstable. Analyses of effect modification were also not possible because of the small number of deaths in each category.

It is estimated that the implementation of IMCI enhanced with pulse oximetry may result in 103,000–145,000 pneumonia deaths averted in the 15 highest-burden countries [36], though this modelling study was based on assumptions about associations with mortality [36]. To help place our results in context and aid policy and decision-making, a cost-effectiveness analysis and economic evaluation of outpatient pulse oximetry—based on our results and cost, benefit (disability adjusted life years averted), and population parameters—is forthcoming. Further research looking at an approach in which chest indrawing episodes without other danger signs are referred or not solely based on SpO2 is also warranted, as is research to evaluate the optimum SpO2 threshold for action. Similar analyses evaluating pulse oximetry in low-mortality LMICs are necessary.

Pulse oximetry use by health workers at HCs can identify hypoxaemic pneumonia patients who go on to die who would otherwise be missed by current referral guidelines, especially if they do not include chest indrawing and pulse oximeters are unavailable. With low-cost portable pulse oximeters becoming much more available now compared to 2014, the WHO/UNICEF IMCI protocol could recommend pulse oximeters for pneumonia case management, instead of making a conditional recommendation of its use when available. As pulse oximetry use increases in outpatient settings, critical next steps include addressing oxygen availability at clinics and during transportation to hospitals. Our analysis also indicates that timely and appropriate care seeking by families and prompt management by health workers is key to survival of children with pneumonia.

STROBE checklist for reporting of observational studies.

STROBE, Strengthening the Reporting of Observational Studies in Epidemiology.

(DOCX)

Click here for additional data file.

Prespecified analysis plan.

(DOCX)

Click here for additional data file.

Additional supporting information and results referred to in the manuscript.

(DOCX)

Click here for additional data file.

17 Feb 2020

Dear Dr Colbourn,

Thank you for submitting your manuscript entitled "Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Helen Howard, for Clare Stone PhD

Acting Editor-in-Chief

PLOS Medicine

plosmedicine.org

19 Mar 2020

Dear Dr. Colbourn,

Thank you very much for submitting your manuscript "Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi" (PMEDICINE-D-20-00470R1) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Apr 09 2020 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Adya Misra, PhD

Senior Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

Title: Please revise your title according to PLOS Medicine's style. Your title must be nondeclarative and not a question. It should begin with main concept if possible. "Effect of" should be used only if causality can be inferred, i.e., for an RCT. Please place the study design ("A randomized controlled trial," "A retrospective study," "A modelling study," etc.) in the subtitle (ie, after a colon).

Abstract- Please structure your abstract using the PLOS Medicine headings (Background, Methods and Findings, Conclusions). Funding is not required here.

Abstract methods and findings- please provide demographics, places where this study took place in Malawi along with dates

Abstract Methods and Findings:

* Please ensure that all numbers presented in the abstract are present and identical to numbers presented in the main manuscript text.

* Please include the study design, population and setting, number of participants, years during which the study took place, length of follow up, and main outcome measures.

* Please quantify the main results (with 95% CIs and p values).

* Please include the important dependent variables that are adjusted for in the analyses.

Abstract methods and findings- the last sentence should include a limitation of your study design

Abstract conclusions- * Please address the study implications without overreaching what can be concluded from the data; the phrase "In this study, we observed ..." may be useful. * Please interpret the study based on the results presented in the abstract, emphasizing what is new without overstating your conclusions. * Please avoid vague statements such as "these results have major implications for policy/clinical care". Mention only specific implications substantiated by the results. * Please avoid assertions of primacy ("We report for the first time....")

Please remove the “research in context” section. At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary.

The Data Availability Statement (DAS) requires revision. For each data source used in your study:

a) If the data are freely or publicly available, note this and state the location of the data: within the paper, in Supporting Information files, or in a public repository (include the DOI or accession number).

b) If the data are owned by a third party but freely available upon request, please note this and state the owner of the data set and contact information for data requests (web or email address). Note that a study author cannot be the contact person for the data.

c) If the data are not freely available, please describe briefly the ethical, legal, or contractual restriction that prevents you from sharing it. Please also include an appropriate contact (web or email address) for inquiries (again, this cannot be a study author).

Square brackets placement- please add a space between text and square brackets followed by a full stop. For example: xxxxxx [3-5].

Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section. a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript. b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place. c) In either case, changes in the analysis-- including those made in response to peer review comments-- should be identified as such in the Methods section of the paper, with rationale.

Please ensure that the study is reported according to the STROBE guideline, and include the completed STROBE checklist as Supporting Information. Please add the following statement, or similar, to the Methods: "This study is reported as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline (S1 Checklist)." The STROBE guideline can be found here: http://www.equator-network.org/reporting-guidelines/strobe/ When completing the checklist, please use section and paragraph numbers, rather than page numbers.

For all observational studies, in the manuscript text, please indicate: (1) the specific hypotheses you intended to test, (2) the analytical methods by which you planned to test them, (3) the analyses you actually performed, and (4) when reported analyses differ from those that were planned, transparent explanations for differences that affect the reliability of the study's results. If a reported analysis was performed based on an interesting but unanticipated pattern in the data, please be clear that the analysis was data-driven.

Role of funding source should be added into the financial statement within the article metadata and removed from the main text

Please conclude the Introduction with a clear description of the study question or hypothesis.

Conclusions must be toned down since this is an observational study

Please present and organize the Discussion as follows: a short, clear summary of the article's findings; what the study adds to existing research and where and why the results may differ from previous research; strengths and limitations of the study; implications and next steps for research, clinical practice, and/or public policy; one-paragraph conclusion.

S1 Appendix table A1- is the date seen relevant? I imagine this is identifying along with the number of episodes, age and whether they were seen by CHW or at HC. Please amend this table. Manuscripts submitted to PLOS should not contain research participants personally-identifying information. In rare exceptions where this is unavoidable and a manuscript does contain PII, the authors should be willing and able to provide PLOS with confirmation of GDPR compliance upon request.

Map of Malawi- you may consider adding this into the main text as it gives an immediate visualisation of where the data were collected from

Comments from the reviewers:

Reviewer #1: This is a very important piece of work that merits consideration by the journal as it tries to assess the "… mortality outcomes of infants and children diagnosed and referred using clinical guidelines with or without pulse oximetry in Malawi". Recent advances in the wider scale up and implementation of pulseoximetry need to go hand in hand with data like the one presented in this manuscript, which are clear-cut. Pulseoximetry helps identify children at risk of dying. Authors conclude that "Pulse oximetry identified fatal pneumonia episodes at HCs in Malawi that would otherwise have been missed by WHO referral guidelines alone", which is a statement with which I do agree. I believe the journal should consider (if finally accepting the manuscript) the inclusion of a comment to go hand in hand with the manuscript. I only have a few minor comments to add:

* The word data should be used in plural

* Current global estimates for Pneumonia mortality are closer to 800,000 than to 900,000/year

* The fact that mortality as an endpoint was assessed at day 7 but also at day 30 means that many of these deaths may not (or may indeed) be related to the initial episode that took the patient to the health system (particularly for those at day 30). This should be further discussed, but it doesn't eliminate the validity of the association and therefore of the use of oxygen saturation as a risk stratification tool

* In this respect, is there any information (for example of verbal autopsy results) regarding the potential underlying causes of those deaths? It would be very helpful to be able to state that an elevated proportion of those deaths were secondary to respiratory problems

* I understand vaccination data were also collected. Was there any association between mortality and lack of adequate vaccination against Hib or pneumococcus?

* The findings that association with mortality is maintained with threshold 93% is also very important but has the risk to overload the (already fragile) system if all patients fulfilling this criterion are recommended for a transfer. Have you been able to conduct any economic modeling of costs associated with the two scenarios (transfer only if sat<90%, or transfer when <93%?). This would be super helpful, as I predict that costs to the system would be massively increased (and be unassumable). Were there other specific variables associated to mortality within the specific group with saturations between 90-93% which were also associated with mortality? Could the recommendation include a need to fulfil at least two of the risk factors, rather than one?

* Noting is mentioned on the absence/availability of emergency oxygen/systems for transfers. Recognizing this is a major deficit in the health systems of LMICS, it may be worth stating it as an important consideration for the future. There is an ethical dilemma of measuring oxygen saturation but not being able to provide life-saving oxygen, at least for the transport. This is perhaps the most neglected field in pneumonia research, how to ensure cheap and durable availability of oxygen for emergency transport

* Pulseoximter readings are very variable and often can produce false positive results of hypoxemia values which are not real. I understand that there was a specific training conduted, but perhaps it may be useful to add in the methods section how values obtained in the peripheral health system were considered "reliable" and robust (i.e whether you had to repeat more than once the reading, or you had to ensure the heart rate was also considered credible etc…). This is particularly important with one of your conclusions in relation to the association with mortality: "the failure to obtain an SpO2 measure using pulse oximeters in identifying otherwise unrecognized fatal childhood pneumonia cases accessing primary care". Could some of those cases be failures of the measurement technique? Of the devices used? It may seem very obvious, but this appears important to me.

Reviewer #2: PLOS Medicine Colbourn SpO2 Malawi review

The lowest denomination of low-cost factors to determine poor outcome is required. Oxygen saturation monitors will be increasingly available for use in developing health and an assessment of their impact is required.

This first report is very useful for informing this field and provides structure to the reporting of future studies.

Comments

Methods

Describe the process for patients to access and move through the HC system (i.e. HCW, HC, OP referral, Hospital in patients).

Results

P10 L227. This manuscript would be helped by a figure or initial text in the results that provide cascade detail to the patients, matching and deaths to provide better context to what is reported.

i.e. N = 13814 pneumonia episodes, 6941 CHW + 5761 HC in which SpO2 <90% found in 86/6941 (1.2%) and 608/5761 (10.5%) respectively. There was matching of pneumonia episodes to mortality cohort in N=1112, 417/6941 CHW and 695/5761 HC. of which there were 29 deaths representing 2.6% of the matched episodes (16/417 CHW and 13/608).

P10 L252 The case for chest indrawing is well made.

Page 11 L259

In light of the finding of a discrepancy in the SpO2 <90% and deaths in CHW cohort (missing 75% of deaths), please provide discussion on the use of an adult probe in a paediatric community setting and the potential for error from extraneous light and the difficulty in positioning. Please also consider discussion that

Table 1

The results identify that an SpO2 threshold of 93% would result in a 6 times higher referral rate by CHW and double referral rate by HC. Please discuss the potential impact of this on the ability to deliver safe and effective care to children with pneumonia with increased sensitivity to identify severe disease, but the potential impact of reduced sensitivity having an economic and hospital bed impact that could reduce the ability to priortise services for those with severe disease.

Discussion

Key finding is that study identifies that clinical criteria including chest indrawing together with SpO2 <90% fail to identify 75% of deaths from pneumonia in children. It is difficult to understand that 75% of deaths occurred in children whose parents were concerned enough to attended services so early that they had no severe signs and no desaturation, but then did not re-attend when things got worse. As this represents such a large proportion of 'inaccessible pneumonia deaths' it would be useful to consider how this may be resolved in a future study, i.e red flag information.

Please discuss that health seeking behaviours are orientated to younger children and it is older children that do not attend out patient services after referral. Whilst this is regrettable, parents so appear to understand that younger children are at greater risk.

Overall the study is of value for providing an indication of the potential value of pulse oximetry in childhood pneumonia - but large missing data and relatively few deaths limit the strength of conclusions despite rigorous analysis. The conclusions could be more circumspect. Line 300 - This evidence 'could' support… The evidence in itself is not strong enough to support the inclusion at this time. Line 420 'should' is inappropriate based on this evidence alone.

Reviewer #3: Alex McConnachie, Statistical Review

Colbourn et al consider the potential for the use of pulse oximetry for identifying children likely to die from pneumonia in Malawi. This review looks at the use of statistics in the paper.

Unfortunately, I have a number of concerns.

A major problem is the very poor linkage rate, which casts doubt on the validity of the results. The authors comment that there were some differences between those matched and not matched, with reference to the tables and appendix, but should perhaps expand on what these differences were in the main text.

My main concern with the analysis is the regression modelling. There were problems with convergence, and the models that it had been intended to fit (allowing for clustering, and including additional covariates and interaction terms) were not possible. There were generally few events, and the models that were fitted produced estimates with very wide confidence intervals. Some models reported in the supplement only worked using a logit link. All of these factors suggest that these models are unstable, and I would not trust these results.

Note that the description of the model is not entirely correct. On line 184, mu.i is not death for subject i, it is the probability of death; Yi is not the log risk of death, it is the outcome, death, for subject i.

Nevertheless, there may be valuable data here, but the tables as presented are quite complex, and are not easy to follow. A simpler approach might be to report more standard measures of diagnostic performance, i.e. sensitivity, specificity, PPV, NPV. That might show, in terms that are easily understood, which combinations of screening criteria (WHO, Malawi, SpO2 <90%, or <93%) worked best. I doubt that the sample size is enough to test say whether any differences are statistically significant, but might make the case for additional research.

Overall, the conclusions of the paper, that SpO2 measurements (or lack of), and chest-indrawing should be included in screening criteria, seems premature given these data.

Minor points on Table 1: p values of "0.000" should be reported as "<0.001", and there is no need to add asterisks - the actual p-values are reported.

Table 2 is confusing - some percentages are in columns, some in rows, but as stated above, if the focus were changed to measures of diagnostic performance, that might help/

Tables 3 and 5 mix up risks and odds, with the constant term referred to as baseline odds.

I note that individual consent was not obtained for this study, but individual patient data is reported in the supplement, which could potentially identify individuals.

Reviewer #4: page 6 the phrase chest in drawing might likely is the same as retractions, a term more familiar to north american. Please add

page 7 it is customary to add the name and address and even model number of a device used in research. Please add

Also, please detail why an adult universal probe was used in young children. did it work better than pediatric one or thats all there was? Thank you.

Page 10. Please define what SpO2 eligible means

Page 13. Please expand the discussion about specific limitation with SpO2 measure in your specific setting. It's what people would likely want to read in the paper.

Any attachments provided with reviews can be seen via the following link:

[LINK]

29 May 2020

Submitted filename: Response_to_reviewers_Outpatient pulse oximetry mortality.docx

Click here for additional data file.

17 Jul 2020

Dear Dr. Colbourn,

Thank you very much for submitting your manuscript "Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: a data linkage study" (PMEDICINE-D-20-00470R2) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Jul 27 2020 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Clare Stone, PhD

Acting Chief Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

As you will see, the statistical referee raises significant issues and advises us to reject the manuscript. We have discussed this and feel that we would like to offer another revision opportunity. There is a sense that the presentation is simply trying to stretch limited data too far in producing clinical recommendations - for example, under "what do these findings mean?" all three points are "should be used" or similar, and it seems that "our findings suggest that pulse oximetry could be beneficial... and should be further investigated", say, would be more sensible.

In addition, please do address all of the points from ref 3 (as well as from other refs). Ref 3 is our statistician and we see these points as important. I realise in some instances toning down will be a compromise.

Comments from the reviewers:

Reviewer #1: all my comments have been addressed, and I'm pleased how authors have rewritten the manuscript. I'm also happy how they have dealt with other reviewer's comments. I still think, however, that this manuscript would benefit from an accompanying commentary.

Reviewer #2: Thank you for responding to the primary review. The responses were very helpful and provide clarity.

I have one remaining comment. Whilst I note that the authors have provided detail on the training for CHW in pulse oximetry, that '12 (75% of the 16 deaths) 'CHW may have missed danger signs, meaning further support for training...' could be inferred that CHW may somehow be considered responsible for these omissions as the children were not 'hypoxemic'.

I would ask that the authors also acknowledge within their limitations, that the use of adult probes on pulse oximeters was a limitation and some of these children may have been hypoxemic at the time of review - we would not use adult probes where a paediatric probe is available - as is increasingly the case globally. The use of adult probes for young children have documented limitations. This potential technological failure to detect hypoxemia does not detract from their core message - but to me adds to it - as it implies that the use of age appropriate sensors may likely provide even greater sensitivity to detect children who may die from lower respiratory tract infection.

Reviewer #3: Alex McConnachie, Statistical Review

I thank Colbourn and colleagues for their responses to my original comments.

Seeing Table 6 (sensitivity etc.) helps me to understand the data a little better. Pulse oximetry adds sensitivity, at the expense of a loss of specificity. The results section of the paper concentrates on sensitivity, but the loss of specificity it is touched on in the discussion. No confidence intervals are provided for any of the diagnostic measures, but the authors do note that none of the differences are statistically significant.

The diagnostic odds ratios in table 6 are:

A1: 1.26, A2: 1.17, A3: 1.32

B1: 7.13, B2: 6.19, B3: 5.24

B4: 6.57, B5: 6.82, B6: 3.43

This suggests that pulse oximetry may be of no real value in the community setting, but might be in a HC setting, particularly when added to the WHO clinical criteria. It also suggests that the Malawi danger signs may perform better than the WHO danger signs on their own.

These data are limited though, because they are derived from real world data in which children are being assessed and treated. The aim of the assessments and subsequent treatment and referrals will be aiming to prevent adverse outcomes. It cannot be known how many of these children might have died in the absence of treatment. So, the actual number of true positives will be higher, and the number of false negatives lower, than suggested by simply looking at death as the outcome. Using referral and hospitalisation data as in Tables 4 and 5 does not really help, because these decisions are being made in the light of the assessments being done; in this study, there is no "gold standard" against which to assess the alternative screening criteria.

I still feel the logistic regression models are pushing the data too far. For death as the outcome, there are only 13 or 16 events being analysed, depending on the dataset, and models are being fitted (or are being attempted) with a total of 6 parameters (intercept, SpO2 (2 df), clinical danger signs, interaction (2 df)), so the events-per-variable ratio is very low. Some of the interaction terms are not estimable due to the lack of events in some subgroups. When looking at referral and hospitalisation as the outcome, there are more events, but still very few in some subgroups. The text of the paper mentions associations with low SpO2 and with clinical assessments, but these are the main effects, and each applies only in the absence of the other. The interaction terms are not taken into account, and these are generally below 1.

I think these data are interesting, and suggest that chest in-drawing and SpO2 measurements may be of value in some settings, but it is quite messy data, which was not collected with this analysis in mind.

The linkage may be highly novel, but the success rate was very low, with many differences between those linked and not linked.

Any attachments provided with reviews can be seen via the following link:

[LINK]

27 Jul 2020

Submitted filename: PMEDICINE-D-20-00470R2 Response to review.docx

Click here for additional data file.

11 Aug 2020

Dear Dr. Colbourn,

Thank you very much for re-submitting your manuscript "Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: a data linkage study" (PMEDICINE-D-20-00470R3) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

Our publications team (plosmedicine@plos.org) will be in touch shortly about the production requirements for your paper, and the link and deadline for resubmission. DO NOT RESUBMIT BEFORE YOU'VE RECEIVED THE PRODUCTION REQUIREMENTS.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Aug 18 2020 11:59PM.

Sincerely,

Clare Stone, PhD

Managing Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

Data – the data statement is now fine, but please remove “We can make available the matched/unmatched data actually used for the analysis (just the relevant variables, with personal identifiable information removed) and could upload that to a free to access public repository such as the one held by the corresponding authors university UCL. Please let us know if this is acceptable.”

Comments from Reviewers:

Any attachments provided with reviews can be seen via the following link:

[LINK]

1 Sep 2020

Submitted filename: response to editorial and production requirements.docx

Click here for additional data file.

11 Sep 2020

Dear Dr. Colbourn,

On behalf of my colleagues and the academic editor, Dr. Quique Bassat, I am delighted to inform you that your manuscript entitled "Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: a data linkage study" (PMEDICINE-D-20-00470R4) has been accepted for publication in PLOS Medicine.

PRODUCTION PROCESS

Before publication you will see the copyedited word document (in around 1-2 weeks from now) and a PDF galley proof shortly after that. The copyeditor will be in touch shortly before sending you the copyedited Word document. We will make some revisions at the copyediting stage to conform to our general style, and for clarification. When you receive this version you should check and revise it very carefully, including figures, tables, references, and supporting information, because corrections at the next stage (proofs) will be strictly limited to (1) errors in author names or affiliations, (2) errors of scientific fact that would cause misunderstandings to readers, and (3) printer's (introduced) errors.

If you are likely to be away when either this document or the proof is sent, please ensure we have contact information of a second person, as we will need you to respond quickly at each point.

PRESS

A selection of our articles each week are press released by the journal. You will be contacted nearer the time if we are press releasing your article in order to approve the content and check the contact information for journalists is correct. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact.

PROFILE INFORMATION

Now that your manuscript has been accepted, please log into EM and update your profile. Go to https://www.editorialmanager.com/pmedicine, log in, and click on the "Update My Information" link at the top of the page. Please update your user information to ensure an efficient production and billing process.

Thank you again for submitting the manuscript to PLOS Medicine. We look forward to publishing it.

Best wishes,

Clare Stone, PhD

Managing Editor

PLOS Medicine

plosmedicine.org