Respiratory rate and pulse oximetry derived information as predictors of hospital admission in young children in Bangladesh: a prospective observational study.

OBJECTIVE: Hypoxaemia is a strong predictor of mortality in children. Early detection of deteriorating condition is vital to timely intervention. We hypothesise that measures of pulse oximetry dynamics may identify children requiring hospitalisation. Our aim was to develop a predictive tool using only objective data derived from pulse oximetry and observed respiratory rate to identify children at increased risk of hospital admission.
SETTING: Tertiary-level hospital emergency department in Bangladesh. PARTICIPANTS: Children under 5 years (n=3374) presenting at the facility (October 2012-April 2013) without documented chronic diseases were recruited. 1-minute segments of pulse oximetry (photoplethysmogram (PPG), blood oxygen saturation (SpO2) and heart rate (HR)) and respiratory rate were collected with a mobile app. PRIMARY OUTCOME: The need for hospitalisation based on expert physician review and follow-up.
METHODS: Pulse rate variability (PRV) using pulse peak intervals of the PPG signal and features extracted from the SpO2 signal, all derived from pulse oximetry recordings, were studied. A univariate age-adjusted logistic regression was applied to evaluate differences between admitted and non-admitted children. A multivariate logistic regression model was developed using a stepwise selection of predictors and was internally validated using bootstrapping.
RESULTS: Children admitted to hospital showed significantly (p<0.01) decreased PRV and higher SpO2 variability compared to non-admitted children. The strongest predictors of hospitalisation were reduced PRV-power in the low frequency band (OR associated with a 0.01 unit increase, 0.93; 95% CI 0.89 to 0.98), greater time spent below an SpO2 of 98% and 94% (OR associated with 10 s increase, 1.4; 95% CI 1.3 to 1.4 and 1.5; 95% CI 1.4 to 1.6, respectively), high respiratory rate, high HR, low SpO2, young age and male sex. These variables provided a bootstrap-corrected AUC of the receiver operating characteristic of 0.76.
CONCLUSIONS: Objective measurements, easily obtained using a mobile device in low-resource settings, can predict the need for hospitalisation. External validation will be required before clinical adoption. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Features characterising pulse rate variability and blood oxygen saturation dynamics automatically derived from pulse oximetry were significantly different in children admitted to the hospital compared to those who were not admitted.

The proposed predictive model uses objective information to provide a risk score, which will allow community healthcare workers with minimal training to identify a child who may require hospital admission.

Adding pulse oximetry derived objective information to the proposed predictive model improved the identification of children at higher risk and provided improved calibration.

The proposed model has been internally validated using a bootstrap method but will require external validation.

Several pulse oximetry recordings (39%) were excluded from the study for low signal quality; thus, the app's user interface will be modified to coerce the user to improve the quality of the data.

Introduction

‘Sepsis’, a syndrome of physiological, pathological and biochemical abnormalities induced by infection, is a major public health concern and is the leading cause of death in children worldwide.1 In 2015 ∼5.9 million children under 5 years died from sepsis caused by conditions such as malaria, pneumonia, diarrhoea and measles.1 Sepsis is associated with several factors such as poverty, malnutrition, poor hygiene, low immunisation rates, poor access to care and under-resourced health systems,2 and hence its burden of mortality and morbidity is substantially higher in low and middle income countries than in the developed world.3

This disproportionate burden of mortality and morbidity from critical illnesses in children in the developing world can be prevented if early accurate diagnosis is followed by prompt treatment.3 4 Targeted interventions are needed if the sustainable development goal of 25 or fewer under-5 deaths per 1000 live births by 2030—down from 43/1000 mortality rate in 2015—is to be achieved.5

A key method to achieve this goal is to improve community case management of critical illnesses.6 In low-resource settings, this can be implemented by enabling community healthcare workers to rapidly identify critically ill children in their community allowing earlier treatment, timely referral and, ultimately, greater survival. Timely and appropriate referral will optimise use of the limited resources and can be greatly facilitated by an objective test, which identifies these children but requires a minimal amount of training or expertise.

A common feature of most treatable childhood illnesses (CI) is hypoxaemia, which correlates with disease severity and is a recognised risk factor for death.7 Including a routine and systematic approach of case finding and treating hypoxaemia is associated with improved quality of care and reduced mortality.7 8 Pulse oximetry, a simple and non-invasive method of measuring blood oxygen saturation (SpO2) and blood volume changes in tissue using the photoplethysmographic signal (PPG), is critical for the accurate detection of hypoxaemia. Thus, the use of pulse oximetry is recommended in the updated integrated management of CI (IMCI) guidelines for low and middle income countries.9 10 In addition, a reduction in heart rate variability (HRV) has been suggested as a possible early indicator of critical illness.11 HRV, traditionally calculated from the ECG signal, can be estimated using the PPG signal through a surrogate measure known as pulse rate variability (PRV).12–14 HRV and PRV have been documented to be altered in the presence of sepsis and correlated with its severity.15

Smartphones have been adopted widely, even in low-resource settings, offering an ideal platform for vital signs assessment and automatic diagnosis.16 The phone oximeter (a pulse oximeter that interfaces a finger sensor with a smartphone), for example, provides a useful tool to record pulse oximetry in low-resource settings. In a previous study, we developed a hospital admission prediction model for children aged under 5 years using clinical predictors of infectious disease and including the SpO2 recorded by the phone oximeter.17 In low-resource settings, the lack of clinical expertise for accurate identification of the signs and symptoms of infection in children is a significant roadblock to reducing mortality.4 Therefore, in this study, we aim to extend our previous work, using the same data set, by developing a tool to predict hospital admission that exclusively uses objective information, including features derived from the analysis of pulse oximetry recordings. We consider the need for hospital admission as an early indicator of critical illness, and thus this model is intended to support community healthcare workers, with limited formal training, to recognise critical illness in children earlier in the course of their disease.

Materials and methods

In order to investigate the viability of developing a hospital admission prediction test, we designed a mobile app to collect data. We then conducted a prospective observational study at the Kumudini Women's Medical College Hospital (KWMCH), a not-for-profit private tertiary-level hospital, located in the Mirzapur subdistrict of Tangail district in Bangladesh. This hospital provides primary level care to a catchment population of ∼500 000 people.

Mobile app design

We designed a customised mobile data collection app, called PhoneOxR2 (figure 1), that uses the phone oximeter for the measurement of SpO2 and HR values (figure 1A) at 1 Hz and PPG signal at 75 Hz.18 19 It also includes a respiratory rate counter interface that registers screen taps (for each inhalation) and calculates the respiratory rate after five taps.20 The hardware of the phone oximeter used in the study was an Xpod OEM pulse oximeter (Nonin Medical, Plymouth, Minnesota, USA; model 3012LP) connected to a fourth-generation iPod touch (Apple, Cupertino, California, USA). The interface of the PhoneOxR2 app displayed the PPG signal with a colour-coded (eight shades of green to red) background, which represents the real-time quality of the signal being recorded. This live quality assessment is based on the amplitude and regularity of the waveform and pulse oximeter module generated alerts.

Figure 1

(A) The pulse oximetry page of PhoneOxR2 showing a completed 1 min good quality (mostly green) recording with the median blood oxygen saturation and HR values from the recording, (B) an in progress recording in the blinded version of the app, used in the study.

Data collection

All individuals were recruited according to a protocol approved by the research review committee and ethical review committee of the international centre for diarrhoeal disease research, Bangladesh and the University of British Columbia/children's and women's health centre research ethics board. A research staff member explained the study and obtained written informed consent from the caretaker accompanying the child.

Children with chronic diseases or with previously documented low SpO2 levels due to other conditions, such as cardiac disease, were excluded.17 All children were assessed and managed by a team of physicians not involved in the study. Pulse oximetry information was blinded (not displayed) within the app (figure 1B) and not provided to the physicians, as this was not routinely available in the facility. Thus, the SpO2 recording did not influence clinical decisions, including the need for admission, which was decided at the physicians' discretion uninfluenced by the study. The children were considered admitted under the following circumstances: (1) admitted and stayed for more than 24 hours in the hospital, (2) sent home initially but admitted at a subsequent visit within 1 week, (3) were advised for admission but the advice was not followed and (4) were transferred or referred to another facility providing a higher level of care. The most common diagnoses included acute lower or upper respiratory infection, and eye or ear infection in admitted children; and acute upper respiratory infection, eye or ear infection, diarrhoea, fever of unknown cause and gastrointestinal symptoms in non-admitted children; see ref. 17 for more details of the distribution of diagnoses and outcomes. From the children sent home, we were able to follow-up 1855/2514 (73.8%) and 14 were admitted on a subsequent visit.

The data acquisition was performed, using the PhoneOxR2, between October 2012 and April 2013 (winter is from November to February). We collected 1 min recordings of the SpO2 and PPG signals along with respiratory rate from a total of 3374 children under 5 years presenting at the outpatient or emergency department. The respiratory rate was estimated from the PPG signal through a previously proposed algorithm21 in 39 cases when the respiratory tapping interface was not used.

Pulse oximetry processing

The quality of each recorded PPG signal was evaluated offline, using an algorithm that assigns a signal quality index (SQI) between 0 and 100 (100 being the best quality). The method is based on pulse segmentation and cross-correlation of consecutive pulse segments.22 PPG segments with an SQI higher than 50 were automatically identified for further analysis. The analysis was limited to children for whom there was at least 30 s of good quality (SQI>50%) PPG signal. PRV and SpO2 derived features were computed for each data recording.

PRV characterisation

We applied a peak detection algorithm based on zero-crossing to locate the pulse peaks in the PPG signal and to obtain pulse-to-pulse interval time series (PPIs).23 Five time-domain parameters were computed from the PPIs: (1) the mean of the PPIs: representing the RR interval or the time interval between two consecutive R waves in the ECG, (2) the SD of PPIs: representing SDNN or the SD of the so-called normal-to-normal (NN) intervals, (3) the root mean square of the successive differences (RMSSD) between adjacent PPIs: representing RMSSD or the RMSSD between adjacent NN intervals and (4 and 5) two standard descriptors to evaluate non-linear short-term and long-term variability (SD1 and SD2) based on the Poincaré plot of PPIs.24

For the frequency analysis, each segment of PPIs was resampled into an equivalent, uniformly spaced time series (sampling rate of 4 Hz), and the power spectral density was computed. The power in each frequency band was computed by integrating the area under the power spectral density curve bound by the band of interest: very low frequency (0.01–0.04 Hz), low frequency (LF; 0.04–0.15 Hz) and high frequency (HF; 0.15–0.4 Hz). LF and HF powers were normalised (LFn and HFn) by the total spectral power between 0.04 and 0.4 Hz. The ratio of the low-to-HF power (LF/HF ratio) was also computed.

SpO2 characterisation

Several oximetry indices proposed in previous studies to assess SpO2 dynamics were computed.25 These included the number of SpO2 desaturations >1%, 2% and 3% below baseline (nb1, nb2 and nb3, respectively), cumulative time spent below an SpO2 of 98%, 96% and 94% (tb98, tb96 and tb94, respectively) and the Delta index (SpO2delta).26 The Delta index quantifies SpO2 variability and was computed as the average of absolute differences of the mean oxygen saturation between successive 12 s intervals. The median (SpO2median) and SD (SpO2std) of the SpO2 within each segment were also calculated.

Data analysis

All data analysis was conducted using R v3.2.0 (R Foundation for statistical computing, Vienna, Austria). To facilitate interpretation, raw heart rate and respiratory rate were standardised with respect to age using population median and SD values.27

Univariate analysis

The relationship between each feature and the need for facility admission was assessed by comparing the median and IQR between the admitted and the non-admitted participants and through univariate logistic regression using the OR (95% CI). The logistic regression analysis for all PRV features was adjusted for age, a known confounding factor.28 For these features, we added age into the univariate models to compute the adjusted ORs.

Multivariate model development

A stepwise selection method was applied to select the most relevant features and to develop the final multivariate logistic regression model, which we refer to as the ‘mobile model’. The stepwise selection method added/dropped one feature at a time and stopped when further inclusion/exclusion no longer improved the model, as determined by the Akaike information criterion.29 All features were included as candidate predictors and a minimum value of 40 events per variable was maintained to avoid overfitting.30 The stepwise selection method was also compared with the least absolute shrinkage and selection operator (LASSO) method (computed via the glmnet R package31). The calibration of the model was evaluated by the Hosmer-Lemeshow goodness-of-fit test. To assess the impact of including objective information derived from pulse oximeter recordings, the performance of the ‘mobile model’ was compared to the ‘baseline-model’, a logistic regression model including only respiratory rate and the median SpO2 value.

Model validation

The model was validated using the bootstrap method, recommended as the optimal technique for estimation of internal validity of a predictive logistic regression model.30 Using the original data set, 100 bootstrap samples were generated using sampling with replacement. The stepwise selection procedure was applied to each bootstrap sample to develop a multivariate model. This model was applied to predict the outcome in the bootstrap sample and in the original data set. The difference in the prediction performance, quantified by the area under the receiver operating characteristic curve (AUC), was computed to estimate the optimism of the ‘mobile model’.

Risk threshold determination

The risk threshold for the estimated probability of facility admission was chosen to maximise a weighted classification score defined as (the number of correct identifications of true positives)+(the number of correct identifications of true negatives). The weighted classification score was computed for various previously established ratios of false negative cases to false positive cases (1:3, 1:5 and 1:10);17 however, the ratio should be predetermined in each specific clinical context.

Model classification performance

Once the risk threshold was set, the performance of the ‘mobile model’ was evaluated in terms of accuracy, sensitivity, positive predictive value (PPV) and negative predictive value (NPV) and compared to the ‘baseline model’. The reclassification technique, offering incremental information over the AUC, was employed to compare both models using the net reclassification improvement (NRI) and integrated discrimination improvement (IDI).32 The NRI quantified the performance of the ‘mobile model’ reclassifying children as compared to the ‘baseline model’. The IDI measured the improvement of the ‘mobile model’ relative to the ‘baseline model’ and was quantified as the amount of increased difference in the average predicted risk between admitted or non-admitted children.32

Results

Excluded cases

In total, 2051 (61%) of the 3374 recruited children had at least 30 s of good quality PPG signal recorded, which permitted the adequate characterisation of SpO2 and PRV. The remaining 1323 children were excluded from analysis. The percentage of excluded cases was similar in the admitted (40%) and non-admitted (38%) groups. The SpO2 values of the excluded cases were not significantly different from those included, evaluated through univariate logistic regression. Thus, there was no clear association between the patient's risk and the PPG signal quality.

Univariate analysis

Pulse oximetry derived features were significantly different in children who required admission compared to those who did not require admission (table 1). Admitted children presented a significantly higher respiratory rate, higher HR and lower PRV (reflected through reduced SDNN, RMSSD, SD1 and SD2 values) than non-admitted children. Admitted children also showed reduced PRV in the spectral domain suggested by lower power in low (LFn) and high (HFn) frequency bands, relative to non-admitted children (table 1).

Table 1

Objective feature distribution for admitted and non-admitted children

Feature	Admission required N=616 (30.0%)	Admission not required N=1435 (70.0%)	p Value	OR (95% CI)
Pulse rate variability analysis*
Heart rate (bpm)	140 (126–154)	125 (112–138)	<0.001	1.03 (1.02 to 1.03)
RR (s)	0.43 (0.39–0.47)	0.48 (0.43–0.53)	<0.001	0.49 (0.42 to 0.57)†
SDNN (s)	0.016 (0.011–0.025)	0.022 (0.014–0.037)	<0.001	0.83 (0.78 to 0.89)‡
RMSSD (s)	0.016 (0.013–0.025)	0.021 (0.015–0.039)	<0.001	0.91 (0.87 to 0.95)‡
SD1 (s)	0.012 (0.093–0.018)	0.015 (0.011–0.027)	<0.001	0.99 (0.98 to 0.99)‡
SD2 (s)	0.018 (0.011–0.029)	0.026 (0.016–0.043)	<0.001	0.98 (0.98 to 0.99)‡
LFn (nu)	0.37 (0.2–0.57)	0.38 (0.23–0.55)	0.003	0.93 (0.89 to 0.98)‡
HFn (nu)	0.14 (0.09–0.24)	0.18 (0.12–0.27)	<0.001	0.86 (0.79 to 0.94)
LF/HF (ratio)	2.3 (1.1–4.7)	2 (0.95–3.9)	0.2	1.00 (0.99 to 1.00)
SpO2 analysis
SpO2median (%)	96 (91–98)	98 (97–99)	<0.001	0.79 (0.76 to 0.81)
SpO2std (%)	0.73 (0.51–1.2)	0.54 (0.4–0.83)	<0.001	0.54 (0.4 to 0.83)
SpO2delta (%)	0.31 (0.19–0.48)	0.22 (0.13–0.34)	<0.001	3.5 (2.5 to 4.8)
tb98 (s)	39 (11–56)	3.3 (0–34)	<0.001	1.4 (1.3 to 1.4)§
tb96 (s)	26 (0–53)	0 (0–3.2)	<0.001	1.4 (1.3 to 1.5)§
tb94 (s)	2 (0–41)	0 (0–0.33)	<0.001	1.5 (1.4 to 1.6)§
nb1 (# times)	4 (1–8)	1 (0–4.5)	<0.001	1.1 (1.08 to 1.12)
nb2 (# times)	0 (0–3)	0 (0–1)	<0.001	1.1 (1.1 to 1.2)
nb3 (# times)	0 (0–1)	0 (0–0)	<0.001	1.1 (1.1 to 1.2)
Non-pulse oximetry data
Heart rate age z-score	0.66 (−0.19 to 1.7)	0.23 (−0.58 to 1.1)	<0.001	1.3 (1.2 to 1.4)
Respiratory rate age z-score	0.47 (−0.48 to 1.9)	−0.046 (−0.8 to 1.1)	<0.001	1.2 (1.1 to 1.3)
Age (days)	230 (70.5–542)	477 (202–1038)	<0.001	0.989 (0.987 to 0.991)§
Gender (% of male)	67%	59%	<0.001	1.43 (1.17 to 1.74) (male vs female)

Values include feature medians (quartiles), p value and OR (95% CI).

*Adjusted for age.

†Associated with an increase of 0.1 times the feature unit.

‡Associated with an increase of 0.01 times the feature unit.

§Associated with an increase of 10 times the feature unit.

HFn, high frequency normalised; LFn, low frequency normalised; RMSSD, root mean square of the successive differences; RR, respiratory rate; SpO2, blood oxygen saturation.

The SpO2 (median) was lower and showed more variability reflected via several features (SpO2std, delta, tb98, tb96, tb94, nb1, nb2, nb3) in admitted compared to non-admitted children. Children who required admission were also significantly younger and more likely to be male (table 1).

Final model validation

The majority of features selected by the stepwise method were also selected by the LASSO selection method. For simplicity, we decided to use the feature selection result from the stepwise method to construct the ‘mobile model’, which includes only information derived from mobile device recordings. All the selected features had p<0.05 and the ‘mobile model’ equation was: logit (probability (admission required))=3.806–0.049×(SpO2 median)+0.012×(time spent below an SpO2 of 98%)+0.018×(time spent below an SpO2 of 94%)+0.152×(respiratory rate age z-score)−0.033×(age in month)+0.200×(heart rate age z-score)+0.280×(male=1, female=0)−0.656(normalised power in the low frequency (LF) band). This model presented an AUC of 0.77 (figure 2A) and was well calibrated (Hosmer-Lemeshow goodness of fit test p=0.84).

Figure 2

The area under the curve of the receiver operating characteristic of (A) the ‘mobile model’, which uses objective information recorded by the phone and derived from pulse oximetry and (B) the ‘baseline model’, which uses respiratory rate and median blood oxygen saturation value.

For comparison purposes, the implemented ‘baseline model’ equation was: logit(probability (admission required))=21 461+0.090×(respiratory rate age z-score)−0.233×(SpO2 median). This model presented an AUC of 0.729 (figure 2B).

With regard to the internal validation, each of the eight final selected predictors was also selected in at least 70% of the bootstrap samples indicating a stable predictive capability. The ‘mobile model’ had optimism in AUC 0.012 (95% CI −0.01 to 0.03) and the bootstrap-corrected AUC was 0.754 (95% CI 0.731 to 0.778).

Model classification

The maximum points indicated in figure 3 illustrate the risk thresholds that maximised the weighted classification score based on the studied false negative to false positive ratios.

Figure 3

Weighted classification score calculated using the (A) ‘mobile model’ and (B) ‘baseline model’, for the full range of thresholds using three different trade-offs between false negative and false positive cases (1:3, 1:5 and 1:10).

Consider a context in which the cost of sending healthy children to hospital is not overwhelmingly high, for example, in a community close to a major hospital. It would be reasonable to allow an unnecessary admission of three healthy children (ie, three false positives) in order to avoid missing any seriously sick child (ie, one false negative). As illustrated in figure 3, the optimal risk threshold in this context (3:1) was 0.265 for the ‘mobile model’ and 0.26 for the ‘baseline model’. The classification results were obtained with an adjusted risk threshold of 0.25 for easier interpretation (table 2). The ‘mobile model’ achieved better classification results in terms of NPV and sensitivity. However, both methods provided a low PPV (table 2).

Table 2

Classification performance (positive and negative predictive values, sensitivity and specificity) obtained with the models using the optimal risk thresholds

Model	Risk threshold	PPV (95% CI)	NPV (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)
‘Mobile’	0.25	0.51 (0.48 to 0.55)	0.85 (0.83 to 0.87)	0.72 (0.68 to 0.75)	0.71 (0.69 to 0.73)
‘Baseline’	0.25	0.51 (0.48 to 0.55)	0.82 (0.80 to 0.84)	0.63 (0.59 to 0.67)	0.74 (0.72 to 0.77)

NPV, negative predictive value; PPV, positive predictive value.

Reclassification results for both models showed that the addition of objective information derived from pulse oximetry recordings (SpO2 dynamics and PRV analysis) exhibited a 28% net improvement in the classification of the admitted (figure 4) and 4% net improvement in the classification of the non-admitted patients (figure 4), at a risk threshold of 0.25. The IDI was 0.0466 (95% CI 0.0368 to 0.0563, p=0) and the categorical NRI was 0.0389 (95% CI 0.0042 to 0.0736, p=0.02799).

Figure 4

Admitted and non-admitted children classification results using ‘baseline’ and reclassification results using the ‘mobile’ model. The height is proportional to the percentage of each classification or reclassification (correct reclassification is represented in green and incorrect reclassification in red).

Discussion

We have developed and internally validated a prediction model that uses objective information to identify under-5 children who would require hospitalisation. The objective information includes age, gender, respiratory rate and features derived from the analysis of 1 min pulse oximetry recordings. In this study, we have shown that HRV estimated using the PPG signal (PRV), SpO2 dynamics and respiratory rate are significantly different in children admitted to the hospital. A high respiratory rate and HR, suppressed PRV-power in the LF band, low median SpO2 value and high SpO2 variability, together with age and gender information, were the strongest predictors of the need for hospitalisation. When a recording with adequate signal quality (>50%) was applied, the model using these features provided a bootstrap-corrected AUC of 0.754. This enhanced-model provided a small but significant improvement in classification performance relative to a model using respiratory rate and median SpO2 value alone. This improvement is most noticeable as an enhanced ability to identify children at risk with only a small increase in the number of false positive results.

In low-resource settings, there is a lack of healthcare workers who are adequately trained for early detection of critical illness in children. One of the advantages of the proposed predictive model is that it exclusively uses objective information extracted primarily from pulse oximetry recordings (PRV and SpO2 dynamics) to generate a risk score. This information can be automatically calculated by a mobile application without requiring any additional effort as long as a good quality pulse oximetry signal is obtained. Thus, community healthcare workers can easily use it without requiring specific additional training for recognising signs or symptoms associated with critical illness. Providing community healthcare workers with this mobile–health approach will facilitate a rapid detection of children that would require admission to a facility and has the potential to modify behaviour towards strengthening emergency response systems and improving decision support information availability at a community level. When implemented within the phone oximeter, this could offer a convenient and easy-to-use platform for improving hospital referral practices in low-resource settings. The current version of the phone oximeter uses a much lower cost finger sensor directly connected to a smartphone or tablet through the audio jack.33

The selection of the risk threshold for predicted probability is a trade-off between acceptable risk and undue burden to the family, patient and health system and should be predetermined according to the clinical context. At a risk threshold of 0.25, the model provided a high-NPV (85%), demonstrating that the model is able to accurately identify children who do not require referral to a facility, without neglecting to refer many patients who should receive further attention. However, it provided a low PPV, suggesting that only 51% of children referred to the hospital would require admission. Increasing sensitivity or negative predictive performance will be at the expense of increasing unnecessary referral for hospital admission. The range of risk could also be used to individualise the frequency of follow-up or to follow the progression of risk overtime.

In a previous study, we developed a clinical prediction model based on continuous oxygen saturation, respiratory rate, temperature and age, combined with additional subjective predictors such as lethargy, irritability, symptoms of cough, diarrhoea and fast or difficult breathing, to predict the need for hospitalisation in children.17 Although this model provided improved performance (AUC=0.89), the ability to accurately identify some of the signs and symptoms may be difficult in settings where frontline healthcare workers are inadequately trained in the early detection of these disease signs and symptoms—a common scenario in low-resource settings. Therefore, in this study, we aimed to provide a predictive model based only on objective measures that can be obtained with minimal specific training.

Our findings from PPG analysis suggest that children with suppressed PRV, both in the time and frequency domains, and increased heart rate were more likely to be admitted. Children at a higher risk of critical illness have been previously demonstrated to have a higher heart rate and lower variability between ECG-based heart beats as measured in the time domain.11 Time-domain features based on heart rate characteristics have also been used in the early diagnosis of neonatal sepsis providing an AUC of 0.73.34 35 Spectral HRV and PRV analysis has also been used to identify patients with severe sepsis in the emergency department12 28 and sepsis shock in intensive care.36 These studies highlighted the value of spectral analysis for monitoring the progression of sepsis. The power within the LF band, which reflects sympathetic modulation of heart rate, was demonstrated to be negatively correlated with sepsis severity and the risk of sepsis shock.12 28 36 This feature was also selected in the current study as the strongest PRV-based risk predictor, confirming the presence of abnormal autonomic control in admitted children. The analysis of the SpO2 signal, recorded once per second, allowed us to assess not only the SpO2 levels but also the short-term SpO2 dynamics, not previously included as sepsis predictors.

The addition of these significant features, extracted from the analysis of PRV and SpO2 dynamics to the prediction model using only the SpO2 median and respiratory rate, improved the classification by 28% in admitted children and by 4% in non-admitted children. Thus, the information derived from pulse oximetry has the potential to provide relevant objective information that improves the identification of children at higher risk. The updated IMCI guidelines recommend the use of pulse oximetry to identify hypoxaemia and improve the diagnosis of critical illness. Using the phone oximeter, pulse oximetry can provide even more objective information, the derived PRV and SpO2 dynamics, without increasing the required effort. The app will automatically calculate this information, unbiased by the community healthcare worker. The result of the app can then be considered together with treatment guidelines for a more informed decision on the level of care necessary for the child.

Limitations and future research

The most significant limitation of this study is that 39% of the cases had to be excluded from the study because they did not meet the signal quality criteria for further analysis. The interface in the PhoneOxR2 software was designed to assist the user in optimising data collection. The colour of the background reflected the current quality of the signal, estimated using amplitude and regularity of the PPG signal and pulse oximeter module generated alerts. Feedback on the overall quality was provided at the conclusion of the recording and users were also trained to optimise the signal quality during the 1 min recording. However, the signal-processing methods applied in this study require a high-quality PPG signal and thus demand a more conservative SQI, such as the one applied offline in this study.26 We have subsequently implemented this more restrictive SQI method in the new app and made significant improvements in the design of new sensors. Any future decision support app that includes the predictive model will include this updated SQI version.26 Additionally, the app will require at least 30 s of good quality data (SQI>50%) to be accepted as a successful recording, a restriction that was not enforced during data collection.

The primary outcome, the need for hospitalisation, was a subjective decision made by the facility physicians, who may have been overly conservative in their decision-making, knowing that their clinical outcomes were being studied. To reduce this possible bias, the cases that were discharged within 24 hours were excluded and all pulse oximetry data were not provided to the facility physicians. Nevertheless, this study showed that there are significant differences in pulse oximetry derived features between admitted and non-admitted children and that this measurable objective information can correlate with a subjective decision, such as hospital admission.

The final model was internally validated through bootstrapping. However, external validation of the proposed predictive model should be performed in the setting of intended use. Owing to the practicality of collecting a large enough sample size and the need for good outcome data, our acquisition was carried out at a tertiary-level facility rather than in the community. The model should be validated in a community setting, with data from a cohort of children assessed by community healthcare providers. The model and thresholds for referral will need to be optimised for the local context.

Another limitation is the modest values of PPV and specificity. While this performance is significantly better than the currently used criteria (such as respiratory rate and SpO2 alone), there are a number of options for improving the model performance. We are investigating the specific characteristics of those participants in the current cohort who were misclassified. We are also looking to identify additional objective information that could be included to improve performance. One option is to add the morphology of the PPG waveform, which also provides information about the cardiovascular system and has been previously used as a measure of fluid responsiveness.37 38

Conclusion

Following further validation and improved quality of signal capture, this prediction tool could be used to improve the health outcomes of children under 5 years in low and middle income countries. The benefit of this predictive model is that it only uses objective information easily recorded from a mobile device, to provide a risk score for the probability of the need for the child to be admitted to a facility. This could allow frontline health workers with a mobile device to rapidly detect the need for referral with limited training overhead.