Post-exertion oxygen saturation as a prognostic factor for adverse outcome in patients attending the emergency department with suspected COVID-19: a substudy of the PRIEST observational cohort study.

BACKGROUND: Measurement of post-exertion oxygen saturation has been proposed to assess illness severity in suspected COVID-19 infection. We aimed to determine the accuracy of post-exertional oxygen saturation for predicting adverse outcome in suspected COVID-19.
METHODS: We undertook a substudy of an observational cohort study across 70 emergency departments during the first wave of the COVID-19 pandemic in the UK. We collected data prospectively, using a standardised assessment form, and retrospectively, using hospital records, from patients with suspected COVID-19, and reviewed hospital records at 30 days for adverse outcome (death or receiving organ support). Patients with post-exertion oxygen saturation recorded were selected for this analysis. We constructed receiver-operating characteristic curves, calculated diagnostic parameters, and developed a multivariable model for predicting adverse outcome.
RESULTS: We analysed data from 817 patients with post-exertion oxygen saturation recorded after excluding 54 in whom measurement appeared unfeasible. The c-statistic for post-exertion change in oxygen saturation was 0.589 (95% CI 0.465 to 0.713), and the positive and negative likelihood ratios of a 3% or more desaturation were, respectively, 1.78 (1.25 to 2.53) and 0.67 (0.46 to 0.98). Multivariable analysis showed that post-exertion oxygen saturation was not a significant predictor of adverse outcome when baseline clinical assessment was taken into account (p=0.368). Secondary analysis excluding patients in whom post-exertion measurement appeared inappropriate resulted in a c-statistic of 0.699 (0.581 to 0.817), likelihood ratios of 1.98 (1.26 to 3.10) and 0.61 (0.35 to 1.07), and some evidence of additional prognostic value on multivariable analysis (p=0.019).
CONCLUSIONS: Post-exertion oxygen saturation provides modest prognostic information in the assessment of selected patients attending the emergency department with suspected COVID-19. TRIAL REGISTRATION NUMBER: ISRCTN Registry (ISRCTN56149622) http://www.isrctn.com/ISRCTN28342533. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY. Published by BMJ.

Post exertional decrease in oxygen saturation can be used to predict prognosis in chronic lung diseases.

Post exertional desaturation has been proposed as a way of predicting adverse outcome in people with suspected COVID-19.

Post-exertion oxygen saturation provides modest prognostic information in the assessment of selected patients attending the emergency department with suspected COVID-19.

Introduction

Guidelines for assessment of suspected COVID-19 recommend measurement of peripheral oxygen saturation to determine the severity of acute respiratory infection.1–3 Clinicians have noted that patients with suspected COVID-19 and a relatively normal oxygen saturation may desaturate after exertion, but the clinical importance of this finding is uncertain. If shown to predict adverse outcome, testing for post-exertional desaturation could be used to identify patients needing hospital admission.

Field walking tests are commonly used to evaluate exercise capacity and assess prognosis in chronic respiratory diseases.4 The lowest arterial oxygen saturation recorded during a 6-minute walk test is an important marker of disease severity and prognosis.5 The rapid 1-minute sit-to-stand test correlates with the 6-minute walk test and the severity of lung disease.6 Exertional tests have shown desaturation in chronic obstructive lung disease,7 8 chronic interstitial lung disease9–11 and Pneumocystis carinii pneumonia.12 A modified 6-minute walk test has been proposed for use in suspected COVID-19 infection13 but not yet evaluated, to our knowledge. A recent review of rapid exercise tests for oxygen desaturation14 identified a number of studies, as outlined above, but found no published studies in COVID-19. The authors suggested that a 3% drop in oxygen saturation on exercise was a cause for concern, based on evidence from other conditions.

The Pandemic Respiratory Infection Emergency System Triage (PRIEST) Study is a multicentre observational cohort study designed to develop and evaluate triage methods for patients with suspected COVID-19 infection. We added evaluation of post-exertion oxygen saturation to the aims of the PRIEST Study in response to reports of its use in the assessment of suspected COVID-19. Our specific objective for this post hoc substudy was to determine the accuracy of post-exertional oxygen saturation as a prognostic factor for 30-day adverse outcome.

Methods

For the PRIEST Study, we collected data from consecutive patients presenting with suspected COVID-19 infection to 70 hospital emergency departments (EDs) from 53 recruiting sites in the UK, where some sites cover more than one hospital. Hospitals used either prospective data collection, through a standardised assessment form for suspected COVID-19, or retrospective data collection, through research staff extracting data from hospital records onto the standardised form.

Patients were included in the PRIEST Study if the assessing clinician used the standardised assessment form or recorded that the patient had suspected COVID-19 infection. The clinical diagnostic criteria used for suspected COVID-19 during the study were1 fever (≥38°C) or a history of fever and2 influenza-like illness (two or more of cough, sore throat, rhinorrhoea, limb or joint pain, headache, vomiting or diarrhoea) or severe and/or life-threatening illness suggestive of an infectious process. We did not seek consent to collect data but information about the study was provided in the ED and patients could withdraw their data at their request. Patients with multiple presentations to hospital were only included once, using data from the first presentation identified by research staff.

The population for this substudy was patients who had post-exertion oxygen saturation recorded as part of routine care. The assessing clinician made the decision to measure post-exertion oxygen saturation and determined the approach to achieving exertion. The study did not influence clinical care, so we were unable to standardise the selection of patients or the approach to measuring post-exertion oxygen saturation. Measurement could have been undertaken deliberately, by asking the patient to exercise in a specified way, or opportunistically, by recording oxygen saturation after the patient had exerted themselves for another purpose.

Research staff reviewed hospital records to identify outcomes up to 30 days after initial presentation. We defined patients who died or required respiratory, cardiovascular or renal support as having an adverse outcome. We defined respiratory support as any intervention to protect the patient’s airway or assist their ventilation, including mechanical ventilation, non-invasive ventilation or continuous positive airway pressure, but not supplemental oxygen alone or nebulised bronchodilators. We defined cardiovascular support as any intervention to maintain organ perfusion, including extracorporeal membrane oxygenation, inotropic drugs or invasive cardiovascular monitoring, but not peripheral intravenous cannulation and/or fluid administration. We defined renal support as any intervention to assist renal function, including haemofiltration, haemodialysis or peritoneal dialysis, but not intravenous fluid administration or urinary catheterisation.

We undertook an initial descriptive analysis of the patients with post-exertion oxygen saturation recorded. This identified a number of patients for whom post-exertion oxygen saturation measurement appeared unfeasible, based on age (less than 3 years), performance status bed/chair bound, baseline oxygen saturation below 85%, post-exertion oxygen saturation below 50%, receiving supplemental oxygen or Glasgow Coma Score less than 14. We excluded these patients from the analysis.

We examined baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change in oxygen saturation (ie, baseline minus post-exertion oxygen saturation). Analysis focused on the latter, because this indicates the additional value achieved by measuring oxygen saturation after exertion. We estimated the accuracy of each index test in terms of the sensitivity, specificity and likelihood ratios of each test across a range of thresholds for positivity, for predicting adverse outcome up to 30 days. CIs for likelihood ratios were calculated using the methods outlined in Bedrick and Koopman.15 Receiving-operator characteristic (ROC) curves were constructed and the c-statistic (area under the ROC curve) was calculated for each index test. We did not attempt to determine an optimal threshold for positivity, because that depends on the relative importance of sensitivity and specificity in the decision that post-exertion oxygen saturation is intended to inform. However, we decided a priori to highlight the performance of a 3% desaturation, as suggested by Greenhalgh et al.14 Analysis was performed on patients with post-exertion oxygen saturation recorded and available 30-day outcome data, as such missing data were not imputed.

To determine whether measurement of post-exertion oxygen saturation adds prognostic information to standard respiratory assessment, we fitted a multivariable logistic regression model with age, baseline oxygen saturation, respiratory rate, heart rate, asthma, other chronic respiratory illness and post-exertional oxygen saturation as covariates.

We undertook a secondary analysis that excluded patients for whom post-exertion oxygen saturation measurement appeared less appropriate, based on age (less than 16 years), performance status of limited self-care, baseline oxygen saturation less than 94%, or heart rate, respiratory rate or systolic blood pressure scoring 3 points on the National Early Warning Score (NEWS2). The rationale for this analysis was that local guidelines3 recommend admission for patients with oxygen saturation less than 94% or a score of 3 points or more on any NEWS2 parameter. It has also been suggested that post-exertional assessment is only undertaken in a patient able to stand safely unaided and whose resting saturation is 96% or above.14

We planned for the PRIEST Study to recruit a sample size of 20 000. The analysis presented here is a secondary analysis, so no sample size was prespecified.

Patient and public involvement

The Sheffield Emergency Care Forum (SECF) is a public representative group interested in emergency care research.16 Members of SECF advised on the development of the PRIEST Study and two members joined the Study Steering Committee. Patients were not involved in the recruitment and conduct of the study. We are unable to disseminate the findings to study participants directly.

Results

The PRIEST Study recruited 22 484 patients across 70 hospitals between 26 March 2020 and 28 May 2020, of whom 39 requested withdrawal of their data. We identified 874 patients who had post-exertion oxygen saturation recorded and excluded 57 in whom measurement appeared unfeasible, leaving 817 for analysis. Adverse outcome occurred in 30 participants (3.7%), of these 9 died, 22 had respiratory support, 5 had cardiovascular support and 4 renal support.

Online supplemental figure 1 shows the flow of patients through the study, and online supplemental table S1 shows the characteristics of the whole PRIEST cohort and the characteristics of those included in this analysis. Participants in this analysis were younger, more likely to have unrestricted performance status, less likely to have any comorbidities, tended to have more normal baseline physiology and had a much lower rate of adverse outcome.

Table 1 compares the baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change between those with and without an adverse outcome. Post-exertion oxygen saturation tended to be lower than baseline oxygen saturation and showed a greater decrease in those who suffered adverse outcome (2.9% vs 1.9% mean decrease). However, figure 1 shows that oxygen saturations increased post-exertion in a proportion of cases and there was considerable overlap between those with and without adverse outcome. Online supplemental figures S2 and S3 show overlaid histograms for baseline and post-exertion oxygen saturation.

Table 1

Comparison of index tests summary statistics between those with and without adverse outcome

Characteristic	Adverse outcome	No adverse outcome	All
	(n=30)	(n=787)	(n=817)
Baseline oxygen saturation
N (%)	30 (100.0)	783 (99.5)	813 (99.5)
Mean (SD)	94.5 (3.5)	97.1 (2.3)	97.0 (2.4)
Median (IQR)	95.0 (92.0–97.0)	97.0 (96.0–99.0)	97.0 (96.0–99.0)
Post-exertion oxygen saturation
N (%)	30 (100.0)	787 (100.0)	817 (100.0)
Mean (SD)	91.6 (5.3)	95.2 (4.2)	95.0 (4.3)
Median (IQR)	92.0 (88.0–96.0)	96.0 (93.0–98.0)	96.0 (93.0–98.0)
Oxygen saturation difference, pre-exercise to post-exercise
N (%)	30 (100.0)	783 (99.5)	813 (99.5)
Mean (SD)	−2.9 (5.3)	−1.9 (3.5)	−2.0 (3.5)
Median (IQR)	−3.0 (−6.0 to 0.0)	−1.0 (−3.0 to 0.0)	−1.0 (−3.0 to 0.0)

Figure 1

Histograms comparing post-exertion change in oxygen saturation from baseline between patients with and without adverse outcome (N=813).

Figure 2 shows the ROC curves for baseline oxygen saturation, post-exertion oxygen saturation and post-exertion change in oxygen saturation. The c-statistic of 0.589 (95% CI 0.465 to 0.713) for post-exertion change indicates poor discriminant value, that is, post-exertion change does not predict adverse outcome. This is partly due to post-exertion increases in oxygen saturation showing some association with increased risk of adverse outcome. Table 2 reports sensitivity, specificity and likelihood ratios for thresholds of post-exertion decrease in oxygen saturation (ie, change less than zero). The positive and negative likelihood ratios of a post-exertional desaturation of 3% or more were 1.78 and 0.67, respectively, suggesting that this finding provides a small amount of additional information in prognostic assessment. Online supplemental tables S2 and S3 show the diagnostic parameters for baseline and post-exertion oxygen saturation, respectively.

Figure 2

ROC curves showing index test accuracies for predicting adverse outcome, primary analysis (n=817). ROC, receiving-operator characteristic.

Table 2

Accuracy of post-exertion change in oxygen saturation from baseline at a range of thresholds for positivity, primary analysis (N=813)

Threshold	Sensitivity(95% CI)	Specificity(95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio (95% CI)
<=−1	70.0 (50.6 to 85.3)	43.6 (40.0 to 47.1)	1.24 (0.97 to 1.58)	0.69 (0.40 to 1.20)
<=−2	53.3 (34.3 to 71.7)	59.3 (55.7 to 62.7)	1.31 (0.93 to 1.85)	0.79 (0.54 to 1.16)
<=−3	53.3 (34.3 to 71.7)	70.1 (66.8 to 73.3)	1.78 (1.25 to 2.53)	0.67 (0.46 to 0.98)
<=−4	40.0 (22.7 to 59.4)	77.3 (74.2 to 80.2)	1.76 (1.11 to 2.78)	0.78 (0.58 to 1.05)
<=−5	36.7 (19.9 to 56.1)	82.2 (79.4 to 84.9)	2.07 (1.26 to 3.39)	0.77 (0.59 to 1.01)
<=−6	30.0 (14.7 to 49.4)	86.1 (83.5 to 88.4)	2.16 (1.22 to 3.83)	0.81 (0.64 to 1.03)
<=−7	23.3 (9.9 to 42.3)	90.5 (88.3 to 92.5)	2.47 (1.25 to 4.89)	0.85 (0.70 to 1.04)
<=−8	20.0 (7.7 to 38.6)	92.5 (90.4 to 94.2)	2.65 (1.24 to 5.65)	0.87 (0.73 to 1.04)
<=−9	16.7 (5.6 to 34.7)	94.0 (92.1 to 95.6)	2.78 (1.19 to 6.48)	0.89 (0.76 to 1.05)
<=−10	10.0 (2.1 to 26.5)	95.1 (93.4 to 96.5)	2.06 (0.67 to 6.30)	0.95 (0.84 to 1.07)

Figure 3 and table 3 show comparable results for the secondary analysis excluding cases where post-exertion oxygen saturation measurement appeared less appropriate. The c-statistic of 0.699 (95% CI 0.581 to 0.817) for post-exertion change in oxygen saturation indicates better discriminant value in this group, that is, post-exertion change predicts an increased risk of adverse outcome. This may be explained by exclusion of patients with lower baseline oxygen saturations who had more potential to show a random post-exertion increase in oxygen saturation. The positive and negative likelihood ratios of a post-exertional decrease in oxygen saturation of 3% or more were 1.98 and 0.61, respectively, indicating some modest value in predicting adverse outcome. Online supplemental tables S4 and S5 show the diagnostic parameters for baseline and post-exertion oxygen saturation, respectively, for the secondary analysis.

Figure 3

ROC curves showing index test accuracies for predicting adverse outcome, secondary analysis (N=655). ROC, receiving-operator characteristic.

Table 3

Accuracy of post-exertion change in oxygen saturation from baseline at a range of thresholds for positivity, secondary analysis (N=652)

Threshold	Sensitivity(95% CI)	Specificity(95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio (95% CI)
<=−1	87.5 (61.7 to 98.4)	44.5 (40.6 to 48.5)	1.58 (1.30 to 1.93)	0.28 (0.08 to 1.03)
<=−2	56.3 (29.9 to 80.2)	60.4 (56.5 to 64.2)	1.42 (0.91 to 2.21)	0.72 (0.41 to 1.26)
<=−3	56.3 (29.9 to 80.2)	71.5 (67.9 to 75.0)	1.98 (1.26 to 3.10)	0.61 (0.35 to 1.07)
<=−4	43.8 (19.8 to 70.1)	79.1 (75.7 to 82.2)	2.09 (1.18 to 3.72)	0.71 (0.46 to 1.10)
<=−5	43.8 (19.8 to 70.1)	83.6 (80.5 to 86.4)	2.68 (1.50 to 4.80)	0.67 (0.43 to 1.03)
<=−6	31.3 (11.0 to 58.7)	87.1 (84.3 to 89.6)	2.42 (1.14 to 5.15)	0.79 (0.57 to 1.10)
<=−7	25.0 (7.3 to 52.4)	91.5 (89.1 to 93.6)	2.94 (1.21 to 7.13)	0.82 (0.62 to 1.09)
<=−8	25.0 (7.3 to 52.4)	92.8 (90.5 to 94.7)	3.46 (1.42 to 8.45)	0.81 (0.61 to 1.08)
<=−9	18.8 (4.0 to 45.6)	94.3 (92.2 to 96.0)	3.31 (1.14 to 9.63)	0.86 (0.68 to 1.09)
<=−10	12.5 (1.6 to 38.3)	95.8 (93.9 to 97.2)	2.94 (0.76 to 11.32)	0.91 (0.76 to 1.10)

In the multivariable model on the primary analysis cohort, post-exertional oxygen saturation did not add prognostic value over other factors in the model (p value for model coefficient 0.368, likelihood ratio test for model with and without post-exertion oxygen saturation 0.78, p=0.376). For the secondary analysis, post-exertional oxygen saturation added prognostic value over other factors (p value for model coefficient 0.019, likelihood ratio test for model with and without post-exertion oxygen saturation 4.82, p=0.078), indicating that post-exertion desaturation may help to predict adverse outcome even when other clinical features are taken into account.

Discussion

Our findings suggest that measurement of post-exertion oxygen saturation adds modest prognostic information to clinical assessment of suspected COVID-19 in the ED. The likelihood ratios in the primary analysis suggest that a desaturation of 3% or more provides a small amount of prognostic information, the c-statistic of 0.589 for post-exertion change suggests little discriminant value, and multivariate analysis suggests that post-exertion oxygen saturation measurement does not add prognostic value once baseline measurements are taken into account. Secondary analysis limited to more appropriate cases suggested better discriminant value, with a c-statistic of 0.699, and additional prognostic value on multivariable analysis. In appropriate cases, post-exertion oxygen saturation measurement should be safe and relatively easy to incorporate into clinical assessment. It may therefore be worth using in selected patients, such as those with normal baseline observations and oxygen saturation. The low risk of adverse outcome in these selected patients means that absence of desaturation could provide reassurance that discharge home is appropriate.

The observation that oxygen saturation increased post-exertion, and that some people with adverse outcome showed an increase, may seem surprising, but is probably explained by random variation. Oxygen saturation varies randomly from one measurement to the next and this variation is likely to be greater in sicker patients with baseline hypoxia. Thus, we might expect greater variation in oxygen saturation to show some association with adverse outcome.

Greenhalgh et al 14 suggested using a desaturation of at least 3% to identify cause for concern in selected patients who are well enough for out-of-hospital management. Our findings suggest that a 3% desaturation indicates a small increase in the likelihood of adverse outcome. Further research could determine whether a more systematic and rigorously controlled approach to post-exertion oxygen saturation measurement can result in more useful prognostic information. The feasibility of such research may be limited by low event rates in people who are able to undertake formal post-exertion measurement of oxygen saturation.

Our study consisted of a clinically relevant population and was recruited across a wide range of settings, but evaluation of post-exertion oxygen saturation was a post hoc secondary analysis and the study was not designed specifically for this purpose. We are unable to say how patients were selected for measurement of post-exertion oxygen saturation, and the method for undertaking exertion was not standardised or recorded. We excluded 57 patients from analysis for whom post-exertion oxygen saturation measurement appeared unfeasible, and excluded a further 162 from secondary analysis for whom measurement appeared less appropriate. These cases may reflect opportunistic oxygen saturation measurement after exertion, such as on attempting to mobilise, but we cannot exclude the possibility of data recording errors. Furthermore, our judgements regarding feasibility and appropriateness of measurement were somewhat subjective, so others may consider that measurement in these cases could be feasible or appropriate. Only 874 out of 22 445 patients (3.9%) had post-exertion oxygen saturation recorded. This may reflect limited awareness and use of post-exertion oxygen saturation, but may also reflect severity of illness in the ED population. Measurement of post-exertion oxygen saturation is only likely to be feasible and clinically indicated in those with milder illness. The relatively small number of adverse outcomes (N=30) limited the precision of our estimates of sensitivity and power to undertake multivariable analysis.

In summary, measuring post-exertion oxygen saturation provides modest prognostic information in the assessment of selected patients attending the ED with suspected COVID-19. It could be added to clinical assessment for patients with normal baseline oxygen saturation when there is uncertainty around the decision to admit to hospital.