Prognostication using SpO2/FiO2 in invasively ventilated ICU patients with ARDS due to COVID-19 - Insights from the PRoVENT-COVID study.

BACKGROUND: The SpO2/FiO2 is a useful oxygenation parameter with prognostic capacity in patients with ARDS. We investigated the prognostic capacity of SpO2/FiO2 for mortality in patients with ARDS due to COVID-19.
METHODS: This was a post-hoc analysis of a national multicenter cohort study in invasively ventilated patients with ARDS due to COVID-19. The primary endpoint was 28-day mortality.
RESULTS: In 869 invasively ventilated patients, 28-day mortality was 30.1%. The SpO2/FiO2 on day 1 had no prognostic value. The SpO2/FiO2 on day 2 and day 3 had prognostic capacity for death, with the best cut-offs being 179 and 199, respectively. Both SpO2/FiO2 on day 2 (OR, 0.66 [95%-CI 0.46-0.96]) and on day 3 (OR, 0.70 [95%-CI 0.51-0.96]) were associated with 28-day mortality in a model corrected for age, pH, lactate levels and kidney dysfunction (AUROC 0.78 [0.76-0.79]). The measured PaO2/FiO2 and the PaO2/FiO2 calculated from SpO2/FiO2 were strongly correlated (Spearman's r = 0.79).
CONCLUSIONS: In this cohort of patients with ARDS due to COVID-19, the SpO2/FiO2 on day 2 and day 3 are independently associated with and have prognostic capacity for 28-day mortality. The SpO2/FiO2 is a useful metric for risk stratification in invasively ventilated COVID-19 patients.

Introduction

Coronavirus disease 2019 (COVID–19) has rapidly spread across the globe, accounting for nearly 5 million deaths worldwide as of November 2021 [1]. While most patients with COVID–19 have only mild symptoms, a substantial number of patients require hospitalization, mostly for supplemental oxygen, and approximately one in every five hospitalized patients needs admission to an intensive care unit (ICU) for escalation of respiratory support, i.e. invasive ventilation [2].

Risk classification may help in projecting the trajectory of individual patients with acute respiratory distress syndrome (ARDS). Typically, the PaO2/FiO2 is being used for mortality risk classification in these patients [3,4]. However, measuring the PaO2 is not always possible or could be too costly to use in resource–limited settings. While in some middle–income countries arterial blood gas measurements are increasingly available, [5] access is far from universal in critical care facilities worldwide [6,7]. In a recent report on COVID-19 in African ICUs, in only 82% of the participating hospitals there was a possibility to perform blood gas analyses [8].

Findings in two recent studies suggest that SpO2/FiO2 could replace PaO2/FiO2 in predicting outcome [9,10]. Several factors, however, could dampen the predictive accuracy of SpO2/FiO2 in patients with COVID–19, including a shift in the oxyhemoglobin dissociation curve due to hypercapnia, fever and metabolic disturbances, [[11], [12], [13]] all frequently present in patients with ARDS due to COVID–19. We performed a post-hoc analysis of a large observational national multicenter study to test the hypothesis that SpO2/FiO2 has prognostic capacity for outcome in critically ill invasively ventilated COVID–19 patients [14]. We also wished to explore the correlation between the SpO2/FiO2 and the PaO2/FiO2.

Methods

Study design

The ‘Practice of VENTilation in COVID–19’ (PRoVENT–COVID) study was an investigator–initiated, national, multicenter, observational cohort study performed in 22 ICUs in the Netherlands [15]. The Institutional Review Board of the Amsterdam UMC (Location AMC), Amsterdam, The Netherlands (Chairperson Prof. Dr. J.A. Swinkels) approved the study protocol on April 7, 2020 (W20_157 # 20.171), and need for individual patient informed consent was waived. The PRoVENT–COVID study was registered at clinicaltrials.gov (on April 15, 2020 with study identifier NCT04346342). The statistical analysis plan of the current analysis was finalized and published [16]. Other study details have been reported before [14].

Patients

Consecutive patients aged 18 years or older were eligible, provided they were admitted to one of the participating ICUs and received invasive ventilation for respiratory failure due to COVID–19, which was confirmed by RT–PCR for SARS–CoV–2. For the current analysis, we had the following exclusion criteria: (1) not fulfilling the criteria for ARDS, according to the current definition; [3] (2) treatment with extracorporeal membrane oxygenation in the first 4 calendar days of invasive ventilation; (3) transfer from or to a non–participating ICU within the first 2 days of invasive ventilation; and (4) an incomplete follow–up until day 28.

Data collection

Detailed information regarding demographic and medical history, disease severity and ARDS classification was collected at baseline. During the first 4 calendar days, ventilator settings, ventilation parameters, use of neuromuscular blocking agents, prone positioning, vital signs, and arterial lactate levels were recorded every 8 h at fixed time points. In the participating hospitals, ventilation variables were continuously recorded in the electronic medical records. SpO2 monitoring, mandatory in invasively ventilated patients in the Netherlands, is performed continuously and without interruptions – these data are captured by the electronic medical records, with hourly validation by trained ICU nurses. Trained data collectors of the PRoVENT–COVID study extracted these validated data at the time-points of interest. Follow–up was 90 days for the timing of extubation, ICU– and hospital discharge, and mortality.

The first and second calendar day that a patient received invasive ventilation were merged and named ‘day 1’––therefore, in theory this day could last from 24 h to 47 h and 59 min. The next two calendar days were named ‘day 2’ and ‘day 3’. In order to minimize the variable effects of prone positioning on the SpO2/FiO2, the lowest SpO2/FiO2 on day 1, 2, and 3 with the corresponding PaO2/FiO2 were used to determine its prognostic capacity. The SpO2/FiO2 in the first hour after the start of invasive ventilation was ignored, since endotracheal intubation and associated hemodynamic instability are likely to influence both SpO2 and PaO2, and because FiO2 is usually not yet adjusted within the first hour.

Endpoints

The primary endpoint of this analysis was 28–day mortality.

Statistical analysis

The sample size was based on the number of available patients. Data are expressed as mean ± standard deviation (SD), median with interquartile range (IQR) or number with percentage, where appropriate. Differences in baseline characteristics between survivors and non-survivors were analyzed using the Pearson Chi–squared or Fisher exact tests for categorical variables and with a one-way ANOVA or Kruskal–Wallis test for continuous variables.

To determine at which day SpO2/FiO2 had the best prognostic capacity for 28–day mortality, we conducted a joint analysis of variance (ANOVA), and a general linear F–test was performed by fitting a logistic multivariable model with SpO2/FiO2 on days 1, 2 and 3 as covariates. Subsequently, the difference in sum of squared errors (partial sum of squares) and the results from the F–test were used to identify which time points had prognostic value [17].

The accuracy of predicting 28–day mortality was analyzed by constructing receiver operator characteristics (ROC) curves. The area under the ROC (AUROC) was calculated and the optimal cut–off value for prediction of 28–day mortality was determined. An AUROC of ≥0.90 was considered excellent, 0.80 to 0.89 was considered good, 0.70 to 0.79 was considered fair, 0.60 to 0.69 was considered poor, and <0.60 was considered a fail [18]. The optimal cut–off point was determined using the Youden index, and differences between ROC curves were tested using a De Long test [19].

A multivariable logistic regression model was used to analyze the prognostic capacity of SpO2/FiO2 for 28–day mortality, while taking into consideration other major confounders. The model was fitted using statistically relevant SpO2/FiO2 values and the following predefined variables: age, PEEP, duration of prone positioning, arterial lactate level, arterial pH, vasopressor use and the presence of kidney dysfunction at admission. PEEP was treated both as a linear and non–linear term, in order to capture a potential threshold effect at varying levels of applied pressure. The variable inflation factor (VIF) was used to test for collinearity between covariates entered in the model, where VIF > 5 suggests moderate collinearity and VIF > 10 great collinearity [20]. A calibration analysis was used to assess the accuracy of the ROC and model overfitting.

Typically, in invasively ventilated COVID–19 patients, the SpO2/FiO2 is approximately 175 (e.g., in a patient ventilated with a FiO2 of 0.5 to 0.6, with a SpO2 of 90%, the SpO2/FiO2 is between 150 and 180), but values near to 100 are easily reached in patients with severe oxygenation problems. To be able to compare SpO2/FiO2 and PaO2/FiO2, we calculated PaO2 from SpO2 using the non–linear formula by Severinghaus–Ellis [21] and the lowest SpO2 on day 1. Then, the correlation between SpO2/FiO2 and PaO2/FiO2 was determined by comparing the calculated PaO2/FiO2 to the measured PaO2/FiO2 using a two–way scatterplot and Spearman correlation analysis. Accuracy was assessed using Bland–Altman plots and a Deming regression [22]. SpO2 values of ≥98% were excluded from this analysis.

Two post-hoc analyses were conducted. One sensitivity analysis of the multivariable model was performed, adding the respiratory system driving pressure as a covariate. Additionally, one analysis using ROC curves was performed to test whether PaO2/FiO2 measurements collected at the same time-points as SpO2/FiO2 would yield similar findings.

All analyses were performed in R (version 4.0.3), in the R studio environment (www.rstudio.com). A P value of <0.05 was considered statistically significant.

Results

A total of 1122 patients from 22 ICU's participated in the PRoVENT–COVID study, out of which 869 were included in the current analysis. Main reasons for exclusion were death or transfer to a non–participating hospital within the first 2 calendar days of invasive ventilation and failing to meet the current definition for ARDS (eFig. 1). The majority of patients were male, overweight and had moderate ARDS (Table 1 ). Mortality at day 28 and at day 90 was 30.1% and 55.6%. Patients that survived beyond day 28 were younger, had lower disease severity scores and higher baseline pH. While ARDS severity was not different between survivors and non–survivors, survivors had higher PaO2/FiO2 and SpO2/FiO2 at baseline and during successive days (eTable 1).

Table 1

Characteristics of invasively ventilated patients with COVID-19 ARDS.

	Survivors (n = 607)	Non-survivors (n = 262)	P-value
Age (Years)	63 [56, 70]	70 [65, 74]	<0.001
Male (%)	70.7	76.3	0.103
Weight (kg)	85.0 [78.0, 97.0]	82.5 [75.0, 95.0]	0.030
BMI (kg/m2)	27.7 [25.2, 30.8]	27.4 [25.0, 29.6]	0.188
Affected quadrants on chest X-ray (%)			0.459
1	7.8	5.2
2	23.1	18.5
3	27.4	28.9
4	41.7	47.4
APACHE II score	15 [12,20]	20 [15, 23]	<0.001
APACHE IV score	54 [45, 66]	65 [50, 81]	<0.001
Tidal volume (ml/kg PBW)	5.7 [5.2, 6.2]	6.0 [5.4, 6.4]	0.002
Respiratory rate (breaths/min)	22 [20,25]	24 [20,26]	0.012
Respiratory system compliance (ml/cmH20)	31 [25, 39]	32 [25, 37]	0.638
Driving pressure	13.0 [10.5–15.5]	13.5 [11.0–17.0]	0.031
Duration of prone positioning (h)	17.0 [0.0, 41.0]	12.0 [0.0, 39.6]	0.798
pH	7.30 [7.24, 7.35]	7.24 [7.18, 7.30]	<0.001
Vasopressor use (%)a	92.1	96.2	0.039
Arterial Lactate (mmol/L)	1.5 [1.2, 2.0]	1.9 [1.5, 2.5]	<0.001
Kidney dysfunction (%)b	235 (39.0)	174 (66.4)	<0.001
PaO2/FiO2	124 (45)	120 (44)	0.291
SpO2/FiO2	153 (48)	143 (42)	0.003
FiO2	0.60 [0.50, 0.80]	0.70 [0.55, 0.80]	0.007
PEEP	12 [10, 15]	12 [10, 15]	0.248
Berlin ARDS category (%)			0.482
Mild	42 (6.9)	13 (5.0)
Moderate	376 (61.9)	161 (61.5)
Severe	189 (31.1)	88 (33.6)
Ventilator-free-days and alive at day 28	11.5 [0.0–18.0]	0.0 [0.0–0.0]	<0.001

Categorical variables: number (percentage); continuous variables: median [25–75 percentile] or mean (SD). Abbreviations: BMI, Body Mass Index; APACHE, Acute Physiology and Chronic Evaluation; PBW, predicted body weight; ARDS, Acute Respiratory Distress Syndrome.

During the first 72 h after ICU admission.

Serum creatininine concentration of >155 umol/l.

Identification of the best day for prognostication using SpO2/FiO2

The SpO2/FiO2 on day 1 had no prognostic value for 28–day mortality (p = 0.721). The SpO2/FiO2 on day 2 and day 3, however, did have significant associations with outcome (p < 0.001 for both days; Fig. 1 ). The AUROC for SpO2/FiO2 on day 2 and day 3 were comparable.

Fig. 1

Predictive capacity of SpO2/FiO2 during the first three days of invasive ventilation. Panel A: SpO2/FiO2 on day 1, Panel B: SpO2/FiO2 on day 2, Panel C: SpO2/FiO2 on day 3.

Legend: AUC, area under the curve.

Best SpO2/FiO2 cutoffs

The best cut-off for SpO2/FiO2 for 28–day mortality was 179 on day 2, and 199 on day 3. The SpO2/FiO2 on day 2 was more specific but less sensitive than SpO2/FiO2 on day 3, and was associated with a higher positive predictive value but a lower negative predictive value (eTable 2).

The multivariable model

In the multivariable regression analysis, SpO2/FiO2 on day 2 and on day 3 had a comparably strong association with 28–day mortality, where an improvement in SpO2/FiO2 was associated with an increase in survival. Higher age, lactate levels, a lower pH, and the presence of kidney dysfunction at ICU admission were significant confounders associated with a higher mortality (eFig. 2 and Table 2 ). The discriminating capacity of the multivariable model for 28–day mortality was fair (AUROC = 0.78 [0.76–0.79], and the calibration was adequate (eFig. 3). No large effect of collinearity was observed in the model for SpO2/FiO2 on day 2 and 3: VIF was 1.77 for SpO2/FiO2 at day 2, and VIF was 1.64 for SpO2/FiO2 at day 3. In a sensitivity analysis including respiratory system driving pressure in the multivariable model, driving pressure was not associated with 28-day mortality (OR 1.13 [0.89–1.42], p = 0.32), whereas SpO2/FiO2 on day 3, but not on day 2 remained associated with outcome (eTable 3).

Table 2

Results from the multivariable logistic model predicting 28-day mortality.

Variable	Odds ratio	Standard error	Lower 95% CI	Upper 95% CI	P–value
SpO2/FiO2 on day 2	0.66	0.19	0.46	0.96	0.027
SpO2/FiO2 on day 3	0.70	0.16	0.51	0.96	0.017
PEEP	1.25	0.14	0.95	1.64	0.235
PEEP (non- linear term)	1.18	0.08	1.01	1.38	0.032
Duration of prone positioning	0.85	0.14	0.61	1.17	0.343
Age	3.21	0.16	2.34	4.40	<0.001
pH	0.72	0.15	0.54	0.96	0.027
Lactate	1.13	0.06	1.01	1.28	0.038
Vasopressor use (yes)	1.25	0.46	0.50	3.01	0.625
Acute Kidney Injury (yes)	2.45	0.20	1.65	3.63	<0.001

Output from the multivariable logistic model using clinically relevant confounders. 28-day mortality as the dependent variable, other variables tested as independent variables.

Correlation of SpO2/FiO2 with PaO2/FiO2

The correlation between observed and calculated PaO2/FiO2 was strong (Fig. 2 ). The Deming regression showed both fixed and proportional bias from the calculated values, with decreasing accuracy at higher oxygenation levels. The estimated PaO2/FiO2 was moderately but systematically lower than the measured PaO2/FiO2 (eFig. 4).

Fig. 2

The relationship between observed and calculated PaO2/FiO2 values using a Deming regression.

Legend: The dashed line depicts the ideal accuracy line; the solid line shows the Deming regression estimation.

Post-hoc analysis of the prognostic value of PaO2/FiO2

PaO2/FiO2 had a comparable prognostic value during the first days of invasive ventilation; alike SpO2/FiO2 on day 1, PaO2/FiO2 on day 1 had no prognostic value for 28–day mortality (eFig. 5).

Discussion

This study investigated the prognostic capacity of SpO2/FiO2 for mortality in critically ill invasively ventilated patients with ARDS due to COVID–19. SpO2/FiO2 had a significant association with 28–day mortality, where SpO2/FiO2 on day 2 and 3 had prognostic capacity for death. The SpO2/FiO2 correlated well with the PaO2/FiO2, even though the estimated PaO2/FiO2 was moderately but systematically lower than the measured PaO2/FiO2.

Our study has several strengths. The analysis included a large cohort of patients from 22 ICUs in the Netherlands, which were located in university hospitals, teaching hospitals and non–teaching hospitals, increasing the validity of the findings. Also, in contrast to previous studies in patients with ARDS, [4,10] this analysis was not limited to arbitrarily defined severity groups. The longitudinal evaluation of outcome prediction across the first 4 calendar days provides information about the early changes that may occur after the initiation of invasive ventilation, and allows pinpointing the most accurate moment to assess patient outcome using this cheap and easily obtainable oxygenation metric.

SpO2/FiO2 on days 2 and 3 was associated with mortality, while SpO2/FiO2 on day 1 had no association with outcome. This is in line with previous findings in patients with ARDS, where reassessment of disease severity and oxygenation after 24 h of invasive ventilation led to improved outcome prediction [10,23,24]. A study in invasively ventilated COVID–19 patients from Spain showed a clear shift in ARDS severity from day 1 to day 2, which remained constant thereafter up to day 28 [25]. This evolution in ARDS severity during the first days has also been reported in a study in a large cohort of patients with ARDS not due to COVID–19 [26]. Additionally, a large recent study in COVID–19 patients did not show an association of oxygenation metrics early after arrival in the ICU with outcome [27]. These findings suggests that re–evaluating patients after 24 h of standard care gives a more accurate picture of ARDS severity, and thus outcome. Recent studies, however, did report significant associations of oxygenation metrics at the beginning of ICU admission with outcomes. One study of a large cohort of COVID-19 patients in three European countries found that non-survivors had significantly lower PaO2/FiO2 values during the first 24 h of ICU admission [28]. However, a significant proportion of patients were not under invasive ventilation at that time. Furthermore, although an association was found between PaO2/FiO2 on the first day and outcome, that study did not evaluate the effect of PaO2/FiO2 during consecutive days. Another study analyzed the association of PaO2/FiO2 with the number of ventilator-free days and alive at day 28 (VFD-28) in invasively ventilated patients with moderate to severe ARDS due to COVID-19 [29]. In their multivariable model PaO2/FiO2 was associated with VFD-28, however, the precise timing of the PaO2/FiO2 was unclear.

SpO2/FiO2 has previously been successfully implemented for predicting mortality and classifying ARDS severity in patients with non-COVID-19 ARDS [9,30]. A few studies have recently evaluated its use in COVID-19, where acid-base disorders and fever might influence prognostic accuracy. A recent analysis in a mixed cohort of ventilated and non-ventilated patients evaluated the use of changes in SpO2/FiO2 for prognostication in patients with COVID-19, where a decrease in SpO2/FiO2 in the first three days was associated with a poor outcome [31]. However, this study did not report on arterial blood gas and ventilation data, and 28-day mortality was 94.7% in patients receiving respiratory support, which is substantially higher than other reports on outcome [2,14]. Another retrospective analysis evaluated SpO2/FiO2 for the prediction of mortality and the occurrence of ARDS, and found a similar cutoff value of 179 as this current study [32]. However, the sample size was limited, arterial blood gas analysis was frequently unavailable, and FiO2 could not be accurately measured in patients with non-invasive oxygen delivery methods. Additionally, a recent study successfully used SpO2/FiO2 as a part of a machine learning model for early prediction of mortality and the need for mechanical ventilation in hospitalized patients with COVID-19 [33]. In contrast to these studies, this current analysis focused on a well-defined large cohort of invasively ventilated patients, evaluated SpO2/FiO2 across multiple days and included several arterial blood gas and ventilator support variables as confounders in the prediction model. Therefore, this study adds to the current understanding of the prognostic capacity of SpO2/FiO2 in mechanically ventilated patients with COVID-19 in the ICU.

The addition of several major confounders related to 28-day mortality improved the prediction capacity, while maintaining the significance of the SpO2/FiO2 as an individual prediction marker. The accuracy of this prediction model was fair, and it could provide a good basis for prognostication after the first day of invasive ventilation, although the evaluation of pH and arterial lactate levels may not be feasible in settings with limited resources [7,8].

PaO2/FiO2 can be accurately calculated from SpO2/FiO2 using the Severinghaus–Ellis equation, but there is a fixed bias across all oxygenation levels, and an increasing proportional bias at higher oxygenation levels. This is in line with previous findings in ARDS not related to COVID–19 [9,10,34,35]. Poor detection of hyperoxemia is a known pitfall of pulse oximetry saturation, caused by the flattening of the oxyhemoglobin dissociation curve [10,34,35]. Several equations have been used to calculate PaO2/FiO2 from SpO2/FiO2, but a validated method is yet lacking that would provide an accurate assessment at all oxygenation levels [21,[34], [35], [36], [37]]. It could be that our findings might have been different if we would have used alternative equations.

Our study has several limitations. Due to the observational nature of the study, selection and performance bias cannot be excluded as confounding factors in the inclusion of participating of ICUs, as well as in the application of lung protective ventilation measures and adjunctive treatments for refractory hypoxemia. We thus used the lowest SpO2/FiO2 of the first days of ventilation, to mitigate the variable effect of prone positioning, recruitment, and other treatments of refractory hypoxemia on the SpO2/FiO2. Although we believe this to be the most accurate depiction of the degree of respiratory failure, we cannot exclude added bias due to this selection. Furthermore, during the pandemic, shortcomings of pulse oximetry have been emphasized especially in patients with a darker skin tone, where under detection of hypoxemia occurs more frequently than in white patients [[38], [39], [40], [41]]. Unfortunately, our database had no data available on patient skin color, impeding any assessment on the effect of skin color on SpO2/FiO2 accuracy in our cohort. Lastly, our study was performed in high–resource settings in the Netherlands, where blood gas analysis and PaO2/FiO2 measurements were exclusively available. As shown in previous studies in non–COVID ARDS, prognostic capacity of PaO2/FiO2 improves in successive days after initiation of invasive ventilation [[23], [24], [25]]. External validation of our findings in low–resource settings is warranted, and we are planning such a study using Asian and African ICU registries.

Conclusion

In this large cohort of invasively ventilated patients with ARDS due to COVID–19, SpO2/FiO2 on day 2 or 3, but not on day 1, had an independently association with 28–day mortality. The prognostic capacity of SpO2/FiO2 alone was poor, while the multivariable model had a fair predictive capacity. A strong correlation between PaO2/FiO2 calculated from SpO2/FiO2 and measured PaO2/FiO2 was observed, however, calculated PaO2/FiO2 values generally underestimated arterial oxygenation. These findings support the use of SpO2/FiO2 as an attractive alternative to PaO2/FiO2, especially in resource–limited settings.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

None.