An internally validated prediction model for critical COVID-19 infection and intensive care unit admission in symptomatic pregnant women.

BACKGROUND: Pregnant women are at an increased risk of mortality and morbidity owing to COVID-19. Many studies have reported on the association of COVID-19 with pregnancy-specific adverse outcomes, but prediction models utilizing large cohorts of pregnant women are still lacking for estimating the risk of maternal morbidity and other adverse events.
OBJECTIVE: The main aim of this study was to develop a prediction model to quantify the risk of progression to critical COVID-19 and intensive care unit admission in pregnant women with symptomatic infection. STUDY
DESIGN: This was a multicenter retrospective cohort study including 8 hospitals from 4 countries (the United Kingdom, Austria, Greece, and Turkey). The data extraction was from February 2020 until May 2021. Included were consecutive pregnant and early postpartum women (within 10 days of birth); reverse transcriptase polymerase chain reaction confirmed SARS-CoV-2 infection. The primary outcome was progression to critical illness requiring intensive care. The secondary outcomes included maternal death, preeclampsia, and stillbirth. The association between the primary outcome and 12 candidate predictors having a known association with severe COVID-19 in pregnancy was analyzed with log-binomial mixed-effects regression and reported as adjusted risk ratios. All the potential predictors were evaluated in 1 model and only the baseline factors in another. The predictive accuracy was assessed by the area under the receiver operating characteristic curves.
RESULTS: Of the 793 pregnant women who were positive for SARS-CoV-2 and were symptomatic, 44 (5.5%) were admitted to intensive care, of whom 10 died (1.3%). The 'mini-COvid Maternal Intensive Therapy' model included the following demographic and clinical variables available at disease onset: maternal age (adjusted risk ratio, 1.45; 95% confidence interval, 1.07-1.95; P=.015); body mass index (adjusted risk ratio, 1.34; 95% confidence interval, 1.06-1.66; P=.010); and diagnosis in the third trimester of pregnancy (adjusted risk ratio, 3.64; 95% confidence interval, 1.78-8.46; P=.001). The optimism-adjusted area under the receiver operating characteristic curve was 0.73. The 'full-COvid Maternal Intensive Therapy' model included body mass index (adjusted risk ratio, 1.39; 95% confidence interval, 1.07-1.95; P=.015), lower respiratory symptoms (adjusted risk ratio, 5.11; 95% confidence interval, 1.81-21.4; P=.007), neutrophil to lymphocyte ratio (adjusted risk ratio, 1.62; 95% confidence interval, 1.36-1.89; P<.001); and serum C-reactive protein (adjusted risk ratio, 1.30; 95% confidence interval, 1.15-1.44; P<.001), with an optimism-adjusted area under the receiver operating characteristic curve of 0.85. Neither model showed signs of a poor fit. Categorization as high-risk by either model was associated with a shorter diagnosis to intensive care unit admission interval (log-rank test P<.001, both), higher maternal death (5.2% vs 0.2%; P<.001), and preeclampsia (5.7% vs 1.0%; P<.001). A spreadsheet calculator is available for risk estimation.
CONCLUSION: At presentation with symptomatic COVID-19, pregnant and recently postpartum women can be stratified into high- and low-risk for progression to critical disease, even where resources are limited. This can support the nature and place of care. These models also highlight the independent risk for severe disease associated with obesity and should further emphasize that even in the absence of other comorbidities, vaccination is particularly important for these women. Finally, the model also provides useful information for policy makers when prioritizing national vaccination programs to quickly protect those at the highest risk of critical and fatal COVID-19.

Introduction

The presentation of COVID-19 is quite variable, ranging from asymptomatic infection to mild respiratory illness with minimal supportive care to hospitalization with multiorgan failure and death. , Given the alterations in physiology and immune function (eg, inflammatory or prothrombotic markers) that may mask or predispose a patient to severe-critical disease, pregnant women represent a unique population compared with their nonpregnant peers.

Why was this study conducted?

Pregnant women are at an increased risk of complications from COVID-19, and pregnancy-specific risk estimation models are lacking.

Key findings

The mini-model including the maternal age, body mass index, and pregnancy trimester can be used to estimate the risk of developing critical COVID-19 before disease onset (area under the receiver operating characteristics curve, 0.73). The addition of inflammatory markers to maternal body mass index (full-model) at the time of diagnosis can accurately predict critical COVID-19 (area under the receiver operating characteristics curve, 0.85), preeclampsia, and the progression time from diagnosis to clinical deterioration.

What does this add to what is known?

This study builds practical tools for risk estimation that can be used to inform the risk of progression to critical COVID-19 along with maternal death, development of preeclampsia, and time to clinical deterioration.

To reduce the burden on the use of healthcare resources and focus them on those in the greatest need, it is important to identify the individual patients who are at an increased actuarial risk of progression to critical COVID-19. Several COVID-related outcome prediction models based on clinical, laboratory, and imaging criteria have been developed for the general population.4, 5, 6 However, they have methodological limitations and do not account for pregnancy, which limits their generalizability and applicability. Furthermore, some models rely heavily on radiologic investigations that are employed less frequently in pregnancy, particularly when symptoms are mild.

Emerging data from the United Kingdom and the United States suggest that pregnant women may be experiencing more severe illness in the second wave of the pandemic than the first. , A recent living systematic review of maternal and fetal outcomes in pregnant women found that although these women are less likely to report the symptoms of COVID-19, they are more than twice as likely as their nonpregnant peers to require critical care or mechanical ventilation; a finding corroborated by large national registries such as the Centers for Disease Control and Prevention (CDC).

The main aim of this study was to develop a prediction model to quantify the risk of progression of infection to critical COVID-19 in pregnant women with symptomatic infection to enable evidence-based triage and effective targeting of diagnostic and therapeutic interventions, including place of care.

Materials and Methods

This was a multicenter cohort study including 8 centers in 4 countries (Supplemental Table 1). Data extraction was from the start of the pandemic in each country to May 1, 2021. The relevant data were extracted from the electronic patient records and were anonymized for statistical analysis.

Supplemental Table 1

Patients included from each center and previous publications including patients from the same cohort

Center	Sample size	Previous publications with overlap
Department of Obstetrics and Gynecology, Koç University School of Medicine, Istanbul, Turkey and American Hospital	30	None
Department of Obstetrics and Gynecology, Sancaktepe Sehit Prof Dr Ilhan Varank Training and Research Hospital, Istanbul, Turkey	530	Kuzan et al,1 2021
		Yassa et al,2 2020
		Tug et al,3 2020
		Kalafat et al,4 2020
Faculty of Medicine, Department of Obstetrics and Gynecology, Istanbul Medeniyet University, Istanbul, Turkey	44	None
Department of Obstetrics and Gynecology, Istanbul Provincial Health Directorate, Prof Dr Cemil Tascioglu City Hospital, Istanbul, Turkey	70	Sahin et al,5 2021
Fetal Medicine Unit, St George’s Hospital, St George’s University of London, United Kingdom.	40	Knight et al,6 2020
Department of Obstetrics and feto-maternal Medicine, Medical University of Vienna, Vienna, Austria	43	None
Third Department of Obstetrics and Gynecology, Attikon University Hospital, University of Athens, Athens, Greece	36	None

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

The inclusion criteria were choosing pregnant and early postpartum women (within 10 days of birth) and reverse transcriptase polymerase chain reaction (RT-PCR)-confirmed SARS-CoV-2 infection. The included women had mild, moderate, or severe illness at the time of diagnosis. The exclusion criteria were asymptomatic infection (positive RT-PCR for SARS-CoV-2 without any clinical symptoms), critical illness at the time of diagnosis, prior COVID-19 infection, or receiving a vaccine against SARS-CoV-2. All the included women were either followed up as outpatients or were admitted as inpatients for supportive care. The women without critical illness and the outpatients were followed up for 14 days following the diagnosis of COVID-19. The patients were managed according to local protocols.

RT-PCR-positive pregnant or postpartum women who were symptomatic but without lower respiratory tract symptoms (eg, dyspnea) or abnormal chest imaging (ie, tomography, lung ultrasound, or chest X-ray) were classified as having mild illness. Moderate illness was diagnosed in RT-PCR-positive pregnant/postpartum women with lower respiratory tract symptoms without significant hypoxia (pulse oximetry saturation ≥94%). Severe illness was diagnosed in RT-PCR-positive pregnant/postpartum women with oxygen saturation <94%, respiratory rate >30 breaths per minute, Po2 to fraction of inspired oxygen <300 mm Hg but not meeting the criteria for critical illness. Critical illness was diagnosed in patients with acute respiratory distress syndrome, which required mechanical ventilation support, septic shock, cardiac dysfunction, hyperinflammatory syndrome, or other organ system dysfunction.

The data on the maternal age, self-reported ethnicity, body mass index (BMI), smoking, chronic comorbidities (pregestational diabetes, chronic hypertension, heart disease [valvular, arrhythmia, or cardiomyopathy] and bronchial asthma), gestational age at diagnosis, number of fetuses, and hospitalization were collected. When available, the complete blood count (CBC) and C-reactive protein (CRP) assessment at the time of diagnosis were also collected. We did not collect the data related to gestational diabetes owing to a variability in screening and diagnosis between the centers. The candidate variables were selected among the factors with known or plausible associations with severe COVID-19 in pregnant and nonpregnant adult populations.

The primary outcome was progression to critical illness requiring intensive care unit (ICU) admission. The secondary outcomes were maternal death, preeclampsia, and stillbirth. Preeclampsia was defined according to the revised criteria of the International Society for the Study of Hypertension in Pregnancy 2014 Statement; hypertension was defined as new-onset systolic blood pressure ≥140 mm Hg and/or diastolic blood pressure ≥90 mm Hg on 2 occasions more than 24 hours apart. Proteinuria was defined as a protein/creatinine ratio ≥30 mg/mmol or a 24-hour urine collection of ≥300 mg per 24 h. Stillbirth was defined as fetal death at or beyond 24+0 weeks’ gestation.

A prediction model was developed and reported as a Type 1b analysis, which uses all the available data for model building with interval validation procedures, as per the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis statement. This is the preferred method of prediction model building when the sample size does not allow dataset partitioning. Moreover, some authors have proposed that the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis 1b analysis is the preferred method of model building regardless of the sample size. Our sample size was 690 women selected based on the following: the need for intensive care in 8.7% of pregnant women with COVID-19, having at least 10 patients with the primary outcome per tested variable, and the ability to test at least 6 variables (50% of the candidate pool) at the same time in a multivariable model. The literature suggests at least 10 adverse outcomes per tested variable to avoid model overfitting. ,

Ethics approval was obtained from the Koç University Institutional Review Board (approval number 2021.264.IRB1.089), which allowed the use of anonymized patient data without individual consent. Approvals were also obtained from the National Health Services Health Research Authority, the University of Vienna (2306/2020) and the University of Athens. The participating centers were Attikon University Hospital (Athens, Greece), Koç University Hospital (Istanbul, Turkey), Medeniyet University Hospital (Istanbul, Turkey), Prof Dr Cemil Tascioglu City Hospital (Istanbul, Turkey), Sancaktepe Education and Research Hospital (Istanbul, Turkey), St. George’s University Hospital (London, United Kingdom), and Vienna University Hospital (Vienna, Austria). All are tertiary care facilities with advanced life-support capabilities. The number of cases collected from each center, and previous publications, including the cases from each center, are summarized in Supplemental Table 1.

Statistical analysis

The continuous variables are presented as mean and standard deviation or median and interquartile range according to the distribution characteristics. The distribution of continuous variables was assessed with quartile-quartile plots, skewness, and kurtosis values. Group comparisons were made using the t test, Wilcoxon signed rank test, chi-squared test, or Fisher exact test where appropriate.

The effect size was reported as mean, median difference, or odds ratio and 95% confidence intervals (CI). The association of variables with ICU admission was analyzed with log-binomial mixed-effects regression and was reported as adjusted risk ratio (aRR). The risk estimates in the regression were reported for one standard unit change in the respective variables. Random intercepts were used to account for study center-level variance.

Prediction models were built using generalized linear models using the logit link function. Two predictive models were constructed from the candidate predictors associated with more severe COVID-19 during or outside pregnancy. The first model used demographic and clinical variables available at disease onset (COvid Maternal Intensive Therapy [miniCOMIT]). The second model used all the variables, including those from investigations in hospital (fullCOMIT). The models were built using the complete case data for each data set (full and laboratory parameters available) while ensuring that the proportion of omitted cases did not surpass 1% of all the available cases in each dataset. Akaike Information Criterion was used to assess the model fit and meaningful improvements at each model iteration. The linearity assumptions were tested using the Box-Tidwell test, and the nonparametric transformation of continuous scale variables was tested for model improvement. The predictive capabilities and change in model fit were considered during the addition or subtraction of a variable. We aimed to achieve the most parsimonious model without sacrificing the predictive capability or goodness of fit using calibration curves. The predictive capabilities were assessed by the area under the receiver operating characteristic curves (AUROC). Optimism-adjusted AUROC values were obtained with repeated k-fold cross-validation. The predictive accuracy measures, including sensitivity, specificity, positive predictive value, negative predictive value, and the positive and negative likelihood ratios (LR) were reported.

The model performances for each of miniCOMIT and fullCOMIT were assessed by the following 3 methods: first, the calibration curves comparing the expected and observed outcome rates by deciles of risk; second, risk stratification tables by risk quintile; and third, Youden index cut-offs that maximized sensitivity and specificity. These were calculated for each model to categorize women into high-risk and low-risk groups. The interval between diagnosis and ICU admission was compared for the risk strata in each model by log-rank tests, and the pregnant/postpartum women not admitted to the ICU at the end of the follow-up period (14 days) were considered as censored. The interval was tested to see whether the classification allowed for a clinically meaningful interval in which interventions can be applied. All the analyses were conducted using R Software for Windows (version 4.0.3; The R Foundation, Vienna, Austria).

Results

Of the 793 pregnant or postpartum women who were positive for SARS-CoV-2 by RT-PCR and were symptomatic, 44 (5.5%) were admitted to the ICU, of whom 10 died (1.3%).

Supplemental Table 2 shows that many baseline characteristics varied among the women admitted to the ICU vs those who were not. The women admitted to the ICU were significantly older and were just over (vs under) 30 years of age. They were more often obese (one-third) and were smokers (almost 7%). There were no differences in either ethnicity (most women overall were White) or chronic morbidities. The women admitted to the ICU were at a more advanced gestational age (by just over 3 weeks) and were more likely to be in their third trimester and have lower respiratory tract symptoms. Most women had singleton pregnancies. There were 658 women (83.0%) who had laboratory assessment with CBC and serum CRP at diagnosis with COVID-19. The women admitted to the ICU (vs those who were not) had significantly higher absolute neutrophil counts, lower lymphocyte counts, and higher neutrophil to lymphocyte ratios in addition to a higher CRP. Most women who were not admitted to the ICU were still hospitalized.

Supplemental Table 2

Baseline characteristics and laboratory parameters of pregnant women with symptomatic COVID-19, stratified according to intensive care unit admission status

Variables	SARS-CoV-2 positive women without ICU admission (n=749)	SARS-CoV-2 positive women with ICU admission (n=44)	Absolute mean, median difference (95% CI)a	P valueb
Maternal and pregnancy variables
Maternal age in y	29.4±5.68	32.0±5.70	2.59 y (0.81–4.37 y)	.0051
BMI in kg/m2	25.7 (23.8–28.5)	28.0 (25.3–31.2)	2.28 kg/m2 (2.00–2.60 kg/m2)	.0006
BMI >30 kg/m2	136 (18.1)	16 (36.4)	18.5% (4.2%–32.9%)	.0038
Smoker	12 (1.6)	3 (6.8)	5.2% (−2.2% to 12.6%)	.023
Ethnicity				.117
– Caucasian	717 (95.7)	40 (90.9)	−5.1% (−7.9% to −2.0%)
– Afro-Caribbean	21 (2.8)	4 (9.1)	6.3% (−2.2% to 14.9%)
– Asian	9 (1.2)	0 (0.0)	−1.2% (−2.0% to −0.4%)
– Not reported	2 (0.3)	0 (0.0)
Chronic comorbidity (≥1)	49 (6.5)	6 (13.6)	7.1% (−3.1% to 17.3%)	.079
– Prepregnancy diabetes	9 (1.2)	2 (4.5)	3.4% (−2.8 to 9.6%)
– Chronic hypertension	8 (1.1)	1 (2.3)	1.2% (−0.9 to 2.9%)
– Heart disease	3 (0.4)	0 (0.0)	−0.4% (−0.8% to 0.5%)
– Bronchial asthma	33 (4.4)	4 (9.1)	4.7% (−0.4% to 13.4%)
Gestational age at diagnosis in wk	27.8 (20.0–34.4)	29.5 (27.4–34.1)	3.22 (1.38–8.99)	.014
– First trimester	82 (10.9)	0 (0.0)	−10.9% (−12.7% to −9.2%)
– Second trimester	260 (34.7)	8 (18.2)	−19.7% (−27.4% to −12.1%)
– Third trimester	400 (53.4)	36 (81.8)	28.2% (16.8%–39.7%)
– Postpartum	7 (1.0)	0 (0.0)
Multiple gestation	19 (2.5)	3 (6.8)	4.3% (−3.2% to 11.8%)	.107
Lower respiratory tract symptoms of COVID-19	454 (60.6)	41 (93.2)	32.5% (24.3%–40.7%)	.0002
Hospitalized for COVID-19	573 (76.5)	44 (100.0)	23.5% (20.4%–26.6%)	.0005
Laboratory variables at diagnosis
Hemoglobin levels in g/dL	11.4±1.36	11.0±1.68	−0.39 (−0.94 to 0.15)	.148
Lymphocyte count (×109/L)	1.27 (0.96–1.72)	0.97 (0.69–1.20)	−0.30 (−0.36 to −0.23)	<.0001
Absolute neutrophil count (×109/L)	5.73±2.41	7.59±2.95	1.87 (0.92–2.82)	.0002
Neutrophil to lymphocyte ratio	4.19 (2.93–5.91)	8.00 (5.40–13.8)	3.81 (3.63–4.00)	<.0001
CRP levels (mg/L)	2.53 (0.71–8.00)	19.0 (10.5–63.1)	16.5 (16.1–17.0)	<.0001

Continuous variables are presented as mean±standard deviation or median and interquartile range according to distribution characteristics. Categorical variables are presented as number and percentage of total.

BMI, body mass index; CI, confidence interval; CRP, C-reactive protein; ICU, intensive care unit.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Parametric or non-parametric bootstrapped CI are reported according to parent distribution

Wilcoxon signed rank, t test, chi-squared test or Fisher exact test where appropriate.

Table 1 shows that by univariable regression analysis, all of the following were associated with ICU admission (P<.05): the clinical characteristics of maternal age, BMI, smoking, chronic comorbidities, gestational age at diagnosis of COVID-19, third trimester pregnancy, and lower respiratory tract symptoms; and the laboratory test results showing anemia, lymphopenia, a higher neutrophil/lymphocyte ratio, and higher CRP levels.

Table 1

Univariable binomial regression analysis of factors associated with intensive care unit admission

Variables	Risk ratio (95% CI)a	P value
Maternal and pregnancy-specific variables
Maternal age in y	1.51 (1.13–2.02)	.0046
BMI in kg/m2	1.46 (1.16–1.78)	.0004
BMI >30 kg/m2	2.47 (1.30–4.51)	.0039
Ethnicity
– Caucasian	Reference
– Black, Asian or Minority Ethnicity	2.22 (0.67–5.52)	.127
Smoker	3.79 (0.92–10.4)	.0258
Chronic comorbidity	1.92 (0.97–3.59)	.0479
– Prepregnancy diabetes	3.38 (0.55–10.9)	.0921
– Chronic hypertension	2.02 (0.11–9.27)	.485
– Heart disease	NE	NA
– Asthma	2.04 (0.61–5.07)	.173
Gestational age at diagnosis in wk	3.04 (1.33–8.31)	.0165
Third trimester pregnancy	3.84 (1.88–8.90)	.0005
Multiple gestation	2.56 (0.62–7.04)	.115
Laboratory and disease specific variables available at the time of diagnosis
Lower respiratory tract symptoms of COVID-19	8.23 (3.00–33.9)	.0004
Hemoglobin levels in g/dL	0.77 (0.58–1.04)	.083
Anemia (Hemoglobin <10 g/dL)	2.96 (1.48–5.60)	.0012
Lymphocyte count (×109/L)	0.40 (0.24–0.62)	.0001
Lymphopenia (lymphocyte count <1000/mm3)	2.60 (1.40–4.83)	.0022
Absolute neutrophil count (×109/L)	1.73 (1.35–2.19)	<.0001
Neutrophil to lymphocyte ratio	1.42 (1.28–1.54)	<.0001
CRP levels (mg/L)	1.38 (1.25–1.50)	<.0001

BMI, body mass index; CI, confidence interval; CRP, C-reactive protein; NA, not applicable; NE, not estimable.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Log-binomial regression. Risk ratios correspond to 1 standard unit change in respective variables.

The miniCOMIT model (based on N=786 women, 7 excluded for missing data for ≥1 of the variables in the model) included the maternal age (aRR, 1.45 [95% CI, 1.07–1.95]; P=.015), BMI (aRR, 1.34 [1.06–1.66]; P=.010), and the third trimester of pregnancy (aRR, 3.64 [1.78–8.46]; P=.001) (Table 2 ). No significant interaction between the variables was detected. The optimism-adjusted AUROC was 0.73 (Figure 1 ). By the Youden index cutoff, 362 of 786 (46.1%) women were at a high risk and 424 of the 786 (53.9%) at a low risk of needing ICU admission. The model had an acceptable goodness of fit according to the Hosmer-Lemeshow test (P=.208) and had acceptable calibration (Supplemental Figure 1). Risk stratification with the quintiles of risk has shown an incremental change in the ICU admission risk with each quintile (Table 3 ). The ICU admission risk was 2.0%, 8.7%, 13.3%, and 27.3% for the first, second, third, and fourth quintile, respectively, and the trend was statistically significant (Cochrane-Armitage P<.0001). The predictive accuracy parameters are presented in Table 3. The women at a high risk according to the Youden index cutoff (vs low risk) were more likely to require ICU admission (38/362, 10.5% vs 6/424, 1.4%; P<.001) and suffer maternal death (8/362, 2.2% vs 2/424, 0.5%; P=.030). They had a shorter diagnosis to ICU admission interval (log-rank test P<.001) (Figure 2 , A). However, preeclampsia did not significantly differ by the risk category (10/362, 2.8% vs 7/424, 1.6%; P=.285), and there were few stillbirths (3/362, 8 per 1000 vs 2/424, 5 per 1000).

Table 2

Multivariable log-binomial regression analysis of factors associated with intensive care unit admission

Multivariable regression	Adjusted risk ratio (95% CI)a	P value
miniCOMIT (optimism-adjusted AUC, 0.73)
– Maternal age in y	1.45 (1.07–1.95)	.015
– Maternal BMI in kg/m2	1.34 (1.06–1.66)	.010
– Third trimester of pregnancy	3.64 (1.78–8.46)	<.001
fullCOMIT (optimism-adjusted AUC, 0.86)
– Maternal BMI in kg/m2	1.39 (1.09–1.71)	.003
– Lower respiratory symptoms of COVID-19	5.11 (1.81–21.4)	.007
– Neutrophil to lymphocyte ratio	1.62 (1.36–1.89)	<.001
– CRP levels (mg/L)	1.30 (1.15–1.44)	<.001

The miniCOMIT was built from variables available before diagnosis and fullCOMIT was built using all variables available at the time of diagnosis

AUC, area under the curve; BMI, body mass index; CI, confidence interval; COMIT, COvid Maternal Intensive Therapy; CRP, C-reactive protein.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Log-binomial regression. Risk ratios correspond to one standard unit change in respective variables.

Figure 1

ROC curves

ROC curves of miniCOMIT (green line) and fullCOMIT (orange line). FullCOMIT, using laboratory parameters, BMI, and respiratory symptoms outperformed miniCOMIT, which includes maternal age, BMI, and gestational age.

BMI, body mass index; COMIT, COvid Maternal Intensive Therapy; ROC, receiver operating curves.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Supplemental Figure 1

The calibration plot of miniCOMIT

The smooth black line represents that the fit of the model predicted the risk of outcome to the observed rate within each decile of predicted probability. The straight red line is used as a reference for perfect fit. The bar chart at the base of the figure presents the distribution of cases with intensive care unit admission (above the line) across the spectrum of predicted probability.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Table 3

Risk stratification table using 5 groups of predicted probability

Predicted risk	Women in range	ICU admission	Sensitivity	Specificity	PPV	NPV	LR+	LR−
miniCOMIT
<5%	454	9 (2.0)	79.5 (64.7–90.2)	59.9 (56.3–63.5)	10.5 (9.0–12.3)	98.0 (96.4–98.8)	1.99 (1.67–2.36)	0.34 (0.19–0.61)
5%–9.9%	231	20 (8.7)	34.0 (20.4–49.9)	88.4 (85.8–90.6)	14.8 (9.9–21.5)	95.7 (94.8–96.5)	2.94 (1.86–4.64)	0.75 (0.60–0.92)
10%–24.9%	90	12 (13.3)	6.8 (1.4–18.6)	98.9 (97.9–99.5)	27.3 (9.3–57.7)	94.9 (94.5–95.3)	6.58 (1.4–18.7)	0.94 (0.87–1.02)
25%–49.9%	11	3 (27.3)	0.0 (0.0–8.0)	100.0 (99.5–100.0)	—	94.4 (94.4–94.4)	—	1.0 (1.0–1.0)
≥50%	0	0 (0.0)	—	—	—	—	—	—
fullCOMIT
<5%	461	6 (1.3)	85.3 (70.8–94.4)	73.7 (70.0–77.2)	17.7 (15.2–20.6)	98.7 (97.3–99.4)	3.25 (2.71–3.90)	0.20 (0.09–0.42)
5%–9.9%	102	9 (8.8)	70.0 (55.4–82.1)	88.8 (86.0–91.1)	33.6 (27.5–40.3)	97.3 (95.9–98.2)	6.23 (4.70–8.34)	0.34 (0.22–0.52)
10%–24.9%	63	12 (19.0)	34.1 (20.0–50.5)	97.0 (95.4–98.2)	43.7 (29.4–59.1)	95.6 (94.6–96.5)	11.7 (6.28–21.8)	0.68 (0.54–0.85)
25%–49.9%	21	5 (23.8)	21.9 (10.5–37.6)	99.6 (98.8–99.9)	81.8 (50.1–95.3)	95.0 (94.2–95.7)	67.7 (15.1–303.2)	0.78 (0.67–0.91)
≥50%	11	9 (81.8)	0.0 (0.0–8.6)	100.0 (99.4–100.0)	—	93.7 (93.7–93.7)	—	1.0 (1.0–1.0)

Predictive values are presented as mean (95% CI). Sensitivity, specificity, and predictive values calculated using the upper limit of the risk range to define a positive test.

CI, confidence interval; COMIT, COvid Maternal Intensive Therapy; ICU, intensive care unit admission; LR, likelihood ratio; NPV, negative predictive value; PPV, positive predictive value.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Figure 2

Diagnosis to ICU admission interval stratified by risk categories

According to miniCOMIT (A) and fullCOMIT (B). Risk stratification by both models was significantly associated with the diagnosis to ICU admission interval (log-rank test P<.0001, both).

COMIT, COvid Maternal Intensive Therapy; ICU, intensive care unit.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

The fullCOMIT model (based on N=658 women with available laboratory data) included the BMI (aRR, 1.39 [1.07–1.95]; P=.015), the lower respiratory tract symptoms of COVID-19 (aRR, 5.11 [1.81–21.4]; P=.007), the neutrophil to lymphocyte ratio (aRR, 1.62 [1.36–1.89]; P<.001), and the CRP levels (aRR, 1.30 [1.15–1.44]; P<.001) (Table 2). No significant interaction among the variables was detected. The optimism-adjusted AUROC was 0.85. By the Youden index cutoff, 174 of 658 (26.4%) women were at a high risk and 484 of the 658 (73.6%) were at a low risk of needing ICU admission. The model had acceptable goodness of fit according to the Hosmer-Lemeshow test (P=.393) and had acceptable calibration (Supplemental Figure 2). The ICU admission risk was 1.3%, 8.8%, 19.0%, 23.8%, and 81.8% for the first, second, third, fourth, and fifth quintiles, respectively, and the trend was statistically significant (Cochrane-Armitage P<.0001) (Table 3). The women at high risk according to the Youden index cutoff (vs low risk) were more likely to require ICU admission (34/174, 19.5% vs 7/484, 1.4%; P<.0001) and had a shorter diagnosis to the ICU admission interval (log-rank test P<.001) (Figure 2). These women more often suffered maternal death (9/174, 5.2% vs 1/484, 0.2%; P<.0001) or preeclampsia (10/174, 5.7% vs 5/484, 1.0%; P=.0003); there were few stillbirths (2/174, 11 per 1000 vs 3/484, 6 per 1000). A spreadsheet calculator is available for both the models for validation (Supplemental Material).

Supplemental Figure 2

Calibration plot of fullCOMIT

The smooth black line represents the fit of the model predicted risk of outcome to the observed rate within each decile of predicted probability. The straight red line is used as reference for perfect fit. The bar chart at the base of the figure presents distribution of cases with intensive care unit admission (above the line) across the spectrum of predicted probability.

Kalafat et al. Prediction of critical COVID-19 in symptomatic pregnant women. Am J Obstet Gynecol 2022.

Comment

Principal findings

In this multicenter international cohort study, we could identify the women at an increased risk of severe COVID-19 based on the variables at symptom onset, particularly those at hospital admission. Risk stratification by either model could classify the women into high- and low-risk categories with systematic differences in the rates of ICU admission, maternal death, and preeclampsia. fullCOMIT has good performance as a rule-out test for ICU admission (LR, ≤0.20), and both miniCOMIT and fullCOMIT have good and very good performances as rule-in tests once the risks are estimated to be 10–24.9%, respectively. High-risk women also had a shorter time from diagnosis to ICU admission. The predictive accuracy of fullCOMIT based on all the available variables, including laboratory tests in hospital (ie, BMI, lower respiratory tract symptoms of COVID, neutrophil/lymphocyte ratio, and CRP levels) was superior to miniCOMIT based on the variables available at symptom onset (ie, maternal age, BMI and third trimester of pregnancy).

Results in the context of what is known

The prediction models are useful for informing patients about their risk and making individualized, data-driven management decisions. Several prediction models have been proposed for use in nonpregnant adults with COVID-19 with varying success. , , , Most models utilized laboratory parameters at the time of diagnosis, whereas some also incorporated imaging studies. A systematic review of the published models criticized the optimistic prediction estimates and poor reporting. Moreover, only 2 prediction models focused on pregnant women with COVID-19, based on very small cohorts (114 and 80 women). , Tutiya et al reported on a similar cohort to ours by including symptomatic disease only, albeit with much smaller numbers (786 vs 114). They reported that comorbidities such as asthma were associated with adverse outcomes, which was not the case in our study. A larger sample size may have allowed for better quantification of variance in our study. Tutiya et al reported that a non-White ethnicity is a risk factor for severe COVID-19. We could not verify this finding, but our cohort mainly consisted of Caucasian ethnicity (90%); there was a two-fold increase in Black, Asian, and ethnic minorities in the ICU admission cohort without statistical significance (risk ratio, 2.22; 95% CI, 0.67–5.52). Although we used most variables on a continuous scale, a linear increase in the risk observed in our cohort may not have external validity, and better modeling approaches may exist in larger datasets. Recently, the CDC data showed that underweight individuals also are at an increased risk of COVID-19 complications. However, the risk increase showed a linear pattern above the normal weight ranges, which is corroborated by our findings as well. Unfortunately, we did not have many underweight individuals in our cohort (0.4%) to reliably model the association.

We employed a large cohort of symptomatic women for whom a prediction model would be most useful. Our findings regarding the serum markers of inflammation and blood count parameters such as neutrophil/lymphocyte ratio are consistent with the published literature. , The miniCOMIT model incorporating only maternal and pregnancy characteristics had a lower predictive accuracy than the results of Tutiya et al (AUROC: 0.73 vs 0.82). However, we obtained optimism-adjusted area under the curve (AUC) values, aimed for the most parsimonious model within the constraints of an adverse outcome group size and employed a much larger cohort. These points may have helped in avoiding overfitting, which is a significant issue for small cohorts and oversaturated models. Yao et al reported a prediction model consisting of dyspnea, heart rate, respiratory rate, fever, CRP levels, and chest imaging. The reported AUC was very high (0.97), but the sample size was inadequate, with only 50 patients in the development cohort and 30 patients in the validation.

We noted an increase in the prevalence of preeclampsia in the cohort predicted to be at a high-risk of ICU admission and death by our model. Studies have reported an increased rate of preeclampsia in women with COVID-19 but did not demonstrate a link with the COVID-19 severity. Finally, the rate of stillbirth was twice as high in the group categorized as high-risk for COVID-19, but this difference was not statistically significant. This finding is likely to be related to low numbers and inadequate statistical power, as larger studies demonstrated a 2–3 fold increase in the stillbirth rates in women with COVID-19. Our results indicate that increased stillbirth rate may be explained by severe and critical COVID-19 infection in pregnant women. The validation of our models in larger cohorts may confirm this association between stillbirth and severe COVID-19.

Strengths and limitations

The strengths of our study include the large sample size of symptomatic pregnant women with COVID-19, adherence to the recommended guidelines for model development, evaluating the independent contribution of recognized risk factors for severe COVID-19 (in and outside pregnancy), and including a simple-to-use spreadsheet calculator for external validation and clinical implementation.

Limitations do apply to our findings. First, we were probably underpowered to look at the impact of ethnicity (Black or other ethnic minority groups) or maternal comorbidities on maternal ICU admission with symptomatic COVID-19 infection. Second, being relatively underpowered resulted in no women being rated with a miniCOMIT risk ≥50%. Third, we did not include chest imaging (using ionizing radiation or alternatives) as a candidate predictor, as it was not routinely included in management protocols in pregnancy, and we aimed to develop a generalizable model. However, the inclusion of imaging modalities would probably increase the predictive accuracy. Fourth, we did not perform external validation because of the constraints of our sample size. Data set partitioning would have caused the model to overfit and yield biased estimates owing to oversaturation. Instead, we opted to use the whole cohort for model building and adjusting for optimism via cross-validation, which is the recommended approach. , There are numerous international cohorts of pregnant women published in the literature, so external validation can be performed in future studies with relative ease. , Fifth, we did not account for treatments applied in each center in the model. However, only a limited number of therapeutic interventions have shown promise for halting progression to critical disease, and limited to no evidence is available for guiding the treatment of pregnant women.26, 27, 28, 29 Therefore, there is little reason to assume that the inclusion of different treatment modalities would have impacted the performance of the fullCOMIT prediction model. Finally, we excluded the asymptomatic cases so that our findings would not apply to such women. However, asymptomatic infection has an excellent prognosis in pregnant women with COVID-19, and the clinical applicability of a prediction model in such populations would be very limited.

Clinical and research implications

The fullCOMIT model can be used at the time of COVID-19 diagnosis in symptomatic pregnant women. Most trials excluded pregnant women, and those who allowed participation had an extremely small number of pregnancies to provide any direct evidence of benefit. The management of pregnant women with COVID-19 is an area currently supported by very little evidence. Therapeutic interventions such as steroids, convalescent plasma, and interleukin inhibitors show some promise, particularly if initiated early in the course of infection.26, 27, 28 The compassionate use of these treatments in pregnant women is common practice in most settings. Our model successfully predicted the need for ICU admission and the time interval between diagnosis and ICU admission, thereby identifying those women at an increased risk of critical disease. This information could be useful to triage pregnant women with symptomatic COVID-19 so that the healthcare resources and potential therapeutic interventions can be focused on those who are likely to benefit most. Symptomatic women who contact the maternity/emergency services should be screened for urgent admission (miniCOMIT score >10%), and all others should be asked to attend, but not as urgently, for blood work, so that fullCOMIT can be used.

The miniCOMIT model can be used to inform pregnant women of their risk of developing critical COVID-19 if infected and symptomatic, however mild. In both the models, obesity was an independent predictor of severe COVID-19 as assessed by ICU admission.

The vaccination of pregnant women is of particular importance, as pregnant women are at increased risk of severe COVID-19 than their nonpregnant peers and unvaccinated peers. , Vaccine hesitancy is a key challenge in pregnant women who are concerned about the risks of any vaccine not just to themselves, but also to their unborn infant. The use of this model to provide an individualized risk assessment for critical COVID-19 can support pregnant women to make more informed decisions around vaccination. This model will also be very useful for healthcare policy makers and vaccine program directors. Although COVID-19 vaccines seem safe and effective in eliciting an immune response in pregnant women, the number needed to vaccinate to prevent a case of severe COVID-19 is very high in young populations. The use of this baseline characteristics prediction model will enable the vaccine program to prioritize those pregnant women at greatest risk. Targeted prioritization for vaccination will be of key importance in all countries around the world, not just in the current vaccine roll out, but also for future iterations of the vaccine directed at new variants of the virus. This will be essential in the settings and populations where the availability of a suitable vaccine or the infrastructure to support a rapid mass vaccination program may be limited.

Nevertheless, external performance of these prediction models is very important for all clinical applications, and future studies should validate our findings. Moreover, our findings related to an increased risk of other adverse outcomes such as preeclampsia in the high-risk group require further investigation. The improved predictive capability of fullCOMIT stemmed from inflammatory markers, and the relationship between a hyperinflammatory state in COVID-19, hypertension development, and stillbirth should be evaluated in future studies.

Conclusions

We propose 2 prediction models for use in pregnant women with symptomatic COVID-19 that accurately predicted ICU admission and maternal death. A practical calculator is available for external validation and clinical application. fullCOMIT includes baseline characteristics and biochemical markers and can aid the focusing of medical resources on those most in need, whereas miniCOMIT includes the baseline and pregnancy risk factors and can support pregnant women in their decision around whether or not to accept vaccination; it can also enable policy makers to prioritize at-risk pregnant women during the current and future COVID-19 vaccination programs.