Performance of diagnostic coding and laboratory testing results to measure COVID-19 during pregnancy and associations with pregnancy outcomes.

BACKGROUND: Large-scale evaluation of COVID-19 is likely to rely on the quality of ICD coding. However, little is known about the validity of ICD-coded COVID-19 diagnoses.
OBJECTIVES: To evaluate the performance of diagnostic codes in detecting COVID-19 during pregnancy.
METHODS: We used data from a national cohort of 78,283 individuals with a pregnancy ending between 11 March 2020 and 31 January 2021 in the OptumLabs® Data Warehouse (OLDW). OLDW is a longitudinal, real-world data asset with de-identified administrative claims and electronic health record data. We identified all services with an ICD-10-CM diagnostic code of U07.1 and all laboratory claims records for COVID-19 diagnostic testing. We compared ICD-coded diagnoses to testing results to estimate positive and negative predictive values (PPV and NPV). To evaluate impact on risk estimation, we estimated risk of adverse pregnancy outcomes by source of exposure information.
RESULTS: Of 78,283 pregnancies, 5644 had a laboratory test result for COVID-19. Testing was most common among older individuals, Hispanic individuals, those with higher socioeconomic status and those with a diagnosed medical condition or pregnancy complication; 52% of COVID-19 cases was identified through ICD-coded diagnosis alone, 19% from laboratory test results alone and 29% from both sources. Agreement between ICD-coded diagnosis and laboratory testing records was high 91% (95% confidence interval [CI] 90, 92). However, the PPV of ICD-code diagnosis was low (36%; 95% CI 33, 39). We observed up to a 50% difference in risk estimates of adverse pregnancy outcomes when exposure was based on laboratory testing results or diagnostic coding alone.
CONCLUSIONS: More than one-in-five COVID-19 cases would be missed by using ICD-coded diagnoses alone to identify COVID-19 during pregnancy. Epidemiological studies exclusively relying on diagnostic coding or laboratory testing results are likely to be affected by exposure misclassification. Research and surveillance should draw upon multiple sources of COVID-19 diagnostic information.

Study question

How well do diagnostic codes measure COVID‐19 during pregnancy?

What’s already known

Many studies examining the perinatal health impacts of COVID‐19 have utilised diagnostic coding from medical records to identify infections. However, the validity of these codes among pregnant patients has not yet been evaluated.

What this study adds

In a large cohort of pregnancies, we found that diagnostic coding without laboratory testing information would miss nearly 20% of COVID‐19 cases, resulting in exposure misclassification. Reliance on either diagnostic coding or laboratory testing data exclusively can bias risk estimates for adverse pregnancy outcomes by up to 50%. To avoid misclassification, epidemiology studies should draw from multiple sources of COVID‐19 information.

BACKGROUND

Large‐scale epidemiological studies often draw from existing medical records. These studies commonly use diagnostic coding systems, such as the International Classification of Diseases (ICD) system, to detect conditions of interest. In the context of the ongoing pandemic, large studies are likely to rely on ICD‐coded diagnoses for documenting COVID‐19. Given growing interest in evaluating the effects of COVID‐19 on maternal and infant health, , pregnant persons are one such group where large‐scale evaluation utilising diagnostic coding is likely. However, the validity and reliability of these codes for measuring COVID‐19, particularly among pregnant persons, remain uncertain.

ICD‐10‐CM codes were released in March 2020 for emergency use in order to clinically document diagnoses of COVID‐19.  These codes include U07.1 (COVID‐19 diagnosis without laboratory confirmation) and U07.2 (COVID‐19 diagnosis with laboratory confirmation). While many countries have adopted both codes, in the United States, only U07.1 was adopted. Guidance on the use of U07.1 is to record only a confirmed diagnosis as (a) documented by the provider, (b) documented through positive COVID‐19 test result or (c) through presumptive positive COVID‐19 test result. Analysis of national commercial insurance claims data presents an opportunity to evaluate the accuracy of ICD‐10‐CM codes for two reasons: (1) commercially insured cohorts are closed with extensive information on contact with the medical health system; and (2) since all individuals are insured, financial access to laboratory testing may have less influence on results.

Using national insurance claims data, we aimed to evaluate (a) the ability of ICD‐coded clinical diagnoses to detect COVID‐19 when compared to laboratory testing results; and (b) how reliance on different methods of COVID‐19 detection could impact measurement of associations between COVID‐19 and pregnancy outcomes.

METHODS

Cohort selection

We conducted a claims‐based cohort study using de‐identified administrative claims and electronic health record (EHR) data from the OptumLabs® Data Warehouse (OLDW).  The database includes longitudinal health information on patients and enrollees across the United States. Claims data in OLDW include medical and pharmacy claims, laboratory results and enrolment records for commercial enrollees. Pregnancies, pregnancy outcomes and gestational age were identified and estimated from medical claims data using a validated algorithm based on ICD‐9, Current Procedural Terminology (CPT) codes, Healthcare Common Procedure Coding System (HCPC) and ICD‐9 procedure codes, which was modified for use with ICD‐10‐CM codes (Table S1). All pregnant individuals with a date of delivery after 11 March 2020 (ie declaration of pandemic start) were selected for analysis. We excluded ectopic and molar pregnancies identified and pregnancies where the gestational age was inconsistent with the pregnancy outcome (ie spontaneous abortion at 32 weeks). To avoid right truncation in the cohort and ensure complete capture of medical events during the pregnancy period, we restricted analysis to those with an estimated date of conception before 30 April 2020. COVID‐19 information and pregnancy outcomes were complete in the dataset.

Variable measurement

We extracted physician, facility and laboratory claims records and EHR data for one year preceding and 30 days following the date of delivery. We identified COVID‐19 diagnoses from diagnosis codes (U07.1) in the physician or facility medical claims records. The presence of the diagnostic code indicated the individual had a COVID‐19 diagnosis, and we considered the absence of such a code as no COVID‐19 diagnosis. COVID‐19 laboratory tests and test results were identified using LOINC codes (Table S2) and free text information in the extracted laboratory claims record. We considered a positive COVID‐19 test result as an indication of COVID‐19 and a negative test result as an indication of no COVID‐19. All types of laboratory tests were considered, including serology, as we assumed a priori that infections at any time during pregnancy were potentially influential; however, we acknowledge that some infections detected through serology may have occurred prior to pregnancy, especially if testing was performed early in pregnancy. An encounter with a date of service within three days of the date of the laboratory test date was considered to be the same medical event. For pregnancies with more than one unique laboratory test result, we included any positive test result. For those with consecutive negative test results, we selected any test result coinciding with a COVID‐19 diagnosis in the medical record. For those with consecutive negative laboratory test results without no clinical diagnoses, we selected one negative result at random.

Information on maternal age, race/ethnicity, residence, educational attainment and household income was derived from enrolment data and linked data supplied by an external vendor. Race/ethnicity was defined as Black, Hispanic, Asian or White and was assigned by an external vendor based on a structured, rule‐based system that combines analysis of first names, middle names, surnames and surname prefixes and suffixes with geographic reference files. Values were then categorised to comply with data de‐identification requirements. Education was estimated based on the average level of education achieved among residents within the census block group. Household income was derived using public and private consumer data for the street address of the enrollee. We identified pre‐existing health conditions using medical claims records (Table S3).

Statistical analysis

We estimated the per cent of individuals tested during pregnancy for COVID‐19, overall and among those with a COVID‐19 diagnosis. We compared the rate of testing by sociodemographic and health characteristics using log‐binomial regression models which controlled for calendar week of conception (as a cubic spline term) and estimated gestational age at pregnancy end.

Among those tested for COVID‐19, we estimated the per cent agreement between ICD‐coded diagnoses and laboratory testing results using Kappa's coefficient. We additionally estimated the positive predictive value and negative predictive value of the U07.1 COVID‐19 diagnostic code. We estimated these values overall and by subgroups. To evaluate the influence of increased testing availability and changes in disease prevalence over time, we estimated values during three time periods: March 2020 to May 2020, June 2020 to September 2020 and October 2020 to January 2021. We additionally estimated values by method of laboratory detection (polymerase chain reaction vs. rapid antigen or serology), gestational age at infection (first, second or third trimester) and presence of medical risk factors including chronic medical conditions, pregnancy complications and advanced maternal age.

To evaluate how different sources of information could influence measures of association, we used Cox proportional hazard models treating COVID‐19 as a time‐varying exposure to estimate the risk of adverse birth outcomes associated with prenatal COVID‐19, using three different methods of identifying COVID‐19: laboratory testing only, diagnostic coding only or a combination of both. Models adjusted for maternal age, race/ethnicity, household income, pre‐existing medical conditions and week of conception (cubic spline).

Missing data

Missing covariate information was imputed using multiple imputation by bootstrapping and expectation‐maximization algorithm with 50 sets of imputations.

Ethics approval

Because this study involved analysis of pre‐existing, de‐identified data, it was considered exempt from Institutional Review Board approval.

RESULTS

Of the 86,111 pregnancies identified, 78,283 pregnancies had a date of conception after 30 April 2020 and were eligible for inclusion in the analysis (Figure 1); 5644 (7.2%, 95% CI 7.0%, 7.4%) pregnancies had a laboratory test result for COVID‐19. Laboratory testing peaked between June and July 2020 (Figure 2). The majority of laboratory tests (75.4%; n=4253) were performed using RT‐PCR; 1391 (24.6%) laboratory tests were performed by serology or rapid antigen testing. COVID‐19 testing rates increased with maternal age (RR ≥40 years vs. <24 years: 1.37, 95% CI 1.23, 1.51) and was more frequent among pregnancies with an asthma diagnosis (RR 1.26, 95% CI 1.18, 1.35), a diagnosed pregnancy complication (RR 1.09, 95% CI 1.04, 1.14) and among pregnancies delivered by caesarean delivery (RR 1.23, 95% CI 1.17, 1.29) (Table 1). Testing was less common among non‐Hispanic Black pregnant individuals (RR 0.91, 95% CI 0.83, 0.99) and those residing in rural areas compared with metropolitan (RR 0.58, 95% CI 0.42, 0.73). Among those with a clinical diagnosis of COVID‐19, similar factors were associated with COVID‐19 testing, with exception to caesarean delivery, age and race/ethnicity.

FIGURE 1

Flow diagram for selection of pregnancies into the study cohort

FIGURE 2

Proportion of pregnancies tested for COVID‐19—OptumLabs Data Warehouse, United States, 11 March to 31 January 2021. Y‐axis values are suppressed to comply with requirements for release of OptumLabs data

TABLE 1

Rate ratios comparing the rate of testing during pregnancy—OptumLabs Data Warehouse, United States, 1 March to 31 December 2020

	All pregnancies			Pregnancies diagnosed with COVID−19
	Number of pregnancies	Tested		Number of pregnancies	Tested
		n (%)	aRR a (95% CI)		n (%)	aRR a (95% CI)
Total	78,283	5644 (7.2)	–	2515	596 (23.7)	–
Maternal age (years)
<24	5684	345 (6.1)	1.00 (Reference)	277	62 (22.4)	1.00 (Reference)
25–29	19,496	1325 (6.8)	1.14 (1.03, 1.26)	773	195 (25.2)	1.14 (0.89, 1.39)
30–34	29,207	2157 (7.4)	1.27 (1.16, 1.38)	834	198 (23.7)	1.11 (0.87, 1.36)
35–39	18,348	1409 (7.7)	1.38 (1.27, 1.49)	481	116 (24.1)	1.15 (0.89, 1.42)
≥40	5548	408 (7.3)	1.37 (1.23, 1.51)	150	25 (16.7)	0.79 (0.38, 1.21)
Race/ethnicity
White, non‐Hispanic	48,969	3509 (7.2)	1.00 (Reference)	1346	309 (22.9)	1.00 (Reference)
Black, non‐Hispanic	9205	603 (6.5)	0.91 (0.83, 0.99)	380	83 (21.8)	0.97 (0.76, 1.18)
Hispanic	13,013	1043 (8.0)	1.11 (1.04, 1.17)	601	154 (25.6)	1.15 (0.98, 1.32)
Asian	7096	489 (6.9)	0.97 (0.88, 1.06)	188	50 (26.6)	1.13 (0.88, 1.39)
Education
≤High school graduate	16,771	1747 (10.4)	1.00 (Reference)	699	167 (23.9)	1.00 (Reference)
Some college	40,135	2714 (6.8)	0.84 (0.77, 0.91)	1222	284 (23.2)	0.96 (0.77, 1.15)
≥College graduate	21,377	1183 (5.5)	0.82 (0.76, 0.88)	594	145 (24.4)	0.93 (0.76, 1.11)
Residence
Metropolitan	70,452	5,266 (7.5)	1.00 (Reference)	2215	549 (24.8)	1.00 (Reference)
Micropolitan	4297	224 (5.2)	0.69 (0.56, 0.82)	164	29 (17.7)	0.66 (0.31, 0.99)
Small town/Rural	3534	154 (4.4)	0.58 (0.42, 0.73)	136	18 (13.2)	0.51 (0.07, 0.95)
Household income
<$40,000	15,564	1079 (6.9)	1.00 (Reference)	568	120 (21.1)	1.00 (Reference)
$40–74,000	17,340	1183 (6.8)	0.99 (0.91, 1.07)	546	116 (21.2)	0.99 (0.77, 1.22)
$75–124,000	21,575	1449 (6.7)	0.97 (0.89, 1.04)	681	182 (26.7)	1.27 (1.07 1.47)
$125–199,000	13,811	1067 (7.7)	1.11 (1.03, 1.19)	417	102 (24.5)	1.15 (0.91, 1.38)
≥$200,000	9993	866 (8.7)	1.27 (1.18, 1.35)	303	76 (25.1)	1.20 (0.95, 1.44)
Medical conditions
Any medical condition	9972	829 (8.3)	1.16 (1.09, 1.23)	366	96 (26.2)	1.13 (0.94, 1.32)
Asthma	5294	480 (9.1)	1.26 (1.18, 1.35)	192	56 (29.2)	1.27 (1.03, 1.50)
Hypertension	3652	267 (7.3)	1.01 (0.89, 1.12)	138	32 (23.2)	0.96 (0.64, 1.27)
Immune disorder	160	14 (8.7)	1.24 (0.75, 1.74)	13	<11	–
Neurological disorder	237	15 (6.3)	0.91 (0.42, 1.40)	<11	<11	–
No medical condition	68,311	4815 (7.0)	1.00 (Reference)	2149	500 (23.3)	1.00 (Reference)
Pregnancy complications
Any complication	23,971	1801 (7.5)	1.09 (1.04, 1.14)	820	196 (23.9)	1.02 (0.87, 1.17)
Pre‐eclampsia	3443	269 (7.8)	1.03 (0.91, 1.15)	117	30 (25.6)	1.03 (0.71, 1.36)
Gestational diabetes	8098	639 (7.9)	1.01 (0.93, 1.09)	279	59 (21.1)	0.88 (0.64, 1.12)
Bleeding in pregnancy	4264	325 (7.6)	1.10 (0.99, 1.20)	149	53 (35.6)	1.60 (1.37, 1.83)
Haemorrhage in early pregnancy	13,061	951 (7.3)	1.13 (1.07, 1.20)	442	107 (24.2)	1.03 (0.85, 1.21)
Hyperemesis	3478	269 (7.7)	1.04 (0.92, 1.15)	118	30 (25.4)	1.07 (0.74, 1.39)
Antepartum haemorrhage	4883	339 (6.9)	1.01 (0.91, 1.11)	190	60 (31.6)	1.40 (1.18, 1.62)
Postpartum haemorrhage	3835	285 (7.4)	0.96 (0.85, 1.07)	150	31 (20.7)	0.83 (0.50, 1.15)
No complications	54,312	3843 (7.1)	1.00 (Reference)	1695	400 (23.6)	1.00 (Reference)
Birth outcome
Miscarriage	7366	320 (4.3)	1.35 (1.16, 1.55)	80	<11	–
Medical termination	1187	22 (1.9)	0.49 (0.06, 0.93)	15	<11	–
Stillbirth	401	24 (6.0)	0.87 (0.49, 1.26)	14	<11	–
Preterm birth	4630	300 (6.5)	0.88 (0.76, 1.00)	190	41 (21.6)	1.05 (0.80, 1.30)
Spontaneous	3526	224 (6.3)	0.85 (0.72, 0.99)	134	26 (19.4)	0.78 (0.43, 1.13)
Clinician‐initiated	1104	76 (6.9)	0.96 (0.73, 1.18)	56	25 (44.6)	1.87 (1.53, 2.22)
Premature rupture of membranes	8716	587 (6.7)	0.88 (0.79, 0.96)	277	60 (21.7)	0.87 (0.64, 1.11)
Caesarean delivery	12,286	1087 (8.8)	1.23 (1.17, 1.29)	405	98 (24.2)	0.99 (0.80, 1.18)

Relative rate of testing, controlling for timing of conception (cubic spline) and estimated gestational age at pregnancy end.

Of the 5644 pregnant individuals with information on COVID‐19 testing, 736 had a record of COVID‐19: 380 (51.6%) had only a COVID‐19 diagnosis, 140 (19.0%) had only a positive laboratory test, and 216 (29.3%) had both a positive test and a COVID‐19 diagnosis code (Figure 3). The remaining 4908 pregnant individuals tested negative for COVID‐19 and had no clinical diagnosis of COVID‐19. Agreement between laboratory testing and diagnostic coding was high (90.8%; 95% CI 90.0, 91.5). When compared to laboratory testing information, clinical diagnoses of COVID‐19 had a positive predictive value of 36.2% (95% CI 33.3, 39.3) and a negative predictive value of 97.2% (95% CI 96.9, 97.5) (Table 2). Positive predictive values increased after May 2020 and were highest for infections occurring during third trimester (42.1%, 95% CI 38.1, 46.3). We observed no differences in positive predictive values by medical risk factors.

FIGURE 3

Source of COVID‐19 identification among pregnant women—OptumLabs Data Warehouse, United States, March 2020–January 2021. Y‐axis values are suppressed to comply with requirements for release of OptumLabs data

TABLE 2

Comparison of clinical diagnosis of COVID‐19 and COVID‐19 testing results for pregnant women (n = 5644)—OptumLabs Data Warehouse, United States, March 2020–January 2021

	Tested positive (n = 356)	Diagnosis (n = 596)	Agreement (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
	%	%
Overall	6.6	10.5	90.8 (90.0, 91.5)	60.7 (55.4, 65.8)	92.8 (92.1, 93.5)	36.2 (33.3, 39.3)	97.2 (96.9, 97.5)
By month of laboratory test
Mar 2020–May 2020	<3.1 a	10.0	89.6 (87.5, 91.5)	40.0 (19.1, 63.9)	90.7 (88.6, 92.5)	8.3 (4.9, 13.9)	98.6 (98.0, 99.0)
Jun 2020–Sept 2020	7.1	9.9	91.3 (90.4, 92.2)	58.7 (52.5, 64.8)	93.8 (93.0, 94.6)	41.9 (37.9, 46.1)	96.7 (96.3, 97.2)
Oct 2020–Jan 2021	7.6	13.4	89.9 (87.9, 91.7)	72.1 (60.9, 81.7)	91.4 (89.4, 93.1)	40.7 (34.9, 46.8)	97.6 (96.5, 98.3)
By laboratory testing method
RT‐PCR	8.1	11.0	90.8 (89.9, 91.7)	61.2 (55.8, 66.4)	93.4 (92.6, 94.2)	44.9 (41.3, 48.5)	96.5 (96.0, 96.9)
Rapid antigen or serology	<3.1 a	9.2	90.7 (89.1, 92.2)	46.1 (19.2, 74.9)	91.1 (89.5, 92.6)	4.7 (2.6, 8.3)	99.5 (99.1, 99.7)
By gestation at time of laboratory test
First trimester	7.8	14.6	86.9 (83.8, 89.7)	59.5 (43.3, 74.4)	89.3 (86.2, 91.8)	32.1 (24.8, 40.3)	96.3 (94.7, 97.4)
Second trimester	6.2	10.8	89.1 (87.5, 90.5)	49.1 (39.5, 58.7)	91.7 (90.3, 93.0)	28.2 (23.5, 33.5)	96.5 (95.8, 97.0)
Third trimester	6.1	10.1	92.3 (91.4, 93.2)	67.3 (60.4, 73.7)	94.0 (93.1, 94.8)	42.1 (38.1, 46.3)	97.8 (97.3, 98.2)
By health and medical factors
High‐risk medical condition
Yes	6.4	11.6	89.0 (86.7, 91.1)	54.7 (40.5, 68.4)	91.4 (89.2, 93.3)	30.2 (23.6, 37.7)	96.7 (95.6, 97.5)
No	6.3	10.4	91.1 (90.3, 91.9)	61.7 (56.0, 67.2)	93.1 (92.3, 93.8)	37.4 (34.2, 40.7)	97.3 (96.9, 97.7)
Diagnosis of a pregnancy complication
Yes	5.6	10.9	91.1 (89.7, 92.3)	67.3 (57.3, 76.3)	92.5 (91.1, 93.7)	34.7 (30.0, 39.7)	97.9 (97.3, 98.4)
No	6.6	10.4	90.7 (89.7, 91.6)	58.0 (51.7, 64.2)	93.0 (92.1, 93.8)	37.0 (33.4, 40.8)	96.9 (96.4, 97.3)
Advanced maternal age
Yes	4.5	7.8	93.1 (91.8, 94.2)	59.3 (47.8, 70.1)	94.6 (93.5, 95.7)	34.0 (28.3, 40.3)	98.0 (97.5, 98.5)
No	7.2	11.9	89.7 (88.7, 90.7)	61.1 (55.1, 66.9)	91.9 (91.0, 92.8)	36.9 (33.6, 40.4)	96.8 (96.3, 97.3)

Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value. a Exact percentages are suppressed to comply with requirements for data release.

Exact percentages are suppressed to comply with requirements for data release.

Based on the two sources of information, although the incidence proportion of COVID‐19 would be higher if relying on laboratory testing alone (6.3%, 95% CI 5.7%, 7.0%) compared with diagnostic coding alone (3.2%, 95% CI 3.1, 3.3%), the proportionate severity would be similar (Table 3). For measures of association, the direction was the same for the outcomes considered; however, the strength of association was not. For example, when caesarean delivery was measured by laboratory testing only, there was a weaker association between COVID‐19 and caesarean when exposure was measured by laboratory testing alone (aHR 1.32, 95% CI 0.95, 1.84) compared with diagnostic coding alone (aHR 2.04, 95% CI 1.74, 2.39) (Table 3). We observed a stronger association between COVID‐19 and preterm birth when exposure was measured by laboratory testing alone (aHR 2.41, 95% CI 1.53, 3.79) compared with diagnostic coding alone (aHR 1.87, 95% CI 1.41, 2.47). Effect estimates for all other outcomes were within 0.23 units from each other.

TABLE 3

Incidence of COVID‐19 and measure of association between COVID‐19 and adverse pregnancy outcomes—OptumLabs Data Warehouse, United States, March 2020–January 2021

	Source of COVID−19 information
	Laboratory test result (n = 5644)	Diagnostic coding (n = 78,283)	Both laboratory testing or diagnostic coding (n = 78,283)
Number of COVID−19 cases	356	2515	2655
Incidence proportion, % (95% CI)	6.3 (5.7, 7.0)	3.2 (3.1, 3.3)	3.4 (3.3, 3.5)
Proportionate severity, % (95% CI) a	3.2 (1.9, 5.5)	3.6 (2.9, 4.4)	3.4 (2.8, 4.2)
Association with birth outcome (aHR b [95% CI])
Preterm birth	2.41 (1.53, 3.79)	1.87 (1.41, 2.47)	2.07 (1.65, 2.61)
Premature rupture of membranes	1.36 (0.84, 2.20)	1.59 (1.30, 1.95)	1.54 (1.27, 1.87)
Spontaneous preterm labour	1.97 (1.15, 3.38)	1.87 (1.41, 2.47)	1.80 (1.38, 2.35)
Caesarean delivery	1.32 (0.95, 1.84)	2.04 (1.74, 2.39)	1.99 (1.71, 2.31)
Foetal growth restriction	2.13 (1.41, 3.24)	2.15 (1.80, 2.57)	2.04 (1.72, 2.43)

Abbreviation: CI, confidence interval.

Proportionate severity indicates the proportion of COVID‐19 cases that were considered severe (ie required admission to intensive care unit, mechanical ventilation or extracorporeal membrane oxygenation, or diagnosis of acute respiratory distress syndrome).

Hazard ratios accounting for COVID‐19 infection as a time‐varying exposure and adjusting for maternal age, race/ethnicity, household income, pre‐existing medical conditions and week of conception (cubic spline).

COMMENT

Principal findings

This large, national study comparing clinical diagnoses to COVID‐19 laboratory testing results provides important insights into the quality of existing data for measuring COVID‐19 infections during pregnancy. In this commercially insured cohort, we found that the rate of COVID‐19 testing did not appear to be random and varied by sociodemographic and health factors. More than one‐in‐five COVID‐19 cases would be missed by ICD‐coded diagnoses alone, and 64% of those with a COVID‐19 diagnosis had a negative laboratory result. These results imply that reliance on diagnostic coding or laboratory testing results alone from medical records will result in misclassification of disease status, and depending on which measurement applied, could change measures of association with perinatal health outcomes. Epidemiological studies relying on existing health data should be aware of these considerations and how they may influence their study findings.

Strengths of the study

Our study had several key strengths. This study drew from a large national cohort of commercially insured US pregnant individuals with comprehensive health and medical information. Analyses within a commercially insured population presented the opportunity to perform our analyses with a population where access to clinical testing was not limited (a major strength of the study). Furthermore, in this population, clinical information from multiple sources can be readily pooled.

Limitations of the data

Despite these strengths, our study had several limitations to consider. First, the reliance on commercial insurance claims records, while useful, meant that our findings may not necessarily generalise to uninsured or publicly insured populations. Second, similar to evaluations of influenza, we opted to use laboratory testing results as the ‘gold standard’ for our comparisons. However, clinical guidance allows for diagnosis of COVID‐19 in the absence of a positive laboratory test if there are clinical observations or epidemiological reasoning to indicate COVID‐19 infection. As a result, it is possible that several of our test‐negative individuals were false negatives, which would have biased our estimates of positive predictive values.

Interpretation

COVID‐19 diagnoses are relatively novel, and few studies have evaluated their validity. A recent Danish study conducted among 710 diagnosed patients with COVID‐19 between February and May 2020 showed that compared with medical record review, ICD‐coded COVID‐19 diagnoses had a positive predictive value of 99%. An important distinction between this study and ours is that for our comparison of diagnoses and laboratory testing information, we restricted our sample to tested individuals to allow comparison. Medical chart review of those who test negative but have a clinical diagnosis would be helpful for elucidating reasons for diagnosis, but was not possible in the current study. A cohort study of 2201 individuals tested for COVID‐19 at the University of Utah Medical Center showed that ICD codes for fever, cough and dyspnoea codes had low sensitivity and negative predictive values. However, this study did not evaluate the validity of COVID‐19 diagnostic codes. A single‐centre validation study among 3,905 paediatric patients indicated U07.1 diagnosis had a sensitivity, specificity, positive predictive value and negative predictive value exceeding 90%. Our findings do not align with these previous studies—which could be due to differences in patient population, clinical behaviours or a combination of both.

Our findings suggest existing data may be prone to several biases. First, in this insured cohort of pregnant individuals, although individuals would have had access to clinical testing, the rate of testing was not independent of sociodemographic and health factors. Given these factors have also been associated with more serious COVID‐19 infection, this could indicate that detection of COVID‐19 is more common among those more prone to severe infection. We believe these results show that observational studies should consider the sociodemographic factors correlated with testing when using large observational databases to perform COVID‐19 research. Second, 64% of those with a clinical diagnosis had a negative COVID‐19 test. Clinical guidance is consistent with giving a diagnosis in the absence of a positive COVID‐19 test if clinical symptoms indicate. Furthermore, false negatives range from 21%–67%, depending on the duration of illness at the time of testing. , , As a result, without medical chart review, it is difficult to say whether these diagnoses reflect true COVID‐19 cases. Since we relied on clinical records, our study would not have detected COVID‐19 among asymptomatic individuals who did not present for medical care or routine screening. As a result, it is possible that we underestimated the true incidence of COVID‐19 among pregnant persons.

Thirty‐nine per cent of COVID‐19 cases detected through laboratory testing had no diagnosis of COVID‐19, indicating that reliance on coded illnesses alone would miss these infections, including asymptomatic infections. Our analyses of how different sources of COVID‐19 information may influence effect estimates highlight that the use of laboratory testing information only or clinical diagnostic information only could result in somewhat different effect estimates for several key outcomes. We hypothesise, therefore, that differences in the measurement of the outcome in published studies  may be somewhat attributable to methods of measurement of the outcome. However, our results did not show consistently weaker or stronger effect estimates when clinical diagnosis was the sole means of identifying infection, suggesting that additional factors may contribute to variation in these estimates.

CONCLUSIONS

We believe our results highlight the importance of considering the source of information on COVID‐19 infection among pregnant individuals and how different sources of information may influence effect estimations. Regardless, utilisation of one data source alone is likely to miss a substantial portion of cases, resulting in measurement error and misclassification bias. It is important for epidemiological evaluations to consider these limitations when conducting research and interpreting findings.

CONFLICT OF INTERESTS

The authors have no conflicts of interest to disclose.

AUTHOR CONTRIBUTIONS

AKR extracted and prepared data and oversaw all aspects of project implementation. OA and SS contributed to the study design. OA and SS contributed to the development of the analytic plan. AKR performed the statistical analyses and prepared study results. AKR, OA and SS contributed to the interpretation of findings. AKR led the drafting of the report, and all co‐authors contributed to revising of the report and approved the final version.

Table S1‐S3

Click here for additional data file.