The validity of self-reported SARS-CoV-2 results among postpartum respondents.

BACKGROUND: Rapid and reliable health data on SARS-CoV-2 infection among pregnant individuals are needed to understand the influence of the virus on maternal health and child development, yet the validity of self-reported COVID-19 testing and diagnosis remains unknown.
OBJECTIVES: We assessed the validity of self-reported COVID-19 polymerase chain reaction (PCR) testing and diagnosis during delivery among postpartum respondents as well as how diagnostic accuracy varied by respondent characteristics.
METHODS: We validated receipt of a COVID-19 PCR test and test results by comparing self-reported results obtained through an electronic survey to electronic medical record data (gold standard) among a cross-sectional sample of postpartum respondents who delivered at four New York City hospitals between March 2020 and January 2021. To assess validity, we calculated each indicator's sensitivity, specificity and the area under the receiver-operating curve (AUC). We examined respondent characteristics (age, race/ethnicity, education level, health insurance, nativity, pre-pregnancy obesity and birth characteristics) as predictors of reporting accuracy using modified Poisson regression.
RESULTS: A total of 276 respondents had matched electronic record and survey data. The majority, 83.7% of respondents received a SARS-CoV-2 PCR test during their delivery stay. Of these, 12.1% had detected SARS-CoV-2. Among those tested, sensitivity (90.5%) and specificity (96.5%) were high for SARS-CoV-2 detection. The adjusted risk ratio (aRR) of accurate result reporting was somewhat lower among Hispanic women relative to white non-Hispanic women (aRR 0.90, 95% CI 0.90, 1.00) and among those who had public or no insurance vs. private (aRR 0.91, 95% CI 0.82, 1.01), controlling for recall time. CONCLUSION(S): High recall accuracy result reporting for COVID-19 PCR tests administered during labour and delivery suggest the potential for population-based surveys as a rapid mechanism to obtain accurate data on COVID-19 diagnostic history. Additional psychometric research is warranted to ensure accurate recall across respondent subgroups.

Study question

How accurately can respondents self‐report the results of a SARS‐CoV‐2 polymerase chain reaction (PCR) test administered during labour and delivery when queried postpartum? Does recall accuracy vary by respondent characteristics?

What’s already known

Self‐reported data on SARS‐CoV‐2 status in the perinatal period is an alternate, potentially more agile method of data collection than use of electronic medical record data. The validity of self‐reported SARS‐CoV‐2 status, however, remains unclear.

What this study adds

This study provides evidence of the accuracy of self‐reported SARS‐CoV‐2 status during labour and delivery among a multiethnic facility‐based sample of postpartum respondents. Findings support the potential of population‐based surveys as a rapid mechanism to obtain accurate data on COVID‐19 diagnostic history among postpartum individuals.

INTRODUCTION

Pregnant individuals are at increased risk of developing severe COVID‐19 if infected with SARS‐CoV‐2, which can lead to adverse outcomes, including severe illness, pregnancy loss, preterm birth and death. , , Access to rapid, yet reliable health data on COVID‐19 diagnosis is critical among pregnant and postpartum individuals in order to understand the potential influence of the virus on maternal and child health, which has disproportionately affected people of color. , Responding to the immediate need for COVID‐related research is challenged by balancing the timeliness of existing health monitoring system data against selection biases in the types of respondents these data likely underrepresent.

Black and Latinx individuals, for example, are more likely to be under‐ or uninsured than their white counterparts which may lower contact with care and inclusion in health record databases. Among those connected to care, people of colour and who are immigrants may be less likely to respond given the pervasive history of racism and discrimination in medical care settings. , , , Fears regarding loss of confidentiality if research is closely linked with medical care institutions may also reduce participation in research affiliated health surveys. Further, institutional procedures concerning data security, privacy and consent, while necessary, may contribute undue delay in accessing clinical data if not expedited in the evolving context of a pandemic.

An alternative and potentially more agile method of data collection for timely research relies on self‐reported data through anonymous self‐administered questionnaires. Some individuals may prefer a web‐based or non‐hospital linked questionnaire which is anonymous and not identifiable to their health record. Anonymous web and population‐based surveys may reduce selection bias by being more inclusive of those outside formal health monitoring systems. Valid inference from self‐administered surveys, however, relies on the accuracy of self‐reported data. While several diagnostic tools for assessing COVID‐19 status through self‐reported symptomology have been validated, , , , the degree to which respondents can accurately report a positive COVID‐19 diagnosis from polymerase chain reaction (PCR) testing, the gold standard in COVID‐19 detection, remains unclear. Further, the same historic and current systems of racism and discrimination which influences inclusion in health monitoring systems ,  may also influence reporting accuracy if associated with level of education and access to care, for example, which warrants exploration.

This study assesses the validity of self‐reported results of a COVID‐19 PCR test administered during labour and delivery in a cross‐sectional cohort of postpartum respondents who delivered at a participating New York City hospital during the COVID‐19 pandemic. We also examine whether reporting accuracy varies by respondent characteristics.

METHODS

Data source, study design and sample population

We analysed electronic medical record data linked with respondent reports of intrapartum care obtained through a bilingual English and Spanish web‐based survey following delivery at four hospitals (two private and two public) in New York City. Records were matched by unique medical record number or (if unavailable), by respondent name and date of birth. Data were collected as part of the coronaVirus Impact on Birth Equity (VIBE) study which assessed the influence of the COVID‐19 pandemic on birth experiences and health outcomes. Eligible participants gave birth between 01 January 2020 and 31 January 2021 at a participating study facility, were aged 18 and older and provided written informed consent to participate. The average time between delivery and the postpartum survey was 16.1 (SD 10.2) weeks (range 1–53 weeks).

Of 1405 eligible participants who received a text message with an electronic survey link, 367 (26%) opened the link, consented to participate and completed the survey. This analysis was restricted to deliveries that occurred after 23 March 2020 (N = 276), when testing began within hospital Labor & Delivery units. Of this sample, 231 participants received a PCR test. Matched medical record and survey data on receipt of test results were available for 193 respondents. Birthing persons were informed of their results prior to discharge.

Outcome

Respondents were asked about SARS‐CoV‐2 testing and results during their labour and delivery stay with the question, “Have you received a test for coronavirus? If so, what were the results (Negative, Positive).” No other questions were asked to aide recall. Responses were matched against laboratory confirmed PCR test results (“Undetected” or “Detected”) obtained at labour and delivery. Two presumptive positive cases, which were laboratory confirmed as positive, but waiting CDC approval, were coded as “Detected”. Both cases were reported “Positive” by respondents.

Exposure

We examined how COVID‐19 reporting accuracy varied by respondent characteristics: 5‐year age group, race/ethnicity, nativity, public or none vs. private insurance, whether respondents were obese (BMI ≥ 30.0 kg/m2) or non‐obese (BMI < 30.0 kg/m2), which may identify individuals at risk of severe complications of COVID‐19, and caesarean section (vs. vaginal delivery) which may pose a salient competing event affecting recall. We examined differences by birth experience using the Birth Satisfaction Scale‐Revised (BSS‐R), dichotomised at the median to represent high vs. low satisfaction, and whether or not any perceived discrimination was experienced during the hospital stay using the Discrimination in Medical Care Settings Scale (any vs. none). Finally, we examined recall time as a predictor of accuracy, defined as 0–8, 8.1–13.0, 13.1–20 and 20.1–53 weeks, based on the empirical distribution of length of follow‐up.

Statistical analysis

Reporting accuracy was calculated by estimating the sensitivity (‘true positive rate’), specificity (‘true negative rate’), positive and negative predictive values (the proportion of respondents who reported a positive test that were truly positive, or the proportion of respondents who reported a negative test that were truly negative respectively) and area under the receiver‐operating curve (AUC) for each indicator. The AUC summarises accuracy in a single number, ranging between 0 and 1.0 (perfect accuracy) and is the trade‐off between sensitivity plotted against (1 – specificity). Confidence intervals were calculated assuming a binomial distribution. Predictors of reporting accuracy were examined using modified Poisson regression with robust standard errors to estimate the relative risk of correct classification. Adjusted models control for potential confounding by recall time. We also mutually adjusted for all predictors to identify variables associated with the largest magnitude of change in reporting accuracy. Analyses were conducted in Stata Version 15.

Ethics approval

The Institutional Review Board for the Icahn School of Medicine at Mount Sinai approved the VIBE study (#20‐00566).

RESULTS

Clinical and demographic characteristics of respondents are shown in Table 1. In total, 83.7% of birthing persons received a SARS‐CoV‐2 PCR test during their labour and delivery stay. Of those tested, SARS‐CoV‐2 was detected in 12.1% of individuals. Validation results show high sensitivity (90.5%) and specificity (96.5%) for self‐reporting SARS‐CoV‐2 test results received at the time of labour and delivery (Table 2). The positive predictive value was moderate (76.0%) and the negative predictive value was high (98.8%).

TABLE 1

Sociodemographic and clinical characteristics of women delivering at four hospitals during the COVID‐19 pandemic (23 March 2020 to 13 January 2021) in New York City (N = 276)

Indicator	Total number (%) a
SARS‐CoV−2 PCR tested b
No	45 (16.3)
Yes	231 (83.7)
SARS‐CoV−2 status b , c (Of those tested)
Negative	203 (87.9)
Positive	28 (12.1)
Respondent characteristics
Age group (years) b
18–24	30 (10.9)
25–29	52 (18.8)
30–34	75 (27.2)
35–39	72 (26.1)
40–49	21 (7.6)
Missing	26 (9.4)
Race‐ethnicity
Hispanic	72 (26.3)
Black	29 (10.6)
White	115 (42.0)
Asian	46 (16.8)
Other/not reported	12 (4.4)
Education
Not college graduate	175 (64.1)
College graduate or higher	98 (35.9)
Pree‐pregnancy body mass index (kg/m2) d
Underweight (<18.5)	7 (2.5)
Normal weight (18.5–24.9)	135 (48.9)
Overweight (25.0–29.9)	63 (22.8)
Obese (30.0–39.9)	43 (15.6)
Morbidly obese (>40.0)	3 (1.1)
Missing	25 (9.1)
Employment
Unemployed/looking for work	39 (14.1)
Employed/not seeking work	220 (79.7)
Missing	17 (6.2)
Nativity
US born	162 (59.1)
Foreign born	112 (40.9)
Insurance status
Public or no insurance	84 (30.4)
Private insurance	169 (61.2)
Missing	23 (8.3)
Caesarean delivery b
No	167 (60.5)
Yes	79 (28.6)
Missing	30 (10.9)
Recall time (weeks) (Mean, SD)	16.1 (10.2)
Recall time (weeks)
0 to 8	29 (12.6)
8.1 to 13	80 (34.6)
13.1 to 20	84 (36.4)
20.1 to 53	38 (16.5)

n of some covariates do not total 276 due to missing data.

Source: electronic medical record. All other variables derived from participant survey response.

SARS‐CoV‐2 status defined from PCR test results.

Women were asked to report their height (feet and inches) and weight (lbs) before their most recent pregnancy. BMI was calculated using the Imperial formula (weight in pounds/(height in inches2)) × 703 and categorised grade obesity.

TABLE 2

Validation of self‐reported SARS‐CoV‐2 results (in those tested) among postpartum women who delivered during the COVID‐19 pandemic in New York City (23 March 2020 to 13 January 2021)

Test characteristics	N (%)	Validity indicators	Estimate (95% CI)
Matched N	193	Sensitivity	90.5 (69.6, 98.8)
True positive	19	Specificity	96.5 (95.8, 99.9)
False positive	6	Positive predictive value	76.0 (54.9, 90.6)
False negative	2	Negative predictive value	98.8 (95.8, 99.9)
True negative	166	AUC	0.93 (0.89, 0.96)
Self‐report prevalence (%)	13.0	EMR prevalence (%)	10.9

Self‐reported prevalence refers to women's self‐report in survey. EMR Prevalence: Electronic medical record prevalence (reference standard). Sensitivity: Proportion of individuals who truly received a positive result who were classified as having received a positive result by survey question. Specificity: The proportion of individuals who truly did not receive a positive result who were classified as not having received a positive result by survey question. Positive predictive value: The proportion of individuals who reported they were diseased who were truly diseased. Negative predictive value: the proportion of individuals who reported they were non‐diseased who are truly non‐diseased. AUC: area under the receiver‐operating curve. The probability that the survey question will correctly classify a randomly selected set of one positive observation and one negative observation.

The likelihood of correct classification of SARS‐CoV‐2 test results was lower among Hispanic relative to white non‐Hispanic respondents (aRR 0.90, 95% CI 0.80, 1.00) and among those with public or no insurance relative to private (aRR 0.91, 95% CI 0.81,1.01), adjusting for recall time (Table 3). Accurate classification was non‐significantly higher among older age groups relative to those aged 18–24 years and non‐significantly lower among those who were obese, or identified as Black non‐Hispanic or ‘Other’. No consistent differences were observed for other variables. Mutual adjustment for all variables demonstrated similar patterns by race/ethnicity and obesity.

TABLE 3

Likelihood of accurate self‐reported SARS‐CoV‐2 PCR test results (among those tested) during hospital delivery by respondent characteristics using modified Poisson regression with robust error

	SARS‐CoV‐2 PCR tested & matched survey and EMR data available N (%)	RR a (95% CI)	aRR b (95% CI)	aRR c (95% CI)
Age group (years)
18–24	22 (11.4)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
25–29	34 (17.6)	1.08 (0.86, 1.37)	1.09 (0.89, 1.34)	1.27 (0.97, 1.67)
30–34	58 (30.1)	1.17 (0.96, 1.42)	1.14 (0.96, 1.34)	1.34 (1.03, 1.75)
35–39	62 (32.1)	1.17 (0.96, 1.43)	1.15 (0.97, 1.38)	1.32 (1.01, 1.71)
40–49	17 (8.8)	1.14 (0.93, 1.41)	1.11 (0.91, 1.36)	1.25 (0.95, 1.63)
Race/ethnicity
White	93 (48.2)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Black	15 (7.8)	0.93 (0.81, 1.07)	0.94 (0.82, 1.07)	0.88 (0.70, 1.14)
Hispanic	47 (24.4)	0.90 (0.81, 0.99)	0.90 (0.80, 1.00)	0.89 (0.76, 1.05)
Asian	30 (15.5)	0.97 (0.90, 1.03)	0.98 (0.93, 1.04)	1.01 (0.94, 1.09)
Other	8 (4.1)	0.88 (0.67, 1.14)	0.89 (0.69, 1.14)	1.02 (0.96, 1.09)
Nativity
US born	125 (64.8)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Foreign born	68 (35.2)	1.00 (0.93, 1.06)	1.00 (0.94, 1.06)	0.99 (0.91, 1.08)
Insurance status
Private	132 (70.6)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Public or none	55 (29.4)	0.90 (0.82, 1.00)	0.91 (0.81, 1.01)	0.94 (0.84, 1.05)
Educational attainment
College degree or higher	135 (70.0)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Not college graduate	58 (30.0)	0.93 (0.86, 1.02)	0.90 (0.80, 1.03)	1.23 (1.01, 1.53)
Obesity status
Normal or underweight	156 (83.9)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Obese/morbidly obese	30 (16.1)	0.88 (0.76, 1.03)	0.91 (0.80, 1.04)	0.88 (0.73, 1.05)
Discrimination during labour/delivery stay
None perceived	98 (60.1)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Perceived at least one event	65 (39.9)	1.02 (0.96, 1.09)	1.03 (0.94, 1.10)	1.00 (0.93, 1.06)
Birth satisfaction
Low satisfaction	106 (56.4)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
High satisfaction	82 (43.6)	1.06 (1.00, 1.12)	0.99 (0.89, 1.08)	1.00 (0.94, 1.06)
Delivery method
Vaginal delivery	128 (66.3)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
Caesarean delivery	65 (33.7)	0.99 (0.93, 1.06)	0.97 (0.90, 1.05)	0.99 (0.93, 1.05)
Recall time (weeks)
0 to 8	24 (12.4)	1.00 (Reference)	1.00 (Reference)	1.00 (Reference)
8.1 to 13	79 (40.9)	0.95 (0.90, 0.99)	0.95 (0.90, 0.99)	0.96 (0.90, 1.02)
13.1 to 20	60 (31.1)	0.98 (0.95, 1.02)	0.98 (0.95, 1.02)	0.97 (0.91, 1.04)
20.1 to 53	30 (15.5)	0.90 (0.80, 1.01)	0.90 (0.80, 1.01)	0.91 (0.73. 1.13)

Abbreviation: EMR, electronic medical record.

RR, risk ratio.

aRR, risk ratio adjusted for recall time (categorical weeks).

aRR, risk ratio mutually adjusted for all variables in model.

COMMENT

Principal findings

Self‐reported results of a SARS‐CoV‐2 PCR test administered during labour and delivery had high recall accuracy among postpartum respondents when queried using a web‐based survey, between 1 week and 13 months following delivery. A pattern of somewhat lower reporting accuracy was observed among respondents who identified as Black or Hispanic, did not have private health insurance or were class 1 obesity or higher.

Strengths of the study

This validation study adds to a little understood area of research: the validity of self‐reported SARS‐CoV‐2 results among postpartum respondents, a potentially more rapid and inclusive data collection method than electronic record data. As we uncover the potential impact of SARS‐CoV‐2 infection on maternal and child health, data on whether a positive PCR test was received during pregnancy will be critical.

Limitations of the data

This study was unable to validate self‐reporting of infection prior to labour and delivery. Future research is needed to understand reporting accuracy for a positive test received at any time during pregnancy, as well as how the wording and manner of result delivery influences respondent comprehension and recall. Selection bias potentially limits the generalisability of findings. Comparison to a pandemic cohort of similar time frame from the same hospital system and inclusive of all deliveries with electronic health data suggests comparable age distribution and a somewhat lower proportion of white women (42% vs. 53%) and respondents with private insurance (61% vs. 76%) in this study. Positive and negative predictive values are dependent on prevalence, which should be considered when applying results to other settings.

Interpretation

High recall accuracy for self‐reported SARS‐CoV‐2 PCR test results is promising for survey‐based data collection. Lower accuracy among some subgroups suggests that additional psychometric research on question wording to facilitate respondent comprehension and accurate recall is needed.

CONCLUSIONS

Results from this validation analysis suggest the potential for population‐based surveys as a rapid mechanism to obtain accurate data on COVID‐19 diagnostic history among postpartum populations – an at‐risk group for severe COVID‐19‐related complications. Further validation research across the antenatal period and among diverse racial/ethnic groups and clinical risk factors for COVID‐related complications is warranted.

CONFLICT OF INTEREST

The authors declare no competing interests.

AUTHOR'S CONTRIBUTION

TJ, SM, SN and PR contributed to the design and implementation of the research. KM analysed the data. KM wrote the draft of the paper. KG, TJ, SM, SN and PR provided critical revisions to the paper. KM, KG, TJ, SM, SN and PR agreed with manuscript and conclusions. ICMJE criteria for authorship read and met by KM, KG, TJ, SM, SN and PR.