A Novel Method to Improve the Identification of Time of Intubation for Retrospective EHR Data Analysis During a Time of Resource Strain, the COVID-19 Pandemic.

Accurate determinations of the time of intubation (TOI) are critical for retrospective electronic health record (EHR) data analyses. In a retrospective study, the authors developed and validated an improved query (Ti) to identify TOI across numerous settings in a large health system, using EHR data, during the COVID-19 pandemic. Further, they evaluated the affect of Ti on peri-intubation patient parameters compared to a previous method-ventilator parameters (Tv). Ti identified an earlier TOI for 84.8% (n = 1666) of cases with a mean (SD) of 3.5 hours (15.5), resulting in alternate values for: partial pressure of arterial oxygen (PaO 2 ) in 18.4% of patients (mean 43.95 mmHg [54.24]); PaO 2 /fractional inspired oxygen (FiO 2 ) in 17.8% of patients (mean 48.29 [69.81]), and oxygen saturation/FiO 2 in 62.7% (mean 16.75 [34.14]), using the absolute difference in mean values within the first 4 hours of intubation. Differences in PaO 2 /FiO 2 using Ti versus Tv resulted in the reclassification of 7.3% of patients into different acute respiratory distress syndrome (ARDS) severity categories.

Introduction

Intubation is both an important outcome measure along the spectrum of all respiratory illnesses and a marker of disease severity. Time of intubation (TOI) is an important point to identify in the electronic health record (EHR) as it allows researchers to study the association of pre- and postintubation variables with mortality and other relevant patient-centered outcomes. This type of knowledge, determined retrospectively, can serve as hypothesis generating to test interventions prospectively. Further, accurately extracting peri-intubation blood gas analyses and postintubation mechanical ventilation parameters from the EHR will allow researchers to understand the state of lung compromise at the TOI, before the impact of human intervention through invasive procedures and mechanical ventilation.[1] Even small discrepancies between derived TOI and actual TOI may alter peri-intubation data, affect outcomes research, and decrease chances that the work could have widespread generalizability and be accurately reproduced.

Previous research has shown that accurate determination of whether a patient was intubated and TOI is difficult.[2,3] Exacerbating this problem, surges of hospitalized patients during the current COVID-19 pandemic have made timely charting in the electronic health record difficult. Surrogates for identifying onset of mechanical ventilation have been employed previously[2,4-7] but have not been evaluated across multiple hospital settings during a time of unprecedented patient volume. The purpose of this study is to develop and validate an improved reference standard for determining TOI across the emergency, internal medicine, and critical care settings of a large health system during a time of resource strain, the COVID-19 pandemic.

Methods

Study Population

The authors conducted a retrospective study of all admitted internal medicine patients who tested positive for SARS-CoV-2 by nasopharyngeal PCR, 18 years or older, intubated with acute respiratory distress syndrome (ARDS), within 12 hospitals of the Northwell Health system between March 1, 2020, and April 30, 2020, to determine whether a new search query (Ti) would more accurately identify the correct TOI than a previous query using only ventilator parameters (Tv).

The internal validation cohort included a randomly selected subset of 1979 patients from the Northwell COVID ARDS (NorthCARDS) dataset (Figure 1). The external validation cohort included 8236 nonsurgical patients, 18 years or older, from the intensive care unit of Beth Israel Deaconess Medical Center, using publicly available MIMIC-III database (Medical Information Mart for Intensive Care III), which is sourced from 2 different clinical information systems, namely CareVue (n = 4657) and MetaVision (n = 3600).[8]

Figure 1.

Inclusion criteria for MIMIC and NorthCARDS. MIMIC, medical information mart for intensive care III; NorthCARDS, Northwell COVID ARDS.

CareVue and MetaVision datasets represent 35 632 and 22 046 ICU admissions, respectively. The authors included adult patients (18 y and older) admitted to a nonsurgical service, which leads to 13 157 and 11 640 admissions in CareVue and MetaVision, respectively. After running Ti, 4657 and 3600 were found to have been intubated in CareVue and MetaVision, respectively (Figure 1).

The Northwell Health Institutional Review Board considered this study as minimal risk and therefore waived the requirement for informed consent.

Data Collection

Data elements used to develop the logic behind Ti include as follows: TOI authored in an endotracheal intubation procedure note (from the emergency department (ED) or as an inpatient); “Start Time” from the mechanical ventilation record; mechanical ventilation parameters—positive end expiratory pressure (PEEP), tidal volume, peak inspiratory pressure, mean airway pressure, plateau pressure, and set/total respiratory rates—extracted from the mechanical ventilation record, excluding average volume-assured pressure support, bilevel positive airway pressure, and continuous positive airway pressure devices; administered medications—etomidate, succinylcholine, cisatracurium, rocuronium, vecuronium, propofol, fentanyl; documentation of “ventilator” in the oxygen delivery method extracted from nursing notes; fractional inspired oxygen (FiO2) extracted from nursing notes and the mechanical ventilation record; oxygen saturation (SpO2) extracted from nursing notes; partial pressure of arterial oxygen (PaO2); arterial pH (pH); partial pressure of arterial CO2 (PaCO2); PaO2/FiO2 (P/F) ratio; SpO2/FiO2 (S/F) ratio. Data elements including mechanical ventilation parameters, medication orders, vital signs and oxygen delivery methods, intubation notes, and lab results were exported from the Sunrise EHR database and Allscripts database to a centralized structured query language (SQL) database for further analysis.

Missing Data

Missing data for FiO2 was imputed using the previous value documented and was continued until a new value was entered into the EHR. Data for SpO2, PaO2, pH, and PaCO2 was considered missing if it was not available within the prespecified time window after TOI, for both Ti and Tv, and, therefore, was left out of calculations of mean and median. These variables were only reported if data were available within 4 hours after TOI using both Ti and Tv. The number of patients that had missing values, and therefore left out of the data reported in Table 2, for either Ti or Tv, can be found in the supplement Table (Supplemental Digital Content 1, available at http://links.lww.com/AJMQ/A65) for the following time windows: 2 hours, 4 hours, 8 hours, and 12 hours.

Table 2.

Summary of Impact of Ti on the Mean of All Recorded Values of Different Variables Within 4 H After Intubation ().

Nonzero    Δ¯ X    *	NorthCARDS	MIMIC-III(CareVue)	MIMIC-III(MetaVision)
FiO_2 (% oxygen)
n (%)	569 (29.1%)	335 (7.4%)	1,018 (29.8%)
Mean (SD)	8.71 (11.29)	10.50 (9.52)	10.48 (13.42)
Median (IQR)	4.00 (1.67–11.50)	8.25 (4.17–12.55)	6.50 (2.92–13.04)
SpO_2 (% oxygen)
n (%)	1,160 (60.4%)	264 (5.8%)	643 (18.6%)
Mean (SD)	1.72 (2.09)	0.71 (0.87)	0.99 (3.62)
Median (IQR)	1.00 (0.40–2.22)	0.33 (0.13–0.98)	0.36 (0.16–0.89)
PaO_2 (mmHg)
n (%)	223 (18.4%)	70 (2.2%)	167 (7.9%)
Mean (SD)	43.95 (54.24)	49.50 (53.95)	38.27 (46.92)
Median (IQR)	26.00 (12.33–52.25)	35.00 (11.38–63.54)	23.00 (9.08–46.00)
PH
n (%)	202 (18.4%)	72 (2.2%)	169 (7.9%)
Mean (SD)	0.07 (0.07)	0.05 (0.05)	0.04 (0.05)
Median (IQR)	0.04 (0.02–0.09)	0.04 (0.01–0.07)	0.02 (0.01–0.05)
PaCo_2 (mmHg)
n (%)	198 (16.9%)	64 (2.0%)	162 (7.7%)
Mean (SD)	9.55 (9.84)	6.87 (8.67)	4.24 (4.47)
Median (IQR)	7.00 (3.00–12.00)	4.00 (1.50–8.12)	2.50 (1.04–5.80)
P/F ratio
n (%)	214 (17.8%)	24 (1.4%)	31 (3.5%)
Mean (SD)	48.29 (69.81)	53.34 (50.81)	45.49 (44.01)
Median (IQR)	27.46 (12.71–56.33)	40.62 (19.00–56.31)	35.83 (13.38–59.62)
S/F ratio
n (%)	1,203 (62.7%)	94 (2.3%)	253 (8.6%)
Mean (SD)	16.75 (34.14)	20.48 (20.98)	23.72 (54.45)
Median (IQR)	5.44 (1.59–15.44)	13.42 (4.31–29.22)	12.11 (2.71–30.75)

(*) Absolute value of difference between (the mean of all recording(s) of the variable within 4 h after TOI using Ti) and (the mean of all recording(s) of the variable within 4 h after TOI using Tv) for a given patient (see Figure 2). The table shows the descriptive statistics for nonzero values across all patients in different cohorts. The mean and median values represent the absolute value of this difference and do not imply direction (ie, higher or lower). Data were only reported if it was available within 4 h of TOI for both Ti and Tv. Table (Supplemental Digital Content 1, available at http://links.lww.com/AJMQ/A65) shows the number of patients that had missing values for either Ti or Tv and therefore left out of the data reported in this table.

Abbreviation: IQR, interquartile range; MIMIC-III, Medical Information Mart for Intensive Care III; NorthCARDS, Northwell COVID ARDS; TOI, time of intubation.

Figure 2.

Illustration of the calculation of primary and secondary outcome measures. is the difference between TOI obtained by the algorithm (Ti) and by the first evidence of ventilator parameter (Tv). Suppose PaO2 has been measured 4 times at the specified points ( to ). The figure illustrates the calculation of 2 measures for the effect of TOI on PaO2: the mean PaO2 within a 4-hour window ( and the first value of PaO2 within the same time frame (). TOI, time of intubation.

Algorithm Generation/Query Development

Initially, 100 randomly selected charts of 3176 intubated patients were manually reviewed to determine TOI (first intubation) and data points that most accurately identified this time point were recorded. Validation of intubation was done by ensuring that mechanical ventilation parameters (ie, PEEP, tidal volume, peak inspiratory pressure) were entered into the mechanical ventilation record, patients were receiving sedatives indicative of being on invasive mechanical ventilation (ie, propofol or fentanyl), and invasive mechanical ventilation was confirmed in the progress note for that specific day.

The following entries in the EHR were determined to be the most common and accurate data points that validated TOI: TOI authored in an endotracheal intubation procedure note (from the ED or as an inpatient); “Start Time” from the mechanical ventilation record; time of first documentation of mechanical ventilation parameters in the mechanical ventilation record; time of first completed injection of medications used for intubation such as etomidate, succinylcholine, cisatracurium, rocuronium, vecuronium; time of initiation of continuous infusion propofol or fentanyl (minimum duration of 5 min); and finally, documentation in nursing ED or inpatient notes regarding use of “ventilator” in the oxygen delivery method for each patient.

Next, a data processing and analytics pipeline were developed using Python with over 2500 lines of code (Ti). Two versions of the code were developed for MIMIC-III and NorthCARDS based on each database’s schema. The code for MIMIC-III is available at https://github.com/amir-gandomi/Time_of_Intubation.

Further refinement of Ti was done after manual review of a total of 50 new charts of randomly selected patients, using Ti to find TOI, found the following issue: patients were included in the data set that were intubated for a surgical procedure. Therefore, Ti was modified to remove all patients intubated for a surgical procedure.

Subsequent validation of Ti was done by manually reviewing a new set of 75 randomly selected patient charts where at least an 8-hour discrepancy between time of first documentation of mechanical ventilation parameters, and any other criteria was found to determine whether TOI was accurate. Of these 75 charts, 63 had the correct TOI. Of the 12 that were incorrect, the following reasons were found: 4 charts had data entry errors; 7 were transfers from another hospital within the health system and were under separate visits—corrected after multiple visits per patient were merged into 1 visit; and 1 was intubated prior to hospital admission.

Data Validation

Internal validation was performed by chart review of a new set of 100 randomly selected intubated patients with COVID-19 ARDS. Three critical care trained physicians and one hospitalist reviewed 25 charts each and determined whether the derived TOI, using Ti, was accurate based on chart review. Ti was found to be accurate in 97% of cases reviewed. Of the 3 cases that did not represent the actual TOI, 2 were data entry errors and one was intubated before hospital arrival. Although Ti identified the earliest time of first documentation of ventilator support for these 3 cases, it did not reflect the actual TOI. There were also 4 other discrepancies found between reviewers and Ti results. Of these disputed cases, the TOI generated by Ti was found to be closer to the actual TOI than the TOI determined by the reviewer, based on an independent review by 2 separate reviewers unfamiliar with the cases.

External validation was performed on intubated adult patients, 18 years or older, admitted to a nonsurgical ICU using CareVue and MetaVision datasets (MIMIC-III) (Figure 1). One author, trained in verifying TOI from the electronic health record, reviewed 50 charts from CareVue and 50 charts from Metavision; Ti was found to be accurate in 94% and 92% of cases, respectively.

Outcome Measures

The main outcome measure is the number of patients with an earlier TOI using Ti. This is reported as the number of patients with a nonzero (hours) time difference between the TOI using Ti and Tv with associated mean (SD) and median (interquartile range [IQR]) time differences. Secondary outcomes evaluated the impact of Ti on the following variables: FiO2, SpO2, PaO2, pH, PaCO2, P/F, S/F. The authors used 2 measures for this evaluation, both using a window of 4 hours from the TOI using Ti and Tv: (1) the absolute value of difference between the mean of all recording(s) of the variable after TOI, using Ti, and Tv, denoted by (2) the absolute value of difference between the first recording of the variable after TOI, using Ti, and Tv, denoted by  Figure 2 illustrates the calculation of outcome measures for a hypothetical case.

Tables 2 and 3 show the number of patients with a nonzero and  respectively, along with their mean (SD) and median (IQR). Other secondary outcomes included the change in ARDS severity classification based on P/F ratios within 12 hours (to decrease missing values) of intubation using Ti and Tv (Table 4). The choice of 12 hours was based on the missingness rate given different window sizes (see Table, Supplemental Digital Content 1, available at http://links.lww.com/AJMQ/A65).

Table 3.

Summary of Impact of Tion the First Recorded Value of Different Variables Within 4 H After Intubation

Nonzero   Δ_1  X  *	NorthCARDS	MIMIC-III(CareVue)	MIMIC-III(MetaVision)
FiO2 (% oxygen)
n (%)	109 (5.6)	18 (0.4)	83 (2.4)
Mean (SD)	29.95 (14.73)	17.50 (14.58)	31.16 (18.88)
Median (IQR)	25.00 (20.00–40.00)	10.00 (10.00–20.00)	30.00 (10.00–50.00)
SpO2 (% oxygen)
n (%)	892 (46.5)	184 (4.1)	334 (9.7)
Mean (SD)	6.39 (6.68)	3.35 (4.04)	4.32 (8.12)
Median (IQR)	4.00 (2.00–8.00)	2.00 (1.00–4.00)	2.00 (1.00–4.00)
PaO2 (mmHg)
n (%)	152 (12.6)	56 (1.7)	99 (4.7)
Mean (SD)	65.31 (68.89)	75.84 (82.88)	82.31 (78.98)
Median (IQR)	41.00 (21.00–88.25)	46.00 (22.00–95.25)	44.00 (21.50–131.00)
PH
n (%)	137 (12.5)	54 (1.7)	97 (4.5)
Mean (SD)	0.10 (0.09)	0.09 (0.07)	0.10 (0.09)
Median (IQR)	0.06 (0.03–0.12)	0.07 (0.04–0.11)	0.07 (0.04–0.14)
PaCO2 (mmHg)
n (%)	136 (11.6)	50 (1.5)	95 (4.5)
Mean (SD)	13.52 (13.10)	10.62 (14.32)	10.40 (9.42)
Median (IQR)	9.50 (4.00–18.00)	6.00 (3.00–13.75)	7.00 (4.00–14.50)
P/F ratio
n (%)	143 (11.9)	31 (1.8)	51 (5.7)
Mean (SD)	73.25 (84.91)	99.15 (120.29)	76.72 (76.28)
Median (IQR)	50.67 (18.33–91.50)	51.94 (27.00–103.83)	41.00 (20.50–119.36)
S/F ratio
n (%)	976 (50.8)	176 (4.3)	300 (10.2)
Mean (SD)	38.83 (67.48)	15.60 (28.91)	21.02 (34.09)
Median (IQR)	19.07 (7.37–29.07)	4.00 (2.00–10.25)	5.00 (2.00–20.48)

(*)Absolute value of difference between (the first recording of the variable within 4 h after TOI using Ti) and (the first recording of the variable within 4 h after TOI using Tv) for a given patient (see Figure 2). The table shows the descriptive statistics for nonzero values across all patients in different cohorts. The mean and median values represent an absolute value of this difference and do not imply direction (ie, higher or lower). Data were only reported if it was available within 4 h of TOI for both Ti and Tv.

Abbreviation: IQR, interquartile range; MIMIC-III, Medical Information Mart for Intensive Care III; NorthCARDS, Northwell COVID ARDS; TOI, time of intubation.

Table 4.

Change in ARDS severity (mean P/F during the first 12 hours after intubation) resulting from use of Ti algorithm for NorthCARDS patients

ARDS severity using Ti	ARDS severity using Tv
	Normal	Mild	Moderate	Severe
Normal	95 (5.4%)*	10 (0.6%)	4 (0.2%)	2 (0.1%)
Mild	5 (0.3%)	271 (15.5%)	23 (1.3%)	5 (0.3%)
Moderate	1 (0.1%)	28 (1.6%)	809 (46.3%)	19 (1.1%)
Severe	-	2 (0.1%)	29 (1.7%)	445 (25.5%)

(*)n (%)

Results

A total of 1979 patients, 4657 patients, and 3600 patients were used for the NorthCARDS, CareVue, and MetaVision datasets, respectively. Using Ti, mechanical ventilation parameters were the determining factor for TOI in 13.6% (n = 300), 86.5% (n = 4250), and 50.2% (n = 2435) for NorthCARDS, CareVue, and Metavision, respectively. In the internal validation cohort, NorthCARDS, Ti resulted in an earlier TOI in 84.8% (n = 1666) of cases with a mean (SD) of 3.5 (15.5) and a median (IQR) of 0.78 hours (0.33–2.08). In the external validation cohort, CareVue, Ti resulted in an earlier TOI in 7.7% (n = 353) of cases, mean 6.4 (34.8), median 0.50 (0.25–1.25) hours. In the external validation cohort, Metavision, Ti resulted in an earlier TOI in 30.2% (n = 1056), mean 2.5 (12.0), median 0.48 (0.18–1.00). Up to 34.8% (n = 688) of patients had at least a 1 hour earlier TOI when using Ti with NorthCARDS. The distribution of earlier TOI thresholds (1, 2, 4, 8, 12, and >24 h) when using Ti in all 3 datasets, along with more detailed information regarding the determining factors for Ti recognition of TOI can be found in Table 1.

Table 1.

Summary of Results of Ti algorithm

	NorthCARDS	MIMIC-III(CareVue)	MIMIC-III(MetaVision)
Intubated patients (n)	1979	4657	3600
Ti’s determining factors[a] (n, %)
Ventilator parameters	300 (13.6)	4250 (86.5)	2435 (50.2)
Paralytics/sedatives	721 (32.6)	661 (14.4)	726 (15.0)
Airway placement procedure	544 (24.6)	1 (0.0)	410 (8.5)
O2 delivery method	240 (10.8)	0 (0.0)	1281 (26.4)
Ventilator start time	408 (18.4)	na	na
Distribution of ΔT[b] (h)
n (%) over 1	688 (34.8)	95 (2.0)	253 (7.0)
n (%) over 2	426 (21.5)	67 (1.4)	151 (4.2)
n (%) over 4	221 (11.2)	47 (1.0)	105 (2.9)
n (%) over 8	92 (4.6)	40 (0.9)	70 (1.9)
n (%) over 12	66 (3.3)	28 (0.6)	49 (1.4)
n (%) over 24	42 (2.1)	17 (0.4)	19 (0.5)
Nonzero ΔT (h)
n (%)	1666 (84.8%)	353 (7.7%)	1056 (30.2%)
Range (Min, Max)	323.20 (0.02, 323.22)	572.98 (0.02, 573.00)	310.07 (0.02, 310.08)
Mean (SD)	3.5 (15.5)	6.4 (34.8)	2.5 (12.0)
Median (IQR)	0.78 (0.33–2.08)	0.50 (0.25–1.25)	0.48 (0.18–1.00)

More than one determining factor may have the same TOI, therefore, total number of determining factors are greater than number of intubated patients.

 (in hrs).

Abbreviation: IQR, interquartile range; MIMIC-III, Medical Information Mart for Intensive Care III; NorthCARDS, Northwell COVID ARDS.

Ti resulted in alternate values for 18.4% (n = 223) of PaO2 values (mean 43.95 [54.24], median 26.00 [12.33–52.25]), 17.8% (n = 214) of P/F (mean 48.29 [69.81], median 27.46 [12.71–56.33]) and 62.7% (n = 1203) of S/F ratios (mean 16.75 [34.14], median 5.44 [1.59–15.44]), in NorthCARDS, when using the absolute value of difference between the mean (SD) and median (IQR) measure within the first 4 hours of intubation using Ti and Tv (Table 2).

Although less patients were included because of an increase in missing data, the absolute value of difference between using Ti and Tv in NorthCARDS, when using the first value within the first 4 hours of intubation with Ti and Tv, resulted in larger differences in PaO2 12.6% (n = 152), mean 65.31 (68.89), median 41.00 (21.00–88.25); P/F 11.9% (n = 143), mean 73.25 (84.91), median 50.67 (18.33–91.50); and S/F ratios 50.8% (n = 976), mean 38.83 (67.48), median 19.07 (7.37–29.07) (Table 3).

Data regarding the differences in oxygenation and ventilation parameters using the absolute value of difference between the mean (SD) and median (IQR) measure within the first 4 hours of intubation with Ti and Tv for CareVue and Metavision can be found in Table 2. Data regarding the differences in oxygenation and ventilation parameters using the absolute value of difference using the first value within the first 4 hours of intubation using Ti and Tv for CareVue and Metavision can be found in Table 3.

Differences in P/F ratios using Ti, when calculated over 12 hours (to minimize missingness) from intubation, resulted in the reclassification of 7.3% (n = 128) of patients into new ARDS categories in the NorthCARDS dataset (Table 4). The largest shift between categories occurred when going from moderate ARDS using Tv to severe ARDS using Ti 1.7% (n = 29).

Discussion

This is the first study, to the authors’ knowledge, that uses multiple levels of redundancy to develop a query (Ti) that identifies an earlier TOI for patients across different hospital settings in a large health system during a global pandemic. Previous studies have either used the first set of mechanical ventilation parameters (eg, PEEP), first record of an airway device (eg, endotracheal tube), first record of ventilator use in the oxygen delivery method section of the EHR, first order of rapid sequence intubation medications (eg, etomidate), written orders for intubation or a combination of 2 of the aforementioned criteria to identify TOI.[2,4-7] The most common measure employed across most of these studies, however, was time of first mechanical ventilation parameter.[2,4,5,7] Based on the cohort, time of first mechanical ventilation parameter provided the closest TOI in only 15% of patients. While this measure may be sufficient within a hospital or health system under normal operating circumstances, when resources are stretched thin, timely documentation may not be possible and thus a query utilizing multiple surrogates for TOI produces results with greater data integrity. The following surrogates were found to be the most accurate in retrospectively identifying TOI in the EHR and, therefore, were used to construct Ti: TOI authored in an endotracheal intubation procedure note (from the ED or as an inpatient); “Start Time” from the mechanical ventilation record; time of first documentation of mechanical ventilation parameters in the mechanical ventilation record; time of first completed injection of medications used for intubation such as etomidate, succinylcholine, cisatracurium, rocuronium, vecuronium; time of initiation of continuous infusion propofol or fentanyl (minimum duration of 5 min); and documentation in nursing ED and inpatient notes regarding use of “ventilator” in the oxygen delivery method for each patient.

Compared with Tv, Ti not only resulted in an earlier TOI in 85% of patients in NorthCARDS, it also resulted in earlier times of intubation for 8% and 30% of patients in the 2 MIMIC-III datasets, CareVue, and Metavision, respectively. The improvement in TOI for the latter 2 datasets using Ti may have been limited, however, by a few factors. First, the MIMIC-III data only represents patients intubated in the intensive care unit[8] where documentation of ventilation parameters may occur with less delay than patients intubated in other settings such as the medical wards or ED. Second, the cohort represents data from New York City and Long Island hospitals during the peak of the COVID-19 pandemic when New York City was the global epicenter. During this period, resources such as ICU space and hospital staffing had to be adjusted to accommodate the large volume of patients. Therefore, data entry may have been delayed to prioritize patient care.

Despite these dramatic differences in environments between the cohort and the patients in MIMIC-III, the use of Ti resulted in substantial differences of PaO2, P/F, and S/F ratios in the sample of patients with differences in TOI between Ti and Tv (Tables 2 and 3). Furthermore, ARDS severity was reclassified between normal, mild, moderate, and severe ARDS (Table 4) in 7% of cases.

The goal in peri-intubation research is to extract the most accurate data points surrounding the TOI in order to determine associations with outcomes of interest. Specific to the COVID-19 pandemic, in addition to patient factors, lab values, respiratory function, type of ventilator used and treatments, the group is also currently evaluating peri-intubation variables in COVID-19 ARDS and their association with index hospital survival and time to liberation from invasive mechanical ventilation.[9] The most proximal data to intubation allows researchers to determine the state of lung compromise at the TOI, before human intervention with invasive mechanical ventilation sets off yet another cytokine cascade[1] that may cloud the picture of true pathology.

Machine learning models are being developed to predict the need for intensive care unit admission, mechanical ventilation, and mortality in patients with COVID-19.[10] However, data quality from an EHR may become a limiting factor for reproducibility and generalizability. In order for a model to be useful in real time, ideally, it should be developed on data that most closely mimics the real life scenario.[11] Therefore, improved EHR data integrity may help improve machine learning algorithms for predicting patient-centered outcomes, not only in COVID-19, but in other disease states as well.

The limitations are in accordance with retrospective studies. All of the data points used to create Ti were surrogates and could not be compared with TOI in real time. As a result, even the most accurate surrogates may be discordant with the actual TOI. Further, the authors did not evaluate the number of patients that did not meet any of their criteria but may have been intubated—this could occur for patients that died shortly after intubation.

In regards to using the first mechanical ventilation parameter as the reference standard for TOI, it did not allow comparison of this measure to actual intubation times. Delays in documentation of mechanical ventilation parameters in the EHR following intubations on the medical wards are possible because a lack of space during the pandemic may have caused delays in transport from the general medicine wards to the intensive care units. Further, the volume of intubations on a daily basis may have also contributed to delays in documentation of mechanical ventilation parameters as patient care was the primary priority.

The summary of impact of Ti on clinical values during the first 4 hours of intubation, Table 2, is limited by overlap between Ti and Tv. The mean values over a 4-hour period starting at Ti will be affected by values that overlap between Ti and Tv. This has the potential to underestimate the impact in the absolute difference between the mean values within 4 hours of Ti and Tv. The impact of using the first variables within 4 hours after intubation (Table 3) also has limitations as there was a significant amount of missing data using this approach (see Table, Supplemental Digital Content 1, available at http://links.lww.com/AJMQ/A65).

Conclusions

In addition to identifying a more accurate TOI, the new query allows for more accurate analyses of peri-intubation changes in patient parameters. Although Ti is designed for the authors’ EHR, the logic behind it can serve as a guiding template for other institutions to follow for identifying TOI, using their own unique EHR. Further, this logic works across multiple hospital settings—ED, medical wards, and ICUs—during a time of significant resource strain, the COVID-19 pandemic. This research highlights the need for institutions to reassess their own data integrity during times of resource strain, where not only patient care may be affected,[12,13] but also data that may inform future care.

Acknowledgments

The authors would like to thank all the frontline workers that risk their lives on a daily basis to provide compassionate care to their patients. The members of the Northwell ARDS Collaborative include as follows: Eric Gottesman, Paul Mayo, Molly Stewart, and Meng Zhang.

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest or financial disclosure relevant to this article.

Funding

Philanthropic funds to the Feinstein Center for Health Outcomes and Innovation Research.

Author contributions

Dr. Makhnevich and Dr Gandomi contributed to the study design, data validation, data interpretation, and writing of the article. Dr Gandomi was responsible for query development and data analysis. Mr. Wu contributed to data analysis and interpretation as well as writing of the article. Dr. Qiu contributed to data collection, analysis and interpretation as well as article writing. Dr. Jafari, Dr. Tsegaye, Dr. Rolston, and Dr. Rolston contributed to the data validation and interpretation as well as the writing of the article. Dr. Makhnevich and Dr. Gandomi had full access to all of the data in the study and take responsibility for the integrity of the data. Dr. Makhnevich takes full responsibility for the content of the aricle.

The authors declare that they had full access to all of the data in this study and the authors take complete responsibility for the integrity of the data and the accuracy of the data analysis.