A quantitative approach for the analysis of clinician recognition of acute respiratory distress syndrome using electronic health record data.

IMPORTANCE: Despite its efficacy, low tidal volume ventilation (LTVV) remains severely underutilized for patients with acute respiratory distress syndrome (ARDS). Physician under-recognition of ARDS is a significant barrier to LTVV use. We propose a computational method that addresses some of the limitations of the current approaches to automated measurement of whether ARDS is recognized by physicians.
OBJECTIVE: To quantify patient and physician factors affecting physicians' tidal volume selection and to build a computational model of physician recognition of ARDS that accounts for these factors. DESIGN, SETTING, AND PARTICIPANTS: In this cross-sectional study, electronic health record data were collected for 361 ARDS patients and 388 non-ARDS hypoxemic (control) patients in nine adult intensive care units at four hospitals between June 24 and December 31, 2013.
METHODS: Standardized tidal volumes (mL/kg predicted body weight) were chosen as a proxy for physician decision-making behavior. Using data-science approaches, we quantified the effect of eight factors (six severity of illness, two physician behaviors) on selected standardized tidal volumes in ARDS and control patients. Significant factors were incorporated in computational behavioral models of physician recognition of ARDS.
RESULTS: Hypoxemia severity and ARDS documentation in physicians' notes were associated with lower standardized tidal volumes in the ARDS cohort. Greater patient height was associated with lower standardized tidal volumes (which is already normalized for height) in both ARDS and control patients. The recognition model yielded a mean (99% confidence interval) physician recognition of ARDS of 22% (9%-42%) for mild, 34% (19%-49%) for moderate, and 67% (41%-100%) for severe ARDS. CONCLUSIONS AND RELEVANCE: In this study, patient characteristics and physician behaviors were demonstrated to be associated with differences in ventilator management in both ARDS and control patients. Our model of physician ARDS recognition measurement accounts for these clinical variables, providing an electronic approach that moves beyond relying on chart documentation or resource intensive approaches.

Introduction

Despite a broad consensus on the virtues of translating evidence into clinical practice, adoption of evidence-based practices remains slow.[1,2] The use of low tidal volume ventilation (LTVV) for the treatment of acute respiratory distress syndrome (ARDS) is a prime example. ARDS is a syndrome of severe acute hypoxemia and non-cardiogenic inflammatory lung injury with high prevalence (10% of intensive care unit (ICU) admissions) and mortality (35–46%).[3-6] In the 2000 ARDS Network trial, it was shown that lowering the volume of each breath–i.e. using LTVV—is an effective therapy for ARDS, with a relative mortality reduction of 20–25%[7]; since then, LTVV has become recommended practice for the management of ARDS. [8] Despite this substantial evidence, LTVV use in clinical practice remains as low as 19%.[9-17] Several studies have examined the barriers to LTVV use, including the primary role of physician under-recognition of ARDS.[3,6,10,12-20] However, the identification of these barriers has not led to a substantial increase in LTVV use, suggesting a need for further investigation.

A hallmark of healthcare quality improvement is the consistent measurement of an outcome (ex: number of infections, checklist use, etc.). In the case of LTVV use for ARDS, measurement of an outcome is challenging for multiple reasons. First, delivering LTVV is a two-step process comprised of i) recognizing ARDS and ii) selecting and adjusting tidal volumes based on patient response. Both steps in this process can be affected by patient characteristics.[3,13,17,20-22] Second, while previous studies have employed LTVV use or physician documentation of ARDS as surrogates for physician recognition of ARDS, these proxies have limitations. The gold standard for measuring physician recognition of ARDS would be to directly ask physicians if a patient has ARDS. While this approach has been used in the past,[3] it introduces important biases such as the observer effect, subjective reporting, and priming clinicians to think about ARDS.[23-25] Furthermore, it is labor and resource intense, making it an infeasible solution for widespread implementation. An alternative method is to collect physician documentation of ARDS from electronic health records (EHR). While this approach is more scalable, physicians may recognize ARDS but not document it in their notes, nor deliver LTVV despite this recognition.

We sought to use data science approaches on EHR data to address the above challenges with measurement of physician recognition of ARDS and build an estimate of physician recognition of ARDS that could be widely implemented. We use tidal volume selection as a proxy for physician decision-making behavior and quantify the factors affecting tidal volume selection in both ARDS patients and a novel control cohort of patients who do not have ARDS. We then build two models of physician recognition of ARDS that account for these factors. Our methods not only address the issues of bias and resource availability, but also provide insights into underlying physician behavior that have implications for effective intervention design.

Materials and methods

Cohort development

We have previously described the development of the ARDS cohort used in this study, which includes 362 patients who met the Berlin Definition of ARDS[5] (see summary below) via independent clinician review and were admitted to an ICU at one academic and three community hospitals in the Chicago region between June 24, 2013 and December 31, 2013.[16]

Berlin Definition Criteria Summary [5]

Hypoxemia: PaO2/FIO2 ≤ 300

Positive chest imaging: bilateral infiltrates (radiological signs in both lungs)

Presence of a known risk factor for ARDS

Respiratory failure not fully explained by cardiac failure or fluid overload

Abbreviations: PaO2 –partial pressure of arterial oxygen; FIO2 –fraction of inspired oxygen

For this study, we developed an additional cohort from the same time period and initial screening population at two of the same hospitals (one academic, one community): 388 patients with acute hypoxemic respiratory failure requiring mechanical ventilation with at least one instance of PaO2/FIO2 ≤ 300 but not with ARDS according to the above Berlin Definition (“control cohort”). We excluded patients with missing key information (predicted body weight [PBW], gender, tidal volumes), intubation duration less than 5.67 hours (the shortest duration of intubation in the ARDS cohort), and PBW less than 25 kg (Fig 1).

Fig 1

Flow of patient screening and enrollment for control cohort.

Patients were not actively recruited for either cohort, but instead all data was mined from the electronic health record. The ARDS and control cohorts were similar across several clinical and demographic measures (S1 Table). These cohorts are representative of the larger population of patients with ARDS and non-ARDS acute hypoxemic respiratory failure due to our broad inclusion criteria, and their similarity to larger cohorts (e.g., LUNG SAFE[3]) with respect to height, weight, and hypoxemia severity. This study was approved by the Northwestern University Institutional Review Board (STU00208049) with a waiver of consent on October 30, 2018.

Data acquisition

All patient data was obtained from the electronic health records serving the participating hospitals. We defined study entry as the start of ARDS for the ARDS cohort and the first instance of PaO2/FIO2 ≤ 300 in the control cohort. Study end was defined as the earlier of extubation, death, or ICU discharge. We recorded gender, height, and all PaO2/FIO2 and weights between ICU admission and study end. We recorded all tidal volumes (VT) and plateau pressures (Pplat) between intubation and study end where available (PPlat was not recorded at two hospitals). Note that 22% of the ARDS cohort and 44% of the control cohort only had one unique tidal volume over their study duration. We recorded which ICU the patient was treated in and whether an ARDS diagnosis was documented in the critical care physician’s notes. For the control cohort, we recorded whether or not bilateral infiltrates were present for all chest radiographs or computed tomography scans between intubation and study end.[16] For data availability for both cohorts and all subgroups, see S2 Table. Patients who met cohort inclusion criteria but were missing other data points were only excluded from analyses that required those missing data points.

In this study, we use PBW as a gender-adjusted and gender-neutral measurement of height because LTVV thresholds are defined using PBW. Any references to patient height refer to PBW (kg) and any references to patient weight refer to a patient’s weight measured at ICU admission (kg). We calculated PBW according to the ARDS Network definition (see below) and defined LTVV as a standardized tidal volume () ≤ 6.5 mL/kg PBW.

Predicted Body Weight Equations [7]:

Significance testing

We used α = 0.01 instead of 0.05 to ensure the statistical strength of our findings [26] and applied the Bonferroni correction for multiple hypotheses. In the regression analyses (see Results), there were 33 comparisons where was the dependent variable, thus we set p < 0.0003 (0.01/33) as the threshold for statistical significance for these analyses. For the covariate analyses, the threshold was p < 0.005 (0.01/2). For the Kolmogorov–Smirnov tests in Model Approach #2, the threshold was p < 0.003 (0.01/3).

Results

Analysis of potential factors in tidal volume selection

Factors assessed

We used the lowest standardized tidal volume () (mL/kg PBW) for each patient as the dependent variable in both univariable and multivariable ordinary least squares (OLS) regressions. was used as a continuous variable. OLS regressions were implemented using the statsmodels (version 0.6.1) Python package.

We determined the relationship between several factors and , choosing variables that have been identified previously in the literature as potential barriers or facilitators of LTVV use[11,18-21,27]: first PaO2/FIO2 ≤ 300, lowest PaO2/FIO2, highest Pplat, patient weight at ICU admission, ARDS documentation in the patient chart, presence of bilateral infiltrates (control only), admitting ICU (ARDS only), and patient height (we used the gender neutral PBW). These factors comprise measures of illness severity (PaO2/FIO2, Pplat, radiographic findings), patient characteristics (height, weight), and physician behaviors (ARDS documentation, patient weight). Plateau pressure was included due to the previously reported practice of physicians not lowering tidal volumes in ARDS patients when Pplat ≤ 30 cm H2O.[19,22] Patient weight was included due to the previously reported barrier of physicians using actual body weight instead of predicted body weight in the LTVV threshold calculation [19]. Note that we use a standardized tidal volume () as opposed to the recorded tidal volume (VT) and PBW is included as a control variable. Since is already normalized for PBW, we expected no additional remaining relationship between PBW and . Input variables were rescaled between 0 and 1 to allow for comparison of coefficients.

Univariable analysis

The relationship between each factor and was investigated through univariable OLS regressions for the ARDS and control cohorts (Table 1). Standardized tidal volume () decreased with worsening hypoxemia (lower PaO2/FIO2) and documentation in the ARDS cohort (p < 0.0003), but not in the control cohort (Fig 2, Table 1). In both cohorts, decreased with increasing PBW (gender neutral height, p < 0.0003, Fig 3 and Table 1)–a surprising result since already takes PBW into consideration. Plateau pressure, weight at ICU admission, PaO2/FIO2 at study start, admitting ICU (ARDS cohort only), and the presence of bilateral infiltrates (control cohort only) were not associated with significant changes in standardized tidal volume in any cohort or subgroup (Table 1, Fig 4).

Table 1

Predictors of lowest standardized tidal volume (mL/kg PBW) (β-coefficient [99% CI]).

Factor	ARDS		Control	Pooled Documented
	univariable	multivariable	univariable	univariable
Predicted body weight	-3.8*[-4.7, -2.8]	-3.7*[-4.8, -2.7]	-5.1*[-6.0, -4.1]	-3.2*[-5.2, -1.2]
PaO2/FIO2 ratio (lowest)	1.3*[0.4, 2.4]	1.1[0.3, 1.9]	0.8[-0.1, 1.6]	2.3[-0.3, 4.9]
Documentation	-1.3*[-1.9, -0.6]	-1.2*[-1.9, -0.6]	-1.2[-2.3, -0.2]
PaO2/FIO2 ratio (first)	0.7[-0.19, 1.5]		0.3[-0.6, 1.1]	1.2[-0.9, 3.2]
Pplat (highest)	-2.2[-4.1, -0.3]		-1.5[-3.2, 0.1]	-3.5[-6.0, -1.0]
ICU admission weight	0.12[-1.8, 2.0]		-0.4[-1.7, 0.9]	-0.6[-3.3, 2.2]
Bilateral infiltrates**			-0.5[-0.9, 0.0]
Admitting ICU	-0.9[-1.9, 0.1]

* p < 0.0003

** At least once after hypoxemia onset.

Empty cells indicate category was not used due to data being unavailable or not relevant.

Fig 2

Effects of lowest PaO2/FIO2 ratio on standardized tidal volume () and ARDS documentation in ARDS and control cohorts.

Top panels show patients with ARDS documented in their chart (purple diamonds) and non-documented patients (tan circles). Gray areas indicate LTVV range from current guidelines[7], with dashed line at 6.5 mL/kg PBW from currently recommended threshold. Solid lines show linear () and logistic (documentation) fits for scatter plot data (shaded regions, 95% confidence bands). Reported beta coefficients are for standardized inputs. * p < 0.0003, ** p < 0.0005.

Fig 3

Effects of predicted body weight (gender neutral height) on standardized tidal volume () and ARDS documentation in ARDS and control cohorts.

Top panels show patients with ARDS documented in their chart (purple diamonds) and non-documented patients (tan circles). Gray areas represent LTVV range from current guidelines[7], with dashed line at 6.5 mL/kg PBW at current recommended threshold. Solid lines show linear () and logistic (documentation) fits for scatter plot data (shaded regions, 95% confidence bands). Reported beta coefficients are for standardized inputs. * p < 0.0003.

Fig 4

Cohort and subgroup definitions.

Covariate analysis

The factors demonstrating a significant association with in the univariable analyses were evaluated for covariance with each other using OLS regression. Three factors were evaluated for covariance: PBW, lowest PaO2/FIO2, and documentation of ARDS. PBW was not associated with increasing documentation probability (Fig 3) in both cohorts, which was anticipated. Documentation and lowest PaO2/FIO2 were significantly correlated (p < 0.005) in both cohorts (Fig 2). This association was also anticipated as sicker patients are easier to recognize. To test the strength of the documentation and lowest PaO2/FIO2 association, we repeated the univariable analysis on the three major subgroups (ARDS non-documented, control non-documented, and pooled documented) (Fig 4). Only PBW was associated with lower in all three subgroups (Table 1, S3 Table). There was no association between PBW and lowest PaO2/FIO2 in both cohorts.

Multivariable analysis

Significant factors from the univariable analyses were included in multivariable regressions comprised of all possible linear combinations of the factors and appropriate interaction terms (see S1 Methods). The “best” multivariable model was selected using the Akaike and Bayesian Information Criterion (AIC, BIC).

In the ARDS cohort, the multivariable regression model that included PBW, lowest PaO2/FIO2, and documentation as independent variables with no interaction terms resulted in the lowest AIC and BIC (S4 Table). In this model, PBW and documentation were significantly correlated with (p < 0.0003), while lowest PaO2/FIO2 was not. Of these variables, PBW had the greatest effect on (β -3.7, 99% CI -4.8 –-2.7). For the control cohort, only PBW was associated with , and therefore no multivariable analysis was performed.

Sensitivity analyses

To test the robustness of our cohort definitions, we conducted two sensitivity analyses: 1) patients with a study duration longer than 12 hours, and 2) patients within the 2.5–97.5 percentiles of PBW. The first sensitivity cohort is intended to capture clinician behavior, which may require longer time scales, such as a shift change and/or patient rounds. The second sensitivity cohort aimed to evaluate a potential disproportionate effect of PBW outliers on linear trends. Neither sensitivity analyses yielded any difference in the regression results.

Construction of model of and estimation of physician recognition of ARDS

Sixty patients—44 (12.2%) in the ARDS cohort and 16 (4.2%) in the control cohort—had a documented diagnosis of ARDS by at least one physician. While documentation of ARDS implies that the physician recognized ARDS, lack of documentation does not imply that a physician did not recognize ARDS. Beyond their mere documentation practices, a physician’s clinical management behavior (e.g., selecting a tidal volume) may shed additional light on whether a physician believes that a patient has ARDS.

We used two approaches to more completely characterize physician recognition of ARDS. To this end, we split the two main cohorts (ARDS and control) into three major subgroups: 1) ARDS non-documented (n = 317), 2) Control non-documented (n = 371), and 3) Pooled documented (n = 60, Fig 4). All patients in the pooled documented cohort have a physician-documented diagnosis of ARDS in their chart. All patients in the non-documented cohorts do not have a physician-documented diagnosis of ARDS in their chart. Our two approaches are based on the assumption that the physician behaviors observed in the ARDS non-documented subgroup represent a mixture of patient-care scenarios in which patients are either recognized by their physician as having ARDS or not recognized as having ARDS. If a patient in the ARDS non-documented subgroup was recognized by their physician as having ARDS, we assume the physician tidal volume selection would be the same as the tidal volume selection seen in the pooled documented subgroup. If a patient in the ARDS non-documented subgroup was not recognized as having ARDS, we assume the physician tidal volume selection would be the same as for control non-documented subgroup. Therefore, the non-documented ARDS subgroup patients can be viewed as a mixture of the pooled documented subgroup and the non-documented control subgroup.

Approach #1: Naïve Bayes

We used a Naïve Bayes model for classifying patients in the non-documented ARDS subgroup as either recognized or unrecognized by their care teams. We used multivariate kernel density estimation (KDE) to characterize the PBW vs clusters for the pooled documented and non-documented control subgroups (Fig 5, top panels). Classifying a patient in the non-documented ARDS subgroup as recognized or unrecognized was based on the following conditional probabilities leveraging Bayes Theorem:

Fig 5

Kernel Density Estimation for control non-documented and pooled documented patients.

Heatmaps of kernel density estimated probability density for data from control non-documented (yellow, top panel) and documented (purple, middle panel) subgroups. Solid line shows boundary separating region with unequal probability of belonging to documented (below line) and non-documented control (above line). (Bottom panel) Normalized gender frequency across PBW for combined patient population of documented and control non-documented. Male and female peaks align with high density regions in above heatmaps.

In the absence of a reasonable prior for P(documented) and P(control), we assign each term 0.5, assuming equal probability of belonging or not belonging to each subgroup. We were able to define a boundary in the PBW vs space where P(documented | PBW, ) = P(control | PBW, ) (Fig 5, top panels, black line). Below this boundary, P(documented | PBW, ) is greater than P(control | PBW, ) and the patient was classified as ‘recognized’. Above this boundary, P(documented | PBW, ) is less than P(control | PBW, ) and the patient was classified as ‘unrecognized’. Due to the size discrepancy between the non-documented control and pooled documented subgroups, we bootstrapped (100 iterations) the “non-documented” control subgroup and repeated this analysis to produce confidence bands (S1 Fig).

The KDE clusters for the pooled documented and control non-documented subgroups as well as the estimated probability equality line are shown in Fig 5. The peaks of male and female PBW (gender neutral height) frequency (Fig 5, bottom panel) align with the two peaks in the pooled documented subgroup (Fig 5, middle panel). Physician recognition of ARDS calculated for each ARDS severity category was: mild, 26%; moderate, 32%; severe, 57% (Table 2).

Table 2

Rates of physician recognition of ARDS by hypoxemia severity.

Severity	ARDSDocumented		Recognition
			Approach #1:	Approach #2:	LUNG SAFE study [3]
	n	%	Naïve Bayes (%)	Mixture Model(% [99% CI])	(% [95% CI])
Mild200 < PaO2/FIO2 ≤ 300	5	6	26	22[9, 42]	51.3[47.5, 55.5]
Moderate100 < PaO2/FIO2 ≤ 200	8	7	32	34[19, 49]	65.3[62.4, 68.1]
SeverePaO2/FIO2 ≤ 100	24	30	57	67[41, 100]	78.5[74.8, 81.8]

Model #2: Mixture Model

In the second model, we incorporate , hypoxemia severity (lowest PaO2/FIO2), and PBW with the goal of calculating the fraction of recognized patients in each Berlin Definition ARDS severity category (mild, moderate, and severe).[5] To calculate physician recognition of ARDS, we estimated the fraction of patients recognized by physicians in each severity category () from the following set of equations: where severity can take the values “mild,” “moderate,” or “severe,” as set forth in the Berlin Definition [5] (Table 2) and we defined the difference between the probability density functions as the L1 norm: where the sum extends over all bins for values of and PBW.

We determined the optimal value of by minimizing Δ. Since the corresponding optimization problem is formulated as a linear programming problem, we used CPLEX (version 12) as a solver. To determine the uncertainty in our estimates of , we used bootstrapping to generate 1000 samples for PARDS (, PBW | severity) and repeated the optimization for the bootstrapped samples. As a result, we generated distributions for the optimal value of for each hypoxemia severity category and tested the null hypothesis that these data were drawn from the same distributions with a Kolmogorov–Smirnov test (Python package scikit-learn (version 0.18.1)).

This approach yielded a mean (99% confidence interval) physician recognition of ARDS: 22% (9%-42%) for mild; 34% (19%-49%) for moderate; and 67% (41%-100%) for severe (Table 2, S2 Fig). All three recognition distributions were significantly different from each other (p < 0.003) when compared via a Kolmogorov–Smirnov test.

Discussion

We quantified the potential impact of patient characteristics and physician behaviors on the decision-making behavior for tidal volume selection by physicians for patients with ARDS and a novel control cohort. This quantification allowed for the construction of a model to measure physician recognition of ARDS that is not confounded by these patient characteristics and clinical factors. These analyses have allowed us to establish several important findings.

First, we corroborated prior studies’ findings that height, hypoxemia severity, and ARDS documentation are associated with the use of lower tidal volumes in ARDS patients.[3,17,19-21,27] We found no evidence for an association between other clinical factors—such as plateau pressure or patient weight—and lower tidal volume use, which have been identified as potential barriers to LTVV use in prior studies.[11,17,19,21] These barriers may still have an impact at the level of the individual physician, but the lack of generalizability to the entire physician population makes them suboptimal for future intervention targets.

Second, our analyses provide additional insight into the previously established relationships between patient height and LTVV use.[21,27] The most common lowest VT reported in the ARDS and control cohorts were identical (450, 500, and 600 mL), and constitute 51% and 63% of the tidal volumes for the ARDS and control cohorts, respectively. This prevalence of a small number of lowest VT suggests that clinicians are not following the canonical relationship between height and lung size originally established in animal studies[28], but instead use a simpler heuristic based on where the patient falls on the height spectrum of their particular gender. This theory is supported by the idea that humans select fast and frugal heuristics under time and knowledge limitations [29], which would both be present in clinical medicine and heightened in critical care. The utilization of this heuristic would translate to a general use of a lower standardized tidal volume () for taller patients that is closer to or, in some cases, below the LTVV threshold; which would lead to our observation of the strong relationship between PBW and , despite that already includes PBW in its calculation. Our findings are strong evidence that at least some delivery of LTVV may be unintentional—i.e., solely of a default VT (450, 500, or 600 mL)—and not based on ARDS recognition or other clinical decision-making factors. While evidence for this physician behavior phenomenon has been previously reported in ARDS patient cohorts [17,27], our findings observe this behavior in a diverse control cohort, implying that the simpler tidal volume selection heuristic use is not restricted to ARDS patients alone.

Alternative explanations for the association between height and LTVV use in both cohorts include some physicians believing in LTVV for patients in the control cohort or some of those patients being classified by physicians as having ARDS. Supporting the latter possibility, 4.2% of control cohort patients had a physician-documented diagnosis of ARDS. Nonetheless, these alternative explanations are less likely because of the low ARDS documentation rate, low use of LTVV in both cohorts, and the strong correlation between PBW (gender neutral height) and in both cohorts. Another alternative explanation is physicians using a non-linear relationship between tidal volume and PBW, but this is less likely given the low variability in chosen tidal volumes in both cohorts. Our results suggest that the relationship between PBW (gender neutral height) and should be accounted for when measuring LTVV use and when designing implementation strategies to improve LTVV use.

Third, our estimate of physician recognition of ARDS for severe ARDS was comparable to previous studies [3], while our estimated physician recognition of ARDS rates for mild and moderate ARDS were lower. We believe our estimated rates are more plausibly representative of real-world practices. Unlike prior studies, the estimated physician recognition of ARDS rates in our study do not rely on subjective reporting [18,19], the observer effect [3,9,10,13-15], or imposed interventions such as additional training of physicians in ARDS recognition or LTVV use.[3] Our results suggest that the potential impact of these biases was limited to mild and moderate ARDS and was neutralized by the presence of severe ARDS. This finding has implications for the selection of implementation strategies: while the mere presence of severe hypoxemia may be enough to trigger physician recognition of ARDS, additional prompting or training is required to improve recognition of mild or moderate ARDS.

Limitations

Our study has several limitations. First, it was conducted in a single metropolitan area, so we were unable to address regional or national differences. Second, we were limited to the patient data recorded in the EHR, which may be overlooked by physicians in lieu of other information, such as a visual estimation of height. [30] Third, we did not evaluate physician knowledge of ARDS or LTVV, specifically the Berlin criteria and what standardized tidal volume threshold they believe qualifies as LTVV. Alternative LTVV thresholds may be justified by the layout of the ARDS Network tidal volume table, which appears to suggest that tidal volumes ranging from 4 to 8 mL/kg PBW qualify as LTVV.[5] Finally, we acknowledge that it is possible that our application of the Berlin definition may have been biased, leading to misclassification of ARDS or control status—this could also explain why some patients classified as control were documented by their physicians as having ARDS.

Conclusions

Our findings could have implications for the design of implementation strategies to improve LTVV use. First, we believe documentation and physician recognition of ARDS should be unlinked. Whereas previous studies rely on documentation of ARDS as the sine qua non of physician recognition of ARDS [11,13,16,20], there are likely to be patients whose physicians recognized ARDS but did not document it. Our study demonstrates two novel approaches for estimating physician recognition of ARDS that consider additional behaviors beyond documentation (e.g., tidal volume selection), providing a more complete characterization of physician recognition of ARDS. Implementation strategies—which commonly rely on behavior change—should account for the multiple facets involved in physician recognition of ARDS, and multiple channels of data required to measure recognition.

Second, our approach provides a measurement of physician recognition of ARDS that is not subject to confounding by patient characteristics and clinical factors. This approach could be integrated into EHR systems to evaluate an arbitrary number of physicians and sites, which will allow for the comparison of physician recognition of ARDS not only between individuals and institutions, but also over different points in time. This methodology could be used to accurately drive interventions, like clinical decision-support or feedback, to create an even stronger structure for improving the adoption of evidence-based practices.[31,32]

This study offers a compelling example of how data science methods can use EHR data to provide new opportunities for measuring and addressing quality of patient care, specifically in a complex setting such as critical care.[33] While traditional implementation studies allow a broad analysis of the situation as a whole, they are constrained by high costs, logistical complexity, and potential for bias. Our approach achieved similar, and potentially more representative results, while minimizing these disadvantages. Furthermore, our methods may be implementable and sustainable on the local level, providing individual institutions the opportunity to continuously assess or track evidence-based medicine implementation.

Multivariable model selection.

(DOCX)

Click here for additional data file.

Characteristics of cohorts and subgroups.

(TIF)

Click here for additional data file.

Data availability for cohorts and subgroups.

(TIF)

Click here for additional data file.

Predictors of lowest standardized tidal volume (mL/kg PBW) in non-documented subgroups (β-coefficient [99% CI]).

(TIF)

Click here for additional data file.

Multivariable models of lowest standardized tidal volume (mL/kg PBW) in ARDS cohort.

(TIF)

Click here for additional data file.

Naïve Bayes boundary between recognized and unrecognized regions with 95% confidence intervals from bootstrapping.

Scatter plot shows pooled documented patients (purple diamonds) and control non-documented patients (tan circles). Size of marker represents number of data points. Solid line shows boundary separating region with unequal probability of belonging to documented (below line) and non-documented control (above line) with 95% confidence bands from bootstrapped data (shaded region).

(TIF)

Click here for additional data file.

Distributions of from CPLEX analysis of bootstrapped data.

(TIF)

Click here for additional data file.

27 Jul 2019

PONE-D-19-17952

A quantitative approach for the analysis of clinician recognition of acute respiratory distress syndrome using electronic health record data

PLOS ONE

Dear Mrs. Bechel,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Sep 10 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Lars-Peter Kamolz, M.D., Ph.D., M.Sc.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

1. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

2. Thank you for including your competing interests statement; "

SP reports grant support from Pfizer, Inc. unrelated to this manuscript.

MB, LANA, and CHW report the related US provisional patent: Systems and Methods for Patient Management Within a Healthcare Facility. US Serial No: 62/457,574.

The other authors declare that they have no conflicts of interest."

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Dear authors,

Thank you for the opportunity to review the manuscript, “A quantitative approach for the analysis of clinician recognition of acute respiratory distress syndrome using electronic health record data”

A few remarks:

Possibly, the already listed Limitations could receive the title, “Limitations.”

I suggest starting with the conclusion section with line 354.

PlosOne’s required citation style is to be used correctly (e.g. line 350, 339)

The references have to be double-checked. The literature has to be updated.

Hardly any current literature is cited.

What are the actual and above all current results of this study, which would ultimately justify a publication in a scientific journal?

The date of the Ethics Statement is missing.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

21 Aug 2019

We have carefully considered all comments and revised our manuscript accordingly. Below please find our point-by-point response to all editor and reviewer comments. We believe these revisions strengthen our manuscript and we hope that it is now suitable for publication in PLOS ONE.

Thank you for your consideration of our manuscript.

Editor Comments

Editor Comment 1: In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

Response: We have added S1 Table: "Characteristics of cohorts and subgroups,” to provide relevant demographic data.

We have also revised the methods to include all of the above details, including the following statement:

“Patients were not actively recruited for either cohort, but instead all data was mined from the electronic health record. The ARDS and control cohorts were similar across several clinical and demographic measures (S2 Table). These cohorts are representative of the larger population of patients with ARDS and non-ARDS acute hypoxemic respiratory failure due to our broad inclusion criteria, and their similarity to larger cohorts (e.g., LUNG SAFE) with respect to height, weight, and hypoxemia severity. [3]”

Editor comment 2: Thank you for including your competing interests statement; "SP reports grant support from Pfizer, Inc. unrelated to this manuscript. MB, LANA, and CHW report the related US provisional patent: Systems and Methods for Patient Management Within a Healthcare Facility. US Serial No: 62/457,574. The other authors declare that they have no conflicts of interest." Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Response: Below is our updated Conflict of Interest statement:

"SP reports grant support from Pfizer, Inc. unrelated to this manuscript. MB, LANA, and CHW report the related US provisional patent: Systems and Methods for Patient Management Within a Healthcare Facility. US Serial No: 62/457,574. The other authors declare that they have no conflicts of interest. This does not alter our adherence to PLOS ONE policies on sharing data and materials.”

Editor comment 3: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers.

Response: After further consultation with our IRB, we have confirmed that we are approved to host a fully de-identified data set on a public repository. We have provided a copy of our data at: https://doi.org/10.21985/n2-33my-2s89

Reviewer Comments

Reviewer Comment 1. Possibly, the already listed Limitations could receive the title, “Limitations.”

Response: We agree that this would improve the structure of the paper and have edited the manuscript accordingly.

Reviewer Comment 2. I suggest starting with the conclusion section with line 354.

Response: We have updated the conclusion section to start where suggested.

Reviewer Comment 3: PlosOne’s required citation style is to be used correctly (e.g. line 350, 339)

Response: The citations have been updated to meet the required style, including the two lines cited.

Reviewer Comment 4: The references have to be double-checked. The literature has to be updated. Hardly any current literature is cited.

Response: We agree that current literature is an important aspect to include. With an eye toward more recent relevant literature, we have added the following three additional citations that focus on ARDS recognition and implementation of appropriate ventilator strategies.

• Spece LJ, Mitchell KH, Caldwell ES, Gundel SJ, Jolley SE, Hough CL. Rate of Low Tidal Volume Ventilation Use Remains Low in Patients with Acute Respiratory Distress Syndrome Despite Improvement Efforts at a Single Center. J Crit Care. 2018 Apr; 44: 72-76.

• Owyang CG, Kim JL, Loo G, Ranginwala S, Mathews KS. The effect of emergency department crowding on lung-protective ventilation utilization for critically ill patients. J Crit Care. 2019; 52: 40-47.

• Matthay MA, Zemans RL, Zimmerman GA, Arabi YM, Beitler JR, Mercat A, Herridge M, Randolph AG, Calfee CS. Acute respiratory distress syndrome. Nat Rev Dis Primers. 2019 Mar 14; 5 (1):18.

The first study provides a confirmation that the low utilization issues demonstrated in our cohort affect other large academic hospitals. This specific study takes place in a former ARDSnet trial hospital, demonstrating adoption drop-off after the clinical trial and the need for a more sustainable implementation strategy.

The second study confirms our tidal volume frequency findings of 450 and 500 mL as well as the lack of tidal volume changes for a single patient. Furthermore, it proposes default values within electronic orders as a source, which supports our conclusion that default tidal volumes may be the source of relationship between patient height and low tidal volume ventilation use.

The third study is a Nature Review Disease Primer, which provides details on the specific challenges of diagnosing ARDS and how that may affect treatment choices.

Reviewer Comment 5: What are the actual and above all current results of this study, which would ultimately justify a publication in a scientific journal?

Response: The main results of this study are twofold. First, we expand on the literature on the associations between patient and physician characteristics and use of low tidal volume ventilation in ARDS. We confirm prior findings that hypoxemia severity and ARDS documentation are associated with a higher likelihood of LTVV use. We demonstrate the novel result that even after normalizing for height, there is still a relationship between height and a patient’s likelihood of receiving LTVV, suggesting that the association moves beyond the height-dependent nature of LTVV threshold calculations. We conclude that when evaluating LTVV utilization, the height of the patient population should be accounted for.

Second, we construct a model for physician recognition of ARDS that accounts for hypoxemia, patient height, and ARDS documentation, which is novel in the literature. The model requires data that can easily be mined from an electronic health record, making it feasible at resource-constricted institutions, where other methods – such as surveys – would not be possible. This model can be integrated into a decision support or audit-and-feedback system for more continual assessment and intervention.

We thank you for your work and feedback.

Submitted filename: Response to Reviewers v2.docx

Click here for additional data file.

10 Sep 2019

[EXSCINDED]

A quantitative approach for the analysis of clinician recognition of acute respiratory distress syndrome using electronic health record data

PONE-D-19-17952R1

Dear Dr. Bechel,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Lars-Peter Kamolz, M.D., Ph.D., M.Sc.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: The authors have responded accordingly to the editor´s and author´s comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

13 Sep 2019

PONE-D-19-17952R1

A quantitative approach for the analysis of clinician recognition of acute respiratory distress syndrome using electronic health record data

Dear Dr. Bechel:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Lars-Peter Kamolz

Academic Editor

PLOS ONE