Accuracy of Laboratory Data Communication on ICU Daily Rounds Using an Electronic Health Record.

OBJECTIVES: Accurately communicating patient data during daily ICU rounds is critically important since data provide the basis for clinical decision making. Despite its importance, high fidelity data communication during interprofessional ICU rounds is assumed, yet unproven. We created a robust but simple methodology to measure the prevalence of inaccurately communicated (misrepresented) data and to characterize data communication failures by type. We also assessed how commonly the rounding team detected data misrepresentation and whether data communication was impacted by environmental, human, and workflow factors.
DESIGN: Direct observation of verbalized laboratory data during daily ICU rounds compared with data within the electronic health record and on presenters' paper prerounding notes.
SETTING: Twenty-six-bed academic medical ICU with a well-established electronic health record.
SUBJECTS: ICU rounds presenter (medical student or resident physician), interprofessional rounding team.
INTERVENTIONS: None.
MEASUREMENTS AND MAIN RESULTS: During 301 observed patient presentations including 4,945 audited laboratory results, presenters used a paper prerounding tool for 94.3% of presentations but tools contained only 78% of available electronic health record laboratory data. Ninty-six percent of patient presentations included at least one laboratory misrepresentation (mean, 6.3 per patient) and 38.9% of all audited laboratory data were inaccurately communicated. Most misrepresentation events were omissions. Only 7.8% of all laboratory misrepresentations were detected.
CONCLUSION: Despite a structured interprofessional rounding script and a well-established electronic health record, clinician laboratory data retrieval and communication during ICU rounds at our institution was poor, prone to omissions and inaccuracies, yet largely unrecognized by the rounding team. This highlights an important patient safety issue that is likely widely prevalent, yet underrecognized.

Critically ill patients generate vast quantities of data that clinicians must gather, interpret, and synthesize to facilitate effective clinical decision making. Daily interprofessional rounds are an integral component of critical care delivery as they provide an opportunity for ICU physicians, nurses, and pharmacists to converge, share data, and formulate the patient’s daily care plan (1, 2). Prior to rounds, each team member engages in their own “prerounding” or data gathering and cognitive processing in preparation to verbally share information (3). Medical errors are now the third leading cause of death in the United States (4). Previous research shows that incomplete data gathering and processing leads to diagnostic error and subsequent patient harm (5, 6). Thus, to minimize diagnostic error, communicated data should be accurate, complete, and current. Surprisingly, little is known about the quality of communicated data during rounds.

There are many reasons to question the accuracy of data communicated during ICU rounds. First, despite the implementation of interprofessional rounds, diagnostic error in the ICU remains a common occurrence. In one study, 28% of ICU deaths demonstrated at least one missed, wrong, or delayed diagnosis with 6% of these potentially fatal (7). Several studies demonstrate that data sharing during patient handoffs is rife with data communication failures (8–10). Poor handoff communication may occur when there is no standardized format, allowing for wide variability in content and quality of information transfer (11). A lack of standardized ICU rounds data reporting practices and individual clinician variations in ICU prerounding workflow may also affect the accuracy of the reported data.

The ICU environment also makes data gathering and communication challenging. With over 1,300 new data points/ICU patient per day, clinicians face information overload and may suffer from cognitive fatigue (12–14). Despite evidence suggesting that physicians who have been in clinical practice longer integrate data more successfully than those with less clinical experience, historic precedence in academic centers relegates the majority of data gathering to the least experienced physician on the team (15, 16). Interruptions and patient care emergencies are also commonplace in the ICU leading to clinician workflow disruptions that impair memory and task completion efficiency (17).

Finally, the health record user interface also influences clinician ability to extract and synthesize data. Paper-based patient records are often disorganized, illegible, and location bound, making serial data assessment and integration time consuming and inefficient (18, 19). Paper records also facilitate data omission and transcription errors given clinicians must manually reproduce information in order to transport it beyond the patient’s chart (20). Electronic health records (EHRs) offer a data management solution by providing a central repository of categorized, legible data that can be accessed by multiple users simultaneously during ICU rounds (21). EHRs also support the creation of rounding tools, which through macros, can automatically populate data, saving time and eliminating transcription errors (22, 23). Unfortunately, EHR use has yet to consistently improve ICU outcomes, which may in part relate to problems with EHR implementation, novel patient safety issues created by the EHR, and suboptimal EHR design and usability (24–27). The extent to which EHRs hinder data gathering and information sharing on ICU rounds also remains unclear (21).

For all of these reasons, we hypothesized that communication of patient data during ICU rounds is prone to errors and inaccuracies. Using the EHR, we created a methodology to evaluate physician data gathering and data communication accuracy on ICU rounds.

METHODS

The study was conducted in a single 26-bed closed medical ICU (MICU) at an urban tertiary care academic medical center. Daily rounds occur in the hallway outside the patient’s room and follow a structured rounding script. Trainees present interval data, and then solicit input from nursing, pharmacy, and respiratory therapy before presenting the plan. At least two EHR-equipped mobile computers are present on rounds at all times. No changes to this process occurred during the study period. Real-time attending physician use of the EHR on rounds is optional but encouraged. Our institution has used EPICcare (Epic Systems, Verona, WI) since 2008 and there were no major system upgrades during the study period. The study was approved by the Oregon Health and Sciences University Institutional Review Board.

Two senior ICU fellows (K.A., E.D.) audited the accuracy of laboratory data communication on daily ICU rounds. To ensure adequate training in the study methodology, personnel piloted data collection, including use of a templated data collection tool (Supplemental Digital Content 1, http://links.lww.com/CCM/C103) and direct observation by the senior author during a 1-month run-in period. Data collection commenced in August 2013 and occurred on weekdays with even sampling between the first and second half of the attending’s time on service (phase 1). Interim analysis after 3 months revealed a very high percentage of omitted laboratory data and that trainees uniformly used their prerounding notes (artifact) as a presentation aid. To understand at what point in clinician workflow loss of data fidelity occurred, the protocol was revised to include obtaining deidentified photocopies of presenters’ prerounding artifacts (phase 2, February 2014 to June 2014). We planned enrollment at 200 patients since a preventable life-threatening or fatal adverse event occurs on approximately one of every 200 ICU patient days (28). To avoid contamination of the prerounding process, investigators arrived minutes prior to the start of rounds. Investigators refrained from participating in rounds discussions. Information on patient census, duration of rounds, order of patient presentation, day relative to attending’s time on service, attending EHR use on rounds, patient disposition, and presenter level of training were also collected.

We audited the communication of 26 laboratory tests relevant to the management of critically ill patients (Supplemental Table 1, Supplemental Digital Content 2, http://links.lww.com/CCM/C104). We limited our study to laboratory data given their instant and accurate population of the EHR from the primary system, their overall ubiquitous presence on ICU patients, and the ease of real-time validation of communication accuracy. Specific laboratories were selected to provide broad representation of multiple organ systems and a balance of frequently and less frequently ordered tests. At our institution, trainees are expected to present all new data since the conclusion of rounds the day prior. Thus, all laboratory results from noon the day prior were eligible for analysis. When serial laboratories were present we audited only the most recent set. Using laboratory values displayed within the patient’s EHR on predefined laboratory screens as a gold standard, observers listened for the oral communication of laboratories during rounds. Spoken values were compared with EHR values captured on printed EHR screenshots taken just prior to the beginning of presentations (data collection protocol, Supplemental Digital Content 3, http://links.lww.com/CCM/C105).

Verbal laboratory communication was scored as accurate when any one of the following conditions was met: 1) laboratory value was accurately reported and not described; 2) laboratory value was omitted but accurately described; and 3) laboratory value was accurately reported and described. Laboratories were considered “misrepresented” if they failed to meet any of these criteria. For every laboratory misrepresentation event, we documented whether or not it was detected, and if so, by whom.

Misrepresentations were classified into one of several categories (Table ). Omissions did not include laboratories described as part of a laboratory set even if not explicitly named. Analysis of the prerounding artifact allowed for further determination of which omissions were due to “artifact creation failure” versus “artifact usage failure.” “Misinterpreted” data were laboratories that were incorrectly described according to the clinical judgment of the observers.

A chi-square test was used to assess for associations between categoric variables and communication accuracy and misrepresentation detection. Relative risk was calculated to assess strength of associations. For continuous variables, linear regression and a Pearson correlation and determination of R2 was performed. A p value of less than 0.05 was considered significant. All data were analyzed with Graphpad Prism (GraphPad Software Inc., La Jolla, CA) and Microsoft Office Excel 2010 (Microsoft Corporation, Redmond, WA).

RESULTS

We observed 34 MICU rounds yielding 301 patient rounding audits (n = 90 phase 1; n = 211 phase 2) and 4,945 laboratory test observations (Supplemental Table 1, Supplementary Digital Content 2, http://links.lww.com/CCM/C104). Interns most commonly presented patients. Eight of eleven attendings regularly viewed the EHR during rounds and 58% of all observations included attending EHR use.

Presenters created and used a paper artifact for 94.3% of presentations. However, the format of artifacts varied (Supplemental Fig. 1, Supplemental Digital Content 2, http://links.lww.com/CCM/C104) both by structure and presenters’ reliance on macros to import data; 82.4% of presenters created an artifact from within the EHR but most of these paper printouts also contained handwritten notes in the margins, and in some cases handwritten duplications of laboratories already present, and 17.6% of artifacts were manually created outside the EHR, most of which were entirely handwritten.

Overall, trainees accurately reported only 61.1% of the laboratories with no observed difference in accuracy between data collection periods (Table ). Most commonly trainees reported the laboratory value without any description (48.9%); 24.6% of laboratories were only described and 26.5% of laboratories values were both reported and described. Ninty-six percent of patients had at least one inaccurately communicated laboratory for an average of 6.3 laboratory misrepresentations/patient. The majority of misrepresentations were omissions (Fig. ). Artifact analysis revealed that 40.3% of omissions were artifact importation failures and 59.7% were artifact usage failures.

A, Types and frequencies of laboratory misrepresentation events. Observation of 301 patient rounds presentations and 4,549 laboratory data points yielded 1,886 inaccurately communicated laboratory test results which were further classified by type of misrepresentation. Most misrepresentation events were omissions. B, Frequency ICU team caught laboratory misrepresentation events varied by type of misrepresentation. Overall frequency of detection of laboratory misrepresentation events was low, with the exception of “pending” type misrepresentation events, which were detected at a significantly higher frequency compared with all other misrepresentation types.

The accuracy of laboratory communication varied by individual laboratory tests (Figs. and 3). The frequency of test ordering correlated strongly with communication accuracy. Infrequently ordered laboratory tests were most likely to be misrepresented (p < 0.00001). Misrepresentations also appeared to cluster by ordering panel such as the blood gas and liver panel tests (Fig. 2).

Figure 2.

Accuracy of laboratory communication on rounds and ICU team detection of misrepresented laboratory data by individual laboratory test. We observed the communication of 4,549 laboratory data points from 26 selected domains on daily ICU rounds. Communication accuracy and detection of misrepresented laboratory data varied by individual laboratory test. alk phos = alkaline phosphatase, ALT = alanine aminotransferase, AST = aspartate aminotransferase, Bcx = blood culture, BUN = blood urea nitrogen, Cr = creatinine, Hb/Hct = hemoglobin or hematocrit, HCO3 = serum bicarbonate, Phos = phosphate, Plt = platelet count, PT/INR = prothrombin time or international normalized ratio, PTT/hep = partial thromboplastin time or heparin level, ScvO2 = central venous oxygen saturation, t bili = total bilirubin, trop = troponin.

The prerounding artifact was strongly linked with accurate laboratory communication on rounds. Overall, any laboratory extracted from the EHR and present on the artifact was more likely to be accurately communicated on rounds compared with a laboratory absent from the artifact (p < 0.0001) (Table 1). Additionally, the more consistently a type of laboratory test was found on trainee artifacts, the more likely it was that communication of the same laboratory on rounds was accurate (p < 0.00001) (Fig. ). Despite the protective association between artifact use and communication accuracy, artifacts were incomplete and contained only 78.5% of audited laboratories. Yet, the more frequently a laboratory was ordered, the more likely it was to be included on the artifact (R2 = 0.82; p < 0.00001). Whether the presenter used an EHR-generated or manually generated artifact had no correlation with communication accuracy (Table 2).

TABLE 1.

Definitions of Types of Laboratory Data Misrepresentation

TABLE 2.

Factors Associated With Accurate Communication and Detection of Data Misrepresentation

A, Correlation between frequency a laboratory test was ordered and accurate communication of the laboratory results. Pearson correlation showed more frequently ordered laboratory tests were more often accurately communicated on rounds. B, Correlation between frequency a laboratory test was present on the artifact and accurate communication of the laboratory results. Pearson correlation showed laboratory tests more commonly present on the artifact were more commonly accurately reported on rounds. Alk phos = alkaline phosphatase, ALT = alanine aminotransferase, AST = aspartate aminotransferase, BUN = blood urea nitrogen, Ca/iCa = calcium or ionized calcium, Cr = creatinine, Hb/Hct = hemoglobin or hematocrit, HCO3 = serum bicarbonate, Phos = phosphate, Plt = platelet count, PT/INR = prothrombin time or international normalized ratio, PTT = partial thromboplastin time or heparin level, ScvO2 = central venous oxygen saturation, T.Bili = total bilirubin.

We next looked at the effect of sociotechnologic factors on laboratory communication accuracy (Table 2; and Supplemental Fig. , Supplemental Digital Content 2, http://links.lww.com/CCM/C104). There was no difference in laboratory communication accuracy based on presenter level of training or what day of an attending’s time on service audits occurred. Laboratory communication accuracy was worse when ICU census was greater than 14 patients, when the attending did not use the EHR, on patients presented later in rounds and when presentations lasted more than 20 min/patient.

Although inaccurate laboratory data communication was common, the interprofessional rounding team recognized only 7.8% misrepresentations. Attending physicians accounted for 56.4% of detected misrepresentations followed by nurses (17.6%), fellows (9.5%), residents (9.1%), and pharmacists (7.4%). An exception to poor team recognition of laboratory misrepresentations was the detection of “pending” type misrepresentations (Fig. ).

Similar to laboratory communication accuracy on rounds, environmental and human factors had a minor impact on the ICU team’s ability to detect laboratory misrepresentations (Table 2; and Supplemental Fig. 2, Supplemental Digital Content 2, http://links.lww.com/CCM/C104). Teams detected more data misrepresentations later versus earlier in the academic year, on patients presented by medical students compared with residents and on patients transferring out of the ICU. Misrepresentation detection was incrementally worse when the attending did not use the EHR, on patients presented at the end of rounds and when presentations lasted less than 10 min/patient.

DISCUSSION

In this study, we developed a simple and reproducible methodology to assess the accuracy of data communication on ICU rounds by studying the intersection of what is present in the EHR with what is generated on paper and finally verbalized on ICU rounds. At our institution, despite well-established EHR use and structured interprofessional ICU rounds, we discovered that laboratory misrepresentation was a pervasive phenomenon. It occurred on almost every patient and multiple times within the same presentation, involving nearly 40% of the laboratories studied. These results are consistent with the reports of communication failures during patient handoffs, which are now acknowledged as a universal patient safety issue (11). Thus, we suspect that data misrepresentation on ICU rounds at other institutions will be equally prevalent. This study provides a framework and methodology to facilitate future research.

Disappointingly, despite multiple studies supporting the positive impact of interprofessional ICU rounding, team-based rounding failed to compensate for individual clinician data communication failures (29–32). Furthermore, teams disproportionately relied on the attending physician to detect data misrepresentation. Possible explanations include unequal EHR access for real-time data viewing to recognize errors, inability to simultaneously listen, process, and verify data, individuals’ unwillingness to prolong rounds, deference to physicians, or perhaps a lack of active engagement by other participants on rounds.

Another key finding of our study was that data omissions were the most common form of data misrepresentation. Data omissions by clinicians may be unintentional (incomplete data gathering and processing) or deliberate (selective data communication); yet both have the potential to negatively impact patient care. Some may argue that intentionally culling data from rounds presentations is justified or even desirable because doing so might improve rounding efficiency and reduce clinician data overload. Yet, other team members on rounds may be unable to, in real time and merely by listening to an oral dialogue, distinguish between appropriate data filtering and lapses in data gathering. Furthermore, the problem with allowing individual providers, especially inexperienced trainees, to selectively present and omit data in a nonstandardized way, is that it allows one individual’s cognitive biases to contaminate the entire group, potentially leading to team consensus around a misdiagnosis (33). Finally, the reduction of unnecessary daily tests in the ICU is a goal of the “Choosing Wisely Campaign” (34). If nonessential test results are never brought to the team’s attention on rounds, wasteful test ordering behaviors will likely persist.

The presence of data on presenter artifacts was the strongest predictor of accurate data communication. This suggests that using a prerounding tool as a presentation aid is an effective data management strategy in our ICU where physicians are expected to gather, interpret, and communicate the entire 24-hour dataset on rounds. Continued use of paper artifacts and reliance on a single individual despite the ability to directly view data on computer screens during rounds may represent a failure to incorporate EHR technology. Alternatively, it represents a coping strategy for an EHR system that does not automatically provide an effective visual display of data needed for daily rounds. Our prior work showed that in order to recognize patient safety issues in simulated MICU patient cases, clinicians had to visit over 30 different EHR screens (15, 35). Thus, the value of the prerounding artifact may be that it gives clinicians a standardized data collection script and creates a single visual display of all rounding data that are otherwise geographically fragmented within the EHR.

Conversely, artifact collection and analysis highlighted the many limitations of electronically generated prerounding tools. Many trainees printed incomplete daily progress note templates that included automatically imported laboratory fields, which may account for the extremely low number of “erroneous” misrepresentations attributable to transcription errors. However, over 20% of audited laboratory data never appeared on artifacts and these templates consistently lacked data fields for less frequently ordered laboratories. Printouts of electronic prerounding tools are also temporally static and fail to automatically incorporate new data resulting after artifact creation. This explains the observation of “pending” and “old data” misrepresentations comprising 14% of misrepresentations. Prospectively designing and ensuring unit-wide adoption of a single, comprehensive prerounding template might mitigate the lack of standardization in prerounding data gathering. Requiring clinicians to present off an electronic prerounding tool that automatically refreshes new data might reduce communication of outdated information. However, these strategies would require further testing including validation of the efficacy of data importation macros. Additionally, reducing the time clinicians spend gathering data and limiting their ability to freely annotate the artifact might also have negative unintended consequences on cognitive processing.

Some of our findings validate existing interprofessional rounds best practices and highlight vulnerable conditions that warrant additional ICU team vigilance (29). For example, data fidelity was worse when census exceeded 14 patients or after the 14th patient presented, further supporting 14 patients as a critical census threshold beyond which the quality of ICU care declines (36).

Our study has important limitations. First, these data represent the experience of a single ICU at one academic institution and need replication in other institutions with different workflows and rounding paradigms. However, the importance of our study is that for the first time, it highlights the potential extent of data misrepresentations on rounds and the multiple variables that may contribute to this. The simple, low-tech methodology used will allow for each institution, with its own unique technology and workflow, to assess both the frequency of data misrepresentation and the impacts of any modifications made to data gathering or rounding processes. A second limitation is the potential Hawthorne effect of in-person observers. However, if the presence of observers on rounds artificially improves communication, then one would predict that our results actually underestimate the frequency of misrepresentations. Third, our study did not link laboratory miscommunication with patient outcomes. Prospectively it is difficult to predict what data ultimately prove critical to effective decision making on individual patients. However, our methodology could be used in EHR rounds simulation exercises in which the diagnosis and desired clinical decisions are known (37). Finally, we limited our communication audit to a select group of laboratory data; thus, we cannot comment on patterns of miscommunication in other important domains such as vital signs, ventilator data, medications, or imaging.

CONCLUSIONS

We developed a methodology to assess ICU physician ability to extract laboratory data from the EHR and accurately present it on interprofessional rounds. At our institution, we found a high frequency of data misrepresentation, especially information omission that was largely unchecked by the rounding team but partially prevented by the use of a prerounding tool. This represents a patient safety issue that may be more widespread than currently recognized and deserves additional study.