Variation in electronic test results management and its implications for patient safety: A multisite investigation.

OBJECTIVE: The management and follow-up of diagnostic test results is a major patient safety concern. The aim of this qualitative study was to explore how clinicians manage test results on an everyday basis (work-as-done) in a health information technology-enabled emergency department setting. The objectives were to identify (1) variations in work-as-done in test results management and (2) the strategies clinicians use to ensure optimal management of diagnostic test results.
MATERIALS AND METHODS: Qualitative interviews (n = 26) and field observations were conducted across 3 Australian emergency departments. Interview data coded for results management (ie, tracking, acknowledgment, and follow-up), and artifacts, were reviewed to identify variations in descriptions of work-as-done. Thematic analysis was performed to identify common themes.
RESULTS: Despite using the same test result management application, there were variations in how the system was used. We identified 5 themes relating to electronic test results management: (1) tracking test results, (2) use and understanding of system functionality, (3) visibility of result actions and acknowledgment, (4) results inbox use, and (5) challenges associated with the absence of an inbox for results notifications for advanced practice nurses. DISCUSSION: Our findings highlight that variations in work-as-done can function to overcome perceived impediments to managing test results in a HIT-enabled environment and thus identify potential risks in the process. By illuminating work-as-done, we identified strategies clinicians use to enhance test result management including paper-based manual processes, cognitive reminders, and adaptive use of electronic medical record functionality.
CONCLUSIONS: Test results tracking and follow-up is a priority area in need of health information technology development and training to improve team-based collaboration/communication of results follow-up and diagnostic safety.

INTRODUCTION

“Diagnostic Stewardship and Test Result Management Using EHRs” has been identified as the first priority on the Emergency Care Research Institute’s list of the Top 10 Patient Safety Concerns for 2019, highlighting health information technology (HIT) enabled test result management as a crucial area for focus in improving patient safety. One aspect of test result management, the follow-up of test results pending at discharge (TPADs), is a major safety concern that has been the focus of numerous studies. This is because TPADs may ultimately be overlooked, face delays in follow-up, or be inadequately communicated.

HIT plays an increasing role in the management of diagnostic test results. A wide variety of HIT-enabled interventions have been designed to assist clinicians in managing test results, including computerized physician order entry, electronic results acknowledgment, automated email notifications of TPADs,, and electronic health record (EHR) alerts., Evaluations of HIT interventions have reported improvements in awareness, communication, and access to clinical test results,,, as well as in efficiency, response time, and the proportions of tests being followed up., Notwithstanding the benefits of HIT, delayed or missed test results, and failure to follow-up test results persist and pose a serious safety issue in healthcare. Furthermore, there is growing recognition that HIT may have occasioned a new category of safety-related issues, with inadvertent consequences potentially arising from usability challenges. An analysis of patient safety reports by Howe et al identified 7 categories of usability issues including, for example, data entry, alerting, and interoperability. The need for understanding and improving safety in the area of HIT systems is becoming an emerging priority for health care,, with evidence from a systematic review by Georgiou et al questioning whether HIT alone is enough to address the issues pertaining to the safe management of diagnostic test results.

In recent years, healthcare safety research has seen the emergence of a new paradigm that shifts the emphasis from addressing what went wrong (Safety I) to understanding the importance of why things go right (Safety II). As a complementary approach to Safety I, Safety II acknowledges the value of understanding and learning in the context of the variability and adjustments that occur in everyday work. It is eloquently summarized by Hollnagel: “the way that work is actually shaped by the working conditions and environment is the best basis for making improvements as well as for identifying hazards.” Achieving an understanding of what goes right can be realized through studying “everyday clinical work,” which has been advocated in the discipline of resilience engineering to learn and understand how clinicians perform and adapt on an everyday basis to deliver safe and effective outcomes. Everyday clinical work has been studied in a number of research contexts including workarounds in nursing and in understanding the gap between work-as-done (WAD) (ie, how work is performed in everyday clinical work) and work-as-imagined (WAI) (ie, how work is expected to be performed).,

Although clinician management of test results in EHR enabled settings has been studied in the literature, the body of evidence is predominantly associated with inpatient or admitted,, or outpatient or primary care settings,, with relatively few studies in the emergency department (ED) context., The ED setting affords the opportunity to explore how diagnostic test results are managed in a fast-paced clinical environment providing urgent medical care to patients presenting with a broad range of conditions. We thus sought to explore test result management in the ED by contrasting the perspectives of WAD and WAI, to gain insight into how clinicians use (or adapt to use) the EHR to manage test results on an everyday basis. The study focused on identifying (1) variations in WAD in EHR-enabled test results management and (2) the strategies clinicians use in everyday clinical work to ensure optimal management of diagnostic test results. By illuminating if and why variations occur, we anticipated identifying potential safety risks which can be used to inform recommendations for improved use of HIT in test results management.

MATERIALS AND METHODS

Study design, participants, and HIT context

Qualitative data used in this study were collected as part of a larger project investigating diagnostic test result communication, management, and follow-up. The study was undertaken across 3 New South Wales public acute care hospital EDs, including two 100-199 bed hospitals and 1 principal referral hospital ∼500 beds. A purposive sample of clinicians was selected in consultation with ED management at each site to include participants from a cross-section of clinical roles and experience including junior and senior medical staff, nursing staff, and clinicians with management roles. J.L. and M.R.D. interviewed 26 participants, including staff at the micro level (nurses, junior and senior doctors; n = 19) and meso levels (clinicians with management responsibilities and ED directors; n = 7). To ensure anonymity with small sample numbers, only aggregated metrics are presented in Table 1.

Table 1.

Aggregated study metrics

Study metric	Quantifier
Number of Participants (Total)	n = 26
• Number of senior medical participants (staff specialists, consultants, career medical officers, medical directors)	n = 11
• Number of junior medical participants (interns, residents, registrars)	n = 7
• Number of nursing participants (registered nurses, advanced practice nurses, nurse practitioners)	n = 8
Interviews (excluding observations/demonstrations) (total) h:min:s	11:43:29
Observation sessions total duration, h:min:s	1:45:26
Demonstrations during interviews total duration, h:min:s	1:04:20

M.R.D. and J.L. conducted semistructured interviews across 3 Australian EDs between October 2016 and November 2017. Open-ended interview questions (Supplementary Appendix 1) allowed participants to describe their test result management work and communication practices, from test ordering through to the return and acknowledgment of test results. Interviews were audio recorded for subsequent transcription.

In addition to interviews, we also observed clinicians performing test results management activities using the electronic medical record (EMR) including both focused observation sessions of 9 of the 26 participants before or after their interviews (n = 9) and demonstrations during interviews (n = 11). Within-interview demonstrations included participants demonstrating ad hoc EMR screens or functions, whereas observation sessions focused on participants performing workflows. In the observations, participants clarified concepts and performed processes discussed during interviews using the EMR, facilitating researcher understanding of EMR functionality and WAD. Observations ranged from <5 minutes (n = 3) for straightforward processes (eg, adding test results to an EMR letter template) or longer for multiple or more detailed processes (eg, actioning results returning to the message center; (n = 6; range 00:06:12 to 00:33:57). Additional data were also collected in the form of de-identified photographs, field notes, and feedback on the draft article by 2 representative staff from the sites studied.

As part of the NSW Health EMR rollout, all sites had completed implementation of the foundation EMR (Cerner; North Kansas City, MO) in the ED by 2011. At the time of data collection, all laboratory and imaging tests were ordered electronically within the EMR, and results were returned electronically to the EMR patient record.

Ethics approval

Ethics approval was granted by the relevant Local Health District Human Research Ethics Committee, and all participants provided written informed consent.

Data analysis

Interview transcripts were de-identified, and J.L., M.R.D. and J.T. coded the data using NVivo 12 (QSR International, Melbourne, Australia), applying a coding classification scheme based on the interview questions, which was reiteratively refined during data immersion. Data coded for descriptions of results management (eg, test result tracking, acknowledgment, follow-up), and associated artifacts, were reviewed for participant descriptions of how test results should be managed in their ED using the EMR. As the focus of the study was on how clinicians use (or adapt to use) the EMR to manage test results, communication of results beyond the EMR (eg, to patients) was out of scope of the present study but has been discussed elsewhere. Based on these data, J.T. constructed workflow diagrams representing WAI at each study site using Microsoft Visio (Microsoft Corporation, Redmond, WA). Once the WAI workflows were determined for each site, M.R.D., J.L., and J.T. then independently reviewed transcripts and artifacts to identify descriptions of WAD in which participants described variations from WAI. The 3 authors then convened to review exemplars and reach a consensus on the data analysis. Guided by principles of thematic analysis, the final dataset of exemplars was subsequently analyzed to identify common themes.

Data quality

Strategies to reduce bias and maximize quality and trustworthiness included data triangulation, which involved the collection of data from interviews, observations, field notes, and de-identified photographs. Investigator triangulation was also used including 2 authors performing data collection, and data analysis was independently performed by 3 authors with findings and disagreements deliberated in joint discussion until a consensus was reached. Interviews were conducted with participants at both the micro level (nurses, junior and senior doctors) and meso level (clinicians with management responsibilities and ED directors). In addition, member checking was undertaken by (1) seeking feedback by presenting key findings from preliminary analysis at an executive meeting that included study site representatives and (2) providing a copy of the draft article to 2 key informants from study sites, for validation and feedback on the data analysis.

RESULTS

In line with the sequential steps of the data analysis, the results of the study are presented commencing with descriptions of WAI and the associated workflow diagrams for each study site. The site-specific variations in WAI-WAD are then detailed followed by a table summarizing all data exemplars. The results of the thematic analysis are then detailed including representative quotes from both interviews and observations.

Work-as-imagined

Diagnostic test orders were placed electronically within the EMR and test results returned electronically to the message center “inbox” of the doctor under whose name the test was ordered. Acknowledgment of test results could be performed after reviewing the result in the doctor’s inbox. Test results also populated the patient’s medical record. In addition to the inbox feature, the EMR functionality included a results “pool” that could be populated with all test results from all ordering clinicians within the ED at each site. The EMR pool included the ability to acknowledge each test result by selecting from an “endorse” (default), “refuse,” or “forward…to” option, thereby clearing that result from the pool. WAI workflow diagrams are presented in Figures 1–3.

Figure 1.

Work-as-imagined workflow diagram for emergency department A (ED-A), showing the workflow for junior doctors, senior doctors, and nursing staff. Dashed lines indicate electronic workflows, solid lines are participant workflows, and dotted lines represent the additional workflow associated with the results pool. The “tick” indicates the end of the workflow at the point at which the test result is removed from the pool. Workflows for junior doctors, senior doctors, and nurses/nurse practitioners are colored blue, gray, and orange, respectively. EMR: electronic medical record.

WAD: Site-specific and individual variations

The EMR afforded functionalities for test result review and acknowledgment via the main patient chart, a personal message center (“inbox”) for medical staff, and a results pool. The pool compiles all unacknowledged ED test results (accessible only to authorized staff) and both site-specific and cross-site variations were identified for the use of this functionality. At ED-A, only senior medical staff (ED specialists/consultants) had access to the results pool and were responsible for reviewing and endorsing pooled results on an ongoing basis. This is depicted in Figure 1, in which senior doctors are shown to have 2 “review results” workflows: for “review results in inbox” (gray-colored workflow) and an additional “review pool” (red-colored workflow). At ED-B, 1 senior doctor was delegated responsibility to review and endorse results in the pool on a weekly basis, with other senior doctors assisting in clearing the pool on an ad hoc basis (eg, if they had downtime on a shift). This is shown in Figure 2, in which an allocated senior doctor (colored purple) performs the “review pool” workflow. ED-B also utilized junior doctor downtime during a shift to check results in the pool but junior doctors were only allowed to approve normal blood results, not abnormal bloods nor any radiology. At ED-C, the results pool functionality was not activated at the time interviews were conducted, as depicted in Figure 3, in which there is a notable absence of any results pool workflow (compared with Figures 1 and 2).

Figure 2.

Work-as-imagined workflow diagram for emergency department B (ED-B), showing the workflow for junior doctors, senior doctors, and nursing staff. Dashed lines indicate electronic workflows, solid lines are participant workflows, and dotted lines represent the additional workflow associated with the results pool. The “tick” indicates the end of the workflow at the point at which the test result is removed from the pool. Workflows for junior doctors, senior doctors, and nurses/nurse practitioners are colored blue, gray, and orange, respectively. EMR: electronic medical record.

Figure 3.

Work-as-imagined workflow diagram for emergency department C (ED-C), showing the workflow for junior doctors, senior doctors, and nursing staff. Dashed lines indicate electronic workflows and solid lines are participant workflows. The “tick” indicates the end of the workflow at the point at which the test result is removed from the ordering doctor’s inbox. Workflows for junior doctors, senior doctors, and nurses/nurse practitioners are colored blue, gray, and orange respectively. EMR: electronic medical record.

Following review of participant descriptions of WAD, a total of 26 excerpts were identified relating to differences between WAD and WAI. Thematic analysis was performed, and 5 themes were generated based on commonalities in participant reasoning underscoring the individual variations in WAD. The themes and results are presented in Table 2.

Table 2.

Summary of results grouped by site and theme

Theme	ED-A	ED-B	ED-C
1. Clinicians use strategies to track test results	Junior doctor leaves results in their inbox as a reminder of patients to follow-up later. Nurse uses manual process (stickers) to track results that require follow-up.	Advanced clinical nurse uses a manual process (diary) to track results expected to return postdischarge.	Senior doctor leaves results in their inbox if they require follow-up or further review.
2. There is variation in use and understanding of EMR functionality	Senior doctor uses the forward action inbox function to refer results to the doctor looking after the patient (eg, inpatient results); however, another senior doctor expresses uncertainty about this functionality. Senior doctor doesn’t use the acknowledgment comments section to record actions taken, as unsure how to retrieve the information after the result is cleared from the inbox (*also theme 3).	Senior doctor reviews and acknowledges admitted patient results in their inbox on the assumption they are being managed by the inpatient team (compared with senior doctor ED-A).	Junior doctor acknowledges results from the EMR results screen when reviewing results. Another senior doctor is uncertain of this. Senior doctor is unsure of EMR inbox functionality.
3. Visibility of test results acknowledgment/action	Senior doctor documents result actions in the patient notes as a form of acknowledgment. Senior doctor uses comments section during acknowledgment to record actions taken. Another senior doctor doesn’t use comments section as unsure how to retrieve the information after the result is cleared from the inbox (also theme 2). Nurse records critical results phone calls in the nurse’s notes.	Junior doctor acknowledges results in both the inbox and progress notes to ensure visibility.	Senior doctor acknowledges clinically significant abnormal test results in patient’s notes and follow-up recommendations.
4. Senior clinicians’ management of the results pool is context specific	Doctors recognize the value of the pool as a safety net for results that return after discharge. Senior doctor has access to the pool but doesn’t really look at it. Senior doctor is aware of the pool but doesn’t have/request access. Senior doctor uses the “refuse” action for blood test results in the pool ordered by other doctors as they should have been reviewed while the patient was in ED. Senior doctor doesn’t review blood test results in the inbox/pool (token acknowledgment) because the results should come back within the timeframe of the encounter.	Senior doctor uses the pool as quality control to cross-check results and appropriate follow-up.	(The results pool functionality was not activated at this site)
5. Advanced nursing staff with ordering rights face challenges with results management in the absence of an inbox	Nurse generates EMR reports as lists of their patients’ requiring follow-up or with results pending at discharge. Nurse acknowledges results and action taken in the patient notes. Nurse reviews each patient’s notes for results/actions as results of the tests they ordered are returned to their supervising doctor’s inbox.	Nurse acknowledges results in the patient notes.	Nursing staff can only access results via EMR.

ED-A: emergency department A; ED-B: emergency department B; ED-C: emergency department C; EMR: electronic medical record.

Clinicians use strategies to track test results—Clinicians at all 3 EDs reported using both manual and electronic strategies to track TPADs and ensure returned results are followed up. In the absence of an inbox, nursing staff used paper-based strategies to track pending test results. These strategies were representative of cognitive reminders to attempt to safeguard against missed results or failure to follow up. For example:

“If it’s something that I need to chase up, perhaps the next day or the patient’s been discharged home and I’m going to call them, I usually put in the note [example of comment in EMR patient notes]. And then I would just make myself a note somewhere…just a Post-It; I usually stick the patient’s sticker on my [mobile device] actually” Observation-N04 (ED-A)

With respect to tracking results that might arrive after discharge, to reduce the risk of missed results:

“…I do have a diary for cultures and swabs and whatnot, for the day that I expect that they will be back. …It’s all a manual process, yeah, and it is all time consuming…” Interview-N05 (ED-B)

And for test results requiring follow-up after discharge, a clinician explained leaving the results in their inbox until the follow-up was completed:

“I wouldn’t sign off on something that needed something…I just leave it, I just close it and then it stays there. …you can ring them and then I’d take it off the system but…It might be midnight at the end of my shift…and you’re not going to ring them up then, so I just leave it there [in the inbox] so it’s one that is not off the system.” Interview-M18 (ED-C)

A similar example of leaving results in the inbox as a cognitive reminder was explained in an anecdote by a senior doctor:

“…one junior said [they were] keeping them [test results] in there [inbox] because it was a way for [them] to remind [themselves] about certain patients to follow them up later. I said “But you can’t keep them in your message center, you’re not actually ticking them off and they’re getting stuck in the pool and then one of the [senior doctors] has to go and do it. You’re creating work for [the senior doctors]”” Interview-M02 (ED-A)

This example describes both an adaption of the inbox (leaving results unendorsed) as a cognitive reminder to follow up test results and also reveals a negative impact of the adaption.

There is variation in use and understanding of EMR functionality—Assumptions and statements including “I have no idea” and “I don’t know” attest to clinicians’ uncertainty regarding different EMR functionalities related to HIT-enabled test result management. Clinicians reported not using some test result acknowledgment functions due to insufficient system knowledge or because of uncertainty regarding the impact of their use. For example, when discussing test results acknowledgment in the EMR inbox, a doctor noted not using the acknowledgment “comments” section to record actions taken, as they were unsure how to retrieve the acknowledgment information after the result is cleared from the inbox:

“I have no idea what happens to those, where they go and where they disappear to and how you find them again. …And how you would ever find out who signed them.” Interview-M08 (ED-A)

Similar uncertainty was expressed regarding inbox “results forwarding” functionality:

“I have no idea whether that actually works or not.” Interview-M08 (ED-A)

as well as escalation functionality:

“…when something is in their message center for a period of time, it then defaults to [specified inbox]. It may be. I don’t know.” Interview-M16* (ED-C)

Furthermore, a clinician explained how they acknowledge results in real-time when reviewing results in the EMR, however, another clinician expressed uncertainty about this functionality:

“I know you can sign them off in the other section somehow but I don’t how to do it.” Interview-M18 (ED-C)

These examples suggest that individual WAD depends on the level of understanding of system functionality.

Visibility of test results acknowledgment/action—Notwithstanding the ability to acknowledge test results in the EMR inbox, clinicians reported using additional strategies to document/follow-up results to ensure visibility of their acknowledgment and actions. Medical staff reported documenting results in the patient progress notes for visibility of acknowledgment and action. For example, when explaining results acknowledgment:

I just usually put it in the progress note…I do [acknowledge from the message center] But, no one can see that I’ve acknowledged it…I think it just signs off in the system…” Interview-M11 (ED-B)

A nurse also described documenting critical results phone calls in the nurse’s notes for visibility of receipt and action:

“…if anything was reported to me and I would document it in the nurse’s notes. That I had received that report and who I’d gone and escalated it to.” Interview-N01* (ED-A)

Senior clinicians’ management of the results pool is context specific—Differences in pool utilization were observed as both intersite WAI and intrasite WAD variations. ED-A recognized the value of the pool as a safety net for results that return after discharge (eg, imaging and microbiology results):

“…because blood cells come back within the time frame of the encounter. The ones that we’re looking at are things that don’t come back within the time frame.” Observation-M09* (ED-A)

To process the large volume of results accumulating in the pool, doctors at ED-A used strategies such as token acknowledgment of test results that should have been reviewed during the patient encounter (eg, blood test results) or inbox “refuse (with reason)” functionality, which leaves the result unacknowledged and removes it from the inbox or pool:

“There’s too many. It’s cognitive dismiss. I just click, click, click.” Observation-M09* (ED-A)

“So the doctor caring for that patient should have been looking at those blood test results before they discharge the patient home” Observation-M02 (ED-A)

In contrast, at ED-B, 1 senior clinician was assigned to use the pool as a quality control safety net to cross check results and ensure appropriate follow-up on a weekly basis:

“… the problem is that endorsement [in the personal inbox] removes that quality control. …we’ve got [number] per cent locums. They don’t know this process anyway” Interview-M15* (ED-B)

whereas ED-C had not implemented the results pool at all.

Advanced nursing staff with ordering rights face challenges with results management in the absence of an inbox—Nurse practitioners in Australia can practice autonomously within their scope of practice which includes ordering and interpreting diagnostic tests. However, inbox functionality was not activated for any nursing staff:

“… when the advanced clinical nurses order something … it will go in under the name of the consultant…or the in-charge of the day. … they [nurses] don’t have the capacity to authorize, so they won’t get the results [directly]” Interview-M16* (ED-C)

These nurses reported time consuming processes to generate EMR patient lists for results follow-up, including a nurse practitioner explaining:

“… but this is also time consuming- is to go and create a…report and actually pull that report up and then go through that list of patients. It’s just time consuming.” Interview-N04 (ED-A)

or reviewing every one of their patients’ notes to search for results or actions and using the patient notes to acknowledge results and the actions they had taken:

“… it’s more a matter of acknowledging within your patient notes that you’ve seen said result.” Interview-N04 (ED-A)

A nurse practitioner noted the safety implications of the absence of an inbox:

“I just order under the [supervising doctor]….But, again, it’s a process that’s flawed because if I miss the result and [the supervising doctor] is not in, that result might not get seen for two weeks. …Because I actually have to go and find the results, rather than the system saying, right, you’ve ordered these…here’s your chance to review them and sign off on them…” Interview-N05 (ED-B)

DISCUSSION

Through exploring WAD to gain a better understanding of how test results are managed by clinicians on an everyday basis, the current study identified 5 key themes relating to electronic test result management. These themes have implications for the safety and design of HIT test result applications as they identify posteriori adaptive strategies used by clinicians (WAD) to optimize test result management outcomes.

Test results tracking

One of the most safety critical aspects of test results management is ensuring that the results of diagnostic investigations are followed up. Despite the availability of an electronic test results management application, clinicians in the current study reported using a combination of both manual and electronic tracking strategies to ensure test results requiring follow-up were actioned (themes 1 and 5). This need to support “memory of pending tasks” for managing test results and using paper-based strategies in HIT-enabled settings such as sticky notes,, lists, and logs, has been previously reported in the literature,,, and characterized as “workarounds.” The paper and electronic tracking strategies we identified meet the definition of workarounds stated in Debono et al as “observed or described behaviors that may differ from organizationally prescribed or intended procedures.” The context surrounding our participants’ adoption of workarounds are indicative of clinicians actively employing strategies to support the cognitive demands of tracking test results through to follow-up and completion. Thus, they also fall under the definition of resilience stated in Smith et al as “positive adaptability within systems that allows good outcomes in the presence of both favourable and adverse conditions.” Although resilience strategies can maximize positive outcomes, they also risk obscuring underlying barriers or threats to safety., Accordingly, the findings of the current study have implications for the safety of electronic test results management as they identify a need for HIT to support clinicians in tracking test results, and especially those that return after a patient has been discharged. Failure to follow-up test results and TPADs are major safety concerns., Our findings reinforce results tracking and follow-up as a priority area for HIT development to improve the safety of results follow-up (by reducing the risk of missed results or failure to follow-up) and provide support for clinician’s cognitive needs. The between-site variations in WAD highlight a need for HIT to provide flexible solutions to meet the results management needs specific to the individual clinical setting. The similarity of our findings with those reported by researchers in the United States and Canada, in both primary care and internal medicine, reinforces test result tracking as a widespread universal concern for HIT.

Notifications and information overload

Although HIT functionalities such as EMR inboxes and alerts have been implemented to assist clinicians in managing test results, there is increasing evidence of the potentially negative impacts of the volume of notifications on the cognitive workload of clinicians. The issue of inbox “information overload” in our study was highlighted in the excerpts of ED-A clinicians with access to the results pool. The demands associated with managing inbox notifications were compounded by the pooling of unendorsed results, triggering strategies to address the volume of the pool. Our study identified strategies clinicians used to manage “information overload” such as token acknowledgment, cognitive dismissal, avoidance or inbox “refusal” functionality. These findings add to the safety concerns associated with managing large volumes of electronic test results-notifications recognized in the literature. One approach to reducing issues associated with test results notifications was a “test management governance model” implemented at an Australian Mothers’ hospital. The model included an electronic results acknowledgment triaging function to allow midwives to review screening tests for the purpose of “vetting” normal results, the use of an algorithm to automatically acknowledge results meeting predefined “normal” criteria, and an escalation process for unacknowledged results. Another approach identified in a study of electronic management of abnormal cancer-related test results in a primary care setting, included a process in which the diagnostic service provider coded the importance of test results to allow primary care providers to prioritize their alerts.

Such approaches offer translational potential to other clinical contexts. For example, a variation to the cited “test management governance model” could include increased responsibilities for nurse practitioners by way of a personal inbox and results triage. A personal inbox would also allow them to check, acknowledge, and action prespecified results of test orders they initiate, thereby overcoming the need for manual test results–tracking strategies and reducing inbox (and pool) notification volumes for supervising medical staff found in the current study.

The variation in workflow models for the pool at our study sites has implications for EMR systems design/development, as it demonstrates how EMR functionality can be implemented and used in multiple ways to support context specific work processes (theme 4). This is a noteworthy exemplar of HIT contextualization and has implications for HIT implementation, as it is evidence of the need for HIT to flexibly support variable workflow and governance models across clinical settings.

HIT usability

Variations in WAD in our study highlighted disparities in the use and understanding of EMR functionality (theme 2), further supporting research which has identified HIT usability as an emerging safety concern in heatlhcare. In addition to current usability concerns, we identified clinicians’ uncertainty about the impact of using EMR functionality. Although these findings may suggest a need for ongoing training, research into root cause analysis (a Safety I perspective) has identified training is a “poor solution” to address serious systems safety issues. Our findings do, however, show that clinicians use of HIT depends on their understanding of the HIT system as a whole. The variable use of the results acknowledgment functionality observed here impacts on the follow-up and communication of test results, as clinicians performed their test result management actions on the assumption that the EMR was processing their actions as anticipated by their understanding of the system. For example, WAD has highlighted the uncertainty and variation in use of the of the results forwarding functionality for managing results that return after a patient has been admitted and the potential impact on electronic results notifications or communication with inpatient teams. Uncertainty also compelled clinicians to avoid using the functionality, or find alternatives, which concurs with the findings of Bodley et al, who concluded, “If physicians are not taught how EHR functions can improve efficiency and patient safety, it seems logical that physicians would not use these functions....” Our findings therefore have implications for EMR training to ensure the scope of training not only covers how to use an application, but also addresses why functionality should be used, the benefits of use (including safety) and how the system processes information. Our findings add to existing literature in describing the challenges clinicians reported in using an electronic test results management application in an ED setting and take a step toward answering the call to “identify the problems” relating to HIT usability.

Workload and communication

Despite using an EMR with electronic results acknowledgment functionality, clinicians in our study reported utilizing additional documentation strategies to ensure visibility of their test result management actions (theme 3). This represents an additional workload undertaken by clinicians to perform dual acknowledgment in the inbox and patient’s notes, providing further evidence of the impact of electronic test results management activities and notifications on clinician’s workload.,, The need for visibility among the clinical team of actions taken after reviewing results represents a challenging area of great importance for the development of safe and effective HIT systems. Through exploring WAD, additional workload was also identified as a challenge faced by advanced nursing staff with ordering rights who practice autonomously in managing test results in the absence of an EMR inbox (theme 5). Their time-consuming workarounds included generating EMR patient lists, reviewing every one of their patients’ notes and acknowledging their own actions in the patient notes. Although this may superficially suggest a need for results to be sent to multiple members of the clinical team, the notion of more than 1 clinician being notified of test results has been studied in the literature and the use of dual alert systems (ie, results notifications sent to 2 providers) has been found to negatively impact the timely follow-up of test results. WAD in our study has highlighted that managing test results notifications and acknowledgment actions in team-based care environments is a complex challenge for HIT design and warrants further investigation for effective solutions to improve team collaboration and communication and thus maximize the safety of results follow-up.

Limitations

The findings from our study have identified several challenges in test results management and the strategies that clinicians use to optimize their management of test results. Although the findings of this study are limited to the Australian ED context and may not be generalizable to other contexts, they do highlight several universal risks in HIT-enabled environments and have implications for the design of HIT that deserve further study. Our findings are based on qualitative data from a sample of ED clinicians, and it was beyond the scope of the study to determine causal relationships between findings (eg, usability of the EHR and level of staff training). As a qualitative study, our findings are limited to the sample of clinicians who participated in the study and additional strategies and results may be identified in studies of larger samples. Owing to the time lapse between data collection for the current study (2016-2017) and the present, it is acknowledged that work processes or EMR functionality may have changed at the study sites and the barriers identified in the current study may have been addressed.

CONCLUSION

From a Safety II perspective of understanding everyday clinical work, the WAI-WAD lens applied in our study identified EMR-related strategies used by clinicians to ensure optimal test result management. Our study highlights the strength of this approach as it not only captured workarounds and resilience strategies, but also cast a wider net to capture HIT usability challenges and context-specific workflows. By investigating the variations in WAD, we have identified potential areas of risk through the identification of workarounds for results tracking and follow-up, which we highlight as a priority area for HIT development for context specific solutions to improve safety. We have discussed the translational potential of governance models (as opposed to HIT solutions) to reduce information overload (and thus improve safety) associated with inbox and pool notifications. Our results add to existing knowledge of HIT usability challenges with implications for HIT training. We have described the complex challenge for HIT design associated with test results management in team-based care environments. Further investigations of effective HIT solutions are warranted to improve team-based collaboration and communication of results follow-up with the aim of enhancing safety. Researching test results management in a previously understudied context (the ED) is a key strength of our study, and our results corroborate with existing knowledge from other clinical contexts and countries.

Analysis of WAD is thus a useful research tool to determine areas of risk that WAI did not anticipate. In this way, WAD analysis provides context specific evidence from the “pointy end” of a system to inform development of HIT solutions and their implementation, with the ultimate aim of improving the safety of diagnostic test result management.

FUNDING

This work was supported by a National Health and Medical Research Council, Partnership Project Grant number 1111925, involving the Australian Institute of Health Innovation at Macquarie University, NSW Health Pathology and the Australian Commission on Safety and Quality in Health Care. Authors JIW and AG are two of the Chief Investigators on the Partnership Project Grant.

AUTHOR CONTRIBUTIONS

AG, JIW, and MRD were involved in the conceptualization and design of the study. MRD and JL performed data collection, and JT, MRD, JL, PS, and JI were involved in the analysis and/or interpretation of the data. JT drafted the article, and AG, JIW, MRD, JL, PS, and JI critically reviewed the article for important intellectual content. All authors approved the article.

Click here for additional data file.