The impact of health information technology on the management and follow-up of test results - a systematic review.

OBJECTIVE: To investigate the impact of health information technology (IT) systems on clinicians' work practices and patient engagement in the management and follow-up of test results.
MATERIALS AND METHODS: A search for studies reporting health IT systems and clinician test results management was conducted in the following databases: MEDLINE, EMBASE, CINAHL, Web of Science, ScienceDirect, ProQuest, and Scopus from January 1999 to June 2018. Test results follow-up was defined as provider follow-up of results for tests that were sent to the laboratory and radiology services for processing or analysis.
RESULTS: There are some findings from controlled studies showing that health IT can improve the proportion of tests followed-up (15 percentage point change) and increase physician awareness of test results that require action (24-28 percentage point change). Taken as whole, however, the evidence of the impact of health IT on test result management and follow-up is not strong. DISCUSSION: The development of safe and effective test results management IT systems should pivot on several axes. These axes include 1) patient-centerd engagement (involving shared, timely, and meaningful information); 2) diagnostic processes (that involve the integration of multiple people and different clinical settings across the health care spectrum); and 3) organizational communications (the myriad of multi- transactional processes requiring feedback, iteration, and confirmation) that contribute to the patient care process.
CONCLUSION: Existing evidence indicates that health IT in and of itself does not (and most likely cannot) provide a complete solution to issues related to test results management and follow-up.

INTRODUCTION

The results of laboratory tests and medical imaging reports directly impact clinical decision-making contributing to the diagnosis, treatment, prevention, and management of patient care. The World Alliance for Patient Safety has identified poor test follow-up as an international priority area for concern in patient care, and in 2017 the US Emergency Care Research Institute flagged inadequate test results follow-up as a key patient safety issue. Many clinicians, aware of the extent of poor test results management, have expressed concerns about systemic shortcomings in organizational follow-up procedures within and across health care settings.

Potential strategies to improve test results follow-up include the use of health information technology (IT) for the communication of test results using automated result notifications.,, The introduction of IT has been supplemented by initiatives to establish guidelines and recommendations for successful implementation, quality improvement, and evaluation. Attention has also been focused on the patient’s role as a partner in the process of enhancing the safety of care., This is particularly relevant to situations where the failure to inform patients of their results has been described as legally indefensible in malpractice claims. Electronic health records (EHRs) are seen as the basis for greater patient involvement, particularly as they provide the means by which patients can access their own information using a secure electronic patient portal, which, in addition to allowing access to personal information, also facilitates communication with health professionals.

There is ample evidence that while IT is capable of helping to prevent medical errors, it also has the capacity to introduce its own class of errors. This is particularly relevant to test results management, where the way that information is collected, reported, and presented can have major safety consequences., Despite a growing evidence base of the diffusion of health IT applications, their impact on test results follow-up, management, and patient engagement has not been widely appraised and is not well understood., This systematic review integrates quantitative and qualitative research findings on how health IT has been used to engage with patients. The systematic review thus provides an overview of the current state of evidence about how health IT has been used to address the test results management and follow-up process and contributes to a better understanding of the gaps and challenges as identified by existing research. The aims of this systematic review were to:

Describe the types of health IT systems that are utilized in the management and follow-up of test results,

Investigate the impact of health IT systems on the rate of missed test results and other outcomes,

Identify the impact of health IT systems on clinicians’ test results management work practices, and

Assess the impact of health IT on patient engagement and the follow-up of test results.

MATERIALS AND METHODS

Study identification

We conducted a search for health IT systems and clinician test results management in the following databases: MEDLINE, EMBASE, CINAHL, Web of Science, ScienceDirect, ProQuest, and Scopus for studies published between January 1999 and June 2018 in accordance with PRISMA guidelines. Our protocol was registered on the PROSPERO register of systematic reviews (CRD42016043148). Search strategies for all databases are presented in Appendix 1. We reviewed reference lists of all literature identified as potentially relevant. Table 1 provides a complete list of peer-reviewed and gray literature sources that were hand-searched.

Table 1.

List of hand-searched sources for peer-reviewed and gray literature

Gray literature/Hand-searching
Google
Google Scholar
Royal Australasian College of Physicians
Royal Australian College of General Practitioners
European Federation for Medical Informatics
International Medical Informatics Association
American Medical Informatics Association
World Health Organization
Article reference lists

Study selection

Two authors (JL, JT) independently reviewed titles and abstracts identified from the search. Papers without abstracts were retrieved and reviewed in full. The identification and selection process of studies is detailed in Figure 1. We resolved discrepancies through discussion or referral to a third researcher (AG). The same authors (JL, JT) retrieved and independently reviewed full text articles of all selected papers for inclusion in accordance with our eligibility criteria.

Figure 1.

Study identification and selection process.

Original studies of all types were included if they reported the impact of any health IT system on the test results follow-up process in hospital and ambulatory care settings. Failure to follow up on a test is defined as failing to take the appropriate next steps after the test. For the purposes of this review, missed test results and/or failure to follow up test results occurs when there is no evidence that the responsible provider becomes aware of a result (laboratory or radiology). We excluded point of care testing. Studies were also excluded if they did not report original research, if the reported outcomes were not directly ascribable to the follow-up of test results (eg, when patients access their entire electronic medical records [EMRs]), or studies which evaluated the accuracy/specificity of a health IT system (eg, algorithm). Studies which explored provider opinions regarding potential patient access to test results, patient preferences, or expectations for potential future electronic access to test results and results of user testing of patient result access applications were excluded. The study selection process is presented in Figure 1.

Data extraction and synthesis

Information regarding health IT, its impact on the rate of missed test results, its effects on clinician test results follow-up work processes, and the patient’s response to electronic access to their own results was extracted from included papers. Due to significant heterogeneity between studies, a meta-analysis of results was not performed. Descriptive statistics were used to summarize the body of evidence including the number of included studies, country of origin, year of publication, and study design. The details of individual studies including primary author, title, year, country, type of health IT, methodology, impact on missed test results follow-up, results type, study site, department/study population, and study size were extracted. Findings pertaining to each of the 4 objectives of the systematic review were extracted.

Two authors (JL, JT) assessed the quality of each included study using a tool applicable to the study design (Table 2). We used the Critical Appraisal Skills Programme (CASP) Qualitative Checklist to appraise qualitative studies. We assessed mixed-methods studies based on the methodological design of the study, or, for studies which included both qualitative and quantitative methods, according to the predominant method. For randomized controlled tests (RCTs) and quasi-experimental study designs, we applied the relevant “Study Quality Assessment Tools” developed by the US Department of Health and Human Services National Institutes of Health (NIH): National Heart, Lung, and Blood Institute. The selection of these tools allowed us to assess all RCTs, controlled trials, and quasi-experimental studies using a similar method with the same quality rating approach. For each study we recorded a quality assessment outcome of either poor, fair, or good.

Table 2.

Quality assessment tools

National Heart, Lung and Blood Institute (Quantitative studies) 25	Critical Appraisal Skills Programme (CASP) (Qualitative Studies) 24
Quality Assessment of Controlled Intervention Studies	CASP Qualitative Checklist
Quality Assessment of Observational Cohort and Cross-Sectional Studies
Quality Assessment of Before-After (Pre-Post) Studies with No Control Group

RESULTS

A total of 57 studies were included in the systematic review, 53 (93%) of which were published between 2006 and 2018. The earliest study was published in 1999 with an apparent rise in studies from 2006 onward (see Figure 2). Most studies (72%, n = 41) were conducted in the US. Figure 2 presents a detailed breakdown of studies by year and country. The studies incorporated a range of research methods which included 7 RCTs (12% of studies), 32 observational studies (56%), 12 mixed-methods studies (21%), and 6 qualitative studies (11%). The quality of the evidence presented in this review overall was rated as mostly fair (n = 35) and good (n = 20) with 2 studies rated as poor quality. A summary of included studies and their quality assessment is presented in Table S1.

Figure 2.

Breakdown of included articles by year and country.

Types of health IT systems utilized in the management and follow-up of test results

The literature reported a variety of health IT systems for the clinical management and follow-up of test results:

Electronic alerts (interruptive and non-interruptive) delivered to clinicians about results with abnormal/critical values,

Computerized provider order entry (CPOE) systems with electronic results viewing, and clinical information systems where results are viewed electronically, but orders are placed manually,

Electronic medical record (EMR)/electronic health record (EHR) systems,

Electronic results acknowledgment systems which require physicians to electronically document that they have seen a test result,

Electronic results tracking systems which allow users to track the progress of tests and status of results (eg, viewed, pending at discharge),,

EHR-based trigger algorithms which identify patients at risk of diagnostic delays, and

Electronic report generation systems for abnormal results.,

The categories were based on how each study reported their intervention. The health IT interventions varied according to whether they reported on the impact of an EMR/EHR system (eg, category 3), or as a dedicated feature of an EMR/EHR system (eg, category 1, 2, and 4). In part these differences reflect the development and increasing specialization of health IT systems over time.

Studies of health IT-facilitated patient engagement and follow-up of results featured two types of electronic systems: 1) patient portals and 2) personal health records (PHRs). Patient portals offer access to personal health information via a secure website, while integrated (tethered) PHRs are institutionally-managed and connected to a health care organization’s EHR system thus offering patients direct access to their medical records., No studies evaluated patient-managed PHRs (ie, freestanding or untethered and not connected to a health care organization). The majority (n = 13) of systems studied offered patients real-time access to test results as they became available. In the remaining 2 studies, patients were able to view results after a delay to allow prior review of results by clinicians.,

Impact of health IT systems on the rate of missed test results

A total of 18 studies reported the impact of electronic results management on the rate of missed test results. The majority of the studies in this section were rated as good,,,,,, and fair,,,,,,, quality. Of all the studies, 1 was rated as poor quality (Table S1).

Alerts

A cluster-RCT of an automated email notification system undertaken by Dalal et al. reported on survey results (152 from hospital physicians and 112 from primary care physicians [PCPs] in the community) which revealed that, compared to the control group, a significantly larger proportion (24–28 percentage point difference) of physicians who used the notification system were aware of actionable test results. A prospective cluster-RCT by El-Kareh used an automated email-based alert system which notified physicians of positive culture results not adequately treated at discharge. This study reported a 15% increase in the documented follow-up of positive postdischarge culture test results.

A cross-sectional study by Wahls et al. involving 106 PCPs found that, despite use of an EMR with a result-alerting function, 37% of primary care physicians reported seeing at least 1 patient with a missed test result. Another cross-sectional survey (of 143 PCP respondents) reported that 30% encountered at least 1 patient with a diagnosis/treatment delay due to a missed test result. The authors noted that the procedures for management of results were not uniform, with only 55% of respondents reporting use of the electronic notification system for results management.

Observational studies by Singh et al., and Bhise et al. reported a range of between 0.2% and 16.7% failure to follow up test results with the use of an integrated, comprehensive EHR with a test results notification system. A cluster-RCT of an email notification system by Dalal et al. reported no significant difference in the rate of documented evidence of follow-up action for test results initially pending at hospital discharge.

The relationship between acknowledgment of electronic alerts and subsequent follow-up action was investigated in 2 studies by Singh et al., Failure to act on abnormal results ranged from 6.4% to 7.3% overall and did not differ significantly between acknowledged and unacknowledged alerts. A before and after study of a mandatory EHR notification system was undertaken by Laxmisan et al. across 2 sites. Logistic regression uncovered a significant intervention effect (preintervention OR 0.7; 95% CI, 0.5–1.0) after accounting for site-specific differences in follow-up, with a lower likelihood of timely follow-up at 1 site (OR 0.4; 95% CI, 0.2–0.7).

Computerized provider order entry (CPOE) and clinical information systems

Studies of CPOE and clinical information systems investigated the impact of electronic transmission of results. These studies reported varying levels of follow-up or physician awareness of results. Provider failure to review or follow up results in the emergency department setting ranged from 1.5% of radiology and microbiology results to 45% of all emergency biochemistry tests. Patient transitions between care settings was also identified as a potential risk factor. It was reported in 1 study that inpatient and primary care physicians were unaware of 61.6% of results pending at hospital discharge. Clinicians in this study deemed 37.1% of missed results as actionable with 12.6% requiring urgent action.

Result acknowledgment systems

An Australian study investigated the impact of an electronic results acknowledgment system which incorporated escalation procedures (based on delineated levels of test follow-up responsibility) for dealing with unacknowledged results in an Australian hospital. The system led to the clinical acknowledgment of all results.

Impact of health IT interventions on clinicians’ test results management work practices

The key findings from studies of the impact of health IT on test results management work practices spanned the following themes: 1) changes in workload, 2) hybrid paper/electronic systems, 3) effect of the organizational context, 4) time to test results follow-up, and 5) implications for patient outcomes.

The quality of studies in this category was rated as either good (n = 11),,,,,,,,,, or fair quality (n = 13). ,,,,,,,,,

Changes in workload

Respondents in a qualitative study conducted in the US noted the additional time burden for clinicians associated with acknowledging clinically irrelevant alerts from EHR-based test results systems. These findings were echoed by a web-based survey of 2590 PCPs, 85.6% of whom reported that they were required to work after hours or over the weekend to address test results notifications.

Hybrid paper/electronic systems

The use of multiple systems within a mixed-media (paper and electronic) environment for managing test results was reported to have negative impacts on test results follow-up work practices by 2 studies., Menon et al. found that 43% of 2554 PCPs surveyed used paper or a combination of paper and computer-based workarounds to support results management. A mixed-method study by Elder across 4 sites concluded that IT alone was insufficient to achieve the highest levels of safety when no site performed better against test results management measures despite varying levels of health IT adoption (partial CPOE to full EHR).

Effect of the organizational context

Li et al. undertook a qualitative study that investigated clinician perspectives of the utilization of an electronic results acknowledgment system on radiology and microbiology results follow-up. Their results showed that contextual factors, such as how the health IT system aligns with existing work practices, and the departmental staff mix can affect the success or otherwise of the new system. Similarly, Menon et al. undertook a mixed-method study of a view alert system for abnormal results and concluded that context-related vulnerabilities (eg, existing test results follow-up policies and escalation procedures) could lead to missed test results in EHR-based settings and advised that interventions should recognize the influence of organizational factors on outcomes of health IT.

Time to test results follow-up

The effects of a real-time, automated paging system for critical laboratory values on internalists’ response times was investigated in 1 RCT. The study found no significant difference in median response time to alerts between the control and intervention groups (39.5 mins vs 16 mins, respectively, p = 0.33). Park et al. reported on a study which examined the effects of sending SMS messages to doctors in wards, in addition to the hospital laboratory ringing doctors with critical results. They found a significant decrease in time to the ordering of treatment in the general wards (249 mins pre to 63 mins post SMS (p < .001) but this combined intervention had no significant effect in changing test results follow-up times in the ICU. Lin et al.’s before and after study in an outpatient department found that episodes of hyperkalemia were more likely to be followed up within 4 days following the introduction of a system which flagged abnormal results and tracked the status of reports (90.0% post vs 62.2% pre; p = 0.003).

Implications for patient outcomes

Studies measuring the impact on patient outcomes reported mostly positive results following the introduction of health IT systems. Benefits included reductions in time to diagnostic evaluation or completion of follow-up action from availability of results,,,, time to receipt of follow-up care for patients requiring referral to other practitioners,, time to diagnostic resolution,, and likelihood of diagnostic resolution.

Impact of health IT systems on patient engagement in the follow-up of test results

Studies that investigated patient engagement tools used 1 or more of the following methods: qualitative interviews,,,,, surveys/questionnaires,,,, or observational data.,,, Mixed-methods were employed in 6 studies.,,,,, While 2 studies were rated as good quality,, 12 were of fair quality,,,,,, and 1 was of poor quality.

Patient utilization of patient portals

Ling et al.’s patient survey involving 429 patients with access to results from a sexually transmitted infection clinic showed that 75% of respondents who accessed results online did so primarily because they could check results at any time of the day. Woywodt et al. reported that from a sample of 295 renal patient portal users (predominantly made up of transplant patients), 42% accessed their results after their clinic appointments and 78% accessed the portal on an average of 1–5 times per month. Most respondents (93%) felt that the portal assisted them in the management of their condition.

Key considerations related to patient access to results

A survey by Christensen examined patient experiences with the use of a tethered PHR. They found that patients associated electronic access to laboratory results with positive feelings including satisfaction and relief and typically engaged in discussions with family and friends about their results following access. Wiljer et al. investigated the clinical, technical, and educational support needs of breast cancer patients with portal access to laboratory and radiology reports. The authors reported that 98% (122/150) of user support requirements were technical in nature (eg, difficulties accessing results). Cimino et al.’s mixed-method study reported that all 4 interviewed patients who reviewed and tracked their laboratory test results believed that it improved communication during physician visits and promoted greater ownership of their care.

Clinician experiences following patient access to results were also positive. A survey exploring the experiences of 508 patients and 48 physicians following direct release of radiology reports to patients found that an equal proportion of patient and physician groups (88%) viewed the ability of patients to access radiology reports as important and useful (although almost half of these patients [49%] received communication from physicians about the result prior to report release). Only 8% of physicians ceased releasing reports online due to confusion and anxiety among patients.

Abnormal or critical test results

Giardina et al.’s qualitative study on the impact of patient access to abnormal test results reported that most patients unequivocally favored access to results electronically. Yet, some respondents mentioned that results of high emotional impact or “sensitivity,” such as those involving life-threatening illnesses, cancer diagnoses, genetic testing, and incurable conditions warranted verbal communication prior to electronic release. Winget et al. surveyed the experiences of 82 oncologists following direct release of results which were potentially indicative of disease progression in cancer patients. Half (49%) of the oncologists reported that sharing online results had negative impacts on their communication with patients. Oncologists generally believed that sensitive information requiring counseling should be delivered in a face-to-face consultation.

DISCUSSION

This systematic review identifies evidence across 2 decades incorporating: 1) multiple research methods (eg, qualitative and quantitative), 2) a range of health IT systems and software applications, 3) investigations into clinical work practices, and 4) assessments of the impact on patient engagement. By doing so the review contributes to a more structured picture of how the broader socio-technical system (technology, clinicians, patients, processes, and organization) impacts on the issue of test results follow-up. The research findings from RCTs provides some indication that health IT systems can increase the proportion of documented follow-up (15 percentage point change) and improve physician awareness of test results that required action (24–28 percentage point improvement). However, taken as a whole, the body of evidence on the impact of health IT on the management and follow-up of test results is not strong. The implication of these findings is that health IT, in and of itself, does not (and most likely cannot) provide a single or complete solution to issues surrounding the inadequate follow-up of test results.

The issue of test results management and follow-up is multi-layered and interwoven. This interconnectedness is conceptualized in Figure 3, which is constructed from the key outcome measures identified within the existing research, as a basis for examining the significance and implications of the research. These layers can be described as: 1) the organizational-communication environment (eg, existing practices about how test results are communicated),, 2) the diagnostic process (eg, the numerous tasks among different people and across different clinical settings that need to be coordinated and synchronised for safe and effective test results management),, and 3) patient engagement in the test results follow-up process (eg, how and when do patients access test results, if at all?).,,

Figure 3.

Key conceptual domains identified from existing evidence with examples of study outcome measures.

The organizational-communication environment

The communication of test results is reflective of existing patterns of accountability, responsibility, and authority that are shaped by clinical governance processes and the contextual characteristics present within different health care settings.,, These communication patterns are not direct one-way processes but multi-transactional processes requiring feedback, iteration, and confirmation., Communication processes are a critical component of the makeup and function of an organization linking people across space and time. The potential disruptive impact of health IT cannot really be appreciated without attention to its ability to change the role that communication plays in linking people and activities across space and time.,,,,

The diagnostic process

The diagnostic process is not a linear set of tasks but rather a series of tasks that involve multiple people (and different clinical settings) across the health care spectrum. This is evidenced by the array of outcome indicators reported by the existing literature to explore this issue.,,,, Health IT systems offer different ways to manage the test results follow-up process, including facilitating the access to and sharing of information, test tracking, and the provision of prompts/alerts., The evidence of the impact of health IT on test results management work practices draws attention to the importance of 1) enhancing the alignment of health IT with the diagnostic process and 2) accounting for the numerous tasks among different professionals and across different clinical settings that need to be coordinated and synchronized for safe and effective test results management., For instance, the evidence in this review indicates that systems which focus on ensuring physician review of results (via CPOE, alerts, results acknowledgment, and tracking systems) is, in some cases, insufficient in ensuring completion of subsequent steps of the follow-up process (eg, actioning a result).,

The failure to properly integrate different electronic systems within and across health care settings and the continued existence of hybrid paper/electronic systems has emerged as a risk to patient safety. Health IT systems that are not maximized to work effectively and efficiently have the capacity to hinder correct diagnosis by contributing to health professionals’ work burdens, resulting in less time to communicate with patients and other health care professionals. For instance, a high volume of alerts (not all of which may be clinically relevant) increases the possibility of clinical cognitive overload which can threaten the value of alerts.

Patient engagement in the test results follow-up process

Many of the studies in this review highlighted the connection between test results follow-up, IT, and patient engagement. This is because attempts to engage patients in the care process invariably involve shared information (enabling patients to read, comment on, and share in decisions about their care) and timely and meaningful communication (enabling consumers to receive, send, and comprehend the information required).

The role of patient-centerd IT systems was investigated across a range (qualitative and quantitative) of studies revealing that patient access to, and ability to check, laboratory results (in real time) is a major reason for patient utilization of portals., The most important reported benefits included improved communication with physicians and the management of the patient condition., There were no findings that directly reported on issues of privacy and security even though the literature often cites these as key sources of concern. In situations involving life-threatening illnesses, cancer diagnoses, and incurable conditions, the evidence highlighted the preference for face-to-face consultations and the initial involvement of the responsible physician., Patient-managed PHRs provide a set of computer-based tools owned and administered by patients with access to personal clinical information., No studies in our review evaluated such a system; yet it is such technologies, organized around a person’s own preferences, that may be emblematic of what is meant by IT-enabled patient-centered health care.,

Limitations

The aim of this systematic review was to examine the impact of health IT on test results follow-up across a number of dimensions (including clinical workflows and patient engagement) incorporating studies that used different methods (quantitative and qualitative). The results yielded numerous outcome measures that ranged from the rate of test results missed, to patient satisfaction rates and the number of workarounds. Although the heterogeneity of the studies makes it hard to provide 1 definitive result about the effect of health IT, the findings from this systematic review do nevertheless identify several factors that collectively contribute to the delivery of safe and effective test results management.

The publication (or non-publication) of research based on the direction (positive or negative) of the results can affect the validity of review conclusions. The scope and variability of findings from this systematic review was accentuated by our incorporation of 1) an array of search engines and databases including of gray literature sources and 2) extensive use of hand searches of relevant research literature.

CONCLUSION

Effective results follow-up is a fundamental part of the diagnostic process, essential to the delivery of quality patient care. Alongside results of laboratory and medical imaging tests, the diagnostic process involves the integration of information (eg, clinical history, physical examination, and consultation) that forms the key to diagnosis and a treatment plan. This process involves multiple tasks often incorporating different medical personnel and usually spread out over time. The central message of this systematic review is that the construction of safe and effective test results management IT systems should pivot on several axes including 1) patient-centerd engagement (involving shared information and timely and meaningful communication), 2) diagnostic processes (that involve the integration of multiple people and different clinical settings across the health care spectrum), and 3) organizational communications and the myriad of multi-transactional processes requiring feedback, iteration, and confirmation that contribute to the patient care process.

FUNDING

This study was supported by an Australian National Health and Medical Research Partnership Grant APP1111925.

AUTHOR CONTRIBUTIONS

AG conceptualized the study and design. AG, JL, and JT were involved with the acquisition, analysis, and interpretation of data. JL and AG drafted the manuscript. All authors made critical revisions to the manuscript for important intellectual content.

Conflict of Interest Statement

None declared.

Click here for additional data file.

Table S1.

Summary of included studies

Database	Search Strategy (4th wk Jun 2018)
MEDLINE (including Epub & In-Process) EMBASE	exp Medical Records Systems, Computerized/ exp “Continuity of Patient Care”/ Communication/ Diagnostic Tests, Routine/ exp Medical Errors/ Clinical Laboratory Techniques/ failure.mp. miss*.mp. lack*.mp. management.mp. exp Patient Satisfaction/ 2 or 3 or 4 or 5 or 6 or 11 view*.mp. access*.mp. 7 or 8 or 9 or 10 or 13 or 14 test*.mp. laboratory.mp. radiolog*.mp. 16 or 17 or 18 1 and 12 and 15 and 19 limit 20 to (english language and humans and yr=“1999 -Current”)
CINAHL	(MH “Patient Record Systems”) OR (MH “Computerized Patient Record”) (MH “Continuity of Patient Care+”) (MH “Communication”) (MH “Diagnostic Tests, Routine”) (MH “Treatment Errors+”) (MH “Patient Satisfaction”) “failure” “miss*” ““follow* up”“ “lack” “management” “view*” “access*” “test*” “laboratory” “radiolog*” (S2 OR S3 OR S4 OR S5 OR S6) S7 OR S8 OR S9 OR S10 OR S11 OR S12 OR S13 S14 OR S15 OR S16 S1 AND S17 AND S18 AND S19
	Limiters—Published Date: 19990101-20180618; English Language; Human Search modes—Boolean/Phrase
Web of science	(TI=(electronic OR computer* OR online)) ANDLANGUAGE: (English) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC Timespan=1999-2018 (TI=(failure OR miss* OR “follow* up” OR lack* OR management OR view* OR access*)) ANDLANGUAGE: (English) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC Timespan=1999-2018 (TI=(test* OR laboratory OR radiolog*)) ANDLANGUAGE: (English) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC Timespan=1999-2018 #3 AND #2 AND #1 Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC Timespan=1999-2018 #3 AND #2 AND #1 Refined by: [excluding] DOCUMENT TYPES: ( LETTER OR BOOK REVIEW ) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC Timespan=1999-2018
ScienceDirect	(pub-date > 1998 and ttl (electronic OR computer* OR online)) AND (pub-date > 1998 and ttl (failure OR miss* OR “follow* up” OR lack* OR management OR view* OR access*)) AND (pub-date > 1998 and ttl (test* OR laboratory OR radiolog* OR result*))[All Sources(Biochemistry, Genetics and Molecular Biology, Computer Science, Immunology and Microbiology, Medicine and Dentistry, Neuroscience, Nursing and Health Professions, Pharmacology, Toxicology and Pharmaceutical Science, Psychology)]
ProQuest	(ti(electronic OR computer* OR online) AND ti(failure OR miss* OR “follow* up” OR lack* OR management OR view* OR access*) AND ti(test* OR laboratory OR radiolog* OR result*) AND la.exact(“English”) AND pd(>19990101)) NOT stype.exact(“Trade Journals” OR “Wire Feeds” OR “Newspapers” OR “Magazines”) AND pd(>19990101)
Scopus	TITLE ( ( electronic OR computer* OR online ) AND ( failure OR miss* OR “follow*up” OR lack* OR management OR view* OR access* ) AND ( test* OR laboratory OR radiology* OR result* ) ) AND PUBYEAR > 1999 AND LANGUAGE ( english ) AND ( LIMIT-TO ( SUBJAREA , “MEDI” ) OR LIMIT-TO ( SUBJAREA , “COMP” ) OR LIMIT-TO ( SUBJAREA , “HEAL” ) OR LIMIT-TO ( SUBJAREA , “NURS” ) OR LIMIT-TO ( SUBJAREA , “BIOC” ) OR LIMIT-TO ( SUBJAREA , “PSYC” ) OR LIMIT-TO ( SUBJAREA , “NEUR” ) OR LIMIT-TO ( SUBJAREA , “IMMU” ) OR LIMIT-TO ( SUBJAREA , “PHAR” ) ) AND ( EXCLUDE ( DOCTYPE , “le” ) ) AND ( EXCLUDE ( LANGUAGE , “Italian” ) OR EXCLUDE ( LANGUAGE , “French” ) OR EXCLUDE ( LANGUAGE , “German” ) OR EXCLUDE ( LANGUAGE , “Turkish” ) )