Automated recommendation for cervical cancer screening and surveillance.

Because of the complexity of cervical cancer prevention guidelines, clinicians often fail to follow best-practice recommendations. Moreover, existing clinical decision support (CDS) systems generally recommend a cervical cytology every three years for all female patients, which is inappropriate for patients with abnormal findings that require surveillance at shorter intervals. To address this problem, we developed a decision tree-based CDS system that integrates national guidelines to provide comprehensive guidance to clinicians. Validation was performed in several iterations by comparing recommendations generated by the system with those of clinicians for 333 patients. The CDS system extracted relevant patient information from the electronic health record and applied the guideline model with an overall accuracy of 87%. Providers without CDS assistance needed an average of 1 minute 39 seconds to decide on recommendations for management of abnormal findings. Overall, our work demonstrates the feasibility and potential utility of automated recommendation system for cervical cancer screening and surveillance.

Introduction

Because of the complexity of cervical cancer screening guidelines and need for a detailed chart review, clinicians may not always follow best-practice recommendations.1,2 Clinical decision support (CDS) systems offer a potential solution,3,4 and they have been reported to improve screening rates for preventive services.5–10 However, most systems are limited to providing screening rather than surveillance recommendations for management of abnormal findings. The latter patients are especially at risk for cervical cancer – 15% of women with cervical cancer were reported not to have appropriate follow-up of abnormal results11 and 27% were reported to have had delayed or no referral for colposcopy.12

Recently, the American Society for Colposcopy and Cervical Pathology (ASCCP)13 has published algorithms and a mobile app to facilitate application of surveillance guidelines. While these tools provide a reference for common consensus-based surveillance recommendations, clinicians have to consult additional sources for guidelines not available in the tools. Moreover, these tools are not integrated with the electronic health record (EHR), which interrupts the clinicians’ workflow. Consequently, the time and effort required to utilize these tools pose a significant barrier for their use.

Given the complexity of the guidelines and lack of a single resource for evidence-based recommendations for cervical cancer screening and surveillance, an automated system that analyzes EHR to provide comprehensive guidance would be clinically useful. Therefore, we developed a system for deployment at the point of care. Our preliminary research has been published previously.14,15 In this paper, we provide an overview of our research, and report on our efforts to model the updated guidelines, along with initiatives to facilitate workflow and usability of the developed system. Our report aims to provide insights to inform development of CDS systems at other institutions and for other decision problems. The unified guideline flowchart can be used to develop similar systems at other institutions.

Methods

This research was approved by the Mayo Clinic institutional review board. The IRB approved waiver of the requirement to obtain informed consent. The research was conducted in accordance with the principles of the Declaration of Helsinki.

Unified model of the guidelines

We reviewed guidelines from the American Cancer Society (ACS), US Preventive Services Task Force (USPSTF), American College of Obstetricians and Gynecologists (ACOG), and the ASCCP.16–20 While the ACS, USPSTF, and ACOG guidelines cover screening, ASCCP guidelines provide the recommendations for surveillance. We constructed a “unified guideline model” as a flow diagram (Fig. 1), which conglomerates the logic embedded in the relevant guidelines. The flow diagram consists of nodes and edges – the nodes represent concepts in patient information (eg, age) and the edges correspond to the possible values of the concepts (eg, age > 30 years). The flow diagram has an inverted tree structure, starting at a single node and ending in multiple leaf nodes. Multiple paths can be traced from the starting node to terminate in the leaf nodes, and each path corresponds to a unique patient scenario encountered in clinical practice. We modified the unified model during our study to incorporate updates to the national guidelines. The latest version of the model is attached in Supplementary File 1.

Figure 1

A portion of the unified guideline flowchart (attached as Supplementary File).

Automated data extraction from HER

The values defined in the flow diagram are extracted from the EHR. A total of 10 concepts are extracted from six different sources including demographic information (date of birth and gender), cervical pathology reports, problem list, laboratory results for high-risk human papillomavirus (HPV) DNA testing, annual patient questionnaire responses, and disposition of previous clinical alerts. In addition, the cervical pathology and clinical notes are in free-text form and are not amenable to computational processing; hence, natural language processing (NLP) is used to determine parameters reported in these notes.

To ensure accuracy of data extraction, reconciliation is performed across multiple sources.21 Figure 2 summarizes the information sources. For example, the history of hysterectomy that determines if a patient should have cervical, vaginal, or no cytology testing is determined by searching for hysterectomy in surgical reports, problem list, clinical notes, and annual survey responses. Similarly, history of cervical cancer is inferred from problem list, searching cervical cytology reports, and the patient survey information.

Figure 2

Workflow and data relationships. Patient interactions with the care providers are depicted on the right side, leading to creation of data, which need to be collectively analyzed for decision making.

Enhancing usability by delivering recommendation and rationale at the point of care

The institutional EHR interface was enhanced to instantly display the recommendation generated by the reminder system. As shown in Figure 3, the recommendation is displayed when the provider navigates to the pathology reports in the EHR. The explanation for the recommendation is also displayed to inform the provider. Figure 4 summarizes a sample of possible explanations. An icon in the reminder tab links to an institutional guideline repository, which also lists local experts for consultation or clarification. The button in the lower right corner opens a pop-up for provider feedback about the CDS recommendation.

Figure 3

The recommendations are displayed in the EHR when providers navigate to the pathology reports section. Along with the recommendation, a brief explanation is displayed, which elaborates on the rationale.

Figure 4

Sample explanations generated by the decision support system.

Validation

Validation of the “unified guideline flowchart” was performed in several iterations/cycles. In each of the iterations, the system was developed using a random set of patients, and then validated on a separate randomly chosen set of patients. The development and validation were performed by comparing recommendations computed by the CDS with the clinical plan of the development and validation patients, respectively. The system was validated on a total of 333 patients. The validations were followed by an error analysis to identify shortcomings in data extraction or in the flowchart to improve the system. Further, to validate the system in the clinical setting, we conducted a pilot study with nine primary care providers for a six-month period and gathered feedback. The validations were performed before we updated the unified model to include the 2013 ASCCP update.

Results and Discussion

The latest version of the “unified guideline” flow diagram covers 51 decision scenarios identified in the practice population. In contrast to the ASCCP flow diagrams that have a particular finding as a reference point, our flow diagram has the most recent cytology as the reference point, which helps to explicitly represent the guideline logic. The ASCCP diagrams aim to be intuitive to clinicians and assume implicit knowledge, which can be challenging for clinicians who are not familiar with the guidelines. It is especially difficult to convey the clinical guideline to nonclinicians. In contrast, our unified flow diagram assimilates and explicitly represents the knowledge that aids computer programmers to develop CDS software for clinicians. The flow diagram reveals the complexity of decision making and visually represents the challenge faced by primary care providers expected to apply the latest guidelines during a brief clinical encounter.4,22 Multiple variables and subgroups can overwhelm busy clinicians.23

Our unified guideline flowchart represents the clinical consensus achieved in 2013. As the “unified model” is able to provide unambiguous recommendations for nearly all decision scenarios seen in practice, we can infer that the guidelines are unambiguous, complimentary, and comprehensive.24 However, a small number of patients who are exceptions to the model remain. These include pregnant women with low-grade squamous intraepithelial lesions, women with atypical glandular cells (AGC) or atypical endometrial cells, and adenocarcinoma in situ (AIS). Pregnancy is difficult to identify from the EHR. Patients with AGC or atypical endometrial cells need endometrial and endocervical sampling in addition to colposcopy, and AIS may be identified during a diagnostic excisional procedure. These rare but important situations are excluded from the scope of the CDS with a generic reminder to refer to the colposcopy clinic.

The iterative validations of the recommendations of the CDS system, with the help of clinicians, revealed shortcomings in guideline flowchart as well as data extraction errors. The flowchart errors were mainly due to scenarios not envisioned by the experts when they reviewed the guideline narratives and generated the decision points. Overall, the system was found to have an accuracy of 87% in 333 validation cases.14,15 This was further improved after the system errors identified during the validation were rectified. The comparison of CDS and clinician recommendations also revealed that providers have difficulty applying guidelines when there are abnormal findings.24 We are currently performing validation of the modifications made to the unified guideline model due to 2013 ASCCP update.

The variables in the flowchart are extracted from multiple EHR screens and sections. An unaided provider needs to scroll through the EHR to view all relevant reports. We determined that for patients with abnormal findings, the unaided providers needed an average of 1 minute 39 seconds to gather information and decide on the recommendation.14 Therefore, the CDS system is expected to save substantial clinical time. Our study also shows that the variables required for the decision making can be collated and reconciled from disparate data sources – this is particularly helpful to identify patients with history of hysterectomy and risk factors for cervical cancer. Four of the 10 concepts required for guideline application are extracted from free-text notes. Hence, NLP25 and data reconciliation techniques21 have a critical role in data extraction. Given the complexity of the decision logic, contextual explanations are needed to justify recommendations generated (Fig. 4) and to direct the provider to additional information sources or institutional experts.

A major challenge in the development of the CDS system is the constantly changing nature of the guidelines. Considerable effort and expertise is required to decipher new guidelines and to identify the resultant changes to the unified model. This is followed by validation of the model by comparing the system with clinicians. Hence, there is lag from publication of new guidelines to update the system. The time lag for system update, and the frequency of guideline updates, renders it essential that the system’s update–validate cycle is a continuous and an iterative process.

Compared to the ASCCP mobile app, our system does not require user input of test results. If the recommendations are visible in the EHR, there is no need for providers to use a separate program. A CDS system that automatically pulls the patient data and generates a guideline-based recommendation for subsequent testing has greater potential to save time and improve implementation of recommended management strategies.

In contrast to our comprehensive guideline model, the decision logic for CDS at most institutions is limited. Specifically, decision support systems for cervical cancer prevention generally suggest a cervical cytology every three years for all female patients. Although this is appropriate for the majority of patients with normal cervical cytology, it is inappropriate for 30- to 65-year-old women being screened with cervical cytology/HPV co-testing. Recent updates suggest five-year interval test frequency for co-tests.20,26 The three-year interval is also inappropriate for patients with abnormal cytology or other risk factors.27,28 Decision making for the high-risk patients requires interpretation of free-text pathology and clinical reports, and existing CDS systems generally lack the capability to interpret free-text reports and cannot supply appropriate recommendations for high-risk patients.25,29 Paradoxically, providers require greater support in order to optimally manage high-risk patients.5,27 With the growing use of EHRs in the United States, the use of advanced decision support systems like ours has a high potential for improving the quality of care.30

Conclusions and Future Work

Our research indicates that the national cervical cancer screening and surveillance guidelines are unambiguous, complimentary, and comprehensive for nearly all patients seen in clinical practice. However, application of the national guidelines for cervical cancer prevention requires determination of multiple variables, and it requires considerable time and effort for an unaided clinician. Overall, our work demonstrates the feasibility of automated CDS system for generating clinical recommendations for cervical cancer screening and surveillance. By automatically extracting the variables from the EHR to provide guideline-based recommendations, the CDS system can potentially save considerable clinical time and improve quality of care. Our unified guideline flowchart can be used to develop similar systems at other institutions. Similar systems are in development for prevention of colorectal cancer21,31 and lung cancer.

Supplementary File 1. Flow chart of unified guideline model for cervical cancer screening and surveillance.

Supplementary File 2. Source code for the clinical decision support system for cervical cancer screening and surveillance.