Structured clinical documentation in the electronic medical record to improve quality and to support practice-based research in epilepsy.

OBJECTIVE: Using the electronic medical record (EMR) to capture structured clinical data at the point of care would be a practical way to support quality improvement and practice-based research in epilepsy.
METHODS: We describe our stepwise process for building structured clinical documentation support tools in the EMR that define best practices in epilepsy, and we describe how we incorporated these toolkits into our clinical workflow.
RESULTS: These tools write notes and capture hundreds of fields of data including several score tests: Generalized Anxiety Disorder-7 items, Neurological Disorders Depression Inventory for Epilepsy, Epworth Sleepiness Scale, Quality of Life in Epilepsy-10 items, Montreal Cognitive Assessment/Short Test of Mental Status, and Medical Research Council Prognostic Index. The tools summarize brain imaging, blood laboratory, and electroencephalography results, and document neuromodulation treatments. The tools provide Best Practices Advisories and other clinical decision support when appropriate. The tools prompt enrollment in a DNA biobanking study. We have thus far enrolled 231 patients for initial visits and are starting our first annual follow-up visits and provide a brief description of our cohort. SIGNIFICANCE: We are sharing these EMR tools and captured data with other epilepsy clinics as part of a Neurology Practice Based Research Network, and are using the tools to conduct pragmatic trials using subgroup-based adaptive designs.

The EMR can capture structured clinical data and provide clinical decision support to promote quality improvement and practice‐based research in epilepsy

We have built into our EMR structured clinical documentation support (SCDS) tools that conform to best practices in epilepsy

SCDS tools write notes and capture hundreds of fields of data including several score tests; brain imaging, blood laboratory, and electroencephalography results; and treatment outcomes

We are sharing our EMR tools and data with other sites, and using the tools to conduct pragmatic trials using the EMR and subgroup‐based adaptive designs

Documentation is an indispensable part of medical practice. This is especially important in neurology because many complex disorders change over time. Serial documentation of the clinical course guides treatment and safety measures. In the subspecialty of epilepsy, it is immensely useful to have details of seizure frequency, seizure severity, brain imaging, electroencephalography (EEG) changes, response to antiseizure medications, drug adverse effects, medication changes, as well as other treatments (e.g., surgeries, devices) routinely documented and summarized.

Epilepsy affects 4% of the population over the lifetime.1 It can have significant negative effects on quality of life.2 Although the American Academy of Neurology (AAN) has proposed quality measures for the management of epilepsy, it is not clear if these are routinely implemented or how often they are being followed. It is also unclear whether compliance to these or other quality measures improves epilepsy outcomes.3

The mainstay of epilepsy treatment is medications. There are multiple antiepileptic medications available to prescribe, but there is little evidence to guide the selection of one agent rather than another in an individual patient, excepting when there are clear contraindications. Comparative effectiveness studies are available, but they are either observational (limited by possible selection biases) or structured literature reviews (indirectly comparing randomized placebo‐control trials in highly selected patients).4, 5, 6, 7, 8, 9 There have been no point of care trials with head to head comparisons of available treatments including not only seizure control, but also cognitive function, mood, and quality of life.

Information documented in the electronic medical record (EMR) for patients with epilepsy is typically unstructured and not discretized, and hence cannot be used efficiently to implement quality measures or to support research. To this end, the NorthShore University HealthSystem (NorthShore) Department of Neurology built within its commercial EMR system (Epic) structured clinical documentation support (SCDS) tools that navigate care according to best practices, generate notes at initial and annual follow‐up visits with simple mouse clicks through electronic forms, and capture several hundred fields of epilepsy data per office visit.10 The clinical decision support (CDS) features of our epilepsy toolkit ensure that care is safely within the parameters of AAN quality guidelines, and support enrollment into clinical research studies (e.g., DNA biobank, pragmatic trials using subgroup‐based adaptive designs). The tools and the data captured are being shared with other neurology practices using the same EMR system, creating a Neurology Practice Based Research Network (NPBRN). The purpose of this article is to describe our epilepsy toolkit and its current uses.

Methods

Figure 1 illustrates our stepwise process of quality improvement and practice‐based research using the EMR, which is also described in detail below.

Figure 1

Structured clinical document support toolkit. Quality improvement and practice‐based research in epilepsy using the electronic medical record (EMR) consists of a stepwise progression from the development and implementation of a structured clinical documentation support (SCDS) toolkit (including note writing and electronic data capture); to enrollment reports; to data quality reports (and data cleaning); to descriptive reports of cohort characteristics; to quality improvement projects (including the creation of benchmark data and quality improvement dashboards); to the use of clinical decision support tools (to hardwire patient safety and improved outcomes); to other research (e.g., biobanking of DNA and the association of genotypes with longitudinal outcomes, pragmatic trials using subgroup adaptive design). The EMR provides a framework for measuring and impacting the three dimensions of quality improvement: structure, process, and outcomes.

Content building

Neurologists in the epilepsy program at NorthShore met every 2 weeks for a period of 3 months to standardize epilepsy office visit types (initial visits, annual visits) according to best practices. We developed consensus on definitions of epilepsy and related disorders, the outcomes of interest to clinicians and patients, valid and feasible outcome measures for point of care assessments, and factors known to influence the outcomes and measures. This was done in conjunction with the development of similar toolkits in other fields of neurology clinical practice.10 We consulted the AAN epilepsy quality measures, the National Institute of Neurological Disorders and Stroke (NINDS) common data elements pertaining to epilepsy, and the International League Against Epilepsy (ILAE) guidelines and classifications.3, 11, 12, 13, 14, 15 We included measures of anxiety (Generalized Anxiety Disorder‐7, GAD‐7),16 depression (Neurological Disorders Depression Inventory for Epilepsy, NDDIE),17 sleepiness (Epworth Sleepiness Scale, ESS),18 cognition (Montreal Cognitive Assessment, MoCA initially; and then the Short Test of Mental Status, STMS),19, 20 and quality of life (Quality of Life in Epilepsy–10 items; QOLIE‐10P).21 We included discrete descriptions for the different auras, ictal states as well as postictal states,22 medication history, surgical history, as well as device history. We discretized the EEG, magnetic resonance imaging (MRI), functional MRI (fMRI), positron emission tomography (PET), magnetic encephalography (MEG), and neuropsychological data, as well as laboratory results including antiseizure medication levels and cerebrospinal fluid (CSF) findings. Many of the fields and variables and forms that we employed were adapted from the NINDS Common Data Elements pertaining to epilepsy.12 We also incorporated the Medical Research Council (MRC) Prognostic Index as a guideline for continuing or discontinuing antiseizure medication.23 For the formulation of the impression, we included the 1989 and 2010 ILAE epilepsy classifications as well as new‐onset seizure and psychogenic nonepileptic seizure definitions.13, 14 For the formulation of the plan, we included statements regarding seizure precautions, drug adverse events, drug interactions, and drug teratogenicity in accordance with the AAN quality measures.3 We designed a workflow that included a medical assistant, nurse, neurologist, and research assistant (when applicable) to accomplish the above activities without extending the face‐to‐face time with the neurologist (typically 60 min at initial and annual follow‐up visits).

EMR building

After we had decided on the content and the workflow to accomplish our best practices, we met with programmer analysts on the NorthShore University HealthSystem EMR Optimization team. They built the SCDS and CDS tools required for clinical standardization, note writing, data capture, Best Practice Advisories (BPAs), and research enrollment. More specifically, the toolkits included custom navigators (effectively, an index of electronic forms), electronic forms (documentation flow sheets, including cascading data elements and autoscoring and auto‐interpreting and other “smart form” features), BPAs (pop‐up boxes), and order sets. We also included optional free text and narrative fields. The epilepsy team met every 2 weeks with the EMR Optimization team for 3 months to develop and test the tools. Although the toolkits were designed primarily for initial and annual follow‐up visits, they also provided the capability to support interval visit types (typically briefer, 30 min face‐to‐face time). Screenshots of these toolkits are provided in Figure 2 and Data S1.

Figure 2

Screenshots of the SDCD toolkits within the EMR, ©2015 EPIC Systems, used with permission. (A) Screenshot of spells/seizure data collection, (B) screenshot of diagnostic imaging data collection, (C) screenshot of EEG data collection, (D) screenshot of MRC Prognostic Index data collection. Additional screenshots are available in Data S1.

Implementation into clinical workflow

After the epilepsy toolkit was built, the neurologists tested them in the EMR development environment. Revisions were made as needed and the toolkit was moved to the EMR production environment to be used during patient visits. The epilepsy team continued to meet every 2 weeks to discuss patient encounters and to make additional changes to the toolkit. We also worked with programmer analysts of the Enterprise Data Warehouse (EDW) team to extract, transform, and load data from the EMR to epilepsy‐specific data marts, and to develop monthly enrollment and follow‐up reports (for patients enrolled in a DNA biobank) and monthly data quality reports (for required fields). The content building, EMR building, and implementation phases of the project took approximately 9 months total.

Specifically regarding our clinical workflow, when patients check in for their office visit for the first time with an epilepsy indication, or annually thereafter in follow‐up, patient‐support agents distribute the patient‐reported outcome measures as paper forms for the patients to complete while in the waiting room. We considered pushing these forms out electronically prior to the office visits using the EMR's patient portal, but less than half of the patients had active accounts, and the wait time for appointments in our department is brief (~1 week), allowing little time for previsit tasks. We considered using tablet or desktop computing devices to directly enter information into the EMR in the waiting areas, but decided against this as patients may have limited cognitive, motor, or sensory abilities, or may be unfamiliar with such electronic devices. Once the paper forms are completed, medical assistants review and enter the patient reported outcomes into an electronic version of the forms. Nurses then perform additional clinical assessments and enter information directly into the electronic forms to which they are assigned, reviewing also the work of the medical assistants who preceded them in the workflow. The neurologists then perform additional clinical assessments, entering information directly into the electronic forms to which they are assigned, reviewing also the work of the medical assistants and the nurses who preceded them in the workflow. We have attached to each initial and annual follow‐up visit, a 30 min pre–neurologist resource visit, allowing the medical assistants and the nurses sufficient time to complete their portion of the customized care navigators. The neurologist is scheduled a 60 min visit to complete their portion of the customized care navigator. We also attach to each initial visit up to 30 additional minutes for care coordination and possible research enrollment into a biobank. We inform patients to plan to spend up to 120 min total for the initial visit, and up to 90 min for the annual follow‐up visits. Note that for interval visits (between initial or annual visits), physicians may elect to use any or all of the electronic forms provided by the custom navigators, allowing flexibility in the use of the tools for such visits (scheduled for only 30 min). By requiring all electronic forms to be completed at initial and annual visits, we ensure clinical standardization according to best practices.

Reporting—enrollment, quality, descriptive, analytical

After SCDS toolkit implementation, epileptologists met every 2 weeks with programmers specialized in extracting, transforming, and loading data from the EMR's relational data repository to project‐specific data marts in NorthShore's EDW. The data marts provide an interface for analytic tools. Up to 1,000 fields of data were captured per office visit. EDW programmer analysts created monthly reports to track research enrollment in our DNA biobanking initiative (231 as of May 31, 2016). The enrollment reports are generated monthly. Reports indicate the number of participants by ethnicity and race or by month and year; the number of initial or follow‐up visits per year; a listing of enrollees (including dates of consent and initial and follow‐up visits, and annotations regarding death, withdrawal from the study, invalid consent, screen failure, or pending blood draw); and a summary of longitudinal follow‐up (including numbers of patients actively followed, past due, pending due, or not due; and follow‐up rates).

EDW programmer analysts created monthly data quality reports that indicate which of the required data were missing for each office visit. These quality reports are distributed to the care team to input missing values. Data not cleaned within 3 months are archived as permanently missing, and those data are not listed on subsequent reports. The care team learned where they were error prone from the data quality reports, and they remediated their use of the toolkits. When systematic errors occurred for many providers, the team had the opportunity to improve their use of the toolkits or to request optimizations or a change in data requirements.

Once our epilepsy toolkit was under regular use, and after cleaned data were captured for at least 100 patients, our statistician developed quarterly descriptive reports of cohort characteristics (providing medians, ranges, and frequencies) to identify possible gaps in care.

We also developed quarterly score test analytic reports (pairwise correlations, principal component analyses) to explore the variation between the measures at initial visits, and to reduce the complexity of the toolkits when able (e.g., by removing highly correlated score tests).

Quality improvement

We also developed quarterly quality improvement dashboards: interactive pivot tables that for defined quality initiatives allowed us to measure trends and stratify performance on factors such as the provider or the practice site, or the age or gender of the patient. For epilepsy we are tracking quality improvement processes such as the following: (1) prescription of antidepressants or anxiolytics or referral to psychiatry or psychology or social services for depression and anxiety; (2) referral for neuropsychological testing in patients with cognitive impairment; (3) vitamin D supplementation for patients on antiepileptic medication to prevent osteoporosis/osteopenia; (4) folic acid supplementation in women of childbearing age and on antiepileptic medication to prevent teratogenesis.

Our multidisciplinary team (epileptologists, EMR programmers, EDW programmers, statistician, practice manager, research personnel) continues to meet every 2 months to improve all structures and processes (EMR tools, clinical workflows, data management, reports), to monitor patient outcomes, and to engage in quality improvement and practice‐based research.

Results

We created SCDS toolkits within the EMR—the first step in our process for quality improvement and practice‐based research in epilepsy (Fig. 1).10 Some examples of the custom navigator and the electronic forms created are shown in Figure 2, and additional screenshots are provided in Data S1.

A complete descriptive cohort report (for 231 patients enrolled in our DNA biobank as of May 31, 2016) is provided in Data S2. These reports also show where there was no information collected/missing for each.

We also performed pairwise correlations (unadjusted and adjusted) of the scored tests at the initial visits of these 231 patients. A full analytical report for this cohort is provided in Data S3. We excluded from the analyses the MRC Prognostic Index, at it is available for only a subset of patients (seizure‐free at 2 years) and as our sample size is limited for that stratum to date. The analyses included data for ESS, GAD‐7, MoCA, NDDIE‐E, and QOLIE‐10P scores. More recently we switched from using the MoCA to the STMS to evaluate cognition (shorter test, fewer false‐positive results, and no copyright restrictions for electronic use). We used published nomograms to convert MoCA scores to MMSE scores, and unpublished nomograms (courtesy of the Mayo Clinic, Dr. Bradley Boeve) to convert STMS scores to MMSE scores.

Discussion

SCDS and CDS tools built into the EMR can be used to standardize epilepsy office visits. This is expected to improve the quality of care by reinforcing adherence to best practices, by improving note writing, by hardwiring patient safety, and by providing descriptive and analytic reports and dashboards. Our initial reports indicate that the score test measures that we are employing provide varying types of information, and our choice of measures is justified (efficiency). Our initial reports also indicate that there are many potential gaps in the quality of epilepsy care, including with respect to the following: (1) detecting and managing anxiety, depression, sleep disorders, and psychosocial stress, which are common in epilepsy patients and impact quality of life; (2) otherwise detecting and managing cognitive impairment; (3) detecting and managing osteoporosis and osteopenia; and (4) preventing teratogenesis. We are building additional BPAs to improve processes and hopefully patient outcomes.

The epilepsy toolkit has the potential to also support clinical research enrollment. For example, for patients with a diagnosis of epilepsy who are 18 or older and who live in Cook or Lake County, Illinois, a BPA pops up, indicating that the patient is eligible for enrollment in a DNA biobanking study.10 If the neurologist selects “enroll,” an electronic message is sent to the research assistant assigned to the practice site alerting them to consent the patient and draw blood at the end of the office visit. The BPA also allows the research assistant to document the consenting process and biobanking procedures. If the neurologist selects “do not enroll,” a drop‐down menu asks that the neurologist document a reason. All DNAs are genotyped for up to 1 million single nucleotide polymorphism markers, and we are associating the genotypes with the clinical data captured by the EMR (“clinomics”), toward the development of pharmacogenomic and molecular prognostic tests (thus far lacking in epilepsy).

We are also using the toolkit to implement pragmatic trials in epilepsy using subgroup‐based adaptive (SUBA) designs.1, 24 We are comparing head‐to‐head, three commonly prescribed first‐line antiepileptic medications, to see which are the most effective treatments for which subgroups of patients (precision medicine).10 Our EMR tools initially randomize the first 100 eligible subjects to any of the three treatments. Our EMR tools then adaptively assign subsequent eligible subjects to the most effective treatment, factoring the independent and dependent variables captured for the individual subject by the SCDS tools, and the response of similar subjects to the compared treatments previously.

We are also sharing our epilepsy toolkit, free of charge, with other neurology departments that use the same EMR platform (Epic), and that agree to share the data captured, de‐identified, in a data repository, thereby creating an NPBRN. The NPBRN currently includes 11 academic departments of neurology, and additional sites are being recruited. Together the NPBRN is engaging in a series of quality improvement and outcomes research studies across several areas of neurology, including epilepsy. This initiative is also being facilitated by the AAN.10

Our approach is not without challenges. The building of EMR tools or data registries can be prohibitively costly. We were able to build our toolkits at NorthShore using philanthropic and departmental research and development funds, but we are making these tools available to participating NPBRN sites at no direct cost. All clinical workflow effort related to this initiative (medical assistants, nurses, neurologists) is paid for from the department's clinical practice account (sustainability). For subjects eligible for enrollment in our biobank, the research assistants' efforts to administer written informed consent and to draw blood, and laboratory expenses for DNA and plasma extraction and storage and for genotyping, are paid for by philanthropic and departmental research and development funds. Additional funds are available from the Agency for Healthcare Research and Quality (AHRQ) to support the implementation of toolkits at NPBRN sites, the sharing of data, and the creation and mining of a registry.

There are also different versions of EMR software in use across institutions. We built our toolkits so that they can be installed in the most recent version and in historical versions of our EMR platform (Epic systems). We are committed to updating our EMR toolkits with each updated version, as they have become the way we practice clinically. We have implemented toolkits in the current and historical versions of the EMR at our NPBRN sites.

Safeguards are also required with respect to the use of the tools and the quality of the data collected. We focused our results on 231 patients who were evaluated initially using the SCDS toolkit and enrolled in the DNA biobank (18 or older, residents of Cook and Lake County, Illinois, final diagnosis of epilepsy, physician accepted the enrollment BPA, patient agreed to participate, and with valid written informed consent; ~96% participation rate). We also used the toolkit to evaluate patients initially who were not enrolled in the biobank, or for interval visits, or for annual visits. In total we have used the epilepsy SCDS toolkit >800 times. The descriptive cohort report and score test analytic report included as appendices (Data S2 and S3) indicate for the 231 enrolled patients the frequency of missing values for each of the summarized fields and score tests. Overall, missing data were few except for components that were added later to the toolkit (e.g., Short Test of Mental Status), or removed from the toolkit (e.g., Montreal Cognitive Assessment), or for components that were not applicable for many patients (e.g., MRC prognostic score). To limit the impact of missing data, we distribute each month to the medical assistants, nurses, and neurologists data quality reports (which required fields were missing, for each initial and annual visit). Most providers have few or no missing values on their monthly reports. Data quality reports are also being distributed to the NPBRN sites via our web‐based and secure data registry portal.

In conclusion we have designed an epilepsy toolkit within a commercial EMR that provides SCDS and CDS and which we hope will significantly improve the quality of epilepsy care and facilitate practice‐based research. We believe that our initiative can serve as a model for similar EMR‐based quality improvement and practice‐based research initiatives.25, 26 We encourage the development of similar epilepsy toolkits using common data elements across different EMR platforms, toward the creation of a network of networks.

Funding

Targeted study funding is provided by the Agency for Healthcare Research and Quality grant 1R01HS024057‐01 “Quality Improvement and Practice Based Research in Neurology Using the EMR,” awarded to Dr. Demetrius M. Maraganore.

Disclosure

None of the authors has any conflict of interest to disclose. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

Data S1. Screenshots of a structured clinical document support toolkit for Epilepsy clinical practice. ©2015 EPIC Systems, used with permission.

Click here for additional data file.

Data S2. Complete descriptive cohort report from the initial visits of 231 epilepsy patients.

Click here for additional data file.

Data S3. Complete score test analytic report from the initial visits of 231 epilepsy patients, revealing the nonparametric bar graph and Q–Q plot distributions of the scores, the pairwise correlations of the score test measures unadjusted and adjusted, and the principal component analyses of the score test measures (without and with Varimax rotation).

Click here for additional data file.