Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education.

Physicians in the 21st century will increasingly interact in diverse ways with information systems, requiring competence in many aspects of clinical informatics. In recent years, many medical school curricula have added content in information retrieval (search) and basic use of the electronic health record. However, this omits the growing number of other ways that physicians are interacting with information that includes activities such as clinical decision support, quality measurement and improvement, personal health records, telemedicine, and personalized medicine. We describe a process whereby six faculty members representing different perspectives came together to define competencies in clinical informatics for a curriculum transformation process occurring at Oregon Health & Science University. From the broad competencies, we also developed specific learning objectives and milestones, an implementation schedule, and mapping to general competency domains. We present our work to encourage debate and refinement as well as facilitate evaluation in this area.

Introduction

Physicians and medical students have been using health information technology (HIT) for decades. During this time, the role of HIT has changed dramatically from a useful tool for data access and occasional information retrieval to a ubiquitous presence that permeates health care and medical practice in myriad ways. As the use of HIT has accelerated and the underlying science of biomedical and health informatics has advanced, medical education has lagged behind, leaving physicians and students alike to learn largely on their own how to make use of these tools. While some medical schools have introduced limited aspects of biomedical informatics into their curricula, most of this is focused on training to perform basic tasks such as accessing knowledge resources or basic use of an electronic health record (EHR).

Twenty-first century clinicians face a much more digital world than their predecessors. The quantity of biomedical knowledge continues to expand, with an attendant increase in the primary scientific literature.1 Secondary sources that summarize this information proliferate as well, not only for use by clinicians but also by patients and consumers. Programs such as the “meaningful use” financial incentives of the Health Information for Technology and Clinical Health Act require comprehensive adoption and use of the EHR.2,3 Patients, especially the aging “baby boomer” generation, want to interact with the health care system the same way they interact with airlines, banks, and retailers, ie, through digital means using technologies such as the personal health record.4 Patients, payers, and purchasers demand more accountability in health care quality, safety, and cost,5 leading to an expectation of measurement and reporting of quality of care as a routine part of participation in new delivery mechanisms such as primary care medical homes and accountable care organizations. At the same time, telemedicine applications extend the reach of health care systems and clinicians in both rural and urban settings. Ideally, for all of these applications, patient data must move readily across organizational boundaries via health information exchange, while privacy and confidentiality are protected.6 The growing quantity of clinical and administrative data in these systems also affords an opportunity for advanced analysis that can enable better deployment of resources and coordination of care, facilitation of personalized and precision medicine, and advancement of clinical and translational research.7 Together, these advances are moving health care toward the global vision of the learning health system put forth by the Institute of Medicine (IOM).8,9

The importance of these developments is further evidenced by the recent establishment of a new medical subspecialty of clinical informatics.10 Practicing physicians are now beginning to become board-certified in this new subspecialty, with fellowship programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) to soon be established. This underscores the need for medical students to be introduced to the concepts and competencies of this new medical subspecialty as part of the undergraduate curriculum.

In this paper, we describe our work to expand the undergraduate medical curriculum to include a comprehensive set of competencies in clinical informatics. We go beyond the usual focus on information retrieval from knowledge sources and data access through the EHR, aiming to address the expanded and diverse roles of HIT in contemporary health care delivery, personal health, public health, and clinical and translational research. We describe the development of a novel biomedical informatics curriculum that is being implemented within a comprehensive transformation of the undergraduate curriculum at our institution. Our future plans include evaluation of the impact of this work, and we encourage others to participate in this evaluation and refinement process.

Previous work developing competencies in informatics

The need to include biomedical informatics in medical education has been recognized in previous publications of competencies, curricula, and other learning materials for informatics. One early, high-profile resource was part of the Medical Student Objectives Project of the Association of American Medical Colleges, which framed proposed objectives in terms of five key roles of physicians: clinicians, life-long learners, educators, researchers, and managers.11 Although current medical students believe these objectives are important to their education,12 uptake of these objectives has been modest.13 More recently, the Alliance for Clinical Education issued a policy statement calling for competencies related to the EHR,14 although also noting that many challenges to implementing such competencies exist.15 Others have also noted the challenges to education about the EHR, citing problems analogous to those of clinicians incorporating EHRs into practice, such as integration into workflows, and concerns about unanticipated and undesirable impacts.16 It is clear, however, that competencies in the use of the EHR are aligned with more general physician educational competencies,17,18 and that informatics curricula can be incorporated into undergraduate medical training19,20 and evaluated.21

Methods

In 2012, the Oregon Health & Science University, School of Medicine began a comprehensive transformation of the undergraduate medical curriculum for the 21st century. From the outset, the Dean and school leadership recognized the importance of informatics to physicians and health care in the future. With participation and support from the Senior Associate Dean for Education (GM), a team of five physician-faculty was convened to lead this effort, including the following:

An informatician involved nationally in informatics education leadership and development of the new clinical informatics subspecialty (WH)

An internist-informatician who is chair of the medical school curriculum committee and actively engaged in the new curriculum development team (PG)

A family physician and innovator in using simulation for teaching and assessing EHR use by medical students (FB)

A critical care physician investigator using EHR simulation to assess residents’ ability to detect important patterns in critical care data (JG)

An informatician and former internal medicine residency associate program director also involved in the previous two above activities (VM)

Using an exploratory qualitative methodology based on action research and working with the curriculum transformation leadership, we developed a broad set of competencies in informatics for medical education. With each competency, we developed a set of learning objectives, a designation of when it would be most appropriate to introduce the concept during the medical school curriculum, and how each mapped to the broader ACGME competencies.

Results

Developing competencies and learning objectives

Before beginning to identify the competencies for student learning, we needed to develop the rationale for including these topics in the curriculum. This led us to map the competencies to the six ACGME core competency domains22 and develop explicit learning objectives within each competency, followed by their implementation in the emerging curriculum. Discerning the rationale for our competencies was a narrative process, with their implementation enumerating the specific areas deemed important to incorporate into the medical school curriculum, as it was being concurrently designed.

Our first step was to provide the rationale to include various topic areas in the curriculum. A first critical concept we agreed upon was that informatics is not the same as computer literacy. Computer literacy is one of many requirements to use informatics successfully, but knowing how to use a computing device (personal computer, tablet, or smartphone) is not the same as having skills in informatics, ie, using that device to improve health, health care delivery, public health, or research.

We concurred that one fundamental skill currently being taught in many medical schools for 21st century clinicians was something taught in many medical schools since the late 20th century, namely, how to find information to apply to patient care. The needed informatics skills start, and do not stop, with the knowledge of how to enter simple queries into common general (eg, Google™, http://www.google.com) or medical (eg, PubMed, http://www.pubmed.gov) search engines. Clinicians must also know how to formulate a clinical question as an answerable one, and then be able to select the appropriate resource and make optimal use of it. This includes knowing what content is in different search systems. He or she must also know about specialized resources such as the Agency for Healthcare Research and Quality Guidelines Clearinghouse (http://www.guideline.gov) and the Centers for Disease Control and Prevention travel site (http://wwwnc.cdc.gov/travel). Clinicians should also have an understanding of the major commercial publishers as well as what professional societies offer. Once clinicians know how to use a search site, they must be able to phrase an appropriate query and know how to maximize the search using advanced query features. To take maximum advantage of available knowledge resources, it is necessary to understand the advanced search features that are available, whether in general web resources such as Google or dedicated biomedical knowledge resources such as PubMed.

Once information is retrieved, clinicians must know how to critically appraise the information retrieved and decide, if warranted, to apply it to the patient or population. As with searching, the type of appraisal varies with the search engine used. With output from general search engines like Google, the trustworthiness of the information must be assessed. Clinicians must also know how to make optimal use of patient data and information. They must know how to use informatics to achieve the triple aim of better health, better care, and lower cost.5 An appropriate framework for applying this emanates from the IOM concept of the “learning health system”,8,9 which presents a compelling vision for a health care system that is patient-centered, population-based, and promotes learning from data.

Being able to use data and information also means understanding that the patient health record is more than “charting”, and that its value goes beyond being able to read it. Clinicians must be facile with all aspects of the EHR, able to easily move from one vendor system to another and to understand the critical role of health information exchange in making any single record as complete as possible.6 This issue is magnified by the increasing amount of data the clinicians must view and manage.23 Further, as the health care focus moves from one of quantity to one of value, clinicians must pay attention to quality, patient safety, and cost. This requires coordination of care, in not just providing medical procedures, nursing interventions, therapies, etc in isolation, but also teamwork and communication.24

From this vantage, the health record is no longer a passive collection of information used mainly to justify billing. Rather, it is a source of data, organized into coherent information that allows the health care team to deliver the best, safest, and most cost-effective care. This means that the 21st century clinician must have a basic understanding of informatics issues, such as capturing data that is correct and complete as well as consistent in its expression.25 He or she must be able to work in partnership with informatics professionals to achieve what we know is so critical in the application of informatics: adhering to standards, achieving system interoperability, appropriately and optimally implementing clinical decision support, and maintaining security to assure privacy and confidentiality.26

Clinicians also require understanding of areas of quality measurement and improvement, especially as it applies to their direct practice.5 The clinician must understand concepts of health care quality and how the EHR and other sources of data are increasingly used to measure it. They must also know how to work with clinical leaders to select, implement, and improve quality measures. This should be introduced as part of the larger learning health system.8,9 This discussion should also include the related issues of patient safety and medical errors,27 including safety of HIT.28

Patient engagement is another component for safe, effective, and coordinated care.29 Not only does patient engagement improve health care, but patients, including Internet-savvy baby boomers, will increasingly demand it. Patients want health care that adapts the online features of other modern industries, allowing them the ability to view their own data and interact with their clinicians and health care system (eg, online scheduling of appointments, prescription refills, and even consultations that are appropriate via the Internet or other electronic media).29 These will likely take the form of a personal health record, accessible from a patient portal that allows access to all information, not just that from the system of the provider organization.4 Because of this natural evolution of health care, clinicians must also have knowledge and understanding of the appropriate use of telemedicine and telehealth (such as e-visits), both for remote locations and as a convenient option locally in patients’ homes and other settings.

Complementary to the patient-centric view, clinicians must also understand population-based care30 and the informatics underlying it.31 The clinician and their team care for populations of patients; they must be able to view their care needs and results across their patient population. When a new test or treatment is determined to be highly effective, the clinician must be able to quickly identify patients who are candidates for it. They must also be able to identify outliers in their populations who require intervention, such as those with excessively high blood pressure or blood sugar, missed appointments or screen tests, or those at risk for hospital (re)admission.

Another area where 21st century clinicians must understand the key issues is in bioinformatics, especially as it relates to personalized medicine.32 Modern clinicians need not understand complicated gene sequencing algorithms, but they should have an understanding of how genomics and related areas are transforming our understanding of maintaining health and diagnosing and treating disease.33 Now that the vision of personalized medicine is coming to fruition, clinicians will increasingly rely on the help of decision support and other tools for assistance in applying it to individual patients. He or she should have a basic understanding of genome-wide association studies and their ramifications.34

Clinicians must also understand the strengths and limitations of clinical and translational research. They must understand the differences between and the value of experimental and observational studies. Ideally, students will have participated in research while in their training. But even if not, they should understand issues such as data quality, study design, and the limitations that come from the sharp focus perspective of a clinical study. Students should participate in the learning health system laid out in the vision of groups like the IOM.

Implementing competencies and objectives

Based on the above narrative, our group of diverse clinician-educators developed through an iterative process a set of 13 competencies in clinical informatics (Table 1). Each competency was mapped to one or more of the six ACGME general competency domains. We then developed more detailed learning objectives and milestones within each competency. We also categorized each learning objective for its presence in the early (at the beginning), middle (during the preclinical portion), or late (during clinical experiences) portion of the curriculum.

Table 1

Competencies in clinical informatics and specific learning objective/milestone within each

Competency	Learning objectives/milestones	When start?	PCPS	MK	PBLI	ICS	Prof	SBP
Find, search, and apply knowledge-based information to patient care and other clinical tasks	Information retrieval/search: choose correct source for specific task, search using advanced features, apply results	Early	x	x	x			x
	Evaluate information resources (literature, databases, etc) for their quality, funding sources, biases	Early
	Identify tools to assess patient safety (eg, medication interactions)	Early
	Utilize knowledge-based tools to answer clinical questions at the point of care (eg, textbooks, calculators, etc)	Mid
	Formulate an answerable clinical question	Mid
	Determine the costs/charges of medications and tests	Mid
	Identify deviations from normal (laboratory tests/X-ray/results) and develop a list of causes of the deviation	Mid
Effectively read and write from the electronic health record for patient care and other clinical activities	Graph, display, and trend vital signs and laboratory values over time	Early	x		x			x
	Adopt a uniform method of reviewing a patient record	Early
	Create and maintain an accurate problem list	Early
	Recognize medical safety issues related to poor chart maintenance	Early
	Identify a normal range of results for a specific patient	Mid
	Access and compare radiographs over time	Mid
	Identify inaccuracies in the problem list/history/medication list/allergies	Mid
	Create useable notes	Mid
	Write orders and prescriptions	Mid
	List common errors with data entry (drop down lists, copy and paste, etc)	Mid
Use and guide implementation of CDS	Recognize different types of CDS	Early	x	x	x			x
	Be able to use different types of CDS	Late
	Work with clinical and informatics colleagues to guide CDS use in clinical settings	Late
Provide care using population health management approaches	Utilize patient record (data collection and data entry) to assist with disease management	Early	x		x			x
	Create reports for populations in different health care delivery systems	Late
	Use and apply data in accountable care, care coordination, and the primary care medical home settings	Late
Protect patient privacy and security	Use security features of information systems	Early	x				x
	Adhere to HIPAA privacy and security regulation	Early
	Describe and manage ethical issues in privacy and security	Early
Use information technology to improve patient safety	Perform a root-cause analysis to uncover patient safety problems	Late	x			x		x
	Familiarity with safety issues	Early
	Use resources to solve safety issues	Late
Engage in quality measurement selection and improvement	Recognize the types and limitations of different types of quality measures	Early	x		x			x
	Determine the pros and cons of a quality measure, how to measure it, and how to use it to change care	Mid
Use HIE to identify and access patient information across clinical settings	Recognize issues of dispersed patient information across clinical locations	Early	x		x	x		x
	Participate in the use of HIE to improve clinical care	Late
Engage patients to improve their health and care delivery though personal health records and patient portals	Instruct patients in proper use of a personal health record	Early	x			x	x	x
	Write an e-message to a patient using a patient portal	Early
	Demonstrate appropriate written communication with all members of the health care team	Mid
	Integrate technology into patient education (eg, decision making tools, diagrams, patient education)	Late
	Educate patients to discern quality of online medical resources (Web sites, applications, patient support groups, social media, etc)	Early
	Maintain patient engagement while using an electronic health record (eye contact, body language, etc)	Mid
Maintain professionalism through use of information technology tools	Describe and manage ethics of media use (cloud storage issues, texting, cell phones, social media professionalism)	Early					x
Provide clinical care via telemedicine, and refer those for whom it is necessary	Be able to function clinically in telemedicine/telehealth environments	Late	x			x		x
Apply personalized/precision medicine	Recognize growing role of genomics and personalized medicine in care	Early	x	x
	Identify resources enabling access to actionable information related to precision medicine	Early
Participate in practice-based clinical and translational research	Use electronic health record alerts and other tools to identify patients and populations for offering clinical trial participation	Mid			x			x
	Participate in practice-based research to advance medical knowledge	Late

Notes: All learning objectives/milestones are designated for time to start (early: start of medical school; mid: later in preclinical years; and late: during clinical experiences); Each is also mapped into one or more of the ACGME core competencies: patient care and procedural skills (PCPS), medical knowledge (MK), practice-based learning and improvement (PBLI), interpersonal and communication skills (ICS), professionalism (Prof), and systems-based practice (SBP).

Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; CDS, clinical decision support; HIE, health information exchange; HIPAA, Health Insurance Portability and Accountability Act.

These learning objectives are now being woven into Oregon Health & Science University’s new curriculum (Table 2). As seen in Table 2, we have designated where the specific learning activities will be placed in the major portions of the curriculum from orientation through preclinical sciences, clinical experiences, and intersession in the fourth year.

Table 2

Informatics in various portions of curriculum in the preclinical (orientation, fundamentals, and preclinical sciences) and clinical (clinical experiences and intersession) years

Competency	Orientation	Fundamentals	Preclinical sciences	Clinical experiences	Intersession
Find, search, and apply knowledge-based information to patient care and other clinical tasks	Aware of OHSU resources	Choose resources; background and foreground questions	Regular use of electronic sources for background questions: textbooks, Web, etc.Occasional use of MEDLINE, PubMed, EBM resources for foreground questions; build awareness of quality, validity (study designs, biases, funding, etc), sprinkled with cutting edge	Competent in ask, access, appraise, and apply approach
Effectively read and write from the EHR for patient care and other clinical activities	Can login to EHRStudents have logins on day 1	All cases simple EHR pageEHR has oral presentation and progress note templates	Progression across basic science course: all cases in EHR, with successive addition of:Laboratory tests, trends, display laboratory results (not recite) at bedside X-rays, multinote EKG, echocardiogramCreate notesFind errors, DDI Problem lists, chart hygiene, chart junkCreate orders, use CDS, DDI	Competent access data, create notes	Create patient templates
Use and guide implementation of CDS				Use CDS and DDI	Build CDS
Provide care using population health management approaches				Use EHR data for population health management
Protect patient privacy and security	Aware of HIPAA, Hippocrates	Secure use of OHSU systems		Competent, secure use of EHRs	Ethical issues, privacy
Use IT to improve patient safety		One medical error case for awareness		Root cause analysis, experiences	Building systems for resilience
Engage in quality measurement selection and improvement			Understand fundamentals of health care quality measures and quality improvement	Use EHR data for quality measurement and improvement
Use HIE to identify and access patient information across clinical settings			Access data over HIE	Competent user of HIE	Recognize HIE issues: clinical, financial
Engage patients to improve their health and care delivery through personal health records and patient portals	Aware of OHSU and VA portals	Patient interaction with EHR (body language, etc)	Send/receive patient messagesEngage, educate patient with EHR, PHR, and other IT	Interact with patients via PHR and portals
Maintain professionalism through use of IT tools	Interact with social media	Cloud storage issues			Professionalism issues with EHR and other IT
Provide clinical care via telemedicine, and refer those for whom it is necessary				Telehealth/telemedicine use on community and rural rotations
Apply personalized/precision medicine		One case to create awareness	One case in any block an example of using genomic information for precision medicine
Participate in practice-based clinical and translational research			Use EHR alerts for clinical trial eligibility in cancer caseRevisit EHR alerts for clinical trial eligibility	Practice-based and translational research integrated in clinical experiences

Abbreviations: CDS, clinical decision support; DDI, drug-drug interaction; EBM, evidence-based medicine; EHR, electronic health record; EKG, electrocardiogram; HIE, health information exchange; HIPAA, Health Insurance Portability and Accountability Act; IT, information technology; OHSU, Oregon Health & Science University; PHR, personal health record; VA, Veterans Affairs.

Conclusion and future directions

Our analysis has shown there are a substantial number of informatics competencies and a large body of associated knowledge that the 21st century clinician needs to learn and apply. From a pedagogical standpoint, there are also issues in how to organize, deliver, and assess this content. Certainly, one approach is to provide this content as a separate course, isolated from the rest of the curriculum. However, a better approach would be to tightly and comprehensively integrate informatics concepts longitudinally into the learning curriculum since clinical informatics is emerging as a core competency of medical practice, applicable in all basic science disciplines and clinical specialties.

A next major step for this work will be to develop evaluation activities for the competencies and learning activities. These will vary based on institutional factors (class size, whether students are asynchronous, etc), faculty preference, and funding limitations (desire to use simulation for many things, but cost may be prohibitive). As such, different learning activities will require different evaluation methods. For example, the competency “Provide care using population health management approaches” might be best measured by having students describe or select effective system change options based on immunizations rates of the clinic to which they are assigned, while the competency “Provide clinical care via telemedicine and refer those for whom it is necessary” might best be measured with an objective structured clinical examination simulation.

The presence of these competencies also indicates a need for educators who are specialists in informatics to (collaboratively with clinical educators) design the learning and deliver learning experiences that are appropriate for lecture, group discussions, self-paced and self-directed methods, and other settings. But informatics is one of those topics that is best infused throughout the curriculum, especially in clinical settings where it is being used.

There are a number of future steps for this work. First, we recognize that this set of competencies represents the views of one medical school’s informatics faculty and must be validated with informatics experts from other institutions. Second, we must evaluate our own implementation of this curriculum to determine how these competencies are delivered to a medical student audience. Finally, we must evaluate this entire process with students from our institution and others to determine which competencies and learning experiences are most valuable for them in their future clinical practice.