Preparing healthcare delivery organizations for managing computable knowledge.

INTRODUCTION: The growth of data science has led to an explosion in new knowledge alongside various approaches to representing and sharing biomedical knowledge in computable form. These changes have not been matched by an understanding of what healthcare delivery organizations need to do to adapt and continuously deploy computable knowledge. It is therefore important to begin to conceptualize such changes in order to facilitate routine and systematic application of knowledge that improves the health of individuals and populations.
METHODS: An AHRQ-funded conference convened a group of experts from a range of fields to analyze the current state of knowledge management in healthcare delivery organizations and describe how it needs to evolve to enable computable knowledge management. Presentations and discussions were recorded and analyzed by the author team to identify foundational concepts and new domains of healthcare delivery organization knowledge management capabilities.
RESULTS: Three foundational concepts include 1) the current state of knowledge management in healthcare delivery organizations relies on an outdated biomedical library model, and only a small number of organizations have developed enterprise-scale knowledge management approaches that "push" knowledge in computable form to frontline decisions, 2) the concept of Learning Health Systems creates an imperative for scalable computable knowledge management approaches, and 3) the ability to represent data science discoveries in computable form that is FAIR (findable, accessible, interoperable, reusable) is fundamental to spread knowledge at scale. For healthcare delivery organizations to engage with computable knowledge management at scale, they will need new organizational capabilities across three domains: policies and processes, technology, and people. Examples of specific capabilities were developed.
CONCLUSIONS: Healthcare delivery organizations need to substantially scale up and retool their knowledge management approaches in order to benefit from computable biomedical knowledge.

INTRODUCTION

Knowledge in the domains of health and health care has grown rapidly over the past few decades,1 and it is well documented that the pace of knowledge generation has exceeded the ability of healthcare delivery organizations to integrate and apply it.2, 3, 4 This gap will exponentially increase as the pace of knowledge generation further accelerates with the rise of data science.5 The range of data science applications that will generate knowledge from real‐world evidence about how to improve health and health care is vast and falls under various emerging initiatives: precision medicine,6, 7, 8, 9, 10, 11, 12, 13 population health,14, 15 learning health systems,16, 17 patient generated health data,18 quality improvement,11, 19 and pragmatic clinical research.19, 20 As knowledge generation expands at an ever‐increasing pace, it is critical to focus on increasing the capacity of healthcare delivery organizations to routinely integrate newly available knowledge into clinical decisions.21

Beginning with a broad definition of health‐related knowledge – any information that is interpreted or understood to have the potential to improve health or healthcare – knowledge management is a term given to the diverse set of activities that serve to capture, distribute, and effectively use knowledge within an organization.3 In healthcare delivery organizations today, knowledge management typically centers on the deployment of a narrow type of knowledge: clinical guidelines, clinical decision support rules, and other protocols. However, as the volume and nature of health‐related knowledge expands – for example to include “deep learning” algorithms – healthcare delivery organizations need to prepare for optimizing the uptake of new knowledge and the associated knowledge management capabilities required.21 This will undoubtedly require healthcare delivery organizations to deepen some capabilities they currently possess, as well as invest in wholly new capabilities. What is less clear is the direction the evolution of biomedical knowledge will take, how healthcare delivery organizations will need to adapt in response, and how to implement those adaptations.

RESEARCH INTERESTS

In light of these uncertainties, the authors convened a group of experts to generate a set of foundational concepts that will speed the evolution of healthcare delivery organizations that are ready to use knowledge generated in the era of data science. These concepts help frame recent progress towards creating technology, and establishing standards and policies, to house and make available biomedical knowledge represented as machine‐executable code in addition to human‐readable words and figures. Such infrastructure is likely to serve as the foundation for future knowledge dissemination and application within healthcare delivery organizations. Next, we sought to envision how this infrastructure may develop and how individual healthcare delivery organizations would utilize it in their efforts to identify and integrate new knowledge into frontline care. Specifically, we sought to identify examples of organizational knowledge management capabilities required to manage, use, and provide feedback on the application of knowledge that will be increasingly available in computable forms. We specified three domains – policies/processes, technology, and people – to serve as an organizing framework for example capabilities because all three domains must exist in alignment in order for any computable knowledge infrastructure to be effectively used. This paper describes the foundational concepts and examples of organizational knowledge management capabilities in order to spur engagement in the critical but challenging work of understanding how to prepare healthcare delivery organizations for the rapidly approaching era of computable knowledge management.

METHODS

Conference participants included experts in three domains that comprise the intellectual basis of the foundational concepts (Table A1). The three domains were: (1) characteristics of, and delivery methods for, new knowledge; (2) application of new knowledge in healthcare delivery, from both technical and organizational perspectives; (3) current state of, and how to evolve, knowledge management and knowledge infrastructures within healthcare delivery organizations. Accordingly, conference participants included leaders in knowledge management, health information technology, biomedical science, frontline care delivery, health system management, and organizational studies.

The conference was comprised of five sessions that covered our two core topic areas: (1) computable knowledge infrastructure at scale and (2) organizational knowledge management capabilities in the domains of policies/processes, technology, and people. Sessions progressed from defining knowledge management, to considering new approaches to knowledge representation, and ultimately to developing examples of required organizational capabilities. Each session included presentations from three experts followed by group discussion.

In Session 1: Computable Knowledge as a “Game Changer” experts defined and described the current and anticipated future state of representing and disseminating healthcare knowledge, with an emphasis on computable representations. In Session 2: Infrastructure to Support Dissemination of Computable Knowledge participants discussed the types of health information technology (HIT) and knowledge management infrastructures currently in use in healthcare delivery organizations and the barriers to their effective use and expansion. Session 3: Use Cases and Initial Capability Discussion introduced specific initiatives and projects demonstrating innovative applications of computable knowledge in healthcare settings. Session 4: Organizational and Implementation Challenges expanded on these use cases to apply lessons learned from HIT adoption to knowledge management. During Session 5: Capabilities, Gaps and Research Agenda, participants collaborated to generate examples of capabilities needed for improved knowledge management in the domains of policies/processes, technology, and people.

Expert presentations and the subsequent discussions were recorded. Transcripts and notes from those sessions were produced and served as the basis for summarization of discussion results. We first analyzed conference materials to generate a set of foundational concepts that captured the commonly understood features of current and anticipated approaches to knowledge management in healthcare delivery organizations. Next, we identified novel findings that emerged from group discussions that served to articulate high‐level organizational capabilities needed to support computable knowledge dissemination at scale and examples of specific organizational capabilities needed to effectively engage with that infrastructure, under three domains of policies/processes, technology and people. The author team expanded on concepts and capabilities discussed during the conference in order to generate a cohesive set of findings.

RESULTS: FOUNDATIONAL CONCEPTS

All healthcare delivery organizations engage in some form of knowledge management that relies on a “pull” model grounded in the mission of biomedical libraries, and few have evolved to enterprise‐scale knowledge management approaches that support “push” models and some forms of knowledge in computable form.

Health‐related knowledge has always been recognized as dynamic and evolving. 21st century knowledge is fundamentally different than 20th century knowledge, which was fundamentally different than 19th century knowledge. Today's healthcare delivery organizations invest, to varying degrees, in efforts to systematically engage in knowledge management – that is, activities that capture, distribute, and effectively use knowledge within the organization. Biomedical libraries are historically the focal points of these efforts, and carry out their missions professionally and effectively, ensuring that knowledge, expressed primarily in journals and books, is organized and accessible. The mission of libraries supports a “pull” model that requires decision makers to initiate action to access knowledge to inform their work. Based on this model, knowledge management capabilities in healthcare organizations are built around non‐computable knowledge representation and dissemination. Knowledge captured in peer‐reviewed journals is translated into practice via clinical guidelines that attempt to turn empirical studies into decision logic. Healthcare delivery organizations then “pull” from the libraries of clinical knowledge by assessing available guidelines then deciding whether and how to deploy them in their frontline clinical systems (e.g., electronic health records or paper‐based reminders or checklists). Decision‐making about what guidelines to deploy is typically accomplished by a clinical content committee and deployment is overseen by the clinical systems department. Frontline deployment may also follow a “pull” model, in which the user needs to seek out relevant guidelines or knowledge, or it may be “pushed” to the user (e.g., a best practice alert that is triggered for a specific patient). However, even if an organization is actively pursuing a “push” model for frontline deployment, if the upstream decision at the organization level about what knowledge to make available for pushing relies on a pull model, then the resulting flow of knowledge to frontline users will still be limited.

When knowledge generation is rapid and diverse, the “pull” model of the traditional library struggles to scale, in both small organizations that lack resources to consistently scan available knowledge and in large organizations for which the volume of applicable knowledge makes its integration into clinical practice a vast and complex undertaking. A relatively small number of organizations have evolved their knowledge management infrastructure to a scalable “push” model at the organization level in which decision makers routinely receive evidence‐based advice generated from knowledge in computable forms without the organization first having to pull it.22 One example of such an effort is work at Partners Healthcare to combine a business rules engine with an ontology engine to implement a scalable CDS system.23 However, beyond a handful of examples, such approaches to computable knowledge management are ad hoc and rare.

The emergence of the concept of Learning Health Systems has created an imperative for management of knowledge.

As shown in Figure 1, Learning Health Systems (LHSs) execute learning cycles that generate new knowledge, deploy existing knowledge, and learn from deployments to iteratively refine knowledge. Accordingly, LHSs require advanced knowledge management capabilities that can capture new knowledge and package it in persistent forms, and then apply that new knowledge, along with related existing knowledge, to inform health‐related decisions. While LHSs do not require knowledge to be represented in computable forms and progress in the use of computable biomedical knowledge objects will still result in improved dissemination of knowledge even absent an LHS cycle, the ability of LHS cycles to exist at scale requires that guidance to decision makers be routinely computed rather than generated by human review and inspection. In particular, the ability to transition between data‐to‐knowledge and knowledge‐to‐practice components of the cycle at scale (blue to red arrow, Figure 1) is essentially impossible without knowledge represented in computable form. In addition, the knowledge‐to‐practice component can only occur at scale if healthcare delivery organizations have capabilities to support computable knowledge management.24

Figure 1

Learning health system

The ability to represent data science discoveries in computable forms that are FAIR (findable, accessible, interoperable, reusable) is fundamental to spread new knowledge at scale.

An additional essential feature of computable knowledge infrastructures to allow them to scale across varied healthcare organizations is that they are “FAIR” by making knowledge findable, accessible, interoperable and reusable. This principle is analogous to data FAIRness, which is a recognized goal of data science. Knowledge FAIRness will facilitate knowledge management and sharing at scale, primarily because of the principle that it be open, free and universally implementable.25 Computable knowledge infrastructures predicated on knowledge FAIRness will allow healthcare organizations to make decisions based on the best available knowledge in an efficient and shareable way. If the information is available in a machine‐executable format it will facilitate broad uptake by minimizing human burden and increasing the value and reusability of the knowledge.25

Knowledge infrastructures that disseminate FAIR knowledge at scale will serve as the necessary foundation for the subsequent work required to translate knowledge into optimized performance on the frontlines of healthcare delivery. However, this will not be a seamless process until the organizations in which the knowledge is deployed have a complementary set of capabilities to engage with computable knowledge infrastructures and integrate them with frontline clinical systems that are the sharp‐end of knowledge application. For example, the ability to match relevant knowledge to specific workflows and clinical decisions in a healthcare delivery organization is crucial to the effective deployment of knowledge. Without that capability and corresponding infrastructure, the knowledge cannot be utilized at scale.

RESULTS: TOWARDS AN UNDERSTANDING OF HEALTHCARE DELIVERY ORGANIZATION CAPABILITIES TO MANAGE COMPUTABLE KNOWLEDGE

Organizational cycles to manage and use computable knowledge

Healthcare delivery organizations and other organizations in the health domain will need to engage in enterprise knowledge cycles, which are supported by specific policy/process, people, and technology capabilities (Figure 2). In a future state in which knowledge is FAIR and readily available in computable form, the knowledge management approach that served the prior era will need to change to effectively select, deploy, and assess (i.e., provide feedback on) computable knowledge. Currently, health systems (and often users working in these systems) must actively search out and ‘pull’ computable knowledge into their information systems and/or workflows. Under this ‘pull’ model, organizations have existing approaches to determine what knowledge to make available (i.e., curation), how to deploy it when requested (i.e., customization, workflow and technical integration), and assess the impact of these efforts These will continue to be important activities in an era of computable knowledge management. However, they are unlikely to be sufficient; to help identify what new capabilities may be required, it is useful to conceptualize the enterprise knowledge cycle as consisting of two components ‐ computable knowledge management and computable knowledge use – and a feedback loop between them.

Figure 2

Enterprise knowledge cycle

The management component of computable knowledge deployment in healthcare delivery organizations involves the selection and technical deployment of computable knowledge. Today, knowledge management activities include decisions about what knowledge to deploy (based on the content of journal articles and clinical guidelines) and how to build knowledge into frontline clinical information systems. As sources of knowledge become computable, there will likely be a larger volume of knowledge that needs to be evaluated for incorporation, new information available about that knowledge (e.g., provenance) that organizations will need to decide how to factor in, and more complex decisions about technical deployment because some knowledge could be designed to automatically update in frontline clinical systems.

The use component of the cycle in healthcare delivery organizations involves frontline application of new knowledge to enable clinicians to integrate new knowledge into decision‐making and care delivery. The use component is not limited to what is traditionally thought of as clinical decision support, but can also include the application of organization‐wide protocols, condition‐specific initiatives and various other types of support for implementing best practices. It is unclear how clinicians will react to guidance or advice derived from computable knowledge that could be updated at a much more rapid pace as well as how workflows and other human‐mediated processes will need to change to ensure routine application of computable knowledge. They will likely depend on how the management component is approached since that will determine how knowledge is presented to clinicians.

The feedback component of the enterprise knowledge cycle involves active and consistent assessment of deployment efforts, and serves as the feedback loop between management and use. It can take many forms, including but not limited to capturing data on clinician utilization. This provides the organization with an understanding of how computable knowledge is being used, which will inform improved management efforts. Other examples of feedback include continuous revision and updates to content that take place locally to better tailor generalized knowledge to the particular clinical setting or patient population. Taken together, the iterative process of selection, deployment, assessment, and adjustment should result in continuously improved application of new knowledge within the given organization.

Examples of new healthcare delivery organization capabilities

To more concretely envision how healthcare delivery organizations need to adapt knowledge management capabilities to engage in the enterprise knowledge cycle, discussions produced examples of policies/processes, technology and people capabilities (Table 1). These capabilities were organized according to the management, use and feedback components of the enterprise knowledge cycle.

Table 1

Example healthcare delivery organization capabilities for computable knowledge management

Components of Enterprise Knowledge Cycle:	Policies/Processes	Technology	People
Management	Create organizational knowledge asset library and policies governing its use   Continuously edit, update, and link knowledge assets	Deploy enterprise‐scale digital library infrastructure software to manage computable knowledge	Employ biomedical librarians to manage the digital library by performing knowledge linking, cataloging and sharing
Use	Prioritize and deploy applications of knowledge to appropriate clinical workflows and decisions	Establish interoperability between knowledge management software and frontline systems (e.g., EHRs) to generate and push advice to inform decisions	Educate stakeholders to function professionally in an environment of practice supported by “pushed” computable knowledge
Feedback	Capture data on the processes and outcomes of clinician engagement with knowledge   Analyze data on knowledge use to continuously improve management and use	Create a data repository that links frontline knowledge deployment, clinical decisions and patient outcomes   Use dashboards or other tools to display “performance” measures derived from repository	Hire knowledge informaticians with expertise in clinical decision making, EHR data, and analytics

Management

Computable knowledge management begins with a set of policies and processes to acquire and manage knowledge assets (i.e., the “content”). These activities require decisions about what knowledge may be appropriate and relevant to the particular healthcare delivery organization, in the same way that librarians select specific journals and books with content that is relevant to their users. This process requires an underlying technology infrastructure where selected knowledge assets can be categorized, searched, etc. so that relevant knowledge can be identified and deployed. Today's libraries have software to manage their knowledge assets but new technical capabilities will be required to manage computable (versus paper/online) assets. Given these new processes and technologies, it is likely that a new type of biomedical librarian will be needed with the specific skills and training to support computable knowledge management tasks.

Use

The use component of the enterprise knowledge cycle will rely on a distinct set of policies/processes, technology, and people capabilities. The key process that must be undertaken is an agile approach to prioritizing problems that would benefit from application of newly available computable knowledge, and then pushing that knowledge into clinical workflows. These processes require nuanced understanding of clinical decision‐making, workflows, and task‐technology fit (i.e., how to integrate automated and human‐mediated decisions). Pushing computable knowledge to frontline decisions requires technology infrastructure in which the computable knowledge is “interoperable” with frontline systems. Without such interoperability, knowledge will be divorced from workflows, which makes it much harder to integrate into decisions. The recent decision to require all EHRs to have application programming interfaces should facilitate such interoperability but we have little experience to‐date to know where the challenges lie. When knowledge is routinely pushed to frontline clinicians and integrated into their workflows, many roles will need to be reconceptualized. For example, clinicians will need to evolve from being seen (and seeing themselves) as possessors of knowledge and instead be seen as managers of knowledge. Training clinicians to assess knowledge provenance, engage in probabilistic thinking, and feel comfortable expressing uncertainty in front of patients will allow clinicians to effectively function in a clinical environment supported by computable knowledge that is continuously pushed. It will be similarly important to identify new skills and expectations for other roles.

Feedback

To build an effective enterprise knowledge management cycle, the process of knowledge management and use needs to be continuously assessed and refined. Such a feedback loop requires that there is a process to capture data on clinician use of computable knowledge (e.g., where was it presented in their workflow, when did they see it) and the decision‐making outcomes (e.g., what was the relevant decision they made or action they took/didn't take) as well as the patient outcomes (e.g., measures of the problem that was attempting to be solved by deploying new knowledge). This data will need to be stored and made available to knowledge informaticians who possess the skills to analyze the data and determine how to refine knowledge management or knowledge use processes accordingly to increase the impact on outcomes (or address unintended consequences). It may also be valuable to develop a dashboard or other reporting tool to make the relevant “knowledge use performance” measures visible to the organization.

DISCUSSION

We are in the very early stages of understanding how healthcare delivery organizations can adapt to apply computable knowledge. Our conference served to identify foundational concepts that reflect current understanding of how knowledge management is evolving in the LHS context as well as new ideas such as the concept of an enterprise knowledge management cycle along with example capabilities that may be needed to support the cycle. Multiple research efforts are necessary to further this work. Priority should be placed on evaluating any efforts to implement enterprise knowledge cycles and associated healthcare delivery organization capabilities. Key to this evaluation will be how implementing organizations have addressed concerns about governance, liability, and costs as well as documenting best practices. Understanding where, how and to what extent new capabilities have been implemented will provide a foundation for analysis of the most effective knowledge‐to‐practice strategies and combinations of capabilities. Assessing specific computable knowledge management tools will also contribute to a fully developed, evidence‐based guide for healthcare delivery organizations working to utilize computable knowledge.

Efforts to pursue adoption of these capabilities at scale will inevitably run into questions about cost and value. Even the example capabilities identified here could be prohibitively expensive for many healthcare delivery organizations and investment in these capabilities will compete for resources that could be devoted to other efforts to improve health system performance. While closing the knowledge‐to‐practice gap is widely viewed as a critical priority, we lack a robust assessment of the costs and benefits, and how these change when knowledge is available in computable form. It will therefore be critical to generate evidence on the costs, benefits, and overall value proposition. In doing so, it will be important to recognize existing efforts to close the knowledge‐to‐practice gap, particularly those targeting smaller, less well‐resourced settings. It likely makes sense to pursue adaption and expansion of these efforts, rather than each healthcare delivery organization investing independently in new capabilities, to achieve economies of scale. Finally, there will be a new set of implementation‐related challenges that are likely to emerge and will also require attention, perhaps calling for an even broader scope of capabilities. For example, what if two types of computable knowledge targeting the same decision are in conflict with one another? At this very early stage in the evolution of computable knowledge management, the concepts and capabilities presented in this work are only the tip of the iceberg.

CONCLUSION

The concepts and capability framework produced by this conference offer a starting point for the critical task of assessing the readiness of the US healthcare system to deliver care in a way that utilizes new knowledge and knowledge infrastructures. Based on the example capabilities, healthcare delivery organizations likely need to substantially scale up and retool their knowledge management approaches, which to date have largely been limited to decisions about how to configure a relatively constrained set of clinical decision support tools. Future research on implementation and best practices as well as the value proposition will contribute to the healthcare system's ability to adapt to a changing knowledge landscape and unlock the potential of knowledge from data science.

CONFLICT OF INTEREST

Mrs. Nong and Dr. Adler‐Milstein have no conflicts of interest to report. Dr. Friedman is the current editor‐in‐chief of Learning Health Systems journal.

Table A1

List of conference participants

Participating expert	Organization
Larry An	University of Michigan
Ashish Dwivedi	Hull University
Allen Flynn	University of Michigan
Jonathan Greenberg	Michigan Medicine
Tonya Hongsermeier	Lahey Health
Gerald Kane	Boston College
Zach Landis‐Lewis	University of Michigan
Nancy Lorenzi	Vanderbilt University
Ann Scheck McAlearney	Ohio State University
Blackford Middleton	Apervita
Jerome Osheroff	TMIT consulting
Jodyn Platt	University of Michigan
Andrew Rosenberg	Michigan Medicine
Shawna Smith	University of Michigan
Walter Stewart	Sutter Health
Douglas van Houweling	University of Michigan
Kevin Ward	University of Michigan