Opportunities and challenges to advance the use of electronic patient-reported outcomes in clinical care: a report from AMIA workshop proceedings.

Despite the demonstrated value of patient-centered care, health systems have been slow to integrate the patient's voice into care delivery through patient-reported outcomes (PROs) with electronic tools. This is due in part to the complex interplay of technology, workflow, and human factors that shape the success of electronic PROs (ePROs) use. The 2018 American Medical Informatics Association Annual Symposium served as the setting for a half-day interactive workshop with diverse stakeholders to discuss proposed best practices for the planning, design, deployment, and evaluation of ePROs. We provide this collective commentary that synthesizes participant feedback regarding critical challenges that prohibit the scale and spread of ePROs across healthcare delivery systems, including governance and leadership, workflow and human factors, informatics, and data science. In order to realize the promise of ePROs at scale, adaptable approaches are critical to balance the needs of individual users with health systems at large.

ePRO: THE ANSWER/THE PROBLEM

Patient-reported outcomes (PROs) are a type of patient-generated data that provide clinically meaningful insight into screening, diagnosis, treatment response, and population health. Examples include improved recognition by clinical teams of chemotoxicity, comparing treatment decisions for osteoarthritis, and improved management of severe depression. PROs enhance the efficiency and patient-centeredness of clinical documentation and facilitate individualized patient care, a key goal of precision medicine. Traditional approaches to PRO data collection focus on paper-based workflows, yet healthcare policy, has prompted advancements in health information technology (HIT) to promote patient engagement and interoperability across electronic health record (EHR) systems. In response to changing healthcare and policy environments, many health systems have prioritized the electronic capture and presentation of PROs (ePROs), leveraging HIT (eg, EHRs, patient portals, third party applications, SMART on FHIR) to enhance patient-centered, personalized care.

However, ePROs have not necessarily been the silver bullet to scale the spread of PROs in clinical care to date. The ability to administer PRO surveys electronically resolves some challenges (eg, auto-reminders and distribution to patients to complete ePROs ahead of visits) and creates new opportunities for improving care delivery (eg, clinical and quality dashboards that present ePROs and clinical data collectively). Yet, ePROs can also amplify existing HIT barriers (eg, low patient portal enrollment and limited functionalities of EHR systems) and introduce others, in particular information overload for clinical teams. Although there is demonstrated value and increasing pressure to incorporate ePROs into clinical care, many health systems have met challenges when trying to bring ePROs to scale and balance the needs of individual users with the system at large. This is due in part to the complex interplay of technology, workflow, and human factors that influence the success of ePRO adoption, as well as the leadership and governance that ensures the sustainability of ePRO implementations.

As efforts to expand the use of ePROs grow, so does the need for collaborative forums where stakeholders and thought leaders can examine critical challenges that continue to prohibit the scale and spread of PROs. These challenges include governance, informatics resourcing, data science approaches, and strategic resource allocation. Such collaborative forums allow stakeholders from a variety of settings to share experiences with successes and failures, discover lessons learned, and identify common strategies as best practices that reflect the needs of diverse populations and clinical contexts. In this commentary, we report on proceedings from a half-day interactive workshop that focused on challenges and recommendations for integration of ePROs across health systems.

BRINGING ePRO STAKEHOLDERS INTO A COLLABORATIVE FORUM

As an extension of an Agency for Healthcare Research and Quality (AHRQ) funded project to identify best practices for the integration of ePROs in care delivery systems, our multidisciplinary research team led a half-day interactive workshop at the 2018 American Medical Informatics Association (AMIA) Annual Symposium. The goal of this workshop was to share ideas and discuss best practices around four key dimensions of ePRO implementation at the health system level: (1) planning, (2) designing, (3) deploying, and (4) evaluating ePRO use (For more detailed information about the workshop content, please review the workshop summary abstract on the AMIA 2018 Annual Symposium website (https://symposium2018.zerista.com/event/member/507887?embedded=1), or contact the corresponding author).

Approximately 100 participants attended the workshop representing 43 unique settings involved in health research and practice, including international representation. Workshop participants held a variety of clinical, administrative, academic, and government roles, with the majority reporting an average range of 1–3 years of experience implementing PROs in clinical care settings. In addition to presentations on the four ePRO implementation dimensions, the workshop provided multiple opportunities for interactive discussion on challenges and opportunities for ePROs. Throughout the workshop, participants engaged in conversation regarding ePRO best practices and shared their experiences with PRO use in the field.

To recognize key nuances in ePRO deployment in clinical settings, workshop activities were organized around three common use cases to compare and contrast how ePROs can be used in different clinical contexts: (1) preventive care (eg, screening for depression), (2) specialty and chronic care (eg, managing chronic pain symptoms), and (3) interventional and surgical care (eg, assessing mobility after total joint surgery). Although these use cases characterize common types of care decisions that are informed by PRO data, PROs are recognizably used in diverse clinical settings and for many different patient care purposes. During the workshop, participants focused on each use case during problem-solving activities to assess the varied ways PROs can support clinical care and decision making and to identify opportunities for standardization. Structured notetaking templates were used to capture participant insights related to (1) system level challenges, (2) patient engagement, and (3) provider engagement. Notes from small- and full-group discussions were compiled and analyzed using content analysis to synthesize recommendations and challenges that emerged throughout the day.

RECOMMENDATIONS TO SUPPORT EPRO

Recent years have shown increasing interest in understanding how to best advance the use of ePROs. The PCORI EHR-Working Group, the ISOQOL taskforce on implementing PROs in clinical practice, AHRQ Technical Expert Panel on opportunities and challenges for PROs and HIT, and EASI-PRO pilot demonstration are just some examples of the concerted efforts to identify best practices that support the translation of ePRO tools into clinical care. Yet, these efforts consistently cite unresolved barriers related to health system infrastructure, readiness of clinical users, and technical capabilities of EHR systems for ePRO use. These challenge areas were also echoed throughout our workshop presentations, discussions, and activities with specific feedback around: (1) leadership and governance, (2) workflow and human factors, and (3) informatics and data science. Table 1 summarizes the recommended strategies presented during the workshop and persisting evidence gaps requiring further study that emerged from workshop discussion.

Table 1.

Recommended strategies and persisting evidence gaps for ePRO use at scale

Area	Recommended strategies	Persisting evidence gaps
Governance and leadership	Assess stakeholder needs for ePROs Establish governance structures and ePRO culture Define PRO measure selection strategy that aligns with health system goals and priorities Understand the capabilities of technical platforms for ePROs	Which PRO governance structures are most effective for different settings or systems? What is the expected impact of ePRO use on care delivery and health system performance?
Workflow and human factors	Define ePRO workflows for various users Design workflows that improve data capture Align ePRO workflows with existing clinical environments Utilize change management strategies to support implementation and ongoing PRO utilization	What workflows best support ePRO use across different clinical settings and use cases? What are the most effective approaches for involving patients and care partners in ePRO workflow design and implementation? How can ePRO process metrics better inform training and implementation monitoring efforts?
Informatics and data science	Understand complex information needs across local (eg, clinical team) and system (eg, population health and payers) stakeholders Align ePRO reporting tools with workflow for clinical decision-making Identify opportunities for HIT to enhance the efficiency of ePRO capture and reporting	How can application programming interfaces (APIs) and emerging HIT tools bridge gaps in EHR system and/or portal functionalities for ePRO implementation? How can predictive analytics improve the efficiency and impact of ePRO use?

Leadership and governance

ePROs, if not governed thoughtfully at the health system level, may contribute to the onslaught of information that both patients and providers must manage as they try to personalize healthcare decisions. Health systems need to establish policies that thoughtfully govern the selection and use of ePRO measures across clinical contexts and create expectations for the responsibilities of clinical teams in review and response (including medicolegal considerations)., As such, the workshop reviewed key areas related to governance for developing repeatable and scalable models for the use of ePROs tools, including: (1) assessing stakeholder needs; (2) establishing governance structures and ePRO culture; (3) defining a PRO measure selection strategy; and (4) understanding the capabilities and limitations of technical platforms.

Workshop participants highlighted that health system governance for ePROs is still an emerging practice and drew on individual experiences to articulate contextual factors that can impact the success of ePROs and inform a system-wide ePRO governance strategy. Workshop participants identified that a critical function of governance is to provide leadership and communicate the value of ePROs. In addition to building a ‘culture’ for ePROs, governance structures create a platform through which continuous learning, feedback, and evaluation can take place. As local healthcare teams start to integrate ePROs into care delivery, governance can augment their work through the identification of opportunities for efficiencies or improvements at the system level. For example, governance teams could recognize the potential to synergize ePRO development efforts with ongoing patient engagement initiatives or address staff barriers to support ePRO workflows. Most importantly, governance teams are well poised to evaluate and disseminate learnings across diverse implementations so that the health system at large can continuously improve. However, workshop participants recognized that no single governance model will serve all health systems. Thus, participants articulated a need for research to describe the features or characteristics of governance models used to support ePRO implementation that best adapt and support goals for diverse healthcare settings.

Workflow and human factors

Successful adoption of technology requires seamless workflow alignment and integration to support the cognitive and physical work of clinical teams providing care. The workshop explored how ePRO workflows often vary across local clinical settings and provided recommendations for how health systems can facilitate efficient and effective ePRO implementations. This includes (1) defining workflow actions for all roles, (2) designing workflows to improve data capture (ie, reduce missing data), (3) aligning ePRO workflows with existing clinical workflows, and (4) utilizing change management strategies.

Small group discussions allowed workshop participants to identify how stakeholders could support ePRO implementations at the system level where diverse use cases need to be considered. There were some similarities across all ePROs use cases (ie, preventive, specialty, and interventional care) and recommendations that apply globally. For example, all use cases emphasized the need to minimize data missingness, and workshop participants recommended using multiple data collection modalities to ensure complete data capture. However, participant feedback also highlighted how clinical user perspectives can vary across the three use cases, warranting tailored implementation strategies to support training and adoption. For example, when ePROs were used for preventive care, workshop participants highlighted the importance of setting patient expectations for completing ePRO measures in preparation for every visit and setting provider expectations for appropriate responses to PRO scores that indicate the need for clinical action. When PROs were used for specialty and chronic care, workshop participants indicated that treatment plans are often individualized and consequently the need for ePROs (both content focus and cadence of deployment) will vary across stages of treatment and recovery. Last, workshop participants noted that when used for interventional and surgical care, ePROs may require concerted efforts to educate patients and providers about how best to leverage ePROs to augment clinical decision making and outcomes assessment over time.

While all use cases consistently reflected the need to have complete and efficient ePRO data collection, the workflows for how clinical teams responded to ePRO data varied significantly by use case. Additionally, the readiness of stakeholders to adopt ePROs into practice is also influenced by factors such as organizational policies, culture, and the availability of resources. Workshop participants therefore recognized that a “one size fits all” approach to ePRO workflow and training will not necessarily meet the needs of all stakeholders, further reinforcing the need to tailor implementation and training needs to local settings. As health systems increasingly collect data from patients, particularly outside of the clinical visit, they may need to adjust resources and policies to support data review and response workflows in new ways.

Informatics and data science

As ePROs add to the volume and variety of data that are introduced into clinical care, it is imperative for ePRO reporting tools to exemplify best practices for visual design. The workshop addressed the human-centered design of health system ePRO tools, including (1) understanding the complexity of information needs across local (eg, clinical team) and system (eg, population health) stakeholders, (2) aligning ePRO reporting tools with clinical decision-making, and (3) identifying opportunities for HIT to enhance the efficiency of ePRO capture and reporting. The workshop highlighted the importance of understanding the complexity of stakeholder information needs for ePRO reporting across such diverse goals as individual care decisions, population health monitoring, comparative performance assessment, and quality improvement needs.

Workshop participants identified additional opportunities for informatics and data science to enhance the effectiveness of ePROs use at scale. For example, increased use of standards for ePRO data storage, exchange, and score harmonization (ie, cross-walking ePRO scores from different measures to allow comparison within similar domains, such as quality of life or functional status) could facilitate the “measure once, cut twice” principle. The goal of employing these standards is to maximize the utility of ePRO data across the various reporting needs for clinical care, quality improvement, and population health. Additionally, as more robust ePRO datasets develop, this growing volume of ePRO data presents opportunities for algorithms to track and flag patients due for ePROs, and then predict and prompt appropriate clinical follow-up given ePRO responses and history. Both of these examples could minimize the burden or potential duplication in ePRO data collection and further align ePROs with clinical decision-making processes.

CONCLUSION

ePROs have significant potential to facilitate more patient-centered, personalized care by aligning healthcare decisions with patient experiences, preferences, and voice. Expanding ePRO integration in HIT introduces both opportunities and challenges, and requires health systems to think strategically about the needs across the organization to ensure efficient design within shared resources and diverse needs.

The 2018 AMIA ePRO workshop created a collaborative forum where ePRO thought-leaders and stakeholders shared experiences and learnings regarding ePRO use and opportunities to advance the field. In reflecting on the ePRO recommended strategies and evidence gaps discovered in this discussion, the workshop leaders called on research and practice to further explore how ePROs can: (1) promote patient empowerment and decision making; (2) improve patient-provider communication; (3) support care coordination across health settings; (4) enhance population health, and (5) play a key role in advancing research and practice around patient-centered care.

ePRO stakeholders participating in the workshop echoed a fundamental premise throughout the day—adaptable HIT systems are critical to balance the needs of large healthcare organizations with individual ePRO users. In order for ePROs to be successful at scale, governance, workflow, and informatics must all align to ensure ePRO tools are designed and deployed to provide actionable data at the right time to the right stakeholders. In this manner, health systems can integrate the patient’s voice into care delivery and further advance patient-centered, personalized care.

FUNDING

This project was supported by the Agency for Healthcare Research and Quality grants number R01HS023785 (PI: Lavallee) and number T32HS013853.

AUTHOR CONTRIBUTIONS

All authors (Austin, LeRouge, Hartzler, Chung, Segal, and Lavallee) have made substantial contributions to the design, analysis, and interpretation of this work. All authors have contributed to the drafting, reviewing, and approval of this work and agree to be accountable for all aspects of this work’s accuracy and integrity.

CONFLICT OF INTEREST STATEMENT

The authors have no competing interests to declare.