Implementation and Feasibility of an Electronic Health Record-Integrated Patient-Reported Outcomes Symptom and Needs Monitoring Pilot in Ambulatory Oncology.

PURPOSE: Describe the feasibility and implementation of an electronic health record (EHR)-integrated symptom and needs screening and referral system in a diverse racial/ethnic patient population in ambulatory oncology.
METHODS: Data were collected from an ambulatory oncology clinic at the University of Miami Health System from October 2019 to January 2021. Guided by a Patient Advisory Board and the Exploration, Preparation, Implementation, and Sustainment model, My Wellness Check was developed to assess physical and psychologic symptoms and needs of ambulatory oncology patients before appointments to triage them to supportive services when elevated symptoms (eg, depression), barriers to care (eg, transportation and childcare), and nutritional needs were identified. Patients were assigned assessments at each appointment no more than once in a 30-day period starting at the second visit. Assessments were available in English and Spanish to serve the needs of the predominantly Spanish-speaking Hispanic/Latino population.
RESULTS: From 1,232 assigned assessments, more than half (n = 739 assessments; 60.0%) were initiated by 506 unique patients. A total of 65.4% of English and 49.9% of Spanish assessments were initiated. Among all initiated assessments, the majority (85.1%) were completed at home via the patient portal. The most common endorsed items were nutritional needs (32.9%), followed by emotional symptoms (ie, depression and anxiety; 27.8%), practical needs (eg, financial concerns; 21.7%), and physical symptoms (17.6%). Across the physical symptom, social work, and nutrition-related alerts, 77.1%, 99.7%, and 78.8%, were addressed, respectively, by the corresponding oncology health professional, social work team member, or nutritionist.
CONCLUSION: The results demonstrate encouraging feasibility and initial acceptability of implementing an EHR-integrated symptom and needs screening and referral system among diverse oncology patients. To our knowledge, this is the first EHR-integrated symptom and needs screening system implemented in routine oncology care for Spanish-speaking Hispanics/Latinos.

BACKGROUND

There will be an expected 22.1 million cancer survivors (ie, any person with a prior cancer diagnosis, from the point of cancer diagnosis forward) in the United States by the year 2030.[1] The experience of survivors across the cancer continuum (ie, from diagnosis to survivorship and end of life) is complex and frequently associated with decrements in health-related quality of life (HRQoL), chronic and debilitating symptom burden, and increases in patient needs including psychosocial, practical, and nutritional needs, among others.[2,3] Historically, as highlighted by the National Academy of Medicine, cancer survivors' psychosocial needs and practical concerns are usually not adequately addressed.[4,5] In response, organizations such as the National Cancer Institute, American College of Surgeons' Commission on Cancer (CoC), and the National Comprehensive Cancer Center Network (NCCN) have set forth mandated standards for psychosocial distress and symptom screening and management.[6,7]

Patient-reported outcomes (PROs) are assessments of health, including psychosocial distress and symptoms, that are directly reported by the patient.[8] Emerging evidence suggests that routine collection of PROs may facilitate responsive, patient-centered care, and are associated with better patient-provider communication, satisfaction with care, and clinical outcomes.[9-11] The use of technology to collect PROs has also facilitated the timely identification of symptoms thereby improving management and preventing adverse events among cancer survivors. For example, a trial by Basch et al[10] demonstrated that the routine collection of PROs, using a web-based platform, improved HRQoL and survival as well as decreased emergency room visits. There has been a recent push to integrate PROs in electronic health records (EHRs)[11-15] whereby integrating other health data and promoting real-time delivery and care coordination.[16-18] However, best practices are still unclear[16] and therefore, studies have focused on identifying barriers/facilitators and challenges/opportunities to implementation of EHR-integrated PROs.[12,14,15,19] Additionally, Garcia et al[11] demonstrated the feasibility of implementing EHR-integrated PROs collection as standard of care in routine ambulatory oncology care at Northwestern Medicine health care system.

Although there is emerging evidence for the routine collection of PROs in ambulatory oncology,[9,10,20-22] implementation guidelines and theoretical frameworks need to be further described to support adoption and scalability of PRO collection. Therefore, the aim of the current paper is to describe the implementation and feasibility and initial acceptability of an EHR-integrated PROs and needs screening and referral system using the Exploration, Preparation, Implementation, and Sustainability (EPIS) framework. Additionally, to date, research examining the routine collection of PROs in oncology has largely focused on English-speaking, non-Hispanic Whites, thereby limiting our understanding of these approaches among Hispanic/Latinos. This is despite the fact that past studies have documented disparities in HRQoL[23,24] and greater unmet supportive care needs[25-29] among Hispanic/Latino cancer survivors, particularly those who are primarily Spanish-speaking. Given the growing Hispanic/Latino population in the United States,[30] and to address the under-representation of Hispanics/Latinos in cancer survivorship research, this quality improvement initiative was implemented at the University of Miami Sylvester Comprehensive Cancer Center (SCCC) ambulatory clinic that serves a very diverse catchment area and therefore adapted for English and Spanish speakers. To our knowledge, this is first time PROs have been implemented in routine oncology care for Spanish-speaking Hispanics/Latinos.

METHODS

Participants and Setting

Initial meetings were held with SCCC leadership and chief physicians to select test clinics to pilot the initiative. Four test clinics were identified and ultimately, the gynecology oncology (GYN-ONC) ambulatory clinic was selected as the lead test clinic for pilot implementation because of its established clinic workflow, available resources, and strong research infrastructure as part of an academic health system. There are a total of four board-certified gynecologic oncologists who perform cancer surgeries, administer chemotherapy, oversee clinical trials, and do survivorship/surveillance care for women with gynecologic malignancies (ovarian/fallopian tube, uterine, cervical, and vulvar/vaginal). Annually, the GYN-ONC clinic sees approximately 1,200 new patients and nearly 9,000 total encounters (ie, visits). The study team met with the chief physician of the GYN-ONC clinic to attain final approval.

Procedures

The My Wellness Check initiative is a quality improvement project implemented in the SCCC GYN-ONC ambulatory clinic. To comply with CoC emotional distress screening standards, My Wellness Check was developed to assess symptoms and needs of patients with cancer in ambulatory oncology before their appointments to triage patients to supportive care services and to address elevated symptoms (eg, depression, anxiety, and fatigue), barriers to care (eg, transportation and childcare), and nutritional needs. To incorporate key recommendations made by patients on the initial acceptability and functionality of the initiative, initial design and implementation was informed by a focus group held with patients from the Patient Advisory Board (n = 9). In the focus group, an overview of the distress screening initiative rationale, workflow, utility, and potential future benefit to patients and health care operations was provided. Then, patients were led through a mock demonstration of completing their assessments in a secured test patient portal environment. The focus group was led by a qualitative methods expert, a cancer care delivery expert, and an information technology (IT) representative, and was audio-recorded for transcription and thematic analysis.

My Wellness Check uses the International Classification of Diseases, Tenth Revision, cancer-related diagnosis codes to identify patients via our electronic data warehouse and EHR systems (Epic). Patients scheduled to visit the GYN-ONC ambulatory clinic for a second or later visit were identified since first visits are generally focused on the diagnosis and treatment planning, which create high levels of anxiety that tend to resolve over time and at this point, as part of standard practice, patient distress is already addressed through various methods, including the Distress Thermometer[31] and Patient Health Questionnaire.[32] Patients were notified by Epic MyChart to complete the assessment via prior stated preferences (ie, via e-mail, MyChart patient portal message, or phone call). Assessments were available in English and Spanish and take approximately 8-10 minutes to complete via the patient's MyChart account (web or smartphone app). Patients were asked to complete these assigned assessments 72 hours before their next appointment. Patients who had multiple visits in a month completed only one PRO assessment within a 30-day period. If the assessment was not completed before their visit, patients were asked by the intake nurse to complete the assessment at the time of the visit via their own smartphone or clinic provided tablet/computer. Moderate or severe symptom elevations in fatigue, pain, and/or physical function as assessed by Patient-Reported Outcomes Measurement Information System (PROMIS) triggered alerts to the medical team within the EHR, whereas depression and anxiety alerts were routed to social work (described in more detail in the Measures section). Endorsement of any psychosocial or practical need, or a nutritional need, triggered best practice advisory (BPA) alerts to the social work or nutrition teams, respectively. Treatment teams were mandated to address the alert with a disposition (eg, referral, intervention, etc) within 72 hours of the alert (72-hour mandate was not in place for physical needs-pain, fatigue, physical function; Fig 1). Within Epic, providers could see the PRO measures administered, completion date, patients' responses to individual questionnaire items, PROMIS T-scores, corresponding severity threshold classifications (normal, mild, moderate, or severe), and score trends. Providers followed up with patients during clinic visits, by telephone, or through MyChart messages. Dispositions, BPA alerts, and clinician notification processes were developed in collaboration with Epic, IT programmers, SCCC GYN-ONC clinical providers, and supportive oncology services (social work/nutrition team members). The described initiative is part of a performance improvement project and was introduced as standard of care; therefore, it was deemed exempt from institutional review board review, and informed consent was waived.

FIG 1.

My Wellness Check initiative workflow. Patients with a scheduled second oncology visit and a confirmed ICD-10 cancer diagnosis who have not received the My Wellness Check assessment within the past 30 days are contacted via text, e-mail, or the patient portal (on the basis of their stated preferences) 72 hours before their appointment and asked to complete the assessments. Patients who do not complete the assessment before their scheduled appointment have an opportunity to complete it within the clinic visit at multiple touchpoints (eg, registration, waiting room, via intake nurse). Once the assessment is completed, the results are populated in real time in the EHR. Best practice alerts are generated and triaged as follows: (1) clinically elevated anxiety, depression, and/or endorsed practical and psychosocial needs (eg, transportation and stress management) generate an alert to Social Work/Cancer Support Services; (2) clinically elevated pain, fatigue, and/or poor physical function generate an alert to the oncology team; and (3) endorsement of nutrition needs generates an alert to the nutrition team. Alerts remain open until a disposition is coded (eg, treatment, referral, etc). The process repeats itself but not more than once every 30 days. CAT, computer adaptive test; EHR, electronic health record; EMR, electronic medical record; HRQoL, health-related quality of life; ICD-10, International Classification of Diseases, Tenth Revision; PRO, patient-reported outcomes.

Measures

Patients completed PROMIS computer adaptive tests (CATs) for depression, anxiety, fatigue, pain, and physical function[33]; a psychosocial (eg, need for stress management), practical (eg, assistance with childcare), and nutritional needs assessment adapted from NCCN Distress Thermometer Problem Checklist[31] and vetted with the GYN-ONC social workers and other clinicians; and the Functional Assessment of Cancer Therapy—General (7-item version; FACT-G7).[34] PROMIS CATs were fully integrated into Epic using the PROMIS Epic App, which is available through App Orchard, Epic's website for sharing apps with the user community.[35] Other PRO measures were incorporated into Epic in collaboration with corresponding key Epic and IT support personnel.

All PROMIS CATs were previously calibrated using item response theory for each domain, thereby tailoring future items to the patient's previously self-reported symptom severity scores.[36-38] PROMIS CATs are brief, reliable, and well-validated adaptive measures of various PROs that have been widely used in ambulatory oncology populations. PROMIS CATs generate T-scores and corresponding severity thresholds (normal, mild, moderate, or severe) on the basis of normative data from patients with cancer and the general population.[39] A T-score of 50 is the mean, and 10 is the standard deviation of the reference population. Furthermore, the FACT-G7 is the brief version of the Functional Assessment of Cancer Therapy[34] used to assess HRQoL.[34] Patient responses on the PROMIS CATs that generate T-scores at a moderate or severe level trigger an EHR in-basket alert, specifically ≥ 70 for pain interference and fatigue, ≥ 65 for anxiety, ≥ 60 for depression, and ≤ 30 for physical function.[11] Moderate or severe elevations in depression and anxiety were defined as emotional needs and trigger alerts to social work, whereas moderate or severe elevations in pain, fatigue, and/or physical function were defined as physical needs and trigger alerts for the medical team. Overall, individual alerts could be triggered for pain, fatigue, physical function, anxiety, depression, psychosocial/practical needs, and nutritional needs for a maximum of seven alerts possible per assessment. The FACT-G7 did not trigger an alert.

Approach to Implementation

Our work was guided by the EPIS framework, a well-established dissemination and implementation model. The EPIS framework is closely aligned with our overarching goal to create a multilevel, comprehensive model of care with My Wellness Check. The EPIS framework accounts for feedback and dynamic processes on the basis of the needs of the clinic setting while emphasizing minimal system burden. It allows for the ongoing identification and modification of key challenges to successful implementation over time, guided across four stages. In the Exploration Phase, a pressing need among patients, clients, or communities that requires attention is identified, or an improved approach to an organizational challenge is acknowledged. In the Preparation Phase, the barriers and facilitators to implementation of the chosen evidence-based practice (EBP) or innovation are assessed. In the Implementation Phase, the EBPs are initiated, and there is ongoing monitoring to determine adjustments that must be made. Finally, in the Sustainment Phase, the limiting or facilitating factors that support the ongoing delivery of the EBP are addressed to promote the public health impact.[40,41] In each stage, the outer system context (eg, policy and interorganizational relationships between entities) and inner organizational context (eg, leadership, organizational structures and resources, and individual adopter characteristics) are considered, and adaptations are made as necessary.[41]

Analyses

Descriptive statistics were used to report patient sociodemographic and clinical characteristics (ie, age, race/ethnicity, marital status, etc). Univariate statistics were used to assess the initial project metric of feasibility, which was defined through patient and provider engagement in accordance with Bowen et al.[42] On the basis of prior data from an EHR-based assessment of PROs among patients with cancer,[13] feasibility was defined as patient assessment initiation rates of at least 40%, including overall assessment initiation rate as well as by assessment initiation rates among Spanish and English speakers. The proportion of BPAs addressed by the corresponding health care professional was deemed to demonstrate adequate feasibility with a response rate of 70% or higher. Among assessments that were initiated, feasibility was defined as an overall completion rate of 70%. Initial acceptability, defined as overall satisfaction/suitability, perceived appropriateness, and perceived positive effects of My Wellness Check on care,[42] was ascertained through qualitative analysis of the Patient Advisory Board focus group. Audio recordings were transcribed verbatim, and qualitative analysis of the focus group was conducted using a thematic analysis approach.[43] This methodology is used to identify, analyze, and report themes within the data and helps to generate insights and interpretations of the data.[44] First, two researchers reviewed the transcriptions and began a thematic analysis by searching for common patterns and themes throughout the text. Second, transcriptions were uploaded to NVivo 12 Plus where parent (themes) and child (subthemes) nodes were identified. Some categories were consolidated, whereas others were expanded upon. The two researchers compared their analyses, discussed differences, and reached a consensus regarding generated themes.

RESULTS

Implementation Plan Analysis

The application of the EPIS framework across each phase of the implementation process of My Wellness Check is shown in Figure 2. A workgroup was established, which was composed of key stakeholders (eg, leadership, investigators, etc; Appendix Table A1, online only). Weekly reporting and fidelity reports informed the current feasibility and uptake of this initiative. As such, the intervention developers and implementers attained an understanding of the challenges and opportunities in implementing My Wellness Check. Considering clinic workflows and patient integration, further modifications were made to improve uptake and reduce patient/clinic burden. The barriers and facilitators to implementation of My Wellness Check were identified according to the phases of the EPIS model and are presented in Table 1. For example, ongoing in-clinic trainings and in-service demonstrations were conducted with nutrition, social work, and the entire SCCC gynecologic oncology staff. Additionally, ongoing collaboration with University of Miami Health System (UHealth; the health system under which SCCC operates) leadership, such as executive directors, clinician early adopters, administrators responsible for oversight, designated clinic workflow managers, and IT programmers, allowed us to gain full buy-in at multiple levels. Furthermore, the availability of institutionally sponsored Spanish language versions of patient materials across UHealth creates the ideal scientific environment to include a diverse, predominantly Spanish-speaking population in this initiative. A detailed description of the EPIS framework across each phase of the implementation process of My Wellness Check is provided in the Data Supplement (online only).

FIG 2.

EPIS framework for My Wellness Check. ACS, American Cancer Society; CoC, Commission on Cancer; EBP, evidence-based practice; EPIS, Exploration, Preparation, Implementation, and Sustainability; IOM, Institute of Medicine; NCCN, National Comprehensive Cancer Center Network; NCI, National Cancer Institute; PCORI, Patient-Centered Outcomes Research Institute.

TABLE A1.

Health System Stakeholders

TABLE 1.

Identified Barriers Throughout the Implementation Process and Associated Strategies

Qualitative Data

Qualitative data analysis from the mock demonstration (to the Patient Advisory Board) revealed four overarching themes, which included previous experience with the patient portal; ability to navigate the patient portal and questionnaires; initial acceptability and functionality of questionnaires; and content-related suggestions. Most participants had used the patient portal MyUHealth Chart before, and thought it was easy to navigate. Participants agreed that the My Wellness Check program was useful if feedback and referrals were obtained in a timely manner by their clinicians. Most preferred to complete assessments before their clinic visit. Addressing the lack of supportive care services during their cancer care experience was identified as a major concern that can be addressed by My Wellness Check.

Initial Project Metrics

Patient sociodemographic characteristics.

Between October 2019 and January 2021, a total of 834 individual patients were assigned an assessment, of which 506 (60.7%) patients initiated the assessment (ie, completed at least one questionnaire). Appendix Table A2 (online only) shows the sociodemographic characteristics of these patients who initiated the assessment. All patients were females, with a median age of 59.8 years (interquartile range: 49.7-67.7 years), and were primarily married (n = 277; 54.7%). This sample included 387 (76.5%) White, 74 (14.6%) African American, and 15 (3.0%) Asian/Pacific Islander women, of which 70.9% and 29.1% had an English and Spanish language preference, respectively, and 54.3% (n = 275) identified as Hispanic or Latino.

TABLE A2.

Sociodemographic Characteristics of Patients Who Initiated the Assessment

Survey assessment data.

A total of 1,232 assessments (an assessment consists of eight assigned PROs questionnaires: PROMIS CAT Depression, PROMIS CAT Anxiety, PROMIS CAT Pain Interference, PROMIS CAT Fatigue, PROMIS CAT Physical Function, Practical Needs Assessment, Nutritional Needs Assessment, and FACT-G7) were assigned in the target GYN-ONC clinic, of which 739 (60.0%) were initiated and 493 (40.0%) assessments were not opened (Table 2). Among assessments that were initiated, 70.2% were completed. Accordingly, 629 (85.1%) of initiated assessments were completed at home via the patient portal and 110 (14.9%) were completed in-clinic via the intake nurse/clinic staff. A total of 595 BPAs were triggered from 44.9% of initiated assessments (ie, 332 out of 739 initiated assessments triggered an alert), the majority were for nutritional needs (32.9%), followed closely by emotional needs (27.8%, ie, depression and anxiety), practical needs (21.7%), and physical needs (17.6%). In terms of nutritional needs, 23.7% had concerns regarding general nutrition counseling; 21.2% wanted information regarding vitamins, supplements, and herbs; and 16.7% noted difficulty losing weight or had unintentional weight gain. Additionally, the most noted practical and psychosocial needs included financial/insurance concerns (22.5%), coping with their illness and/or managing stress (17.6%), and overall general education and information (15.5%). Of the physical symptom, social work, and nutrition BPAs, 77.1% (n = 81), 99.7% (n = 293), and 78.8% (n = 154), respectively, were addressed by the corresponding GYN-ONC health professional, social work team member, or dietitian. Moreover, 43.4% (n = 127) and 86.4% (n = 133) of the social work and nutrition BPAs were addressed within the 72-hour time frame.

TABLE 2.

Survey Assessment Statistics

Among the total assigned assessments, 801 (65.0%) were in English and 431 (35.0%) in Spanish. For assigned English assessments, exclusively, 524 (65.4%) were initiated, of which 393 (75.0%) were completed. For Spanish, 215 (n = 49.9%) assessments were initiated and of those initiated, 126 (58.6%) were completed. In terms of mode of completion, 87.0% and 84.4% of Spanish- and English-initiated assessments, respectively, were completed at home via the patient portal. For triggered BPAs, there was a higher endorsement of emotional needs among those who completed the assessments in Spanish (31.2%) versus English (26.0%) but lower endorsement of nutritional needs (31.6% v 33.6% for Spanish and English, respectively).

DISCUSSION

Our goal was to implement My Wellness Check, an EHR-integrated screening and referral system, guided by the EPIS framework. Our results demonstrate feasibility and initial acceptability in implementing an EHR-based symptom screening tool in a culturally diverse ambulatory care setting while also operationalizing the implementation of the EPIS framework components used in the process.

My Wellness Check uses feasible measurement instruments that capture clinically relevant outcomes (eg, NIH PROMIS measures) in a highly ethnically/racially diverse sample of patients with cancer and survivors. For example, nearly 20% of our patient population sample consisted of African American women. Additionally, this sample consisted primarily of Hispanic/Latino women (ie, more than 50%), and about one in every four patients were predominantly Spanish-speaking. This quality improvement initiative was adapted from previous work, which demonstrated the successful implementation of distress screening in routine ambulatory cancer care using EHR integration.[11] To our knowledge, My Wellness Check is the first EHR-integrated platform to incorporate Spanish language PROMIS CATs. Historically, non–English-speaking populations have been excluded from such initiatives, given the lack of availability of Spanish-translated PRO measures. Therefore, we have built on past studies that have called for PROs to be integrated in the EHR in other languages, allowing patients to select their desired language.[45] Technology implementation was also a central approach in this model. Novel and emerging technologies such as use of the patient portal (My Chart) were implemented. My Wellness Check is fully integrated within Epic rather than using third-party applications. Together, all these components contribute to the larger scalability unique to this quality improvement initiative.

In our study, more than half (60%) of the total screening assessments assigned to patients were initiated. Similarly, almost 50% of assessments specifically assigned in Spanish were initiated. This modest response rate is similar to that observed in the study by Garcia et al.[11] However, to our knowledge, our study is the first to demonstrate uptake of PROs screening initiatives among Spanish speakers, thereby informing future efforts to reach racial/ethnic minority populations. Moreover, overall, 70.2% of initiated assessments (ie, 519 of 739) were completed, which suggests that patients who initiate the assessment are more likely than not to complete it. However, lower rates of initiation and completion were observed among Spanish speakers, suggesting targeted efforts are needed to increase engagement among this population. The majority, nearly 90%, were completed at home. Furthermore, 44.9% of assessments triggered a BPA, which is higher than previously observed for oncology outpatients.[11] These metrics, as a whole, demonstrate the feasibility of My Wellness Check.

Notably, almost 100% of the BPAs intended for the social work team members were addressed accordingly. This is crucial, given the fact that social work is responsible for addressing the majority of BPAs, which not only include practical and psychosocial needs (eg, financial/insurance concerns, coping with illness/managing stress, general education, transportation, etc), but also those pertaining to anxiety and depression. Past studies documenting provider perceptions on the EHR integration of PROs have shown that medical team personnel as well as social workers may experience task overload[45]; however, social workers reported reviewing PROs as part of their routine care and using results to guide upcoming patient clinic encounters.[45] In our pilot study, our goal was to minimize disruptions to clinic workflow, provider/patient burden, and maximize the use of clinically meaningful data by engaging with providers and stakeholders early on and having recurring meetings and trainings/demonstrations to incorporate feedback from GYN-ONC clinical providers, social work, and nutrition team members. Although there is currently no counteraction in place for providers who do not address alerts accordingly, weekly reporting/fidelity reports and educational newsletters were shared with providers in an effort to promote provider compliance. Given the lower rate of compliance with the 72-hour response mandate, future efforts should aim to increase the response time to BPAs.

Similar to past research,[11] the most commonly triggered alert at the UHealth GYN-ONC test clinic was for nutritional needs. Although studies have demonstrated that nutrition is an important concern for cancer survivors and that a high proportion experience diet-related problems and lack of access to dietitians,[46] it still remains an often overlooked component of cancer care. For example, in a survey of dietitians participating in the Oncology Nutrition Dietetic Practice Group of the Academy of Nutrition and Dietetics, 47% of cancer centers were not screening for malnutrition.[47] In accordance with this, there has been a call for the implementation of comprehensive nutritional screening and referral programs to dietitians in ambulatory oncology settings to identify and facilitate early management of nutritional concerns.[46] Aside from weight and muscle loss, overnutrition is also equally as important as indicated by our study results, with people wanting to know more about weight loss and weight gain. Therefore, our study informs efforts regarding the importance of nutritional needs among cancer survivors, including those of diverse racial/ethnic backgrounds.

Given the overlap between implementation science and quality improvement science, past research has demonstrated that to improve the quality of cancer care, methodology from both disciplines should be used in the research design phase.[48] The EPIS framework provided a structure for our scientific team to document the delivery and monitoring of the initiative in a systematic way.[40,41] EPIS has been previously used to successfully implement EBPs. The framework demonstrates high adaptability, given its application in a diverse array of settings and topic areas that include mental health, substance use disorder treatment, and child welfare.[41] In fact, EPIS has successfully guided the implementation of EHR-integrated distress screening initiatives at large, multisite health care centers.[49] In our initiative, during the Adoption and Preparation phase of the EPIS model, the study team had a meeting with stakeholders (eg, administrators, clinicians, and patient advocates) to explain the EPIS framework and attain initial buy-in. Additionally, our strategy targeted multiple levels of the delivery system, which included frequent meetings with key provider stakeholders (eg, GYN-ONC, nutrition, and social work) regarding logistics (eg, triaging), content build (eg, dispositions and needs checklist items), roles, training, and for educational purposes to create initiative awareness/increase uptake. Aside from EPIS, other implementation frameworks used previously in the integration of PRO measures include the Knowledge-to-Action framework, Consolidated Framework for Implementation Research (CFIR), Theoretical Domains Framework (TDF), and integrated Promoting Action Research in Health Services (i-PARIHS) framework.[50,51] Advantages unique to using EPIS include the emphasis on iterative strategies and inner/outer contextual factors throughout different phases of implementation, the latter of which is also true for CFIR.[52] Although frameworks such as EPIS place more emphasis on implementation activities (eg, continual trainings, stakeholder meetings, and informing EHR integration processes), future work incorporating/combining multiple models that can also emphasize evaluation such as Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) and CFIR as well as those used for widespread dissemination such as the Framework for Spread are crucial.[52] Adhering to a strategy informed by implementation science will facilitate our ability to accelerate translation, develop scalability, and promote sustainability of the My Wellness Check initiative.[48]

We note limitations to the implementation of this quality improvement initiative. My Wellness Check was first launched in the GYN-ONC clinic at SCCC, which limited our sample to women only. Previous research has noted differences in attitudes, reported needs (eg, dietary problems), and distress by sex in the cancer setting,[53] with women experiencing higher levels of health problems (including nutritional problems), depression, anxiety, distress, and lower HRQoL.[54,55] Additionally, adherence to distress screening protocols has been shown to be better for female patients with cancer.[56] Therefore, this could limit the generalizability of our implementation to other clinics and patient populations throughout the University of Miami Health System. Future research needs to identify differences in patient sociodemographic/clinical profiles in regards to completion of PRO measures.[57] Training documents have been developed to adapt the initiative procedures for implementation to other clinics systemwide, taking into account their specific workflow processes and needs. Moreover, protocols were modified, taking into consideration COVID-19 contingencies and new limited-contact clinic workflows. Full integration of PROs within the EMR and CAT technology is costly and complex, and may vary according to the current institutional infrastructures and additional interfaces needed; however, the costs to maintaining an independent PRO system may be higher.[16,58]

In conclusion, our results demonstrate promising feasibility and initial acceptability in implementing My Wellness Check within an ambulatory oncology clinic with a diverse patient population. In using the EPIS framework, strong institutional, administrative, and clinician support were instrumental to the successful implementation of this initiative. Although patient engagement was modest among both English and Spanish speakers alike, future systemwide implementation and marketing efforts can improve uptake. Future efforts focus on tracking the impact of this initiative on PROs, patient-identified needs, and system-level outcomes (eg, emergency room/urgent care visits, hospital admissions, days of stay, unscheduled visit count, etc). Additionally, the EPIS framework will be used to guide sustainability efforts. Data received from pilot implementation will be used to develop reports for presentations to clinical operations leadership and stakeholders to justify expansion into other test clinics. The results from the initiative can be used in the planning and implementation of a systemwide EHR-based symptom and needs screening, monitoring, and referral model. My Wellness Check data generated will be used to inform and guide patient-centered care, clinical decision making, and health policy decisions.