Feasibility of Tablet-Based Patient-Reported Symptom Data Collection Among Hemodialysis Patients.

INTRODUCTION: Individuals receiving in-center hemodialysis have high symptom burdens but often do not report their symptoms to care teams. Evidence from other diseases suggest that use of symptom electronic patient-reported outcome measures (ePROMs) may improve outcomes. We assessed the usability of a symptom ePROM system and then implemented a quality improvement (QI) project with the objective of improving symptom communication at a US hemodialysis clinic. During the project, we assessed the feasibility of ePROM implementation and conducted a substudy exploring the effect of ePROM use on patient-centered care.
METHODS: After conducting usability testing, we used mixed methods, guided by the Quality Implementation Framework, to implement a 16-week symptom ePROM QI project. We performed pre-, intra-, and postproject stakeholder interviews to identify implementation barriers and facilitators. We collected ePROM system-generated data on symptoms, e-mail alerts, and response rates, among other factors, to inform our feasibility assessment. We compared pre- and postproject outcomes.
RESULTS: There were 62 patient participants (34% black, 16% Spanish-speaking) and 19 care team participants (4 physicians, 15 clinic personnel) at QI project start, and 32 research participants. In total, the symptom ePROM was administered 496 times (completion rate = 84%). The implementation approach and ePROM system were modified to address stakeholder-identified concerns throughout. ePROM implementation was feasible as demonstrated by the program's acceptability, demand, implementation success, practicality, integration in care, and observed trend toward improved outcomes.
CONCLUSIONS: Symptom ePROM administration during hemodialysis is feasible. Trials investigating the effectiveness of symptom ePROMs and optimal administration strategies are needed.

Individuals receiving maintenance hemodialysis have high symptom burdens that negatively affect their health-related quality of life and dialysis care experiences.1, 2, 3 Patients often underreport their symptoms, and nephrologists tend to underestimate patient symptoms. Evidence from individuals living with cancer demonstrates that symptom assessment through routine patient-reported outcome measure (PROM) administration can improve patient-provider communication, symptom distress, health-related quality of life, and survival.5, 6, 7, 8, 9, 10, 11, 12 Conceptual frameworks synthesizing the existing evidence posit that PROMs support patient care through changes to patient-care team communication, detection of unrecognized problems, changes to patient behavior and clinical management, and improved patient experiences and health outcomes. However, in most dialysis practices, there are no standardized approaches for routine symptom collection outside of required annual health-related quality of life assessments.

Although there is growing interest in incorporating PROMs into clinical care, there are numerous perceived implementation barriers. Studies of nephrologists’ perspectives on PROMs have revealed concerns about patient ability and/or willingness to complete them, care team capacity to meet patient follow-up expectations, workflow disruptions, and uncertainty about optimal administration frequency and appropriate response thresholds for follow-up., In addition, paper-based questionnaires are the most common mode of PROM administration. However, ePROM capture may be more advantageous because of its capacity to (i) generate alerts to notify providers of problems, (ii) track data longitudinally, and (iii) facilitate integration of PROM data with the electronic health record.16, 17, 18 Existing data suggest that tablet-based ePROM capture is acceptable to individuals with kidney disease, including those receiving home hemodialysis.19, 20, 21 Less is known about ePROM administration to in-center hemodialysis patients, who experience greater burdens of cognitive dysfunction and comorbidities affecting dexterity.

We converted a paper-based symptom PROM to an ePROM and assessed its usability. We then conducted a QI project with research substudy to improve symptom communication and, simultaneously, assess feasibility of routine collection of ePROM-based symptom data at a U.S. hemodialysis clinic. In addition, we developed care processes to support routine ePROM administration in clinical practice. We used a mixed methods approach to assess symptom ePROM implementation feasibility and its potential to improve outcomes.

Methods

Overview

We executed a 2-phase project. In the first phase, we converted an existing, content-valid, paper-based symptom PROM to a tablet-based ePROM and evaluated its usability. In the second phase, we implemented the resultant ePROM system in routine care through a QI project and conducted a research substudy. We relied on principles of human-centered design for interactive systems to guide conversion of the paper PROM to an ePROM, and the Quality Implementation Framework to guide ePROM implementation.

Conversion of a Paper-Based Symptom PROM to a Tablet-Based ePROM

We converted a paper-based, dialysis-related physical symptom PROM with demonstrated content validity to a tablet-based ePROM using an agile software development approach. Agile methodology uses incremental, iterative cycles of development (sprints) to adapt the user interface to end-user needs, enhancing the end-technology’s effectiveness, efficiency, and usability in a real-world clinical environment.,, Usability testing, a component of agile methodology, evaluates how an individual responds to, understands, and navigates application questions, while capturing problems with the application interface, navigation prompts, question wording, and/or difficulties in question completion. First, we completed a series of 2-week sprints to identify end-user needs and develop the tablet user interface. Thereafter, we conducted 2 rounds of interviews and usability testing with hemodialysis patients, iteratively refining the interface in response to feedback. Participants completed the ePROM independently, and then research personnel reviewed the interface with participants using a think-aloud technique and verbal probing.

We recruited usability testing participants from 2 central North Carolina clinics (University of North Carolina Institutional Review Board 18-1531). Individuals were eligible to participate if they were ≥18 years old and received in-center hemodialysis for ≥6 months. We excluded individuals who were unable to read and converse in English and those with cognitive impairment (as identified by treating nephrologists). We used purposeful sampling to capture individuals of varying ages, symptom experiences, education levels, and comfort with technology, stopping recruitment on reaching data saturation. Participants received $20 remuneration.

Tablet-Based ePROM System Description

Usability testing resulted in the tablet-based ePROM system, Symptom Monitoring in Renal Replacement Therapy–Hemodialysis, SMaRRT-HD. SMaRRT-HD is a 14-item instrument that measures 13 symptoms (12 specific + free response) with 5-point severity Likert scales, and hours for dialysis recovery time (open-ended) (Supplement 1), available in both English and Spanish. The ePROM is administered during the first 30 minutes of hemodialysis and specifies a recall period of the last hemodialysis treatment for each symptom. The system sends designated care team members e-mail alerts for symptoms meeting prespecified severity thresholds at the time of instrument completion (Supplementary Table S1) and generates longitudinal symptom reports displaying reported symptoms from up to the last 8 ePROM administrations.

QI Project and Research Substudy

We implemented the SMaRRT-HD system as a QI project with the goal of improving patient-care team symptom communication. In addition, we sought to (i) assess feasibility of routine patient-reported symptom collection via an ePROM in a dialysis clinic (QI) and (ii) explore the effect of such data collection on patient-centeredness of care (research). We used a pre- and postproject design over 24 weeks, with a 4-week preproject period, 16-week intervention, and 4-week postproject period. A 7-member steering committee (patient, researchers, dialysis organization leaders, and implementation expert) supported the initiative.

The QI project was approved by the participating dialysis clinic’s leadership and was determined to be nonhuman subject research by the University of North Carolina Institutional Review Board (17-0193). We performed, analyzed, and reported the QI project in accordance with the Standards for Quality Improvement Reporting Excellence guidelines (Supplementary Table S2). The research substudy was approved by the University of North Carolina Institutional Review Board (19-0303), and participants provided informed consent.

Setting and Participants

The project took place in 2019 at a North Carolina hemodialysis clinic, a joint venture between the University of North Carolina and a large dialysis organization. All clinic hemodialysis patients who were able to respond to questions about their health and personnel (nurses, patient care technicians, and medical providers) were eligible to participate. Patients were informed about the QI project via waiting room signs and letters and asked to notify their care team if they desired to opt out (n = 0). All patient QI project participants were eligible to participate in the research substudy, except those lacking cognitive ability as identified by their treating nephrologist. Research recruitment methods included study informational signs, letter, and in-person recruitment by research personnel. Research participants received $25 remuneration.

Symptom ePROM Implementation

We relied on the Quality Implementation Framework, an implementation science framework, as the conceptual model for SMaRRT-HD implementation. The Quality Implementation Framework is organized into 4 temporal phases (Figure 1) and synthesizes 25 implementation models, focusing on actions constituting quality implementation in real-world environments. In phase 1 (host setting considerations), we assessed clinic needs, resources, and readiness for SMaRRT-HD implementation, optimized the SMaRRT-HD interface through the previously described usability testing, and built capacity by engaging with local stakeholders and fostering a supportive clinic climate. In phase 2 (creating a structure for implementation), we collaborated with clinic stakeholders to create an implementation plan and provided clinic personnel trainings. In phase 3 (ongoing structure), we iteratively updated the implementation plan to respond to end-user feedback and address encountered barriers. In phase 4 (improving future applications), we collected clinic stakeholder perspectives on barriers to and facilitators of long-term use and sustainability of SMaRRT-HD. At project start, the QI support team provided in-clinic assistance, gradually withdrawing support over time.

Figure 1

Quality implementation framework and related symptom monitoring in renal replacement therapy–hemodialysis (SMaRRT-HD) implementation strategies. The figure displays the 4 phases of the Quality Implementation Framework with associated activities from the SMaRRT-HD system implementation. aIn phase 1, clinic personnel received an overview of the project (rationale, objectives, timeline) in a 20-minute presentation at a routine monthly clinic personnel meeting. In phase 2, clinic personnel participated in 1 “lunch-and-learn” session during which they reviewed a draft implementation plan and provided feedback. Nurses (5) and patient care technicians (8) received a 1-time 5-minute individual training on how to administer the SMaRRT-HD electronic patient-reported outcome measure (ePROM) on the tablet by the research assistant. Medical providers (4) were e-mailed instructions for accessing the online system for longitudinal symptom reports. QI, quality improvement.

Data Collection

Overview

We collected data to assess SMaRRT-HD implementation feasibility, including acceptability, demand, implementation, practicality, integration, and limited efficacy testing (Table 1).

Table 1

Assessment of SMaRRT-HD implementation feasibility

Aspect of feasibilityDefinition29	Outcome assessed (data source)a	Results
Acceptability
How the intended recipients view the program	• Intent to continue use (interview)	• (+) Ongoing use by clinic
	• Perceived appropriateness (interview)	• (+) Appropriate; see Table 3
Demand
Perceived need for the program	• Fit within clinic culture (interview)	• (+) Fit; see Table 3
	• Perceived effects on clinic (interview)	• (+) Benefit; see Table 3
	• Actual use (ePROM metricsb)	• (+) Use; see Figure 3
	• Expressed interest or intention to use (interview)	• (+) Ongoing use by clinic
Implementation
Extent, likelihood, and manner in which the program can be fully implemented as planned	• Degree of execution (ePROM metricsb)	• See Figure 3
	• Amount and type of resources needed to implement (interview)	• (+) Fit with existing resources, minimal workflow disruption; see Table 3
Practicality
Extent to which the program can be delivered when resources and time are constrained	• Factors affecting implementation ease or difficulty (interview)	• (+) Implementation ease; see Table 3
	• Efficiency and quality of implementation (ePROM metrics,b interview)	• (+) Efficiency; see Figure 3, Table 3
	• Positive/negative effects on target participants (interview)	• (+) Effects; see Table 3
	• Ability of participants to carry out program activities (ePROM metrics,b interview)	• See Figure 3, Table 3
Integration
Amount of system change needed to integrate the new program into an existing infrastructure	• Perceived fit with infrastructure (interview)	• (+) Fit; see Table 3
	• Perceived sustainability (interview)	• (+) Sustainable; see Table 3
Limited efficacy testing
Use of a single clinic to assess shorter follow-up intermediate outcomes with limited statistical power	• Intended effects of program on key intermediate variables (interview, ePROM,c EHR,d symptoms)	• (+) Trend toward improved outcomes; see Table 3
	• Maintenance of changes from initial change (ePROM metrics,b interview)	• (+) Ongoing use; see Figure 3, Table 3

(+), area of focus affirmatively demonstrated; EHR, electronic health record; ePROM, electronic patient-reported outcome measure; PPPC, Patient Perception of Patient-Centeredness; SMaRRT-HD, symptom monitoring in renal replacement therapy–hemodialysis.

Pre-, intra-, and postproject implementation semistructured interviews were conducted with patients and care team members. Interviews assessed the clinic’s needs, resources, capacity, and support for implementing SMaRRT-HD; acceptability, perceived demands, and barriers to and facilitators of SMaRRT-HD implementation; and perceived sustainability, plans for ongoing use, and perceived effects of the project.

Proportion of completed SMaRRT-HD ePROMs, proportion of ePROMs requiring staff or quality improvement support team assistance, and time for ePROM completion, among others.

Modified PPPC Scale (Supplementary Table S2). The PPPC Scale is a valid and reliable 14-item instrument with 4-point Likert scales that assesses patient-centeredness of care.

Difference in missed treatments, shortened treatments, and hospitalizations in the pre- to postproject periods.

Table 3

Clinic stakeholder interview findings, responsive updates, and future recommendations

Key findings	Updates/Recommendations
Preproject
Existing clinical practices for symptom reporting and communication
• Clinic staff inquire about general patient well-being and perform a nonstandardized ROS before HD start. MDs discuss symptoms with patients intermittently; no standardized approach.	—
• Patients variably report symptoms spontaneously (severe or changing symptoms most common).
• Clinic staff and/or MDs may document symptoms but usually only severe or perceived high-risk symptoms. Documentation is nonstandardized, and clinic staff rarely review MD notes.
• No formal process for assessing longitudinal symptoms/symptom change or following up with patients.
Perceived importance of symptom reporting and communication
• Care team members and patients consider symptoms of high importance.	—
• Appreciate need for interdisciplinary approach to symptom management but acknowledge lack of formal approach.
• Recognize that some patients report symptoms more freely than others do, noting a small subset of patients who “never” report symptoms even when directly asked. Patients confirmed this.
• Viewed symptom discussions as potential gateways to more meaningful relationships with patients. Patients noted that standardized symptom reviews would demonstrate care team investment in their well-being.
SMaRRT-HD system
• No concerns about ePROM content (included symptoms felt to be most important and frequent).	—
• Care team and patients thought tablet-based approach would yield more complete symptom reporting.	—
• Care team opinions about administration frequency varied (weekly, every other week, and monthly were suggested).	• Administer weekly
• Clinic staff and MDs desired to receive SMaRRT-HD alerts and reports but worried about burden.	• Send alerts (in real time) and reports (every other week) to RNs and MDs
• Patients cited importance of care team follow-up about reported symptoms; they had variable interest in the reports.
SMaRRT-HD implementation
• Concern about patient ability to complete due to low vision, low literacy, dexterity, and/or cognitive challenges. Patients had similar concerns but most thought they would be able to learn to use the tablet after some initial assistance.	• Provide assistance at project start
• Concern about job duty/treatment interruption from ePROM administration.	• Administer at patient’s HD start
• Concern about infection control issues with shared tablets.	• Provide staff training
• Care team and patient concern about potential for inadequate follow-up of reported symptoms.	• Send alerts to clinic e-mail
Intraproject
Overall impressions
• Clinic staff and patients generally found the ePROM easy to administer and complete; minimal workflow disruptions.	• Continue use of ePROM
• Improved symptom awareness by care team, which was more pronounced for a subset of patients who tended to be more “withdrawn” per clinic personnel. Patients confirmed this.
• Greater symptom vigilance from care team, regardless of symptom reports. Patients confirmed this.
SMaRRT-HD system
• No concerns about ePROM content. Patients noted that intra-HD (vs. post-HD) symptoms most important to capture.	—
• (Week 5) Care team concern that alerts were too infrequent and important symptoms missed (iteration A).a	• Changed alert thresholds (B)a
• (Week 8) Care team concern that alerts were too frequent and difficult to handle without care disruption (iteration B).a	• Changed alert thresholds (C)a
• MDs found reports helpful, but often forgot to review; RNs found reports too long to be reviewed regularly.	• Provided printed reports to MDs
• Clinic staff concerned that MDs not following up with patients about symptoms.	• Provided printed alerts to MDs
• Reports were not shared with patients.	• Shared finding with care team
• Some patients tired of weekly administration and requested monthly. Staff also interested in monthly administration; however, many expressed concerns about missing symptoms and desired a longer recall period (1 wk).	• Changed to monthly administration with 1-wk recall
SMaRRT-HD implementation
• Most patients could complete ePROM on own after assistance with 1–2 administrations.	—
• Some PCTs preferred to assist patients for time efficiency (including some patients who could complete on own).	—
• Missed administrations due to lack of system for make-up ePROM when patient absent on planned administration day.	• Created checklist system
• Patient concerned that care team not responding to mild symptoms and non–fluid-related symptoms.	• Shared finding with care team
Postproject
Overall impressions
• ePROM raised awareness about symptoms and improved communication about symptoms and other topics.	• Continue use of ePROM
• Patients better understood importance of symptom reporting and more inclined to report symptoms (including on non-ePROM administration days). Patients who had not previously reported symptoms began reporting symptoms.
• Patients felt more informed about how their care team was trying to address their symptoms.
• Care team and patients valued being able to link symptoms to specific treatments (enabled follow-up and intervention).	• Use single treatment recall
SMaRRT-HD system
• ePROM includes the most important symptoms and is user-friendly (e.g., good balance of content and patient burden).	—
• Care team satisfied with alert frequency (iteration C).a	• Use iteration Ca
• Care team found reports for all patients not useful (too much information), but reports for patients with (+) symptoms were useful in establishing context and assessing longitudinal change/response to intervention.	• Perform targeted review of reports based on alerts
• Care team desired a formal process for obtaining input on symptom management from full interdisciplinary team.	• Incorporate into QAPI meetings
• Improved MD follow-up over course of project, but all thought this could be improved further with EHR integration.	• Link SMaRRT-HD to EHR
• Reports were not shared routinely with patients (patients interested in receiving a simplified report).	• Develop patient-friendly report
• Monthly administration not burdensome for anyone; however, all concerned about missed symptoms and frustrated by inability to link symptoms with treatments (resulting in extra work when RNs followed up to determine symptom timing).	• Use biweekly administration with single treatment recall + PRN
SMaRRT-HD implementation
• Minimal difficulty with ePROM system. Implementation most efficient when ePROM administered early in HD treatment.	• Administer early in treatment
• Care team acknowledged more frequent follow-up for fluid-related symptoms and expressed concern about lesser follow-up for more difficult to modify symptoms (e.g., restless legs, thirst).	• Develop suggested symptom management algorithms
• Patients generally were satisfied with follow-up, noting that they did not need “repeat” follow-up if no new actions/changes were recommended.	—

EHR, electronic health record; ePROM, electronic patient-reported outcome measure; HD, hemodialysis; MD, medical doctor; PCT, patient care technician; PRN, pro re nata (as needed); QAPI, Quality Assurance and Performance Improvement; RN, registered nurse; ROS, review of systems; SMaRRT-HD, symptom monitoring in renal replacement therapy–hemodialysis.

Semistructured interview data summarized and reported in aggregate to protect participant privacy. Clinic staff includes clinic manager, nurses, patient care technicians, dietitian, and social worker at the participating dialysis clinic. Medical providers include nephrologists, nephrology fellows, and nephrology advanced practice providers providing care at the participating dialysis clinic. Care team includes both clinic staff and medical providers.

Alerts refer to e-mail alerts generated at prespecified thresholds of symptom severity (Supplementary Table S6). Reports refer to summaries of longitudinal symptom data.

The thresholds for sending e-mail alerts to specified recipients were changed over the course of the project. E-mail alerts were sent according to the following paradigms: Threshold A: severe or very severe racing heart, chest pain, or shortness of breath or very severe cramping, nausea, vomiting, dizziness, thirst, headache, itching, restless legs, tingling, or write-in symptom or a new symptom reported as moderate, severe, or very severe that has not been reported over the past 3 administrations; Threshold B: mild, moderate, severe, or very severe racing heart, chest pain, or shortness of breath or moderate, severe, or very severe cramping, nausea, vomiting, dizziness, thirst, headache, itching, restless legs, tingling, or write-in symptom or a new symptom reported as moderate, severe, or very severe that has not been reported over the past 3 administrations; and Threshold C: mild, moderate, severe, or very severe racing heart, chest pain, or shortness of breath or moderate, severe, or very severe cramping, nausea, vomiting, dizziness, headache, tingling, or write-in symptom or severe or very severe thirst, itching, or restless legs (Supplementary Table S6).

Figure 3

Symptom monitoring in renal replacement therapy–hemodialysis (SMaRRT-HD) completion and assistance rates. The figure displays SMaRRT-HD completion and assistance rates (clinic staff and quality improvement [QI] support team, separately) as documented by the QI support team. There were missing data on staff assistance rates in weeks 2 and 16, as no QI support team member was in the clinic on at least 1 of the 2 days of electronic patient-reported outcome measure (ePROM) administration. The stars depict the 2 weeks when ePROM completion rates fell below 80%. During week 6, there was a clinic water issue on 1 of the 2 ePROM administration days, necessitating treatment stoppage, and during week 16, the tablets were out of charge at the start of ePROM administration on day 1.

Interviews and Observations

A trained interviewer conducted pre-, intra-, and postproject semistructured interviews with clinic stakeholders (patients, clinic personnel, and medical providers) to capture end-user needs, experiences, and recommendations for change. Interviews occurred in-person at the clinic, and responses were recorded on standardized note templates. Preproject interviews assessed clinic needs, resources, capacity, and support for implementing SMaRRT-HD. Intraproject interviews were conducted every 2 weeks to assess program acceptability, perceived demands, and barriers to and facilitators of implementation. Postproject interviews assessed perceived sustainability and effects of the program, as well as plans for ongoing use. We supplemented interviews with field observations.

Quantitative Outcomes

The primary quantitative QI outcomes were implementation-related measures including the proportion of symptom ePROMs completed, proportion of ePROMs requiring assistance, and ePROM completion time. Exploratory QI clinical outcomes included pre- to postproject change in shortened treatments, missed treatments, and hospitalizations. We collected data on symptoms and associated e-mail alerts throughout the project.

The research outcome was pre- to postproject change in patient-reported patient-centeredness of care as captured by a modified Patient Perception of Patient-Centeredness (PPPC) Scale (Supplementary Table S3), a valid and reliable 14-item instrument that measures patient-centeredness of care.30, 31, 32, 33 Lower PPPC Scale scores indicate perception of more patient-centered care, correlating with better emotional health and patient satisfaction.,,

Data Analysis

Symptom ePROM Usability Testing

During ePROM usability testing, 2 trained cognitive interviewers took detailed notes on standardized templates. Data were entered into Research Electronic Data Capture (REDCap) and organized by usability testing domain (i.e., navigation/use, understanding). We created overall data summaries in table format, collectively reviewing summaries and notes to confirm accurate data summation.

Symptom ePROM Implementation Interviews and Observations

Implementation interview data were entered into tables organized by content (i.e., SMaRRT-HD system, program implementation) and interviewee type (patient, clinic personnel, medical provider). Field observations were organized similarly. Throughout the project, 3 research personnel (JHN, MJT, and JEF) met every other week to review SMaRRT-HD implementation challenges and successes. We used thematic analysis and investigator triangulation (i.e., iterative discussions) to categorize semantically related concepts into common themes in the qualitative data.

Descriptive statistics (e.g., count [%], means ± SDs) were used to report participant characteristics, ePROM response and assistance rates, patient-reported symptoms, and clinical outcomes. We calculated pre- and postproject PPPC Scale scores according to instrument scoring instructions (mean of individual item scores).,, We used paired Student’s t-test to compare pre- and postproject means for PPPC Scale scores and clinical outcomes.

Results

Conversion to Tablet-Based ePROM and Usability Testing

See Supplementary Tables S4 and S5 for participant characteristics and complete findings from symptom ePROM usability testing. In brief, 13 patients (mean age 54 ± 17 years, 77% black, 31% with less than a high school education) participated. Of the 13 participants, 7 (54%) had never used a tablet before the interview. All participants displayed good understanding of the symptoms, recall period, and time-to-recovery question. In response to interview findings, we updated the interface to improve navigation and appearance. Specifically, we (i) removed the progress tracking bar to decrease clutter, (ii) replaced the auto-advance feature with a “next” button so the user could control screen advancement timing, (iii) added a pop-up keypad for recovery time to ease entry, (iv) changed the phrase “write in” to “type in” to align terminology with the administration mode, and (v) altered the dialysis machine graphic to make it more realistic.

Round-2 participants (n = 3) confirmed understanding of the symptoms, recall period, and time-to-recovery question. All were able to navigate the updated ePROM, including the pop-up keypad for recovery time entry. An 83-year-old woman with no prior tablet experience commented, “I caught on easily.” A 32-year-old man found the application “simple and easy.”

QI Project and Research Participant Characteristics

Table 2 displays participant characteristics, and Supplementary Figure S1 displays patient participant flow diagrams. At QI project start, there were 62 patients with a mean age of 61 ± 15 years and dialysis vintage of 6 ± 5 years; 21 (34%) were women, 21 (34%) were black, and 10 (16%) were Spanish-speaking only. There were also 19 care team participants (4 medical providers, 5 nurses, 8 patient care technicians, 1 dietitian, and 1 social worker). Overall, the 32 research participants had similar characteristics to the QI project patient participants.

Table 2

Participant characteristics

Characteristic	QI project	Research substudy
	Patients (n = 62)	Patients (n = 32)
Age (yr)	61 ± 15	62 ± 14
Female	21 (34)	10 (31)
Race
Black	21 (34)	19 (59)
White	20 (32)	13 (41)
Missing/unknown	21 (34)	0
Spanish-speaking only	10 (16)	6 (19)
Highest level of education completed
8th grade or less		5 (16)
Some high school but did not graduate		5 (16)
High school graduate or GED		15 (47)
Some college		6 (18)
4-yr college degree or more		1 (3)
Dialysis vintage (yr)	6 ± 5	6 ± 6
Diabetes	13 (21)	6 (19)
Heart failure	17 (27)	8 (25)
Cancer	2 (3)	1 (3)
Depression	1 (2)	1 (3)
Vascular access type
Fistula	39 (63)	22 (69)
Graft	9 (15)	4 (13)
Catheter	14 (22)	6 (18)
Prescribed dialysis treatment time (min)	228 ± 24	224 ± 21
Pre-HD systolic BP (mm Hg)	149 ± 23	143 ± 22
Nadir intradialytic systolic BP (mm Hg)	120 ± 20	117 ± 20
	Care Team (N = 19)a
Professional role (%)
Medical provider	4 (21)
Nurse	5 (26)
Patient care technician	8 (42)
Dietitian/social worker	2 (11)
Female	15 (78)
Race
White	15 (78)
Black	2 (11)
Other	2 (11)

BP, blood pressure; GED, graduate equivalency degree; HD, hemodialysis; QI, quality improvement.

Participant characteristics at time of QI project start. Values are listed as n (%) or mean ± SD. Demographic data about QI participants were obtained from the electronic health record, which has a high degree of missing race, whereas demographic data from research participants were self-reported.

The participating clinic was an average-sized central North Carolina clinic with 22 stations and a 12:1 nurse:patient ratio and 4:1 patient care technician:patient ratio.

Symptom ePROM Implementation Findings

Figure 2 displays the project timeline. We assessed clinic needs, resources, and readiness for SMaRRT-HD implementation and built capacity among clinic stakeholders through staff meeting presentations, individual interviews, and personnel trainings. Clinic personnel and the QI support team (MJT, JHN, and JEF) codeveloped an initial SMaRRT-HD implementation plan. Data from intraproject interviews and field observations informed iterative updates throughout the project.

Figure 2

Quality improvement (QI) project implementation timeline and data collection.The figure depicts the implementation timeline including changes in the symptom monitoring in renal replacement therapy–hemodialysis (SMaRRT-HD) system. Preproject data (interviews, clinical outcomes) were collected in the 4 weeks before and after the 16-week implementation period. Iterative changes were made in response to end-user feedback during the 16-week project (e.g., changes to symptom severity thresholds for e-mail alerts [weeks 5 and 8], recall period in question stem [week 8], and administration frequency [week 8]).

Stakeholder Feedback and Responsive Changes

Table 3 displays findings, responsive project updates, and future recommendations from the pre-, intra-, and postproject interviews. In brief, the clinic had no preexisting formal approach to symptom assessment but considered symptoms important. Care team members noted variability in symptom reporting across patients, recognizing a subset of patients who never to rarely reported symptoms, even when asked. The care team opted for weekly ePROM administration frequency with symptom alert e-mails sent to (i) a central clinic e-mail account accessed by all nurses and (ii) individual medical provider e-mail accounts. To minimize burden, the care team selected a higher symptom severity threshold to trigger e-mail alerts.

After project implementation, care team members reported e-mail alerts as too infrequent, leading to missed symptoms and inadequate follow-up. In response, the symptom severity alert thresholds were lowered to generate more alerts (week 5); however, care team members then found the alerts too frequent, and an intermediary threshold was implemented (week 8) (Supplementary Table S1). In addition, clinic personnel noted poor symptom follow-up by medical providers. To prompt this follow-up, clinic personnel provided printed e-mail alerts and longitudinal symptom reports for medical providers to use while rounding. Overall, patients found care team symptom follow-up acceptable but noted there was no follow-up for some symptoms, particularly itching and thirst. Similarly, 1 medical provider observed less follow-up for non–fluid-related symptoms. Clinic personnel confirmed this, acknowledging uncertainty about how to address such symptoms. All agreed that future implementations should include suggested care team guidance for symptom management; however, despite the challenges, all agreed that ePROM administration improved patient-care team communication by facilitating conversations on symptoms, a high-priority patient issue.

At week 8, clinic stakeholders suggested changing the administration frequency from weekly to monthly to decrease patient burden; however, all were concerned about missing symptoms with this approach, and requested concurrently changing the recall period from “last treatment” to “last week” to capture symptoms over 3 treatments. Monthly administrations with a 1-week recall period were thus used for the remaining 8 weeks of the project. Although this approach was less burdensome, clinic personnel were frustrated by their inability to link reported symptoms to specific treatments. To address this concern, on receiving e-mail alerts, nurses discussed the reported symptoms with patients to identify the associated treatment. The care team viewed the ability to link symptoms to individual treatments as essential to symptom management. As such, all concluded that a single treatment recall period was the preferred approach and ultimately recommended twice-monthly administration.

Although providing written longitudinal symptom reports to medical providers was helpful, all desired report linkage to the electronic health record for point-of-care accessibility. Patients and nurses requested simplified reports that omitted symptoms reported as “none,” citing a preference for fewer data to interpret. Finally, all agreed that reviewing symptom reports at monthly Quality Assurance and Performance Improvement meetings would better facilitate full interdisciplinary team input.

Symptom ePROM Implementation Outcomes

During the 16-week project, SMaRRT-HD was administered 496 times to 66 unique patients (398 weekly and 98 monthly administrations). The overall completion rate among patients present for treatment was 84% with varying completion rates across time (Figure 3). The completion rate was <80% at 2 administrations: week 6, 73% (clinic water problem on day 1 of administration) and week 10, 70% (uncharged tablets on day 1 of administration). Reasons for missed ePROMs included patient late arrival, sleeping, refusal, and busy with medical team; and clinic personnel forgot and were too busy. The system-recorded median [quartile 1, quartile 3] time to completion was 3 [2, 4] minutes. Actual completion times were usually shorter than system-recorded times, however, as tablets were often “started” and set down before being handed to the patient to complete.

In general, patients were able to complete SMaRRT-HD without assistance. Support with ePROM administration from the QI support team decreased over time (Figure 3). Reasons for assistance included poor vision (28%), patient care technician preference (23%), trouble with tablet (e.g., poor dexterity), hemodialysis machine alarming with arm movement or patient care technician concern that machine could alarm (19%), tablet unfamiliarity (10%), fell asleep (9%), requested assistance (6%), and isolation room utilization (5%). Patient need for assistance due to tablet unfamiliarity and requested assistance decreased to 0% by week 8. In some cases, patient care technicians preferred to help patients with ePROM completion regardless of patient ability, because they felt their assistance speeded administration.

The care team made numerous changes to patient management in response to SMaRRT-HD symptom reports. Example interventions included changes in target weight, dialyzer and phosphate binder prescriptions, and ultrafiltration rate; patient education about salt/fluid restrictions and thirst management; transition from profiled ultrafiltration to conventional ultrafiltration; and administration of saline and medications (e.g., antiemetic, antipruritic). Of these interventions, target weight changes were most common. In 1 case, a follow-up conversation about physical symptoms prompted patient disclosure of mood symptoms, resulting in depression treatment.

Clinical Outcomes and Patient-Centeredness of Care

Symptoms and Alerts

Figure 4a displays reported symptoms over the 16-week project, with thirst, cramping, and itching reported most frequently. Of the 495 ePROMs with complete data (a computer system error resulted in incomplete data on 1 ePROM), 121 (24%) had “none” for all symptom items, and 306 (62%) had no reports of symptoms above mild severity. Fifteen (3%) of the ePROMs (12 unique patients) had write-in symptoms. Reported write-in symptoms included back pain, chills, cough, diarrhea, and pain in both hands, among others (Supplementary Table S6). The time-to-recovery question was completed on 495 (99.8%) ePROMs, with a median [quartile 1, quartile 3] response of 2 [1, 4] hours and a range of 0 to 96 hours. Of the 66 patients, 64 (97%) reported at least 1 symptom during the QI project, and 49 (74%) reported at least 1 symptom at a severity of moderate or greater.

Figure 4

Symptom monitoring in renal replacement therapy–hemodialysis (SMaRRT-HD)–reported symptom severity findings (a) and triggered alerts (b). Over the 16-week implementation period, SMaRRT-HD was administered 496 times to 66 unique patients (398 weekly and 98 monthly administrations). (a) The number of times each symptom was reported and associated severity across the 496 electronic patient-reported outcome measures (ePROMs). (b) The number of completed ePROMs at each administration week and number of e-mail alerts that were generated at the 3 tested symptom severity thresholds (Supplementary Table S6).

The number of system-generated e-mail alerts for symptoms meeting the specified threshold ranged from 2 to 22 per week, depending on the alert threshold paradigm (Figure 4b).

Adherence and Patient-Centeredness of Care

Among the 55 patients with pre- and postproject data, the number (%) of patients having at least 1 unexcused hemodialysis absence, shortened treatment, and hospitalization declined from pre- to postprogram, but these declines did not reach statistical significance (unexcused hemodialysis absences, 14 [25%] vs. 11 [20%] patients [P = 0.5]; shortened hemodialysis treatments, 29 [53%] vs. 25 [45%] patients [P = 0.4]; hospitalizations, 5 [9%] vs. 2 [4%] patients [P = 0.4]). Among the 30 research participants with complete data, there was no change in pre- to postproject PPPC Scale scores, 1.3 ± 0.4 and 1.3 ± 0.4, respectively (P = 0.7).

Overall Feasibility Assessment

Overall, symptom ePROM administration was feasible as demonstrated by affirmative evidence of acceptability, demand, implementation, practicality, integration, and limited efficacy testing (Table 1). Specifically, qualitative data indicated perceived benefits from ePROM administration for both patients and care team members, and pre- and postproject quantitative data showed a non–statistically significant trend toward improved clinical outcomes. Data from interviews, observations, and ePROM completion and assistance rates suggested that SMaRRT-HD was practical and could be integrated into clinical workflows with minimal added burden. The system’s perceived value, overall feasibility, and potential for sustainability were underscored by the clinic’s decision to continue using SMaRRT-HD postproject.

Discussion

We demonstrated that symptom ePROM data collection during routine hemodialysis care is feasible. Our findings suggest that individuals receiving in-center maintenance hemodialysis are able and willing to complete ePROM symptom assessments during dialysis without clinical care interruption. Moreover, such data collection has the potential to improve patient-care team communication about symptoms and associated clinical outcomes. Notably, our findings underscore the importance of patient and care team engagement and flexibility when developing and implementing new clinic processes, such as ePROM data capture in the dialysis environment.

Collection of patient-reported symptoms is associated with improved patient-care team communication, symptom management, and health-related quality of life, as well as reduced hospitalizations and mortality among individuals with advanced cancer and those receiving palliative care.5, 6, 7, 8, 9, 10 ePROM-collected data can facilitate patient-care team discussions about symptoms, promoting shared decision-making and demonstrating care team appreciation for a patient-prioritized aspect of care. Such interactions can also strengthen therapeutic relationships, extending beyond symptoms. Moreover, symptom recognition facilitates interventions aimed at symptom alleviation, potentially improving treatment tolerance and adherence and subsequent clinical outcomes.

Interest in tailoring treatment plans to individual patient priorities has fueled interest in incorporating PROMs across the spectrum of kidney disease., Prior studies among individuals with kidney disease indicate that ePROMs may be usable by patients,,, but pragmatic implementation concerns remain., As such, it is necessary to rigorously study the impact of ePROMs on outcome and implementation strategies. Studying the two in parallel has the potential to expedite translation of effective clinical interventions into practice. Therefore, we first established the usability of SMaRRT-HD using an agile methodology approach and then evaluated its implementation using the Quality Implementation Framework, a framework that supports implementation through a series of steps including assessment, collaboration, monitoring, and self-reflection. We used capacity-building strategies, intervention fit assessments, and codevelopment of an implementation plan to increase likelihood of successful SMaRRT-HD implementation. Moreover, by obtaining early buy-in and iteratively integrating input from clinic personnel, we empowered them to take ownership and identify solutions to encountered challenges.

Recognizing that modifications are often needed to accommodate host settings, we allowed mid-project, stakeholder-informed changes. For example, we adjusted the symptom severity threshold for e-mail alerts twice to balance the need for real-time information with the associated follow-up burden. We also changed the administration frequency from weekly to monthly and the recall period from last treatment to the last week of treatments. Although these latter changes reduced burden, patients and care team members were frustrated by the associated loss of linkage between reported symptoms and specific treatments. In the end, these changes increased burden, as they resulted in the additional step of nurses asking patients to recall with which treatment the reported symptom occurred.

The desire to link symptoms to individual hemodialysis treatments led QI project participants to conclude that SMaRRT-HD may be optimally administered on a twice-monthly basis using a single treatment recall period. This also underscored the importance of linking symptoms to specific treatments for management considerations. For example, when patients reported cramping, nurses subsequently examined the ultrafiltration volume/rate and target weight–post-weight differential from the associated treatment, informing intervention selection. Finally, studying implementation highlighted the need for guidance about symptom management strategies. Care teams were less likely to follow up on symptoms such as itching and thirst than they were cramping and shortness of breath, acknowledging uncertainty about how to manage some symptoms. Provision of symptom management guidance algorithms may be helpful in future implementations.

Finally, our data confirm the previously reported burden of symptoms experienced by individuals receiving dialysis.1, 2, 3, It is striking that 97% of patients reported at least 1 symptom during the project, and nearly 75% reported at least 1 symptom at moderate severity or higher. Care team members reported feeling more informed about patient symptoms and made numerous responsive management changes. They also noted that communication about symptoms and other topics increased, even on days when SMaRRT-HD was not administered. However, there was no change in patient-reported patient-centeredness of care, possibly because of the clinic’s favorable preproject PPPC Scale scores. Our treatment adherence and hospitalization data showed nonsignificant trends toward improvement, suggesting potential for symptom ePROM administration to affect clinical outcomes, but these findings should be interpreted with caution given their exploratory nature.

Strengths of our project include use of an established implementation framework under which we engaged key stakeholders throughout, ultimately achieving clinic ownership of the new care process, and rigorous evaluation of both qualitative and quantitative outcomes. However, our project has limitations. First, we lengthened the recall period mid-project in response to stakeholder input, potentially introducing recall bias. As such, the symptom data should be interpreted with caution. In addition, findings may not transfer to environments with different implementation barriers and practice climates. For example, we conducted our project in a university-affiliated clinic in a rural setting with a substantial Spanish-speaking population, staffed by large dialysis organization–employed personnel who use corporate clinical algorithms. Preproject data suggest a positive practice environment in which most patients felt comfortable discussing their symptoms with the care team, despite having no established processes for symptom ascertainment, documentation, or follow-up. The positive preproject clinic environment may also explain the lack of change in pre- to postproject PPPC Scale scores. As implementation processes must be tailored to individual environments, we present our experience not as a recommendation for a universal implementation strategy, rather as an illustration of implementation science guiding process development in dialysis. Finally, this was a pilot feasibility project; we neither sought nor were powered to detect intervention effects on outcomes.

In conclusion, we demonstrated that routine symptom ePROM administration to in-center hemodialysis patients is feasible and has the potential to improve outcomes. We propose the next step in SMaRRT-HD evaluation is a randomized trial investigating its effectiveness at improving outcomes while simultaneously evaluating optimal implementation strategies to expedite its potential, future clinical practice uptake.

Disclosure

In the past 2 years, JEF has received speaking honoraria from American Renal Associates, American Society of Nephrology, Dialysis Clinic, Incorporated, National Kidney Foundation, and multiple universities, as well as investigator-initiated research funding related to and unrelated to this project from the Renal Research Institute, a subsidiary of Fresenius Medical Care, North America. JEF is on the medical advisory board to NxStage Medical, now owned by Fresenius Medical Care, North America, and has received consulting fees from Fresenius Medical Care, North America and AstraZeneca, Inc. In the past 2 years MMA has received honoraria from the International Society of Nephrology and investigator-initiated research funding unrelated to this project from the Renal Research Institute, a subsidiary of Fresenius Medical Care, North America. All the other authors declared no competing interests.