A Patient Safety Educational Tool for Patients With Chronic Kidney Disease: Development and Usability Study.

BACKGROUND: Chronic kidney disease (CKD) is a health condition that threatens patient safety; however, few interventions provide patient-centered education about kidney-specific safety hazards.
OBJECTIVE: We sought to develop and test the usability of a mobile tablet-based educational tool designed to promote patient awareness of relevant safety topics in CKD.
METHODS: We used plain language principles to develop content for the educational tool, targeting four patient-actionable safety objectives that are relevant for individuals with CKD. These four objectives included avoidance of nonsteroidal anti-inflammatory drugs (NSAIDs); hypoglycemia awareness (among individuals with diabetes); temporary cessation of certain medications during acute volume depletion to prevent acute kidney injury (ie, "sick day protocol"); and contrast dye risk awareness. Our teaching strategies optimized human-computer interaction and content retention using audio, animation, and clinical vignettes to reinforce themes. For example, using a vignette of a patient with CKD with pain and pictures of common NSAIDs, participants were asked "Which of the following pain medicines are safe for Mr. Smith to take for his belly pain?" Assessment methods consisted of preknowledge and postknowledge surveys, with provision of correct responses and explanations. Usability testing of the tablet-based tool was performed among 12 patients with any stage of CKD, and program tasks were rated upon completion as no error, noncritical error (self-corrected), or critical error (needing assistance).
RESULTS: The 12 participants in this usability study were predominantly 65 years of age or older (n=7, 58%) and female (n=7, 58%); all participants owned a mobile device and used it daily. Among the 725 total tasks that the participants completed, there were 31 noncritical errors (4.3%) and 15 critical errors (2.1%); 1 participant accounted for 30 of the total errors. Of the 12 participants, 10 (83%) easily completed 90% or more of their tasks. Most participants rated the use of the tablet as very easy (n=7, 58%), the activity length as "just right" (rather than too long or too short) (n=10, 83%), and the use of clinical vignettes as helpful (n=10, 83%); all participants stated that they would recommend this activity to others. The median rating of the activity was 8 on a scale of 1 to 10 (where 10 is best). We incorporated all participant recommendations into the final version of the educational tool.
CONCLUSIONS: A tablet-based patient safety educational tool is acceptable and usable by individuals with CKD. Future studies leveraging iterations of this educational tool will explore its impact on health outcomes in this high-risk population. ©Cassandra Bowman, Joseph Lunyera, Aviel Alkon, L Ebony Boulware, Jennifer St Clair Russell, Jennie Riley, Jeffrey C Fink, Clarissa Diamantidis. Originally published in JMIR Formative Research (http://formative.jmir.org), 28.05.2020.

Introduction

Chronic kidney disease (CKD) is characterized by progressive loss of kidney function and increases the risk of adverse patient safety events [1]. As kidney function declines, renal clearance is reduced, which can result in elevated and potentially hazardous circulating levels of medications. For example, individuals with CKD and concomitant diabetes who are on antidiabetic medications are at heightened risk of developing hypoglycemia due to the delayed renal clearance of these therapies [2]. Furthermore, as CKD progresses, susceptibility to hemodynamic changes or nephrotoxins such as nonsteroidal anti-inflammatory drugs (NSAIDs) is amplified, significantly increasing the risk of acute kidney injury (AKI) development and accelerated renal function decline [3,4].

Despite these well-established risks, patient awareness of CKD and of its potential safety hazards remains low, which limits the effectiveness of current CKD treatment strategies [5,6]. A high prevalence of limited health literacy among individuals with CKD [7] and poor patient-provider communication about CKD and its risk factors all contribute to low awareness and ineffective patient self-management among individuals with kidney disease [8]. Consequently, behaviors linked with CKD risk and progression, such as NSAID use, are commonly reported by individuals at high risk of adverse outcomes [9]; however, few interventions have been developed to address these challenges and attenuate risks.

We hypothesized that educational tools tailored to audiences with low health literacy can increase disease awareness and improve self-management among individuals with kidney disease. Therefore, we sought to develop a digital educational tool to promote patient awareness of relevant patient safety issues in CKD. Here, we describe the development of this educational tool; our experience usability testing its digital platform among the target patient population; and the results of our assessment of the users’ experience with the educational tool.

Methods

Educational Tool Development

This tablet-based educational tool was developed to support patient awareness of important disease-specific patient safety topics relevant for individuals with CKD. The tool was designed to evaluate participants’ awareness of common safety risks associated with reduced kidney function and to provide education regarding optimal patient-centered behavior to minimize risk. Materials were designed for low health literacy audiences, as it is estimated that the health literacy of approximately 25% of the general CKD population is inadequate [10]. The educational content was developed using Adult Learning Theory and plain language principles [11]. The andragogy theory of adult learning by Malcolm Knowles (Adult Learning Theory) postulates that adult learning is motivated by 6 principles: (1) the need to know, or “why” a person should learn; (2) a foundation or experience that provides the basis for learning activities; (3) a self-concept, or the idea that adults need to be responsible for and take charge of their learning; (4) readiness to learn stemming from perceived relevance of the knowledge to be gained; (5) orientation, or the concept that adults learn best with task-oriented learning that exercises their problem-solving ability; and (6) internal motivation [11]. The educational content of the tool draws on this framework to promote CKD awareness and self-management.

The educational objectives of the tool were based on patient safety topics relevant to individuals with reduced renal function based on prior work examining adverse safety events in CKD [1,12-14]. Educational content was created using previously derived educational materials addressing patient safety in CKD [15]; this content was reviewed and refined by adult educational curriculum experts using a rubric based on Adult Learning Theory [16]. The educational material encompassed four patient safety objectives pertaining to CKD: (1) avoidance of NSAIDs [17,18], (2) hypoglycemia awareness (among individuals with diabetes) [19], (3) temporary cessation of certain medications during acute volume depletion (ie, “sick day protocol”) [5,20,21], and (4) iodinated contrast risk awareness [22-24] (Table 1).

Table 1

Patient safety objectives, aims, instructional content, teaching strategies, and assessment methods of the developed tool.

Patient safety objective and aim		Instructional content	Teaching strategies	Assessment methods
NSAIDsa
	To distinguish safe pain medications for individuals with CKDb from unsafe ones	Description of common pain medications available over the counter, their common names/brands, and their safety for use by individuals with CKD	Auditory explanations with complimentary graphics that include photographs of common brands of pain medication and their safety for CKD, linked with a clinical scenario and questions regarding a hypothetical patient experiencing pain	Clinical scenario of a second hypothetical patient experiencing pain, interactive questions that use photographs of common brands of pain medications and ask the learner to distinguish between safe and unsafe pain medications
“Sick day protocol”
	To be aware of the symptoms of volume depletion and the potential danger associated with taking certain medications during volume depletion	Description of common scenarios linked with volume depletion (eg, diarrhea, vomiting, fever); list of different medications that can harm the kidneys if taken during volume depletion (eg, diuretics and ACEc inhibitors)	Auditory explanations with complimentary graphics that include photographs of medications to be withheld during volume depletion, linked with a clinical scenario and questions regarding a hypothetical patient experiencing volume depletion	Clinical scenario of a second hypothetical patient with volume depletion and interactive questions that ask the learner to choose which medications should be withheld during a “sick day”
Hypoglycemia
	To understand the concept of hypoglycemia and the elevated risk of hypoglycemia in individuals with CKD	Description of the link between CKD and diabetes, how diabetic therapies can lead to hypoglycemia, and common symptoms of hypoglycemia	Auditory explanations with complimentary graphics that include photographs of a glucometer with a low blood sugar reading, linked with a clinical scenario and questions regarding a hypothetical patient experiencing signs and symptoms of hypoglycemia	Clinical scenario of a second hypothetical patient with hypoglycemia and interactive questions that ask the learner to choose medications that could lead to hypoglycemia and steps that should be taken if the patient suspects they have hypoglycemia
Contrast dyes
	To understand how medical tests that use contrast dyes can further harm already weak kidneys	Description of common medical tests that use contrast dyes and emphasis on the importance of informing health care providers about their CKD	Auditory explanations with complimentary graphics that include descriptions of common medical tests, linked with a clinical scenario and questions regarding a hypothetical patient undergoing a medical test	Clinical scenario of a second hypothetical patient undergoing a medical test and interactive questions that ask the learner to distinguish between safe and unsafe medical tests for individuals with CKD

aNSAIDs: nonsteroidal anti-inflammatory drugs.

bCKD: chronic kidney disease.

cACE: angiotensin-converting enzyme.

The developed teaching strategies optimized human-computer interaction and content retention. The hypothetical experiences of 2 diverse patients (Mr. Smith and Mrs. Johnson), with Hispanic and African American backgrounds, respectively, were used to provide clinical context (Figure 1). Simple animations accompanied the audio presentations in real time and emphasized key concepts [11,25]. Visual images and audio were chosen in consideration of information clarity, participant involvement, and captions to meet participant comprehension needs [26,27] (see Multimedia Appendices 1 and 2).

Figure 1

Example of an educational tool vignette.

Usability Testing

We recruited 12 patients receiving care from outpatient CKD clinics at Duke University Hospital in fall 2016 and again in early 2018. Individuals who were non–English-speaking, illiterate, physically unable to use a tablet (ie, individuals with limb amputation, debilitating arthritis, or legal blindness), on dialysis, or under the age of 18 years were excluded. If interested, patients were taken to a private room, where written consent was obtained and testing was initiated. The study was approved by the Institutional Review Board of Duke University. All participants were compensated for their time.

Testing Protocol

The usability testing was based on the performance of the users and their ability to answer questions and navigate the educational tool on a tablet device. Participants were given a pair of headphones connected to an Android tablet (Samsung Galaxy Tab A, model SM-T550). Once seated comfortably, participants were asked to complete a series of tasks designed to evaluate their ability to interact with the technical aspect of the safety tool (Multimedia Appendix 3). All participants were asked to complete a minimum of 52 tasks and a maximum of 67 tasks based on their diabetes status. The study facilitator remained in the room with the participant and monitored the ease or difficulty with which tasks were completed as well as participant commentary about the activity. Task completions were categorized as no error, noncritical error, or critical error. If the participant had no difficulty completing the task independently, the task was marked as no error (easily completed). A noncritical error was defined as a task that was completed with an alternative strategy. For example, if a participant chose a wrong answer but made the correction before selecting “check my answers,” the error was categorized as noncritical. A critical error was defined as a task that could not be completed independently by the participant without assistance from study personnel. After completing the tool, participants were asked to evaluate their user experience with the tool.

Statistical Analysis

No formal hypothesis testing was performed in this analysis due to the qualitative study methods of usability testing. Participants’ demographic characteristics and their typical use of the internet and mobile devices were described using counts and percentages. Participant satisfaction using the educational tool and the tablet was recorded. Critical and noncritical errors were evaluated for each subject along with the time to complete each exercise. Errors and completion times were analyzed to recognize any significant differences between participants who regularly use the internet and mobile apps and participants who do not.

Results

Twelve participants took part in the usability testing of the patient safety educational tool. The majority of participants were aged 45 years and older (11/12, 92%) and were female (7/12, 58%) (Table 2). The races of the participants were diverse. All participants had at least completed high school and owned a mobile device that they used daily. Half the population owned a tablet they used for internet and mobile apps. Most participants used mobile apps (9/12, 75%) and the internet (8/12, 67%).

Table 2

Usability study participant demographics (N=12).

Characteristic		n (%)
Age (years)
	<45	1 (8)
	45-64	4 (33)
	≥65	7 (58)
Gender
	Male	5 (42)
	Female	7 (58)
Race
	Black/non-Hispanic	6 (50)
	White/non-Hispanic	5 (42)
	Hispanic	1 (8)
Education
	High school (grades 9-12) or GEDa	3 (25)
	Some college but did not graduate	4 (33)
	College	1 (8)
	Graduate or professional school	4 (33)
Mobile devices used
	Cell phone	12 (100)
	Tablet	6 (50)
Frequency of mobile device use
	Daily	12 (100)
Type of mobile device use
	Apps	9 (75)
	Internet	8 (67)

aGED: General Educational Development

Figure 2 illustrates user performance by displaying the numbers of critical errors, noncritical errors, and easily completed tasks out of the total assigned tasks. Out of 725 tasks, 15 critical errors (2.1%), 31 noncritical errors (4.3%), and 679 easily completed tasks (93.7%) were recorded. Of the 12 participants, 10 (83%) easily completed at least 90% of their tasks. The median time to complete all tasks was 19 minutes (SD 3.4 minutes, range 11-26 minutes). The median time for internet users to complete all tasks was 18 minutes (range 11-21 minutes); in contrast, participants who did not use the internet completed all tasks with a median time of 21 minutes (range 20-26 minutes). There was 1 outlier who completed all tasks in 11 minutes. The participants who used the internet had fewer errors than those who did not: 4/725 errors versus 42/725 (0.55% versus 5.8%, respectively). The 3 participants who did not use the internet or mobile apps were all aged 65 years or older. Five participants completed the testing without any errors, and 2 others completed it with only 1 error. Participants 4 and 8 had the most difficulty, with 7 and 30 errors, respectively. Both of these participants were over the age of 65, did not own a tablet, did not use the internet, and did not use mobile apps. All subjects completed more than 50% of the exercises without any errors, and 11/12 subjects (92%) completed more than 90% of the tasks without critical errors.

Figure 2

Usability testing results by participant.

User Experience Survey

The Likert scale used to evaluate user satisfaction using the tablet application is shown in Table 3. Participants were asked questions about their experience using the app and provided a ranking for the activity from 0-2 or 0-4, with lower numbers indicating higher satisfaction. The median and range for the ranking of the tool were used to determine the results. The median ranking for the overall activity was 0 (range 0-4), which describes the activity as “very easy.” Participants found the tablet to be very easy to use based on the median ranking of 0 (range 0-4). The length of time to complete the activity had a median ranking of 1 (range 0-2), indicating that the length was “just right.” The subjects ranked the patient story examples and audio as 1 (range 0-4), stating they agree that these aspects of the app helped them to understand the information. When participants were asked if they would recommend the activity to others, the median ranking was 1 (range 0-4), indicating that the participants agree they would recommend the app to others. Lastly, the participants were asked how they would rate the activity on a scale of 1-10 (10 being the best); they provided a median rating of 8 (range 0-10).

Table 3

Survey indicating user satisfaction with the patient safety tool.

Question	Response, median (range)
Overall, this activity was…a	0 (0-4)
The mobile tablet was…a	0 (0-4)
The length of time it took to complete the activity was…b	1 (0-2)
Using stories about patients helped me to understand the information.c	1 (0-4)
The use of audio helped me to understand the information.c	1 (0-4)
I would recommend this activity to others.c	1 (0-4)
How would you rate this activity?d	8 (0-10)

aPossible responses: 0=Very easy, 1=Somewhat easy, 2=Neither easy nor difficult, 3=Somewhat difficult, 4=Very difficult.

bPossible responses: 0=Too long, 1=Just right, 2=Too short.

cPossible responses: 0=Strongly agree, 1=Agree, 2=Neutral, 3=Disagree, 4=Strongly disagree.

dPossible responses: 1-10, where 1 is the worst and 10 is the best.

After completion of the activity, participants were given the opportunity to provide feedback and comments about the tool, such as what changes they would make, what they liked or did not like, and their overall thoughts about the exercise. Most comments reiterated the helpfulness and quality of the material learned using the app. Other comments suggested ways to improve the tool by stating that the exercise should be faster to navigate, while others expressed the need to simplify some medication names. The majority of participants found the educational tool to be a worthwhile experience that helped them learn about adverse safety events (Multimedia Appendix 4).

Discussion

The CKD educational tool was designed to help audiences with low health literacy learn about potential safety hazards associated with kidney disease through an interactive and simple-to-use digital interface. Our findings demonstrate that the majority of participants found the digital tool to be helpful and easy to navigate and would recommend it to others. Most participants were proficient in using the tablet without significant guidance and without errors; those with critical errors were older than 65 years and less familiar with digital technologies in general but still predominantly completed the programming without difficulty. The participants’ ability to complete the tasks did not appear to be related to their degrees of education, which supports our intent to provide an educational tool for populations with low health literacy regardless of educational background. Although some participants did not regularly use the internet or mobile apps, the task completion times in this group did not greatly differ from those of regular internet users. Together, these findings suggest that further testing to evaluate the impact of the CKD educational tool on outcomes is warranted.

CKD places individuals at high risk for developing adverse safety events, and effective tools that attenuate this heightened risk are needed. The principal feature of CKD that can influence patient risk is the reduced level of renal function or glomerular filtration rate (GFR), which is typical of the condition. A low GFR affects the clearance of many drugs, confounds therapeutic interventions, and increases susceptibility to acute kidney injury [28]. Moreover, CKD is frequently associated with several comorbidities, such as diabetes and cardiovascular disease [28]. The clustering of these health conditions leads to increased inpatient and outpatient medical encounters, with potential consequences of polypharmacy and high self-management burden. Compounded by low patient awareness of kidney disease [29] and poor knowledge of contributory health behaviors [5,30], potential adverse outcomes in CKD are abundant.

Our previous work demonstrated high acceptance of digital tools and general ease of use among individuals with CKD, supporting mobile health (mHealth) as a feasible mechanism through which patient-centered programming can be delivered to CKD populations [31-33]. Other studies using mHealth tools in non-CKD populations have demonstrated similar results for provision of tailored educational programming across an array of health conditions, such as obesity, asthma, and diabetes [34-37]. For individuals with CKD, responsiveness to the high degree of limited health literacy in the CKD population [10,38] is fundamental to the development of any educational curriculum, and use of audio and visual support can address differences in learning styles. We developed our app using plain language principles to address this need. Similarly, user experience testing of such digital tools in the CKD population can unearth interface difficulties by users and inform refinements to optimize user understanding and retention. The importance of such assessment is evidenced by one recent study that systematically evaluated all available and updated patient-facing CKD mobile apps. Of the 174 unique applications found, only 38 were related to kidney disease, were patient-facing, and had been updated in the previous 4 years. The quality of app content widely varied, and the authors found high levels of discordance between patient and provider app reviewers regarding value and usability [39]. These findings highlight the need for digital health content that is developed with a patient-centric approach and tested for ease of use in the target population prior to implementation in clinical care and broad dissemination. Similar content should also be made available through alternative platforms, such as on paper or via a website, to address differences in digital readiness and access.

Empowering patients with proper information and education in a manner they can comprehend can improve self-efficacy and self-management and, in turn, improve health outcomes [40]. The patient-actionable tool we developed emphasizes the importance of a patient-centered approach to educating CKD patients about self-care at home. Further, developing digital tools for patient education can bridge the digital gap between younger and older generations. When provided with online tools to manage chronic illnesses, older individuals access the digital content more frequently and sustain its use over longer periods than their younger counterparts [41]. This engagement by older populations is particularly relevant in CKD, as its onset typically occurs later in life. Digital tools can also narrow disparities in accessibility between rural and urban CKD populations, especially with the widespread availability of smartphones and the mobile web as well as the increasing use of telehealth applications for clinical care [42,43]. However, in kidney disease, few tools have been appropriately developed for use in a low health literacy population or have undergone formal usability testing. Fewer still tools are available outside of the research setting, which creates opportunities for such tools to be integrated into clinical care and evaluated in pragmatic studies [44,45].

Our study has limitations that are worthy of mention. Although our sample size is within the recommended range of 5-7 participants for usability testing [46,47], our sample size is small, and we recognize that this limits the inferences was can draw from our findings. Further, our participants were recruited from the Duke University Hospital nephrology clinics and may not be representative of the general CKD population, particularly given that only half our study participants were elderly. We also restricted our usability testing to individuals with CKD, which may not be generalizable to individuals with other kidney-related (eg, acute kidney injury) or non–kidney-related conditions. Finally, the specific patient safety conditions included in our educational curriculum may not be comprehensive, although they do represent many common patient safety events reported and detected in patients with kidney disease [1,48].

In summary, usability testing of a patient-centered digital tool to promote safety among patients with CKD demonstrated general ease of use and acceptability. The impact of the intervention on mitigation of safety events in a CKD population has yet to be determined. However, the information derived from this study strengthens the growing consensus that tailored digital tools can be used effectively by high risk populations, including older persons and persons with low health literacy. Future studies are needed to evaluate the benefit of such tools in clinical care.