A parent-science partnership to improve postsurgical pain management in young children: Co-development and usability testing of the Achy Penguin smartphone-based app.

BACKGROUND: Young children are at risk for poorly managed pain after surgery, with significant negative consequence to their quality of life and health outcomes. Mobile applications offer a highly accessible, engaging, and interactive medium to improve pain assessment and management; however, they generally lack scientific foundation or support. AIMS: The aims of this study were to describe a successful parent-science partnership in the development and testing of Achy Penguin, a parent-developed iOS app to help assess and manage acute pain in young children, and to evaluate and refine the usability of Achy Penguin in young children with acute postoperative pain.
METHODS: Three cycles of iterative usability testing were conducted with 20 4- to 7-year-old children (M = 5.8 years) in hospital who had recently undergone surgery (n = 6-7 children/cycle). Semistructured qualitative interviews were analyzed using simple content analysis.
RESULTS: Feedback from children and further integration of evidence-based pediatric pain knowledge led to refinements in app pain assessment and management content, as well as app flow and functionality. Changes improved children's ease of use and understanding and satisfaction by simplifying language in app instructions and content, adding audio and pictorial instructions, and increasing the engagement, interactiveness, immersiveness, and general appeal of pain management strategies.
CONCLUSIONS: This article showcases the value of collaborative partnerships between various stakeholders (parents, app developers, and researcher/health care providers) to address gaps in pediatric pain care. The Achy Penguin app shows promise for improving pain assessment and management in young children, although further evaluation of app effectiveness and implementation is warranted.

When children undergo surgery, the potential for moderate to severe pain lasting days to weeks is high.[1-3] Poorly managed pain after surgery is associated with increased child behavioral problems, anxiety, school absences, and difficulty sleeping, eating, and drinking, as well as greater unplanned health care use and parent absence from work.[2-4] Younger children are particularly vulnerable to undertreated pain while in hospital[5] and may be at risk for greater opioid-related adverse effects after surgery.[6]

Managing postoperative pain remains challenging once home after hospital discharge. Identified barriers to effective pain management include parents’ difficulty in assessing their child’s pain, lack of knowledge and access to evidence-based pain management advice, and problems with pain medications (for example, child refusal, parental misconceptions or attitudes, difficulty accessing, or inadequate dosing).[1,2] Challenges to pain assessment are increased in young children, who typically lack the cognitive abilities needed to validly and reliably use existing self-report pain measures until 5 or 6 years of age.[7] Furthermore, the primary reliance on medications to manage pain after surgery misses an opportunity to increase children’s self-management through utilization of other effective physical and psychological pain management strategies.[8,9] The degree of pain experienced after surgery is greater than expected by children and parents,[2] further highlighting the need to better equip families with evidence-based pain management tools.

Smartphones offer a highly accessible, engaging, and interactive medium to improve pain assessment and management in children.[10,11] Recommended multimodal pain management after surgery includes pharmacological, psychological, and physical strategies, as well as education about pain and surgery.[8,12] Available smartphone apps for postoperative pain management have offered information about all of these strategies.[13] Yet, despite a growing abundance of available pain apps,[14,15] only a handful have been tested for self-reported pain assessment by children under 12 years old.[16-21] To our knowledge, no apps have focused on management strategies for pediatric postoperative pain.[13] Children as young as 4 years old find smartphone-based pain apps easy to use; however, they continue to experience difficulties self-reporting pain using electronic versions of existing pain measures (such as the Faces Pain Scale–Revised and Color Analog Scale).[18,19]

Research suggests that well-designed digital games, including those for health, can provide powerful interactive experiences that promote learning, skill building, healthy development, and positive health outcomes in children as young as 3 years old.[22,23] In order to do so, these games must be developmentally appropriate, evidence based, and designed and tested with young children and their families to ensure that they are fun and beneficial.[23] To date, empirical evidence for pain apps remains sparse, with minimal involvement of end-users in their development.[13-15] This suggests that any app designed to improve postoperative pain management in young children should consider their developmental abilities, involve multiple stakeholders in its design (such as young children, parents, app designers, health care providers, and researchers), and be empirically evaluated.

The purpose of this article is to (1) describe a successful parent–science partnership in the development and testing of Achy Penguin©, an iOS app for iPhone or iPad to help assess and manage acute pain in young children, and (2) present a study evaluating and refining the usability of Achy Penguin in 4- to 7-year-old children with acute postoperative pain. Usability assessment of the Achy Penguin app pain assessment and management features focused on ease of use and understanding, app flow and functionality, and patient satisfaction.

Materials and methods

Achy Penguin mobile application

Achy Penguin is a free mobile iOS application developed by Lesley Baker (For Jack and Jill, LLC; http://forjackandjill.com) targeted toward young children with acute pain. This innovative tool uses animal-based body maps to help children self-report pain location (selecting the corresponding section on body map modified to resemble a bear or monkey cartoon[24]) and pain intensity (cartoon bear or monkey faces scale illustrating different levels of pain modified from the Faces Pain Scale–Revised[25,26]). Children can select their preference of a bear or monkey. The app also includes four psychological pain self-management activities, including (1) gamified deep breathing where children blow into the microphone area of the device to move a sailboat, with progressive game levels; (2) an interactive visualization where children take a photo of their painful area and add white shimmering “Icy Magic” by tapping on the photo to pretend to freeze the area; (3) a brief guided progressive muscle relaxation story during which children pretend that they are a penguin while they tense and release different muscles throughout their body; and (4) distraction through an interactive “Pop-It” game or access to YouTube videos. Version 4.0 of the Achy Penguin is currently publicly available for free download on the Apple iTunes store for use on iPhone or iPad.

Development of the parent–science partnership

Lesley Baker’s inspiration to develop the Achy Penguin app came from her experience as a parent of a young child with pain. Lesley drew from her educational and professional background in computer science and technology to create a possible solution. In Lesley’s words:

Truthfully, I felt so helpless waiting for his pain medicine to work. I noticed if I queued up a movie on YouTube, he would calm down and watch it. It inspired me to look at other ways and strategies that could help him cope and incorporate them into an iOS app. I spent a lot of time researching different pain coping techniques online that I could incorporate into Achy Penguin. I wanted him to have a tool that didn’t just distract him but would empower him to do something about his pain. A tool that would teach him strategies that he could use even if he didn’t have the app in front of him.

In addition to her own research online about effective pain management strategies, Lesley connected with two child life specialists at the local children’s hospital (Seattle Children’s Hospital), who reviewed and tested her initial app prototype before it was first publicly launched through the Apple iTunes store.

The current partnership between Lesley and the research team at The Hospital for Sick Children in Toronto (led by Dr. Jennifer Stinson) was initiated when Achy Penguin was identified in a systematic review evaluating available pain apps.[15] Dr. Stinson’s research program focuses on improving outcomes in child health through technology, including apps for pain assessment and management in a variety of pediatric populations.[10,11,27] The chance to create this partnership between parent and app developer and researchers and pain experts emerged as a unique opportunity to facilitate greater access to evidence-based pain care for young children by addressing critical gaps with existing pain apps.[14,15] As such, this partnership set out goals of refining Achy Penguin with end-user input (that is, young children with pain who would use the app) and ensuring its foundation in evidence-based pediatric pain assessment and management strategies and planned empirical evaluation. Given the focus of Achy Penguin on acute pain and the challenges to assessing and managing pediatric pain after surgery,[1-3] the decision was made to conduct initial app testing with young children with postsurgical pain.

To meet these goals, the project team expanded the partnership to involve multidisciplinary experts in pediatric pain assessment and management (psychology, nursing, anesthesiology), as well as research project staff. As the app developer and owner, and as a parent, Lesley has been engaged as an equal member of the research team, including as co-investigator on grant funding and research ethics for the study, as well as all dissemination of findings. Design or content changes to the Achy Penguin app throughout the usability testing study described below were jointly determined.

Study design and participants

A qualitative user-centered design approach was used to conduct iterative cycles of usability testing and interviews with young children who have undergone outpatient surgery. Eligible children aged 4 to 7 years, able to speak and read English, and experiencing postoperative pain of any intensity (per self-report). Children were ineligible to participate if they had significant cognitive impairment that could preclude pain self-report or participation in a brief qualitative interview (per their health care provider).

Based on recommendations and previous usability studies,[28-30] two to three cycles of usability testing were planned for refinement of the app. A target of 10 to 21 participants was planned over the three iterative usability testing cycles, because five to seven participants per usability cycle is generally recommended in order to reach data saturation and ensure that no further usability issues are identified.[31] Changes to the app were planned after each testing cycle based on data analysis of participant feedback. A purposive sample method was used to ensure variability, participant age, and sex. This study was conducted at The Hospital for Sick Children with local institutional research ethics approval.

Study procedure

Children were recruited from day surgery units by a research assistant who was introduced by the unit charge nurse. If interested, further details were provided to children and their parents, who respectively provided assent and informed consent. Parents reported child demographic characteristics and relevant health information, as well as their child’s level of comfort using smartphones and mobile applications. Participating children then completed an individual usability testing session and interview in their hospital room lasting approximately 20 min. The child progressed through all aspects of the app in a stepwise manner with the guidance of the research assistant, including pain assessment (pain location and intensity) and pain management strategies (deep breathing exercises, freezing visualization, progressive muscle relaxation story, and distraction game and videos). Participants were asked to “think aloud” as they went through the app and comment on their likes, dislikes, and any difficulties using the app. Having participants verbalize their thoughts while they use the app is a common approach to usability testing of health-based technology.[32,33] General questions were asked about each section of the app (e.g., “What did you like best about the ‘how much do you hurt’ question?”) and probes (e.g., pain scale) to encourage participants to elaborate. Questions about ease of use (e.g., “What did you find hard about the games?”) and suggested change to the app (e.g., “What would you like to change about the games?”) were asked. Interviews were audio-recorded and transcribed verbatim with identifiers removed. Research assistants took field notes on children’s responses, as well as to record the length of time children engaged with each app feature (pain intensity, pain location, and pain self-management strategies). The app was shown to the first three participants on an iPhone 6 (screen size 4.7 in.); however, this was switched to an iPad Mini 4 (screen size: 7.9 in.) for all subsequent participants to increase ease of use in interacting with app content.

Data analysis

Child demographic characteristics and smartphone usage data were summarized using descriptive statistics. Transcribed usability testing interviews and research assistant field notes were analyzed after each usability testing cycle to inform necessary modifications to the app. These qualitative data were analyzed independently by two members of the research team using content analysis,[34] followed by discussion with other study investigators regarding specific changes to the app between usability testing cycles. Content analysis focused on fundamental qualitative summary descriptions of participants’ feedback, sticking closely to the participants’ own words.[34]

Results

Participants

Participant demographic characteristics and smartphone usage are summarized in Table 1. Twenty children aged 4 to 7 years old (M = 5.8 years; SD = 0.95) participated across three usability testing cycles (Cycle 1: n = 6; Cycle 2: n = 7; Cycle 3: n = 7). All participants had access to a smartphone at home and 85% of parents reported that their child was comfortable or very comfortable using a smartphone.

Table 1.

Participant demographics and smartphone use.

Demographics	Cycle 1 (n = 6)	Cycle 2 (n = 7)	Cycle 3 (n = 7)
Mean age (SD) in years	5.5 (1.0)	6 (1.1)	5.8 (0.70)
Sex, n (%)
Male	2 (33)	3 (43)	4 (57)
Female	4 (67)	4 (57)	3 (43)
Type of surgery, n (%)
Dental	0	0	0
Ear, nose, and throat	1 (17)	1 (14)	1 (14)
Orthopedic	1 (17)	1 (14)	2 (28)
Plastic	0	0	0
Urology	1 (17)	0	1 (14)
Other	3 (50)	5 (71)	3 (42)
Grade in at school, n (%)
Junior kindergarten	2 (33)	3 (42)	0
Senior kindergarten	1 (16)	1 (14)	1 (14)
Grade 1	2 (33)	3 (42)	4 (57)
Grade 2	1 (16)	0	2 (28)
Smartphone usage each week, n (%)
Not at all	2 (33)	1 (14)	0
Once a week	0	0	0
Two times per week	0	0	1 (14)
Three times per week	0	1 (14)	2 (28)
Four times per week	0	1 (14)	0
Five times per week	0	1 (14)	1 (14)
Six times per week	0	1 (14)	0
Every day	4 (67)	2 (28)	3 (42)
Time daily smartphone use, n (%)
Not at all	2 (33)	1 (14)	0
Less than 1 h	0	3 (42)	2 (28)
1–2 h	1 (16)	2 (28)	3 (42)
3–4 h	2 (33)	0	2 (28)
5–6 h	1 (16)	1 (14)	0
Smartphone use comfort level, n (%)
Not at all comfortable	1 (16)	1 (14)	0
A little comfortable	1 (16)	0	0
Comfortable	3 (50)	4 (57)	4 (57)
Very comfortable	1 (16)	2 (28)	2 (28)

Usability findings

Qualitative content analysis revealed three main topics surrounding app usability, including pain assessment (self-report pain intensity and location), app flow and functionality, and pain management content. Findings and corresponding app changes across usability cycles are summarized in Tables 2–4 with illustrative screenshots. See Table 5 for data on the length of time that participants engaged with each app feature.

Table 2.

Summary of usability issues and refinements to app pain assessment.

Testing cycle	Usability issues	App changes	Illustrative screenshots
Cycle 1	Pain intensity rating:● Animal faces depicting different pain levels were hard for users to understand	Pain intensity rating:● Animal facial expressions adjusted● Changed from four face scale (i.e., no hurt, little hurt, some hurt, lots of hurt) to three face scale (i.e., little hurt, some hurt, lots of hurt)	Pain intensity ratinga:
	Pain location:● Challenging for participants to indicate where they had pain on animal body map due to difficulty placing the indicator on correct body part and lack of discrete regions on body map	Pain location:● Removed the original red dot indicator● Added discrete regions to the body map and a larger surface area requiring less fine motor skills● Added a pinch-to-zoom feature allowing user to make selected body part larger	Pain location:
Cycle 2	Pain intensity rating:● Animal faces depicting different pain levels was still unclear and users were still unable to complete a pain self-report● Pain self-report question states: “Which bear shows how you feel,” which asks for an emotional response	Pain intensity rating:● Removed secondary frown lines on animal faces to aid in understanding● Modified pain self-report question to “Choose a face to show how much you are hurting right now” to better capture level of pain versus emotions	Pain intensity rating:
	Pain location:● No issues identified.	Pain location:● No changes made.	N/A
Cycle 3	Pain intensity rating:● No issues identified	Pain intensity rating:● No changes made	N/A
	Pain location:● Participants unaware of the pinch-to-zoom feature	Pain location:● Added visual cue of two magnifying glasses to indicate zoom-in and zoom-out feature of the body map	Pain location:

aThe brown face depicts which face has been selected.

Table 5.

Time engaged with app features.

Usability testing cycle	Pain intensity “How much do you hurt” (M, SD, range in seconds)	Pain location “Show where you hurt” (M, SD, range in seconds)	Self-management strategies (M, SD, range in minutes)
Cycle 1	44.2, ±22.9, 30–90	90.0, ±53.7, 30–180	7.7, ±2.7, 3–10a
Cycle 2	21.7, ±20.1, 10–60a	23.6, ±18.4, 15–60a	9.1, ±3.5, 6–16a
Cycle 3‡	24.7, ±18.2, 10–60	40.0, ±30.3, 10–90	14.0, ±1.8, 11–16

aData missing from one participant.

Pain assessment

Pain Intensity Scale

In Cycles 1 and 2, participants had difficulty identifying the level of pain intensity that the faces portrayed. One participant stated, “… It’s hard to tell without the words, cause it looks like, ‘Oh, I’m super tired,’ and this one looks like, ‘Oh, I’m fine.’ …” (Cycle 1, age 7, female). Changes to the facial expressions and a simplification from four faces to three faces were made for Cycle 3. In Cycle 2, one participant preferred not to complete the pain rating scale because he was not currently experiencing pain, and an initial screen was added assessing current pain intensity. Participants reported that it was easy to identify levels of pain intensity: “It was easy” (Cycle 3, age 6, male). Children spent less time on this app feature in later cycles, suggesting an increased ease of understanding (see Table 5).

Pain location

In Cycle 1, participants had difficulty accurately selecting the location of their pain on the body map because the function (dots) was too small and the body sections were unclear. Modifications were made in Cycle 2 to add discrete lines to indicate body sections and in Cycle 3 to add a zoom-in feature. This improved children’s abilities to accurately indicate pain location, as supported by less time spent completing this app feature (see Table 5).

App flow and functionality

App flow and aesthetics

The overall flow of the app was improved through usability testing cycles to ensure smooth transition between the app features and app ease of use. In Cycle 1, participants who self-reported “no pain” were still required to complete the pain scale and reported this as confusing. Based on participant feedback, Cycles 2 and 3 integrated a more customized algorithm based on child report of pain presence or absence. Specifically, if they reported no pain, children were no longer shown the pain rating scale or body map and moved directly to other app features.

App functionality

In Cycle 1, participants found the navigational icons and font sizes too small. In addition, the language was overly complex and difficult for children to understand. Changes made during Cycles 2 and 3 included increasing the size of app icons and text font, simplifying language throughout the app, and adding audio instructions to accommodate younger users who were not yet able to read.

Pain management content

Participants spent more time engaging with the app pain management content as modifications made the app more child-friendly and appealing through usability testing cycles (for example, increasing from average of 7 to 14 min engaging with pain strategies from cycles 1 to 3; see Table 5).

Gamified deep breathing

In Cycle 1, participants felt that the game was too long with five levels. This was shortened to three levels.

“Icy Magic” visualization. In Cycle 1, one participant chose not to complete the “Icy Magic” visualization because she was concerned that it would cause real freezing to her skin. Changes were made in Cycle 3 to modify activity instructions from “Freeze where you have pain” to “Freeze anything” to accommodate participants who self-reported no pain. Visual instructions were added and simplified for children to better understand how to engage with this activity. Furthermore, when children tapped “Icy Magic,” a shimmer was added to increase fun and imagination of the visualization.

Progressive muscle relaxation penguin story

In Cycle 1, participants reported that the story was difficult to comprehend, with one participant stating the story was “hard to understand” (Cycle 1, age 6, female). One participant in Cycle 2 chose to skip the story due to lack of interest. The story was revised for Cycle 2 to enhance the penguin theme of the app, simplify the language, slow the pace of the story, and add corresponding animated visuals. Participants subsequently reported greater understanding and benefit, for example, stating that “… the penguin story made me so relaxed, and made my ear feel better when it was kinda hurting” (Cycle 3, age 7, female) and that “I like that one because you can have a big stretch” (Cycle 3, age 6, male).

Distraction “Pop-It” game

Participants generally enjoyed this distraction game across all cycles. Based on participant feedback, minor modifications were made after Cycle 1 to add customization for children to self-select their preferred animal character while playing the game and to add visual icons allowing participants to control game speed. One participant stated that he liked the game because “… it has so much animals! I want to play it again. I think that game is so fun” (Cycle 2, age 4, male).

At the completion of his usability testing interview, one participant also reported perceived benefit of the pain management content, stating “… Before I was kinda feeling like … sad. But when I had this, I feel a little better” (Cycle 3, age 6, male).

Discussion

Achy Penguin is a mobile iOS application designed to improve acute pain care in young children. This article reports on the initial parent-led development of the app and resulting research team collaboration to provide an evidence-based foundation for the app and its content. A usability study was conducted with children aged 4 to 7 years to evaluate the app’s pain assessment and management features for ease of use and understanding, app flow and functionality, and user satisfaction.

Usability

As is typically required in usability testing of mobile health apps,[28-30] three cycles of iterative testing were conducted to identify and address all usability concerns reported by participants and as observed by study research assistants. Across areas of pain assessment, app flow and functionality, and pain management content, key changes improved the app’s suitability to young children’s cognitive abilities. Such modifications included simplification and use of more child-friendly language in app instructions and content and the addition of audio and pictorial instructions. Several refinements were needed to the newly developed self-report animal faces pain scale to ensure its comprehensibility to this younger age group. To increase the scale’s developmental appropriateness, a preceding simple yes/no question about the presence of pain or hurt was added, the number of faces was reduced from four to three, and refinements were made to the facial images themselves to increase construct validity by better capturing pain/hurt versus other emotions (e.g., sadness). These changes were informed by participant feedback and are consistent with guidance from previous empirical studies and systematic reviews of self-report pain intensity measures in young children.[7,18,19,21,26,35] Beyond improving children’s ease of understanding, additional changes to the psychological pain management strategies presented in the app from usability testing focused on improving each strategy’s level of engagement, interactivity, immersiveness, and general appeal. These are qualities that are likely tied to treatment mechanisms and resulting efficacy for reducing pain and distress.[9]

Public–science partnerships

The Achy Penguin app reflects one parent’s self-initiated efforts to improve pain care for her child by applying her own professional skills and expertise. Her drive, and arguably need, to do this highlights several critical issues in current pediatric pain care: most notable, the poor availability and translation of effective pediatric pain management information for parents,[36] as well as the continued underutilization of pain management strategies in pediatric pain care despite scientific support for their efficacy.[5] Partnerships between parents and researchers are emerging as a valuable strategy for addressing gaps in pediatric care in an empowering and inclusive manner[37-40] and have the potential to implement pain science into practice more effectively and efficiently.[41] Recent emphasis on patient-oriented research and patient engagement has shifted the contemporary health research landscape toward meaningful and active collaboration with patients and their families in the research process.[42,43] This involves members of the public (such as patients or parents) as partners in health research rather than as research subjects or participants.[44]

Although an ever-increasing number of apps are available for pain, most lack any scientific foundation or evaluation.[13-15] This is particularly problematic for health apps that are freely available to the public, who may understandably assume that apps will be helpful and based on credible information. The current research team and resulting study showcase the potential partnerships that can emerge to effectively address this problem. App developers and researchers alike should seek opportunities to create new partnerships involving other relevant stakeholders (such as parents, child app end-users, health care providers, and policymakers) to ensure that health apps are high quality, appealing, feasible, and effective. This user-centered approach of engaging children as app users throughout the development of mobile health and learning apps is recommended.[22,23,45,46] The parent–science partnership described here reflects close collaboration or leadership of members of the public in the research process; however, parents (or other nonresearcher stakeholders) can partner across a spectrum of engagement dependent on goals, interest, time, knowledge, and funds.[47]

Limitations and future directions for Achy Penguin

The usability study reported here completes one of several phases needed to rigorously evaluate Achy Penguin as a mobile health intervention.[45] Future research should assess the measurement properties of the newly developed self-reported, animal-based body map and faces pain rating scale. Although researchers have begun evaluating smartphone-based self-reports of pain in young children,[17-21] there remains a need for well-validated and reliable measures of pain intensity that are suited for children as young as 3 to 4 years.[35] Studies evaluating the feasibility and effectiveness of Achy Penguin should consider relevant patient-reported outcomes of pain and distress, assess continued engagement with the app after hospital discharge, as well as employ innovative research designs to simultaneously address implementation outcomes (such as acceptability, adoption, appropriateness, fidelity, penetration, and sustainability).[41,48] Effectiveness studies can contribute to the promising but somewhat limited research assessing psychological interventions for pediatric postoperative pain.[9]

Generalizability of the current study’s findings is limited by testing at only one pediatric center with a focus on children with acute postoperative pain. The pain assessment and psychological pain management strategies included in Achy Penguin could also be used by young children in other acute and chronic contexts, such as pain from medical procedures or from disease. Consistent with recommendations from the Canadian Paediatric Society regarding healthy use of screen time for young children, the app should be considered complementary to face-to-face pain management support from parents and health care providers and not a replacement.[49] The young children in this study were less verbose in their responses than might otherwise occur in qualitative interviews with older children or adults. As such, iterative design changes also drew from research assistant observations during testing. Younger children may be prone to increased social desirability when asked by adults to provide feedback and may have been less likely to verbalize negative feedback; however, verbal reports from participating children did identify and result in several app refinements.

Conclusions

This article presents the successful initiation and activities of parent–science partnership to refine and evaluate a parent-developed pain self-management app for young children. Results from the usability study led to changes in the app to ensure its integration of evidence-based pediatric pain content, as well as developmental appropriateness and appeal to young children with acute pain after surgery. A continued parent–science partnership will draw from diverse expertise to innovatively address gaps in current pediatric pain care. Future collaboration will focus on evaluating the Achy Penguin app for its measurement properties, effectiveness, and implementation. Overall, Achy Penguin shows promise as a mobile tool to improve pain assessment and management in young children.

Table 3.

Summary of usability issues and refinements to app flow and functionality.

Testing cycle	Usability issues	App changes	Illustrative screenshots
Cycle 1	App flow:● When launched, app immediately asked users to indicate where they were hurting and to provide a self-reported pain rating. Users with “no current pain” were confused	App flow:● To account for users with no current pain, the app algorithm was modified to launch with a screen asking “Are you hurting right now?” If the child indicates “no pain,” then it skips the pain rating scale● Reordered body map and pain scale	App flow:
	App functionality:● Navigational icons and fonts size were too small● Overly complex language used	App functionality:● Increased size of fonts and icons● Simplified reading level, minor word changes to make more child-friendly● Updated names of games to be more playful and child-friendly (i.e., “Deep Breathing” changed to “Belly Breaths”)	App functionality:
Cycle 2	App flow:● App algorithm does not capture range of pain levels that ausers can report (e.g., only captured “yes” or “no” when asked if in pain)	App flow:● Modified algorithm to provide users with three options when asked if they are hurting: (1) Yes and I would like tips to help, (2) Yes but I don’t want tips to help, and (3) No	App flow:
	App functionality:● No issues identified	App functionality:● No changes made	N/A
Cycle 3	App flow:● No issues identified	App flow:● No changes made	N/A
	App functionality:● Younger participants had difficulty because they are unable to read the text throughout the app● Participants did not understand what to do when the activities began	App functionality:● Added a landing page to each activity and audio instructions to the entire app● Increased pictorial instructions● Simplified activity instructions	App functionality:

Table 4.

Summary of usability issues and refinements to app pain management content.

Testing cycle	Usability issues	App changes	Illustrative screenshots
Cycle 1	Content:● Deep breathing activities too long (i.e., five levels)● Progressive muscle relaxation story was too difficult for participants to understand● In the “Pop-It” distraction game, the animal is preselected and cannot be modified	Content:● Shortened length of deep breathing activity (i.e., three levels)● New progressive muscle relaxation story was created based on a penguin theme to promote child-friendly engagement● Added option for users to select one out of six animals for “Pop-It” distraction game	Content:
Cycle 2	Content:● No issues identified	Content:● No changes made
Cycle 3	Content:● “Icy Magic” activity instructions confusing if the user selects “no pain” (e.g., users are still asked to “Freeze where you have pain”)● Users were not aware that they could speed up or slow down the rate at which bubbles with animals appear in the “Pop-It” distraction game	Content:● Updated “Icy Magic” to allow participant to “Freeze anything”● Added shimmer to “Icy Magic”● Added visual icons to indicate ability to speed up/slow down the “Pop-It” game in increments	Content: