Remote after-care using smartphones: A feasibility study of monitoring children's pain with automated SMS messaging.

BACKGROUND: Monitoring children's recovery postoperatively is important for routine care, research, and quality improvement. Although telephone follow-up is common, it is also time-consuming and intrusive for families. Using SMS messaging to communicate with families regarding their child's recovery has the potential to address these concerns. While a previous survey at our institution indicated that parents were willing to communicate with the hospital by SMS, data on response rates for SMS-based postoperative data collection is limited, particularly in pediatric populations. AIMS: We conducted a feasibility study with 50 completed pain profiles obtained from patients at Perth Children's Hospital to examine response rates.
METHODS: We collected and classified daily average pain (0-10 parent proxy score) on each day after tonsillectomy until pain-free for two consecutive days.
RESULTS: We enrolled 62 participants and recorded 50 (81%) completed pain profiles, with 711 (97.9%) of 726 requests for a pain score receiving a response. Two families (3%) opted out of the trial, and 10 (16%) were lost to follow-up. Responses received were classified automatically in 92% of cases. No negative feedback was received, with a median (range) satisfaction score of 5 on a 5-point Likert scale (1 = very unhappy, 5 = very happy).
CONCLUSIONS: This methodology is likely to generalize well to other simple clinical questions and produce good response rates in further similar studies. We expect SMS messaging to permit expanded longitudinal data collection and broader investigation into patient recovery than previously feasible using telephone follow-up at our institution.

What is already known about the topic

Families of children in the hospital report willingness to communicate with the hospital about their child's postoperative condition via SMS. Using SMS to distribute online surveys for follow‐up has been reported in one study which demonstrated high levels of acceptability.

What new information this study adds

The use of interactive SMS to collect longitudinal postoperative pain scores was well accepted by parents, with a large proportion (81%) replying each day until the pain had resolved completely. This methodology required substantially reduced researcher time compared to conventional telephone follow‐up.

INTRODUCTION

Postoperative pain is well established as a significant issue in the perioperative care of children. Routine care, research, and quality improvement often include monitoring how patients recover at home and how their pain resolves over time. Collecting this data is laborious and time‐consuming, particularly for longitudinal studies. Historically data collection for these activities has been completed by telephone in our institution with high patient attrition rates.

In other institutions, a wide variety of technological interventions have been studied including interactive SMS messaging, online surveys, telephone calls, , and smartphone applications. , , One study measured postoperative pediatric pain scores using SMS and found it to be reliable with a high response rate of 88%. This study messaged parents once a day for 14 days and manually collected responses, without automated alerting or interactivity.

We have previously identified a scope for technical intervention to streamline our data collection processes and found interactive SMS messaging as having the potential to make data collection more efficient and less intrusive for families, with broad availability across socioeconomic groups. In this setting, interactive SMS messaging describes sending a question via SMS directly to parents and collecting data with automatic classification of responses and predefined automatic replies sent when criteria are met. This has the potential to replace telephone calls as a data collection approach to postoperative pain in children. Our results from surveying 642 families of children who had received care at our hospital gave strong support for using interactive SMS messaging in our patient population, with 93.2% of families happy to receive SMS follow‐up. A previous study of follow‐up in the pediatric emergency department setting also found widespread acceptance and positive attitudes toward electronic follow‐up including SMS.

We conducted a feasibility study using automated interactive SMS messaging to collect pain scores in the postoperative period. Tonsillectomy is a common pediatric surgical procedure which is well recognized as being associated with significant postoperative pain and an area of concern for pediatric anesthetists. We have previously assessed parent‐proxy daily average pain following this procedure at our institution using telephone follow‐up. , We aimed to confirm support for SMS messaging follow‐up in our population, and determine the feasibility of automated collection of variables with minimal researcher input compared to our previous studies.

METHODS

This study was reviewed and approved by the Child and Adolescent Health Service Human Research Ethics Committee (RGS0000003157), recognized by the Human Ethics Office at The University of Western Australia (RA/4/20/5606), and registered with ANZCTR (ACTRN12619001028190). The ethics approval included the use of an opt‐out methodology.

Study design

We recruited patients from Perth Children's Hospital aged 0–16, who had undergone tonsillectomy and where the mobile phone number of the child's parent or guardian was known. Recruitment with replacement of participants who opted out or were lost to follow‐up continued until 50 sets of serial pain scores were collected. Eligible families were provided with an information sheet prior to enrolment by mail which included the study aims, contact details, and information on opting out of the study.

A custom software platform was developed by the team to manage the list of enrolled participants. This involved triggering SMS messages to be sent, classifying replies, and storing the pain scores and feedback received. SMS messages were sent via the Twilio Studio messaging service, which was also used to handle inbound messages and send automated responses to participants.

On the day of their child's surgery families who had not opted out were enrolled and an introductory SMS was sent to the primary caregiver which also included information regarding how to opt out of the trial via SMS. Following this, an SMS was sent on each postoperative day asking the caregiver to provide a numerical 0‐to‐10 parent proxy score for their child's average pain over the preceding 24 h. These are available in Appendix S1. As part of standard practice at our institution parents are educated on the assessment of pain by nursing staff during routine care and prior to discharge, using the Numerical Rating Scale and Faces Pain Scale‐Revised pain rating scales as appropriate, and were provided with an education sheet on assessing their child's pain. They also received standardized written information at discharge on recognizing and scoring pain in their child.

Responses were automatically classified when families responded with an integer pain score, and an acknowledgment was sent. When any other response was received, the family was informed that the response would be reviewed, and a researcher was notified to classify the response and reply to the family if required. If no response was received, reminder messages were sent after 2 h and then after a further 5 h.

Daily messages continued until the family responded with two consecutive zero pain scores or did not respond for three consecutive days—in which case they were considered lost to follow‐up. Each family was asked for optional feedback via SMS after completing their involvement in the study. Families were asked to rate their satisfaction with the SMS messages on a five‐point Likert scale from one (very unhappy) to five (very happy) and had the option to send additional qualitative feedback.

Outcomes

Our primary outcome was the response rate from enrolled families, calculated overall and for each day of follow‐up. Our secondary outcomes were the proportion of families who were lost to follow‐up each day, the proportion of responses that could be automatically detected as a pain score as well as the proportion requiring researcher intervention, the median (range) daily pain score, and the proportion of mild (1–3), moderate (4–6), and severe (7–10) scores on each postoperative day compared to previous data derived by telephone follow‐up, quantitative feedback score proportions, and qualitative feedback grouped by theme.

Statistical analysis consisted of descriptive statistics and was performed using pandas version 1.0.3 (The Pandas Development Team, Texas, USA), R version 3.5.3 (R Foundation for Statistical Computing, Vienna, Austria), and JupyterLab version 2.1.0 (NumFOCUS). , , Analysis for response outcomes was performed for all enrolled participants, and pain outcomes were analyzed on completed pain profiles.

RESULTS

This study commenced on September 24, 2019 but was paused from March 24 to December 16, 2020 due to elective surgery restrictions caused by COVID‐19. An outage affecting the data center hosting the study software also disrupted the study between December 11, 2019 and January 6, 2020. There were minor technical issues at the start of the study that affected the recruitment of 2 patients. We achieved the planned recruitment of 50 completed pain profiles.

We identified and sent information regarding the study to 97 potential participants. There were 22 potential participants excluded from enrolment due to COVID‐19 (10) and technical outages (12). The remaining 75 were sent an introductory SMS, with no potential participants opting out prior to this point. Of the 75 introductory SMSs sent, 13 participants were not recruited—10 due to changes to operating lists made after SMSs were scheduled, two due to technical error, and one recruited and subsequently excluded due to unplanned pediatric intensive care unit (PICU) admission. There were 62 participants enrolled to receive daily postoperative pain score queries (Figure 1).

FIGURE 1

Study profile.

Of the 62 enrolled families, 50 (81%) completed the pain profile, 2 (3%) opted out of the trial, and 10 (16%) were lost to follow‐up. The children with complete pain profiles had a median (IQR) age of 5 years (4–8). Of those lost to follow‐up, there were 5 (50%) who never replied to SMS messages, and 4 (40%) families who stopped responding to messages after sending at least one pain score—one on each of postoperative days one, three, six, and eight. One family responded on postoperative day three but did not provide a pain score, and then did not reply further.

Overall, 711 (97.9%) of the 726 requests for a pain score received a reply, 82.1% (584 of 726) of which were received after the initial prompt, with subsequent first and second reminders resulting in 15.9% (113) and 2% (14) additional replies. Table 1 details daily response rates.

TABLE 1

Response and loss to follow‐up rates and mean number of prompts sent to each participant for each postoperative day during SMS‐based follow‐up for daily average pain in children following tonsillectomy

POD	n	Reponses (%)	Lost to follow‐up (%)	Mean prompts
1	61	53 (86.9)	‐	1.37
2	60	53 (88.3)	‐	1.21
3	60	53 (88.3)	6 (14)	1.21
4	54	51 (94.4)	1 (2.7)	1.19
5	52	51 (98.1)	0 (0)	1.15
6	52	51 (98.1)	0 (0)	1.25
7	50	49 (98.0)	0 (0)	1.20
8	48	47 (97.9)	0 (0)	1.13
9	46	44 (95.7)	1 (3.2)	1.14
10	44	42 (95.5)	0 (0)	1.09
11	41	39 (95.1)	0 (0)	1.15
12	35	35 (100)	0 (0)	1.17
13	28	28 (100)	0 (0)	1.14
14	24	24 (100)	0 (0)	1.08
15	22	22 (100)	0 (0)	1.32
16	13	13 (100)	0 (0)	1.15
17	11	11 (100)	0 (0)	1.09
18	8	8 (100)	0 (0)	1.00
19	3	3 (100)	0 (0)	1.33
20	2	2 (100)	0 (0)	1.50
21	2	2 (100)	0 (0)	1.50
22	2	2 (100)	0 (0)	1.50

Note: Only one participant was enrolled past postoperative day 22, subsequent days were omitted. Values are count (proportion) and mean.

Abbreviation: POD, postoperative day.

Of the responses received, 107 (13.5%) required researcher intervention to be classified to a numerical pain score, with the remaining 685 (86.5%) classified automatically. Researchers were also notified in 33 cases where severe pain scores were reported, and in four cases where surgery had been rescheduled. We responded to these alerts on a case‐by‐case basis, updating or excluding patients who had procedures rescheduled and advising patients with multiple severe pain scores or who provided additional information to seek medical attention.

Mean daily average pain peaked on postoperative day six (Table 2), with a median (range) duration of 12.7 (1–50) days (Figure 2). Mild pain lasted for a median (range) of 12.5 (1–49) days, whereas moderate pain lasted for a median (range) of 7 (1–31) days and severe pain for 4.5 (1–26) days (Figure 3), including three patients with pain lasting for more than 18 days.

TABLE 2

Daily average pain scores and rates of mild (1–3), moderate (4–6), and severe (7–10) pain in children following tonsillectomy

POD	n	Pain	Pain group
			Mild	Moderate	Severe
1	49	5 (0–9)	9	25	11
2	50	4 (0–9)	19	25	5
3	50	5 (0–10)	14	23	9
4	50	4.5 (0–8)	17	24	8
5	49	5 (0–9)	17	19	12
6	49	5 (0–10)	10	24	12
7	48	4.5 (0–10)	14	25	5
8	46	3 (0–9)	25	15	2
9	44	2 (0–8)	28	11	1
10	42	2 (0–9)	26	5	2
11	39	1 (0–7)	23	2	1
12	34	1 (0–6)	20	3	0
13	28	1 (0–5)	18	1	0
14	24	1 (0–5)	12	1	0
15	22	0 (0–5)	7	2	0
16	13	1 (0–4)	6	2	0
17	11	0 (0–4)	2	1	0
18	8	0 (0–3)	2	0	0

Note: Only three participants had pain longer than postoperative day 18, subsequent days omitted. Values are median (range) and count.

Abbreviation: POD, postoperative day.

FIGURE 2

Pain scores for each postoperative day in children following tonsillectomy. Only three participants had pain longer than postoperative day 18, subsequent days omitted. The solid line shows the mean pain score each day, and bars show the 10–90 percentile range. Dashed lines show mean pain scores each day from previous cohorts at our institution (short dashes, Wilson 2016; long dashes, Lagrange 2021).

FIGURE 3

Proportion of pain scores categorized into mild, (1–3), moderate, (4–6) and severe (7–10) for each postoperative day in children following tonsillectomy. Only three participants had pain longer than postoperative day 18, subsequent days omitted.

Some participants were unable to complete the study due to postenrolment issues. One parent replied after several days to say that they had been unable to reply due to not having prepaid credit for their mobile phone. In another case, a patient was admitted to the PICU unexpectedly but had already received an automated pain score query. Another parent who received an SMS during this study needed to contact another parent to find out their child's pain score for that day before replying. One patient was also admitted for a tonsil bed bleed during the follow‐up period, but their family was happy to continue replying to the messages during their admission and this did not impact how they participated in the study. No families contacted the follow‐up service requesting help with a medical emergency.

There were 37 families (74%) who responded to feedback prompts—36 who completed the pain profile, and one who was lost to follow‐up. A median (range) satisfaction score of 5 (4–6) was received, with 70.2% (26) of scores were a five. Qualitative feedback was received from five participants of which three were positive, and two were neutral overall. Generally, parents appreciated being in contact with the hospital but felt that additional specific advice on pain management could have been useful. One response commented on the ease of use, saying that it was “easy just to send a number.” Two responses specifically noted that the messages felt supportive, of which one additionally found being advised to give analgesia irritating and wanted additional advice beyond what they already knew to do. Three responses expressed a desire to be able to contact the hospital more easily.

DISCUSSION

We demonstrated that SMS follow‐up for the collection of longitudinal research data in the postoperative period for a single quantitative variable is feasible in a pediatric population and highly acceptable to parents. This aligns with a previous survey of parents from our institution which found the majority were willing to communicate with the hospital via SMS. The high completion and response rates seen in this study demonstrate that this reported willingness translates well into practice. These results compare very favorably with our previous experience with telephone follow‐up in similar populations—in one previous study of canceled procedures, we allowed up to 10 attempts to contact each family being followed up, calling every phone number listed for the family on each attempt, and still only achieved an 83% (810 of 979) contact rate (authors' unpublished data).

There is limited previous evidence on response rates for SMS‐based postoperative data collection solutions. One Australian study using a 15‐question online survey after day surgery with a link sent via SMS found a response rate of 77.5% compared to 66% for telephone follow‐up. Our methodologies differ, however, as they used SMS messaging to distribute a link to an online survey rather than direct SMS interaction. Separately reported sub‐group analysis of that study found a response rate of 85.3% (852 of 999) in pediatric patients. Another study on posttonsillectomy pain in 24 children aged 4–16 years where parents were messaged for 14 days found a response rate of 88%. These response rates are similar to the completion rate seen in our study. High response rates from caregivers of pediatric patients may be related to the intrinsic motivation of caregivers to communicate with the hospital about their child's health. The drop‐off seen in response rates in our study following the initial request for pain score, first reminder, and second reminder suggests that any more than two reminders was unlikely to improve response rates significantly.

The level of researcher intervention required was low, with the time required to handle alerts orders of magnitude less than the time that would be required to perform telephone follow‐up for the whole population being monitored. Most alerts were either severe pain alerts early in the postoperative course, or surgery scheduling issues that were easily resolved. Further, several types of replies that were not automatically classified in this pilot were noted by the researchers and could be automatically classified in future studies—for example, noninteger pain score responses such as “4.5”, split‐level responses such as “4 to 5” or “4/5”, and responses that contained pleasantry that could be safely discarded such as “4 thanks.”

There were a small number of cases that highlighted specific challenges for SMS communication with families, such as identifying a child's full‐time carer from hospital records and ensuring that parents are able to respond every day to SMS messages. These cases were each handled without issue in our study but presented an opportunity for misunderstanding if automated messaging systems are not overseen by clinical staff who can respond appropriately to these situations. Having parents contact the service asking for urgent medical assistance remains a potential concern for future work. Parent feedback was an optional step in this study exposing sources of bias including poor response rate for qualitative feedback, negative bias toward issues encountered in the process, and reduced feedback rate in less technologically literate caregivers.

We collected parental proxy pain scores for our secondary outcome, which we have previously shown to be a reliable indicator of children's pain. The pain scores recorded follow a distribution that is similar to that seen in our previous audits of postoperative tonsillectomy pain (Figure 2), including a second pain peak roughly 5 days after surgery. , This is reassuring, as it shows that parents report pain scores consistently and in a similar range for the same surgery when communicating via SMS.

As this is a feasibility study, we did not perform a direct comparison to telephone follow‐up and our analysis is restricted to descriptive nonparametric statistics. Historically, a key barrier to the adoption of telephone follow‐up has been the labor cost of calling families and manual data entry after patient contact. Future work could include a formal cost analysis, although we are confident that the efficiencies seen for SMS messaging represent savings in researcher time that outweigh opposing costs.

The use of simple numerical parental proxy scores to assess pain rather than self‐reported scores aligns with our previous work on posttonsillectomy pain but is also a limitation of this study. Future work could include a more comprehensive assessment of pain, including the use of validated tools, and consideration of self‐report. This would require additional logistics such as whether SMS messages should be sent directly to older children and collecting scores from young children who may not understand SMS prompts.

Collecting patient data using computer software has challenges, both technical and ethical. During our study, we had interruptions due to unforeseen technical issues including a data center outage and a minor logic error in the study software. Any SMS data collection platform being used to provide clinical care would require failsafe procedures, testing to ensure any errors were caught, and monitoring for outages. Another consideration is the appropriate transport and retention of electronic patient data, particularly when collecting more sensitive data. In this feasibility study, we used an Australian regulation‐compliant data center and SMS platform to ensure that private healthcare data was stored and transmitted securely to patient phones. Care was taken to ensure that no patient health information was transmitted to patients directly and our team did not store any personally identifying information beyond a phone number, first names, and date of surgery.

Our results should generalize well to a variety of areas with similar data collection objectives. While we collected numeric pain scores it is likely that yes/no responses or other numeric ratings are equally feasible as long as the information requested is familiar to caregivers (e.g., a scoring system that was used while their child was in hospital). As noted by the parents who provided qualitative feedback, a key frustration was that they were not able to provide enough information or receive specific advice when caring for their children at home. A system for clinical after‐care incorporating automated SMS follow‐up with a broader range of automated or clinician‐triggered responses targeted to specific circumstances (e.g., nausea, throat pain, etc.) may be well received by parents. During the study, families often provided additional information such as feeding status, other symptoms such as crying or vomiting, the type of analgesia given, or the effect of analgesia. These variables were not captured as part of this study. The high response rate and willingness of parents to provide additional information are good indicators for use of SMS follow‐up for further studies.

There is a need for a highly scalable remote research follow‐up solution that is efficient, inexpensive, feasible, and acceptable to the parents of pediatric patients, and our findings support the use of SMS messaging in this capacity.

CONFLICT OF INTEREST

No external funding and no competing interests were declared.

Appendix S1

Click here for additional data file.