Age Significantly Affects Response Rate to Outcomes Questionnaires Using Mobile Messaging Software.

PURPOSE: To investigate the demographic factors that influence time to respond (TTR), time to completion (TTC), and response rate when using a text messaging-based system and to determine the feasibility and applicability of mobile messaging-based services for collection of patient-reported outcomes among orthopaedic sports medicine patients.
METHODS: On the day of surgery, patient mobile phone number was collected and the automated mobile messaging service (MOSIO, Seattle, WA) messaged patients for 10 ``days postoperatively. Patient visual analog scale (VAS) scores were collected 3 times daily, side effects were asked each evening, and Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference (PI) Short Form was collected on postoperative day 3 and 7.
RESULTS: A total of 177 patients were enrolled in the study. The overall response rate to the survey questions was 75.0%. For all patients, the average TTR of questions was 35.09 ± 12.57 minutes. The TTC was 2.75 ± 3.56 minutes for PROMIS-PI, 3.51 ± 1.26 minutes for VAS, and 3.80 ± 6.87 for side-effect questions. When patients were stratified into age groups, the youngest group, 16 to 32 years, had the greatest response rate of 85.2% and patients in the 49 to 59 years group had the lowest response rate of 68.4% and 69.1%, respectively (P < .001). There was no significant difference in the TTR or TTC for VAS, PROMIS-PI, or side-effect questions when patients were stratified by age or sex groups (P > .05).
CONCLUSIONS: Collectively, all age groups successfully achieved a mean response rate of 75%; however, significantly lower response rates were observed for patients >49 years old. Differences in age and sex did not impact the overall TTR or TTC for VAS, PROMIS-PI, or side-effect questions. Mobile-based applications present as an emerging opportunity to track postoperative outcome scores and reduce clinic survey load. LEVEL OF EVIDENCE: Case series, level of evidence IV.

The importance of value-based care is increasing for physicians, particularly as greater emphasis has been placed on patient-reported outcomes (PROs) by clinicians, insurers, and government-based enterprises., PROs serve as an important surrogate measurement for clinical outcomes and patient satisfaction, especially in the realm of physical, mental, and social function., Although the use and value of PROs has increased in recent years, there has been a lack of consensus regarding the optimal logistics and use within clinical practice.

In the United States, cell phone ownership among adults has increased from 62% of the general population in 2002 to 96% in 2019. Communicating with patients via mobile devices has become an important modality in health care. It allows for easier communication between providers and patients through appointment and medication reminders, web-based surveys, and medical counseling.5, 6, 7 In fact, more than one-quarter of community health programs currently use cellphones for health care purposes. Administering PROs via mobile-based applications has been successful and valid. Investigations into the validity of collecting PROs of orthopaedic patients using an automated mobile messaging based service have demonstrated a high intraclass correlation between paper and text messaging-based delivery. In addition, it is critical to evaluate not just the validity of mobile messaging-based PROs collection but also the time to response to question and to complete the entire survey collection, as shorter survey has been demonstrated to have increased completion rates.

The purposes of this study were to investigate the demographic factors that influence time to respond (TTR), time to completion (TTC), and response rate when using a text messaging–based system and to determine the feasibility and applicability of mobile messaging–based services for collection of PROs among orthopaedic sports medicine patients. We hypothesized that there would be a direct correlation between patient age and the length of time patients take to respond to text messages and complete surveys. We also hypothesized that patients’ race, sex, and body mass index (BMI) would not correlate with response time and response rate of patients using a text messaging–based service.

Methods

The study was reviewed and approved by our institutional review board. Between February 2019 and January 2020, patients who presented to a fellowship-trained sports surgeon were considered for inclusion (K.R.O). Inclusion criteria included age older than 16 years and the following commonly performed procedures: primary anterior cruciate ligament reconstruction, primary arthroscopic shoulder labrum repair, primary meniscectomy or meniscal repair, and primary rotator cuff repair. Exclusion criteria included non-English–speaking or non-English–reading patients, and patients without access to a cellphone. Nineteen patients declined participation.

During the preoperative consent process, patients were informed that postoperative follow-up information will be collected using automated text messaging service (MOSIO, Seattle WA). On the day of surgery, patient mobile phone was verified for SMS capabilities, and they instructed on how to respond to questions (Appendix Table 1). Their phone number was collected and the automated mobile messaging service (MOSIO) messaged patients for 10 days postoperatively. Patient visual analog scale (VAS) scores were collected 3 times daily (9 am, 1 pm, and 7 pm), side effects were asked each evening at 7 pm, and Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference (PI) Short Form was collected in the evening on postoperative day 3 and 7. In addition, each evening at 7 pm, patients were asked about how many narcotics they had consumed in the last 24 hours. Both the VAS score and the PROMIS PI form have been previously validated in the literature as metrics for assessing postoperative pain., The mobile-based messaging application sent an automated message to patients at the predefined intervals. The time at which each patient began responding was recorded. The difference between these values was used to calculate TTR for each day. TTC was calculated based on how long it took a patent to complete the entirety of each individual survey. The response rate was defined as the number of responses received divided by the number of survey questions administered over the course of the study period.

Table 1

Demographic Comparison of Patients Based on Age Groups

	0-32 years (N = 79)	33-48 years (N = 37)	49-59 years (N = 39)	60+ years (N = 22)	P Value
Procedure					<.01
ACL	41 (52%)	14 (38%)	2 (5%)	1 (5%)
RCR	2 (3%)	6 (16%)	14 (36%)	12 (55%)
Labrum	20 (25%)	5 (14%)	1 (3%)	0 (0%)
Meniscus	16 (20%)	12 (32%)	22 (56%)	9 (41%)
Race					.01
White	30 (39%)	15 (42%)	25 (64%)	13 (62%)
African American	13 (17%)	8 (22%)	8 (21%)	6 (29%)
Other	19 (25%)	11 (31%)	5 (13%)	1 (5%)
Unknown	15 (19%)	2 (6%)	1 (3%)	1 (5%)
Sex					.21
Male	57 (72%)	20 (54%)	23 (59%)	15 (68%)
Female	22 (28%)	17 (46%)	16 (41%)	7 (32%)
BMI	27.72 (5.92)	28.37 (5.22)	30.93 (5.88)	29.53 (3.40)	.03
Presurgery PROMIS-PI	57.23 (7.60)	59.20 (5.16)	60.02 (6.17)	61.89 (5.82)	.08
Postsurgery PROMIS-PI	59.69 (10.13)	58.56 (7.23)	59.38 (8.62)	62.67 (7.19)	.69

ACL, anterior cruciate ligament; BMI, body mass index; PROMIS-PI, Patient-Reported Outcomes Measurement Information System Pain Interference; RCR, rotator cuff repair.

Bold values are significant at P < .05.

The following variables were collected from the patient medical record: demographic data (age, sex, BMI, and race) and workers compensation status. Patients were stratified into cohorts based on age and sex. In accordance with methodology used in previous literature patients were grouped into quartiles based on age (16-32, 33-48, 49-59, and ≥60 years old).,

Statistical Analysis

All continuous data were reported as mean ± standard deviation, whereas categorical data are reported as counts and column percentages, N (%). For continuous variables, group comparisons were performed using one-way analysis of variance if the variable was normally distributed and using Kruskal–Wallis tests if the variable was non-normally distributed. Multiple comparisons were performed using a Bonferroni P value. For categorical variables, univariate 2-group comparisons were performed using χ2 tests when expected cell counts were >5 and using Fisher exact tests when expected cell counts were <5. Multiple variable regression is performed for each significant outcome and adjusted for each significant demographic variable. Statistical significance was set at P < .05. All analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC).

Results

Demographics

A total of 196 patients were considered for inclusion during the study period. Of these, 177 patients were enrolled in the study. Nineteen patients declined participation due to concerns regarding involvement with any form of research. The overall response rate to the survey questions was 75.0% (Fig 1). The response rate per postoperative day is demonstrated in Figure 2. The average time from automatic text message to initial response (TTR) was 35.09 ± 12.57 minutes. For all patients, the TTC for each question was 2.75 ± 3.56 minutes for PROMIS-PI, 3.51 ± 1.26 for VAS, and 3.80 ± 6.87 for side effects. Patient demographics are reported in Table 1. Fifty-eight (32.8%) patients underwent anterior cruciate ligament reconstruction, 34 (19.2%) rotator cuff repair, 26 (14.7%) shoulder labrum repair, and 59 (33.3%) meniscal debridement or repair (Table 1). No significant difference was present in the TTR, TTC, or response rate when patients were stratified by operative procedure (P > .05). When patients were stratified based on age groups, 79 patients were younger than the age of 32 years, 37 patients were age 33 to 48 years, 39 patients were age 49 to 59 years, and 22 patients were older than the age of 60 years. When patients were stratified by sex, there were 116 male patients and 77 female patients.

Fig 1

Distribution of response rate based on age grouping.

Fig 2

Overall response rate for all patients on each postoperative day.

Response by Age Group

Patients were stratified into 4 cohorts based on age (Table 1). When we evaluated age groups, reported race (P = .01), BMI (P = .03), and procedure (P < .01) significantly differed between the groups. When we evaluated the cumulative response rate across all postoperative days, the overall response rate to the survey questions was 75.0% (Fig 1). The youngest group, 16 to 32 years, had the greatest response rate of 85.2%, and patients in the 49 to 59 years’ group had the lowest response rate of 68.4% and 69.1%, respectively (P < .001).

Figure 3 represents the mean TTR to all survey questions for patients in each age group. No significant difference was present between age cohorts. When evaluating the average TTR on each postoperative day, there was no significant difference found between the age groups (Table 2). When we evaluated TTC between patients in each age cohort, there was no significant difference in the mean TTC for VAS scores (P = .53) (Fig 4A), PROMIS-PI questions (P = .89) (Fig 4B), or side-effect questions (P = .33) (Fig 4C).

Fig 3

The mean TTR for all patient stratified by age groups. (P = .94). (TTR, time to respond.)

Table 2

Average TTR in Minutes Per Day for Age Cohorts

	16 to 32	33 to 48	49 to 59	60+	P Value
Day 1	21.38	13.82	19.44	23.97	.8
Day 2	22.15	20.95	24.43	15.35	.8
Day 3	11.04	11.89	12.00	9.59	.76
Day 4	17.63	32.80	19.62	32.03	.81
Day 5	20.76	24.65	28.91	27.10	.95
Day 6	26.76	22.27	28.30	16.89	.69
Day 7	12.15	14.67	11.07	12.18	.98
Day 8	22.14	23.33	14.16	30.31	.63
Day 9	26.00	25.15	26.87	17.01	.87
Day 10	22.71	26.06	29.15	20.53	.94

TTR, time to respond.

Fig 4

(A) The mean time to respond in minutes for all VAS questions when stratified by age group. (B) The mean time to respond in minutes for all PROMIS-PI questions when stratified based on age group. (C) The mean time to respond in minutes for all side effect questions when stratified based on group. (PROMIS-PI, Patient-Reported Outcomes Measurement Information System Pain Interference; TTC, time to completion; VAS, visual analog scale.)

Response by Sex

After mean TTR was stratified by sex, overall TTR was not found to significantly differ between sexes (Fig 5). When mean TTR was evaluated on each postoperative day, there was no significant difference found between sexes (Table 3). When we examined the response rate by sex, the total response rate was 79.17% for male patients and 76.36% for female patients. When evaluating TTC based on sex, there was no significant difference seen for VAS scores (P = .11) (Fig 6A), PROMIS-PI (P = .49) (Fig 6B), or side-effect questions (P = .95) (Fig 6C).

Fig 5

The mean TTR for all patient stratified by sex (P = .50) (TTR, time to respond.)

Table 3

Average TTR Average TTR in Minutes Per Day for Sex Cohorts

	Male	Female	P Value
Day 1	22.03	22.83	.94
Day 2	33.29	19.18	.23
Day 3	11.88	10.29	.81
Day 4	38.82	35.38	.86
Day 5	27.88	32.3	.74
Day 6	28.43	29.06	.96
Day 7	14.79	11.33	.66
Day 8	27.85	25.54	.87
Day 9	31.18	27.14	.78
Day 10	31.58	26.79	.74

TTR, time to respond.

Fig 6

(A) The mean time-to-response in minutes for all VAS questions when stratified by sex (P = .12). (B) The mean time to respond in minutes for all PROMIS-PI questions when stratified by sex (P = .49). (C) The mean time-to-response in minutes for all side- effect questions when stratified by sex (P = .95). (PROMIS-PI, Patient-Reported Outcomes Measurement Information System Pain Interference; TTC, time to completion; VAS, visual analog scale.)

Discussion

The present study demonstrates that age has a significant effect on the response rate for a text-message based service collecting postoperative outcomes in patients undergoing routine sports surgery. Collectively, all age groups successfully achieved a mean response rate of 75%; however, significantly lower response rates are observed for patients >49 years old. Patient age and sex did not significantly affect TTR or TTC for VAS, PROMIS-PI, or side-effect questions.

A majority of studies reporting data on implementing clinic-based PRO measures and mobile applications for postoperative follow-up have enrolled young patients with an average age less than 60 years old15, 16, 17, 18, 19, 20; however, there is suggestion that older age may influence response times of patient reported outcomes. In a cross-sectional study of 3658 PROMIS forms administered in ambulatory clinics, Kadri et al. found that patients from the oldest age quartile (mean ± standard deviation, 70.3 ± 7.5 years) had a statistically significant longer TTC as compared with the second quartile (41.2 ± 4.7 years) (3.70 vs 2.87 minutes; P < .05). They suggested that PROMIS tools can still be used efficiently among older patients; however, they may require more time. Similarly, Morgan et al. examined the performance of PROMIS Physical Function computer adaptive tests compared with traditionally used PRO measures in patients with proximal humerus fractures. Forty-seven patients, 60 to 88 years old, completed the PROMIS PF survey rather quickly, with a median TTC of 98 seconds (range 45-203 seconds). Beletsky et al. sought to compare VAS measures of function, strength, and pain against legacy PRO measures in a group of rotator cuff repair patients 55.6 ± 10.9 years old. Patients quickly completed surveys with an average TTC of 1.36 ± 1.12 minutes for the 3 VAS measures and 1.72 ± 1.48 minutes for the PROMIS UE computer adaptive test. These studies together highlight that patients in older age groups are still able to complete surveys rather efficiently. The present study collected PRO measures using a mobile text message–based application. We found that the average time for patients to begin responding to questions after the first message was sent was 35.09 ± 12.57 minutes, and the average TTC for PROMIS-PI, VAS, and side-effect questions was unaffected by age. While we cannot directly compare TTC times with these aforementioned studies due to different pathologies studied and survey tools used, our results suggest that mobile-based outcome collection efficiently gathers data without differences in completion time in older patients. The average TTR was unaffected by age and showed that patients took just more than a half hour (average 35 minutes) from text message receipt to responding. These results in combination emphasize that that mobile-based collection tools can perform efficiently in terms of TTC and TTR regardless of age.

Conflicting evidence exists on the effect of age on response rate with a mobile application. Premkumar et al. implemented a text message-based platform to track pain and opioid use for 6 weeks after common orthopaedic procedures for patients on average 59.4 ± 10.9 years old. Their group reported an excellent response rate of 96.1% (173 of 180 participants) with further logistic regression showing that age was not associated with nonresponse. However, in their study, adequate response was defined as answering greater than 50% of text messages received. This threshold of 50% may be too low to capture key differences in response rates between groups of patients. In contrast, Geerds et al. examined the use of a mobile application for monitoring outcomes after hip fractures. Patients in this study were on average 80.5 years old. Only 26.4% of participants downloaded the application and 3.4% of those patients completed the questionnaire within 12 to 18 weeks after the operation. Reasons for not completing the questionnaire included technical problems with the app, cognitive disorders, patient dependency on caregivers, and forgetting about the study after discharge. Despite high levels of smartphone expertise and self-reported intention to use the app, the application was not successfully implemented in this elderly age group. Although our cohort of interest differed in pathology and follow-up length, our findings similarly suggest that increasing age hinders response rate within 2 weeks after surgery, with patients 49+ years old have significantly lower response rates compared with the youngest group, 16 to 32 years old (85.2% vs 68.4%, P ≤ .001). Our findings, which specifically focused on sports procedures, are in contrast to the results of Premkumar et al. who studied a wide variety of pathologies. Although studies have shown that patients >55 years old are willing to use mobile applications for postoperative follow-up,, there is still a barrier in response rate with application use in older age groups. As suggested by Geerds et al., older age groups may require further education on the importance of the application. Nonetheless, our findings demonstrate that the majority of patients were compliant with the application, with an average response rate of 75%, but further attention to these older age groups is warranted to improve their compliance.

The effect of sex on performance of PROs has been reported in the literature. Teresi et al. sought to examine performance of PROMIS PI items based on demographic factors in a cancer and palliative care populations. Differential item functioning or item bias refers to participants with the same underlying trait (such as a specific pain level) will have different probabilities of reporting a response, suggesting that a participants membership in a group or demographic may influence how they respond to a question. Their analysis found no difference in terms of sex. Differential item functioning also was observed for PROMIS Physical Function, PI, and Pain Behavior in patients with osteoarthritis, rheumatoid arthritis, or undergoing physical therapy for a musculoskeletal condition. However, the magnitude and impact of these items were also negligible. Furthermore, Kadri et al. further explored the role of sex by assessing factors impacting TTC of PROMIS scores for common arthroscopic surgeries. The authors performed their study with surveys delivered in the clinic setting rather than at home from a mobile device. They found no significant difference in TTC between male and female participants. In line with previous literature, our study did not show differential effects TTR or TTC based on sex. Response times to PROMIS-PI, VAS, and side-effect questions remained prompt regardless of sex, race, or ethnicity. In sum, these findings suggest that PROMIS tools and questionnaires perform reliably and efficiently despite racial or ethnic differences, especially in a mobile based application.

Limitations

This study was not without limitations. The follow-up in the study was limited to 14 days, thus preventing us from making conclusions beyond this timeframe and impacting the generalizability to other investigations that last longer. In addition, factors linked with poor compliance such as cognitive disabilities, technologic literacy, patient-perceived usability, and satisfaction were outside the scope of this study. There is also potential for observational bias, as patients were made aware of study participation; however, in an effort to reduce potential bias, patients were not aware that the response rate, TTR, and TTC were being recorded. In addition, heterogenicity in operative intervention exists within the observed cohort. This variability may lead to additional bias in response rate as rotator cuff repair patients are more likely to be older and have reduced upper-extremity function that may limit mobile device usage. Future studies should assess for barriers to using the text message application with patient surveys after the completion of the study and should evaluate for mobile device use competency preoperatively.

Conclusions

Collectively, all age groups successfully achieved a mean response rate of 75%; however, significantly lower response rates are observed for patients >49 years old. Differences in age and sex did not impact the overall TTR or TTC for VAS, PROMIS-PI, or side-effect questions. Mobile-based applications present as an emerging opportunity to track postoperative outcome scores and reduce clinic survey load.