Collection of the International Hip Outcome Tool-12 Using a Smartphone Application Format Is Faster and Preferred When Compared With the Paper Version: A Pilot Study of rHip.

PURPOSE: To evaluate the agreeability between the mobile application-based International Hip Outcome Tool-12 (iHOT-12) survey with the paper version, as well as compare the time it takes patients each of the versions, and patient preferences between the two.
METHODS: Patients seen with symptomatic femoroacetabular impingement syndrome were prospectively enrolled in February 2019 and completed both the paper and application-based iHOT-12, in randomized order. Outcomes scores and time to completion were recorded for each version, and patients were also asked which they preferred. Intraclass correlation coefficient was calculated to assess for absolute agreement between the 2 versions. Bland-Altman plots were constructed to evaluate the agreeability between paper and application-based iHOT-12 scores. Bland-Altman plots were evaluated to identify systematic bias and data stratification was performed to identify sequence bias between the application and paper-based collection modalities.
RESULTS: Twenty-nine patients (aged15-56 years) completed both the paper and application-based versions of the iHOT-12. Between the application-based and paper versions, the intraclass correlation coefficient was 0.98, and Bland-Altman analysis showed agreement without bias between versions. There was no sequence bias. Accounting for completion order, the application-based iHOT-12 was faster for patients when compared to the paper version (61.4 ± 20.3 vs 71.9 ± 23.6 seconds, P = .02). Twenty-two patients reported a version preference where 19 of 22 (86%) chose application-based (P < .001).
CONCLUSIONS: The application-based iHOT-12 demonstrated absolute agreement with the paper iHOT-12, and is faster for patients to complete. Patients preferred using the application-based iHOT-12 over the paper-based version. Application-based PROs allow for collection of patient data at more frequent time points, which may be helpful in tracking the recovery progress of patients and predicting outcomes. CLINICAL RELEVANCE: As electronic-based outcome surveys become more common, it is important to know how the results may differ from traditional paper-based surveys.

Patient-reported outcomes (PROs) are widely used to track functional outcomes of patients following hip arthroscopy for the treatment of femoroacetabular impingement syndrome (FAIS).1, 2, 3 While there is no consensus on a standardized set of PROs collected following hip arthroscopy for FAIS, the International Hip Outcome Tool-12 (iHOT-12) is a reliable, responsive, and validated PRO that is specific to capturing the function of nonarthritic hips and is widely accepted in the FAIS literature. The iHOT-12 has been recommended for widespread adoption, as it was found to be 1 of the 2 most responsive hip-specific PROs. In addition, the responses to the iHOT-12 are on a continuous scale, rather than having discrete values, making it suitable for a smartphone touchscreen interface. Therefore, we decided to adapt this PRO to a smartphone application to assess patient function following hip arthroscopy.

Smartphones have become ubiquitous, with more than 70% of the U.S. population owning one. According to recent data, people spend more than 3.5 hours per day interacting with their phones. Applications on these devices allow for easy data storage that is remotely accessible. Additional benefits of smartphones are the ability to send scheduled application reminders as well the ability to send push to increase patient engagement. Orthopaedic surgeons have taken advantage of this to track activity data following procedures such as spinal fusion. Furthermore, smartphone applications have shown to increase medication adherence as well as adherence to home-based exercise programs.8, 9, 10, 11

Given the prevalence of smartphone usage, we developed an application that collects the iHOT-12. The purpose of this study was to evaluate the agreeability between the mobile application-based iHOT-12 survey with the paper version, as well as compare the time it takes patients each of the versions, and patient preferences between the two. We hypothesized that the scores between the 2 versions would demonstrate excellent agreement without bias and that patients would prefer the application-based version of the iHOT-12.

Methods

Patient Selection

Following institutional review board approval, patients seen in a high-volume hip-preservation practice with a chief complaint of hip pain related to FAIS during the month of February 2019 were approached to participate in the present study until 29 patients were enrolled. Inclusion criteria consisted of patients being treated for hip pain with clinical and radiographic diagnosis of FAIS. Exclusion criteria consisted of hip pain related to any other cause than FAIS (osteoarthritis, gluteus medius tendinopathy, hamstring injury, etc.).

Application Development

Following institutional review board approval, a smartphone application was developed for the iOS Operating System (Apple Inc., Cupertino, CA) using the xCode Console (Apple Inc.) and distributed on the iTunes App Store, where users could download it onto their smartphone. A corresponding secure database communicating with this patient-facing application was configured to store application data. The application contains a digital version of the iHOT-12, which is a PRO consisting of 12 visual analog scale questions. For this study, an iPhone 7 (Apple Inc.) with the application pre-installed was used by patients.

In the standard paper version of the iHOT-12 (Appendix Fig 1, available at www.arthroscopyjournal.org), each question addresses a certain activity or quality of life domain. The questions are accompanied by a continuous 100-mm line with the labels on the left end representing “extreme” and the right end representing “none.” The instructions ask the patient to mark where they fall on the line, which represents a continuum from either extreme to no difficulty with the function or quality of life domain (Appendix Fig 2A, available at www.arthroscopyjournal.org). The distance along the line is recorded as a percentage of the entire length, and the final score is the average of all the responses.

In the smartphone version of the iHOT-12, users are first presented with an instruction screen with similar text as the paper version instructions. Each of the questions is accompanied by a slider, with the left end labeled “extreme” and right end “none,” and the patient is able to drag the slider to their desired location on the line (Appendix Fig 2B, available at www.arthroscopyjournal.org). In the software, the value of the slider is a continuous scale from 0 on the left end to 1 on the right end. Each time the patient adjusts a slider, the new value for that question is stored in memory; when the patient reaches the end of the questions and taps a “Submit” button, the score is calculated by averaging the values of the sliders of each question, then multiplying by 100% (Appendix Fig 2C, available at www.arthroscopyjournal.org). The result is instantly accessible to both providers and the patient. For this study, the application was modified to present a page showing the result of each question, the overall score, and the duration of the survey immediately after the patient tapped “Submit.”

Comparison of Application- and Paper-Based Versions of the iHOT-12

In the clinic, patients were randomized into 2 groups via an online randomization tool: one group completed the paper iHOT-12 first and the other completed the app-based iHOT-12 first. Following completion of the first survey, patients proceeded to complete the version they had not already completed. The paper version consisted of a 100-mm scale for each item, which was scored using a ruler with the distance corresponding to the score. Total score was obtained by an average of the 12 components and was performed by a resident surgeon. The application-based version produced scores instantly through the application (Appendix Fig 2C, available at www.arthroscopyjournal.org). The application also tracked the duration of the survey, beginning when patients tapped a start button to be presented with the initial instructions page and ending when patients tapped “Submit.” The patient was timed with a stopwatch for the paper version: timing began when the patient indicated they were ready and stopped when they put down the pen. Patients also were asked afterwards if they preferred one of the versions, and their response were recorded along with the times.

Statistical Analysis

The intraclass correlation coefficient (ICC) for paper based and application based iHOT-12 surveys was evaluated using a 2-way mixed-effect model for absolute agreement. Bland–Altman analysis was performed to evaluate the agreeability between survey version. The mean difference between application and paper based iHOT-12 scores was determined. Limits of agreement were defined by the standard deviation of the differences between application and paper based iHOT-12 scores. Agreeability required fulfillment of 2 criteria; (1) the difference between application and paper based iHOT-12 score must encompass zero within the established limits of agreements; and (2) greater than 95% of the differences between measurement method must be within the established limits of agreement. Patient preference of the different versions was compared using a 2-proportion z test. In addition, the time it took patients to complete each version of the survey was compared using a paired-samples t test. To control for test familiarity, the average difference of the amount of time required to complete the first and second surveys was subtracted from the first survey performed. Statistical significance for all analysis was set at an α ≤ 0.05. All statistical analysis was performed using the MedCalc (version 14; MedCalc Software Ltd., Ostend, Belgium).

Results

Patient Demographics

A total of 29 patients seen in clinic for FAIS completed both the paper and application-based iHOT-12, in randomized order. The study population was 62.1% (N = 18) female with an average age of 31.3 ± 10.7 years. Fourteen patients (48.3%) completed the paper survey first, whereas 15 (51.7%) completed the smartphone version first.

iHOT-12 PRO Correlation

Between the application-based and paper versions, there was excellent agreement, with ICC of 0.98 (Fig 1). When results were stratified by order of completion there was no sequence bias with an ICC of 0.97 and 0.98 for paper first group and application first group, respectively (Fig 2). Bland–Altman analysis showed absolute agreement with no systematic bias between versions (Fig 3).

Fig 1

Scores on the paper and application versions of the International Hip Outcome Tool-12 (iHOT-12) demonstrate excellent reliability.

Fig 2

Paper and application versions of International Hip Outcome Tool-12 (iHOT-12) display excellent reliability when stratified by groups completing (A) paper and (B) application versions first.

Fig 3

Bland–Altman analysis displaying agreeability between application and paper versions of the International Hip Outcome Tool-12 (iHOT-12).

Completion Time and Preference

The average completion time of the application and paper-based surveys were 70.7 ± 22.9 seconds and 80.5 ± 25.7, respectively (P = .09). On average, patients completed their second survey 18.7 seconds faster than their first. Subtracting this from patients’ first survey times to control for test familiarity, the application-based iHOT-12 was faster for patients when compared with the paper version (61.4 ± 20.3 vs 71.9 ± 23.6 seconds, P = .02). Of 22 patients with a version preference, 19 (86%) chose the application-version of the iHOT-12 (P < .001). The application based iHOT-12 was scored instantly versus the paper form which required scoring by a surgical resident.

Discussion

Our study found that a mobile application-based measurement of the iHOT-12 had excellent reliability, demonstrated absolute agreement, was faster to complete, and was preferred by patients when compared to the paper survey. Mobile-based administration of PROs can save time for both patients and physicians and make it easier to collect, store, and analyze data than through the use of paper-based surveys. Electronic-based applications such as rHip also offers the ability to obtain PROs before the visit, or in cases of missed or canceled visits. Mobile applications also have the ability to send the user “notifications” at regular postoperative intervals and facilitate improved long-term follow-up. A demographic study of 3,447 patients found that 20- to 39-year-old patients were the most common age group that underwent hip arthroscopy. Specifically, this may support collection of PROs from younger patient populations, particularly those who may be from out of state, or who attend college.

Similar studies have been performed in other areas of orthopaedics with regards to the validation of electronically based PROs. Sabatino et al. evaluated the reliability of electronically administered PROs including the Pediatric International Knee Documentation Committee, Hospital for Special Surgery Pediatric Functional Activity Brief Scale, Tegner Activity Level Scale, visual analog scale, and PedsQL Teen. Similar to the present study, all participants completed electronic versions and paper versions. Overall, a high degree of reliability was found for every PRO except the visual analog scale, with electronic surveys taking less time than paper (10.0 vs 11.2 minutes). Of all participants, 69.8% preferred the electronic, 67.4% found the electronic faster, and 93.0% reported that they would complete the electronic forms at home prior to their appointment. Our study similarly found reduced time with the rHip application, as well as finding that 86% of patients preferred the rHip application to traditional paper testing.

Limitations with paper-based PROs have been well documented in the literature., Surveys can be time consuming, which can lead to survey fatigue, and reduced patient compliance. For instance, the iHOT-12 was originally a 33-item questionnaire (iHOT-33), which was subsequently shorted to increase responsiveness and decrease time burden. Paper PRO completion rates can severely limit the quality of patient reported data and patient reported studies. One study of total joints patients at a large academic medical center found that only 30.6% of patients completed their paper-based PROs annually. On the contrary, Slover et al. used a web-based version of the Euroqol-5D and Knee Osteoarthritis Outcome Score to 666 patients at 2 different centers over a 9-month period and reported a completion rate of 93% and 95% with electronic survey administration. Similar to our study, the authors reported that this method of PRO administration was both feasible, and effective.

Outcome tracking in the medical field is trending toward the eventual usage of electronic-based platforms, not only to improve the rate and ease at which data can be collected, but to help standardize and customize patient surveys on a patient-by-patient basis. Our mobile-based application goes a step further in that the survey itself is administered on the patient’s own personal cellular device, allowing them to complete the survey anytime, anywhere, as long as they have a simple, free, mobile application on their device. By administering the iHOT-12 in an easy-to-use mobile-based platform, we were able to support its usage compared with traditional paper-based questionnaires. Overall, future studies are needed on the long-term usage of rHip; however, we believe this is an easy way to empower patients to record their PROs, which can be both time-saving for the patient and physician while also giving patients greater transparency into their own progress.

Limitations

This study is not without limitation. This was a pilot study with a relatively small sample size; however, a post-hoc power analysis demonstrated we were adequately powered at a sample size of 5 was required to obtain statistical power at the recommended 0.80 level. Furthermore, test familiarity is a limitation due to patients completing both versions. However, test familiarity was controlled for by subtracting the average difference of the amount of time required to complete the first and second surveys from the first survey performed. In addition, we did not assess PROs at various outcomes for the same patients, which could lead to differences in time between the 2 methods. There is also inherent bias with regards to the patient’s preference for rHip over the paper-based study, as they were clearly informed that they were participating in a study, which does introduce an element of bias into their evaluation.

Conclusions

The application-based iHOT-12 demonstrated absolute agreement with the paper iHOT-12, and is faster for patients to complete. Patients preferred using the application-based iHOT-12 over the paper-based version. Application-based PROs allow for collection of patient data at more frequent time points, which may be helpful in tracking the recovery progress of patients and predicting outcomes.