Prospective Randomized Cohort Study to Explore the Acceptability of Patient-Reported Outcome Measures to Patients of Hand Clinics.

Purpose: The purpose of this randomized prospective cohort study was to assess the acceptability of administering patient-reported outcome measures (PROMs) in the waiting room of hand clinics.
Methods: Participants were randomly assigned to receive the Patient-Rated Wrist and Hand Evaluation (PRWHE), Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire, or Michigan Hand Questionnaire (MHQ). Acceptability was measured in terms of data quality, participation, and retention rates. Data quality was determined by the number of incomplete questionnaires, unanswered questions, and unscorable questionnaires. Most frequently unanswered questions were identified. The participant-reported time taken to complete the questionnaires was collected.
Results: A total of 491 participants enrolled in this study. A participation rate of 85% with a retention rate of 94% indicated that patients found the administration of PROMs in the waiting room of the clinic to be acceptable. The proportion of missing data for each questionnaire was 4.2% for PRWHE, 3.9% for DASH and 6.3% for MHQ. Whether a questionnaire could be used to generate a score was determined by the scoring rules of each instrument. The proportion of questionnaires that were not completed sufficiently for a score to be generated was 0% for PRWHE, 9% for DASH, and 4% for MHQ. No association was found between whether a questionnaire could generate a score and participants' sex, age, or the nature of the condition. Over 80% of participants reported taking 10 minutes or less to complete the questionnaire. Conclusions: This study shows that hand clinic patients will complete PROMs while waiting for the clinical review. The PRWHE and MHQ groups demonstrated good usability, because less than 5% were unable to produce a score. The usability of the DASH group was lower, because 9% were unable to be scored. This indicates that PRWHE and MHQ are more suitable than DASH to application in a hand clinic setting. Type of study/level of evidence: Therapeutic III. Crown

Patient-reported outcome measures (PROMs) are being increasingly integrated into clinical practice for routine outcome measurement at both the individual and cohort levels. The field of hand surgery has multiple PROMs available. The literature reflects the use of PROMs in research, but less is available on their application in routine clinical care. Measuring comparative effectiveness allows the evaluation of interventions in a broad, unselected population, in contrast to formal research trials that often have tight inclusion and exclusion criteria and in which the participants may not be broadly representative of the population served. If it is accepted that both clinicians and patients can benefit from the routine use of PROMs in clinical care, it is important to assess whether patients are willing to participate in the completion of PROMs in the clinical setting. This study examined the acceptability of PROM administration in the waiting room of a busy government-funded hand clinic in Australia.

Acceptability is often assessed by participation, retention and withdrawal rates of participants, as well as the quality of the data. Many factors may affect the quality of participants’ responses, including patient characteristics and questionnaire characteristics. In terms of questionnaire characteristics, poor acceptability can indicate that a PROM is too complicated or too long, contains irrelevant questions, or has other design problems. Our hypothesis was that a shorter questionnaire would have a lower number of incomplete responses and a higher rate of scorable questionnaires, and thus higher acceptability in the hand clinic setting.

Materials and Methods

This study was a prospective randomised cohort study with broad inclusion criteria: any hand clinic patient aged 14 years or older who could read English, understand the questionnaires, and give consent to participate was eligible. The study was conducted at the hand clinic of a tertiary Australian hospital between February and June 2017. The Southern Adelaide Clinical Human Research Ethics Committee granted approval for the study. Participants were recruited by a member of the research team after they had checked in with clerical staff for the scheduled appointment. The study was explained to participants, who were provided with the participant information and consent form and were given the opportunity to ask questions about the research. Participants were advised that the purpose of the study was to compare the 3 questionnaires, to decide which was most suitable for future use in the clinic. Participants were not specifically told that the completeness of responses would be a variable of interest in the study. An on-line random number generator performed randomization, with the participant enrollment number deciding each participant’s PROM allocation.

Three validated questionnaires were used to determine whether routine clinical PROM data collection was acceptable to patients in the clinics. The PROMs selected for use in this study were the Disabilities of the Arm, Shoulder, and Hand questionnaire (DASH), the Michigan Hand Questionnaire (MHQ), and the Patient-Rated Wrist and Hand Evaluation (PRWHE). These questionnaires were chosen because they are commonly cited in hand surgery literature and they vary in terms of the length and detail of items., The full version of each questionnaire was selected, and participants completed the PROMs on paper. With the first survey administration, each participant was also asked, “How long did it take you to complete this form?” The following response options given: “less than 5 minutes,” “5 to 10 minutes,” “10 to 15 minutes,” and “greater than 15 minutes.”

Once participants were assigned to receive a PROM, they would receive the same questionnaire on each subsequent clinic visit, with a minimum of 2 weeks between questionnaire administrations. For the purposes of the study, a maximum of 5 questionnaires were administered to any participant.

Participants were left to read the directions independently on the questionnaire and complete the form to the best of their ability. In some cases, participants had family members aid them by acting as a scribe. Questionnaires were not reviewed for completeness at the time of collection. Participant demographics including age, sex, and hand pathology were recorded by a member of the research team. Information relating to the affected side was collected retrospectively from the participant’s case notes. Data were collated in a database (Microsoft Access 2010; Microsoft, Redmond, WA).

We established the relationship between a questionnaire response that could generate a score and participants’ sex, age, and whether they had an elective or traumatic condition. The proportion of missing data was compared among PROMs. Individual questions with high rates of missing data were noted as potentially irrelevant or poorly functioning questions. The incidence of questionnaires that were unable to generate a score was compared between groups. Participants’ reported time taken to complete each questionnaire was also compared between groups.

Patient-reported outcome measures

The DASH is a 30-question PROM that measures disabilities of the upper limbs as a single functional unit. The PRWHE is a 17-question PROM that measures pain and function. The MHQ has 62 questions and is the only PROM included that can give a separate score for the left and right hands. The mean of these scores results in the total score, which is relevant only for patients with bilateral hand problems. For unilateral hand problems, the MHQ has 37 core questions. The side or sides affected are required to calculate the relevant MHQ score.

Statistical methods

Sample size calculation was made a priori, based on the null hypothesis that the proportion of scorable questionnaires was not dependent on the questionnaire group. A sample size of 100 (per group, total n = 300) produced a 2-sided 95% confidence interval with a width of 0.178 when the sample proportion was 0.25. Descriptive statistics were used to describe the study population, the completeness of the data, and the time taken to complete the questionnaires. Independent-sample 2-tailed t test was used to compare means between groups. The relationship between qualitative (ie, categorical) variables was assessed with chi-square tests of independence using Yates’ correction for continuity (when correlation between greater than 2 dichotomous variables was performed). Bonferroni adjustment of the α level was used for post hoc analyses.

Results

Population

Table 1 lists the demographics of study participants and nonparticipants. In terms of age and sex distribution, there was no significant difference between the participating sample and the sample that declined to participate. Overall, 85% of those approached were willing to participate in the study. Of those who declined, they often cited practical reasons, such as that they did not have reading glasses or were in too much discomfort to complete the forms. Recruitment took place over 4 months. Figure 1 displays enrollment and randomization. Demographics of participants in each PROM group are shown in Table 2 and details of participants’ conditions are listed in Table 3.

Table 1

Demographics of Participants and Nonparticipants

	Participants	Nonparticipants
Population (n = 577)	491	86
Age, y (mean [range])	49 (14–91)	52 (18–89)
Sex, n (%)
Male	282 (57)	53 (62)
Female	209 (43)	33 (38)

Figure 1

Flow diagram of study participant enrollment and randomization into groups.

Table 2

Characteristics of Participants in Each Questionnaire Group

Participants (N = 491)	Questionnaire Group
	PRWHE	DASH	MHQ
n	161	165	165
Age, y (mean [range])	47 (14–88)	46 (14–90)	49 (14–90)
Sex, n (%)
Male	86 (53)	98 (59)	97 (59)
Female	75 (47)	67 (41)	68 (41)
Pathology, n (%)
Trauma	82 (51)	96 (58)	94 (57)
Elective∗	79 (49)	69 (42)	71 (43)

Further details of elective conditions are listed in Table 3.

Table 3

Elective Conditions in Each Questionnaire Group∗

Elective Pathology	Questionnaire Group
	PRWHE	DASH	MHQ
Arthropathy	8 (5)	6 (4)	7 (4)
Trigger finger	7 (4)	8 (5)	11 (7)
Nerve compression	30 (19)	25 (15)	23 (14)
Tumor	4 (2)	4 (2)	1 (1)
Dupuytren disease	8 (5)	11 (7)	11 (7)
Ganglion	0	1 (1)	2 (1)
Undefined	22 (14)	14 (8)	16 (10)

Data are shown as n (%).

A total of 139 participants attended the clinic for a second visit during the study period, 131 of whom agreed to continue in the study, thus giving a retention rate of 94%. Eight participants withdrew from the study after round 1 and a further person withdrew after round 2. Five of those who withdrew were from the DASH cohort; the remaining 4 were from the MHQ cohort. There were no withdrawals from the PRWHE cohort. Participants who withdraw gave reasons such as a lack of interest, physical discomfort, and the length of the questionnaire.

Incomplete questionnaires

A total of 673 questionnaires were collected, 328 of which were missing at least one response (49%). The PRWHE and DASH had similar rates of total incomplete responses (35% and 30%, respectively), whereas the MHQ had a much higher rate (81%). The proportion of missing data for each questionnaire is shown in Table 4. When including the MHQ questions relating to only the side or sides of interest, the number of incomplete MHQs decreased to 33% (Table 5). The number of unanswered questions related to how many times the questionnaire was completed is provided in Table 6.

Table 4

Questionnaires Completed

Questionnaire	PRWHE	DASH	MHQ	Total
Items per PROM	15	30	62	107
PROMs returned	214	234	225	673
Items administered	3,210	7,020	13,950	24,180
Items missing response, n (%)	135 (4.2)	271 (3.9)	1,644 (11.8)	2,050 (8.5)
Items left unanswered in questionnaires, n
0	139	164	42	345
1	37	38	16	91
2–3	29	11	5	45
≥4	9	21	162	192
Incomplete questionnaires, n (%)	75 (35.0)	70 (29.9)	183 (81.3)	329 (49)
Questionnaires that could not generate score, n (%)	0	22 (9)	25 (11)	47 (7)

Table 5

Subanalysis of MHQ Responses

	All MHQ	Side of Interest Only
		Unilateral Condition	Bilateral Condition	Unilateral or Bilateral Condition
Items per PROM, n	62	37	62
PROMs completed, n	225∗	199	8	207
Items administered, n	13,950	7,363	496	7,859
Items missing response, n (%)	1,644 (11.8)	442 (6.0)	55 (11.0)	497 (6.3)
Incomplete questionnaires, n (%)	183 (81.3)	65 (32.7)	4 (50.0)	69 (33.3)
Questionnaires that could not generate score, n (%)	25 (11.1)	8 (4.0)	1 (12.5)	9 (4.3)

For 18 of the 225 MHQ PROMs, the side of interest was not known; these cases were included in the overall analysis.

Table 6

Completion Rate of Each Questionnaire Round

Questionnaire	All	PRWHE			DASH			MHQ
		Responses, n	Incomplete, n	Incomplete (%)	Responses, n	Incomplete, n	Incomplete (%)	Responses, n	Incomplete, n	Incomplete (%)
Total	673	214	75	35	234	70	29.9	225	183	81.3
Questionnaire round
1	486	160	57	35.6	161	56	34.8	165	133	80.6
2	131	38	14	36.8	49	12	24.4	44	38	86.4
3	42	12	3	25	16	1	6.3	14	10	71.4
4	11	4	1	25	5	2	40.0	2	2	100.0
5	3	0			3	0		0

The most commonly unanswered questions in each PROM were determined by the frequency of missing data per question (7). The MHQ had multiple questions that were unanswered equally often; as a result, these data could not be provided in Table 7. The likelihood of a question being unanswered in the MHQ increased with a later position in the questionnaire (Fig. 2).

Table 7

Unanswered Questions

Questionnaire	Item	Item Wording	% Unanswered
PRWHE	3	Rate your pain … When doing a task with a repeated wrist movement	19/214 = 9%
	14	Rate the amount of difficulty you experienced performing … Work (your job or usual everyday work)	19/214 = 9%
DASH	21	Rate your ability … Sexual activities	37/234 = 16%

Figure 2

Missing data in relation to question order.

Scoring

Questionnaires were scored according to the published guidelines relevant to each instrument. Whether a questionnaire could be scored depended on the scoring algorithm for that specific instrument. The PRWHE questionnaires all resulted in a score, because any missing values can be substituted with the mean value of that scale. The DASH had 22 questionnaires that could not generate a score (9.4%), because more than 3 answers were missing. A total of 25 MHQ questionnaires (11%) were unable to produce a score for either one or both hands; however, when the side of interest was considered, only 9 questionnaires could not be scored (4%). The DASH is statistically more likely to result in a questionnaire unable to be scored (χ2[1, n = 673] = 15.21; P < .001). The PRWHE is statistically more likely to be able to generate an overall score (χ2[1, n = 673] = 4.30; P < .001).

Time

Figure 3 shows the participant-reported time taken to complete each questionnaire. In total, 82% of participants reported that completing the questionnaire took less than 10 minutes, regardless of which questionnaire they received. The PRWHE was the quickest to complete; 60% of respondents reported being able to complete the questionnaire within 5 minutes.

Figure 3

Patient-reported time taken to complete questionnaires.

Participant characteristics

Chi-square test for independence showed no relationship between participants’ age or sex and whether they would return a score-generating questionnaire (χ2[3, n = 491] = 6.788; P > .05; and χ2(1, n = 491) = 0.06; P > .05, respectively). Whether participants had an elective or traumatic hand condition was also unrelated to the likelihood of them returning a questionnaire that could generate a score (χ2(1, n = 491) = 0.789; P > .05).

Discussion

Administration of PROMs in the clinic waiting room resulted in a participation rate of 85%, which is similar to other studies with hand surgery patients. Nota and colleagues performed a study comparing follow-up response rates with various forms of PROM administration. The participation rate in this study, in which PROMs were collected in the waiting room, was considerably higher than PROM data was collected by mail (34%) or e-mail (24%), and even slightly higher than when participants were asked to complete the PROM by phone interview (80%). The high participation rate in a study population with minimal exclusion criteria implies that this result may be applicable to the whole hand clinic population. This proposition is further supported by the finding that age and sex did not influence the likelihood of participation. The ability to collect longitudinal PROM data is helpful in tracking the progress of individuals and when trying to gauge the success of any intervention. This study demonstrated high retention rates of participants (94%), which may show that if patients are initially willing to complete a PROM, they will continue to do so throughout the course of treatment.

Although we did not directly measure the number of participants with a hand immobilized in a cast or orthosis in this study, it is reasonable to infer that a substantial proportion of those attending clinic would have had their hand(s) immobilized. The high participation rate despite the physical challenge associated with a hand injury and immobilization suggests that these physical barriers are less of an obstacle to PROM administration and completion than might have been expected. Participants were willing to use their time waiting in the clinics to complete the PROMs. It has been demonstrated that electronic administration of PROMs improved survey completion compared with traditional pen and paper., Electronic administration is associated with fewer omitted questions, which is attributed to the device prompting patients to complete unanswered questions. This study did not use electronic data collection; incorporating this technology may result in increased data quality.

It was expected that longer questionnaires would be more difficult and tiresome to complete, resulting in proportionally more missing data. The results of this study were that all 3 instruments had similar rates of incomplete responses (35% for PRWHE, 30% for DASH, and 33% for MHQ) despite the varying length of each PROM. This is based on the core questions of the MHQ related to the side or sides of interest. When the MHQ is considered in its entirety, the number of incomplete questionnaires increased to 81%. This shows that participants did not engage with questions that related to the unaffected limb even though the instructions on the questionnaire asked them to complete all questions. The question most commonly left unanswered in the DASH was about participants’ sexual activities. In the PRWHE, the problematic questions asked about pain with wrist movement and difficulty at work. These items seem not to engage with the respondent, potentially because the content was not relevant to the participant or was too personal for the respondent to disclose.

Although the MHQ had a higher amount of missing data overall, the number of questionnaires unable to generate a score for the affected side was lower than for the DASH (4% and 9%, respectively). The tolerance for missing data within the scoring rules for each PROM had an impact on the number of questionnaires that could be scored. The PRWHE scoring rules allow for any number of missing items to be substituted by a mean score. As a result, only those who failed to answer an entire section were unable to be scored. In contrast, the DASH and MHQ scoring rules are more stringent in their management of missing data. This unlimited substitution of the mean for missing answers may compromise the overall accuracy of the PRWHE; this should be critically considered before implementing this PROM.

One limitation of this study is that the time taken to complete the questionnaires was not objectively measured with a stopwatch; rather, it was self-reported in categories. As such, the time data may not be an accurate representation of the actual time taken by respondents. We chose this element of the study’s design to minimize the labor involved in administering the study, because timing each individual would have limited the number of total participants approached to enroll in the study. Although the time taken for questionnaire completion was not objectively measured, the perceived length of time taken from the patient’s perspective is relevant.

This study did not include PROMs in languages other than English, which may have caused non–English speaking participants to be excluded from the study. Therefore, the results of this study may not be applicable in a clinic with participants with diverse languages. This study was carried out in a government-funded hand clinic; thus, the results may not be transferable into the private medical sector where patients might spend less time waiting before the consultation.

This study measured the acceptability of a PROM-based research study to patients. Participants were approached and informed about the purpose of the study; they provided consent and were enrolled in the research project. As such, the acceptance rates could reflect patients’ willingness to participate in the study, rather than to complete a PROM for clinical care. If PROM administration were to become part of routine practice in the setting of hand clinics, participants may be even more willing to participate, especially if they could see that clinicians were reviewing the PROM data during the clinical consultation. This will be the focus of further research after routine PROM implementation.

According to this study, PROM administration in the hand clinic waiting room resulted in a high participation rate and good quality data collection; however, this mode of administration may be prone to bias. Dawson et al suggest that collecting follow-up PROM data at outpatient appointments is not ideal because of inconsistent follow-up schedules and owing to selection bias, in that patients with ongoing problems are more likely to return to clinic appointments. Therefore, any studies reporting on PROM data collected in the waiting room should consider these potential biases.

The practical implementation of PROMs into routine clinical care remains a challenge. The time, labor, and resources needed to design and implement PROM collection can be a barrier in a high-volume clinical practice setting. In this study, 2 members of the research team were present at the hand clinics to approach patients and collate the data, which may not be economically feasible in the long term. When considering routine administration of PROMs in hand clinics, rigorous exploration of acceptability, efficacy, and practicality is necessary to ensure optimal practice adoption. This study demonstrated that routine PROM data collection in the hand clinics of a busy tertiary referral hospital is widely acceptable to patients.