Hip and knee replacement patients prefer pen-and-paper questionnaires: Implications for future patient-reported outcome measure studies.

OBJECTIVES: Electronic forms of data collection have gained interest in recent years. In orthopaedics, little is known about patient preference regarding pen-and-paper or electronic questionnaires. We aimed to determine whether patients undergoing total hip (THR) or total knee replacement (TKR) prefer pen-and-paper or electronic questionnaires and to identify variables that predict preference for electronic questionnaires.
METHODS: We asked patients who participated in a multi-centre cohort study investigating improvement in health-related quality of life (HRQoL) after THR and TKR using pen-and-paper questionnaires, which mode of questionnaire they preferred. Patient age, gender, highest completed level of schooling, body mass index (BMI), comorbidities, indication for joint replacement and pre-operative HRQoL were compared between the groups preferring different modes of questionnaire. We then performed logistic regression analyses to investigate which variables independently predicted preference of electronic questionnaires.
RESULTS: A total of 565 THR patients and 387 TKR patients completed the preference question. Of the THR patients, 81.8% (95% confidence interval (CI) 78.4 to 84.7) preferred pen-and-paper questionnaires to electronic questionnaires, as did 86.8% (95% CI 83.1 to 89.8) of TKR patients. Younger age, male gender, higher completed level of schooling and higher BMI independently predicted preference of electronic questionnaires in THR patients. Younger age and higher completed level of schooling independently predicted preference of electronic questionnaires in TKR patients.
CONCLUSIONS: The majority of THR and TKR patients prefer pen-and-paper questionnaires. Patients who preferred electronic questionnaires differed from patients who preferred pen-and-paper questionnaires. Restricting the mode of patient-reported outcome measures to electronic questionnaires might introduce selection bias. Cite this article: Bone Joint Res 2013;2:238-44.

Based on the recent literature, we hypothesised that patients would prefer electronic questionnaires to pen-and-paper questionnaires after total hip and total knee replacement

We hypothesised that elderly patients would prefer pen-and-paper questionnaires

The vast majority of patients prefer pen-and-paper questionnaires

Patients who prefer electronic questionnaires are generally younger and have completed a higher level of schooling

Strengths: a large sample size and a limited extent of optimism in our statistical models

Limitations: This study was performed using pen-and-paper questionnaires, which might lead to an overestimation of the actual preference of the pen-and-paper questionnaires

Introduction

Traditionally, the assessment of outcome in orthopaedics has focussed on technical aspects. In total hip (THR) or knee replacement (TKR), the cumulative incidence of revision surgery is often used to compare the outcome of different implants or surgical techniques.[1] The underlying assumption of the traditional orthopaedic approach is that the technical aspects are the most important determinants of clinical success. However, a technically well-performed joint replacement does not guarantee clinical success, as no information is provided on functional status and pain. Additionally, the indication for revision surgery varies widely between orthopaedic surgeons.[2] Patient-reported outcome measures (PROMs), defined as questionnaires that are completed by patients, provide complementary information as they give an impression of a patient’s experience of the surgical procedure and their concerns with regard to health status, health-related quality of life (HRQoL) and the results of the treatment received.[3]

PROMs can be measured using traditional pen-and-paper questionnaires or various electronic counterparts including touch screens,[4] personal digital assistants,[5,6] tablets or mobile phones.[7] Expected advantages of electronic questionnaires include more complete data capturing, immediate availability of results and lower costs of administrating and entering data.[6,8]

On the other hand, electronic questionnaires may induce selection bias. A meta-analysis performed in 2008 showed that mail surveys had higher response rates than those based online.[9] A recent randomised controlled trial, in which 2400 patients were randomised to receive either a pen-and-paper questionnaire or an internet-based questionnaire at four years after THR, revealed an enormous difference in response rate: 92% for the pen-and-paper group versus 49% for the internet-based group.[10] Selection bias can occur if the association between exposure and outcome differs between participants and all eligible patients.[8]

To our knowledge, no study has investigated patient preference for electric questionnaires after THR and TKR. The majority of members of a senior citizens club prefers electronic to pen-and-paper questionnaires.[11] Given the similar age of THR/TKR patients, we would expect a preference for electronic questionnaires. We aimed to estimate the proportion of patients who prefer pen-and-paper questionnaires to electronic questionnaires and to estimate predictors of electronic questionnaire preference.

Materials and Methods

The current study is part of a multi-centre cohort study of HRQoL after THR/TKR (NTR2190), performed from August 2010 to August 2011.[12-15] Institutional review board approval was obtained from all participating centres and all patients gave written informed consent (CCMO-Nr:NL29018.058.09;MEC-Nr:P09.189). The data used in this report constitutes a subset of patients who underwent primary THR or TKR and who completed pre-operative HRQoL questionnaires along with a question regarding their preference for a mode of questionnaire at a mean of three years (1.5 to 6) after surgery.

We performed this study in order to investigate the preference for a mode of questionnaire for future studies in HRQoL after THR or TKR in a Dutch population. A prerequisite for such future studies is that patients can participate without outpatient department visits, thereby facilitating participation and forestalling the occurrence of selection bias. We selected a web-based questionnaire as the most feasible electronic option. At follow-up, we asked all THR and TKR patients which mode of questionnaire they preferred: pen-and-paper questionnaires or web-based electronic questionnaires, each completed at home.

In order to judge whether patients who preferred pen-and-paper questionnaires differed from patients who preferred electronic questionnaires, we compared age, gender, highest completed level of schooling, body mass index (BMI) categories (< 25 kg/m2, 25 to 30 kg/m2, 30 to 35 kg/m2, > 35 kg/m2), comorbidity, indication for joint replacement (osteoarthritis vs other indications) and pre-operative HRQoL between both groups.

We have aggregated the levels of schooling into an approximation of the social classes, on the assumption that level of schooling indexes the type of qualifications obtained, which in turn indicates the type of occupations available to the subject and hence their own adult social class.[13] Thus: ‘university, higher vocational education and preparatory higher vocational and scientific education’ have been aggregated as indicating the professional and managerial social classes; ‘middle vocational education and preparatory middle vocational education’ have been aggregated as indicating the skilled non-manual and manual social classes; and ‘lower vocational education, elementary schooling and no formal education’ have been aggregated as indicating the semi- and unskilled manual social classes.

Comorbidity was measured using a patient-reported Charnley classification (A, patients in which the index operated hip or knee are affected only; B, patients in which the other hip or knee is affected as well; and C, patients with a hip or knee replacement and other affected joints and/or a medical condition which affects the patients’ ability to ambulate).[16,17]

HRQoL was measured two weeks before TKR/THR, using the Dutch version of the Short-Form 36 (SF-36).[18,19] This questionnaire comprises 36 items covering eight domains (physical function, role physical, bodily pain, general health, vitality, social function, role emotional and mental health), for each of which a subscale score is calculated (100 indicating no symptoms and 0 indicating extreme symptoms). Additionally, these scales are incorporated into two summary measures: a physical component summary (PCS) and a mental component summary (MCS). Missing items were imputed whenever possible according to Ware et al.[19] We compared pre-operative PCS and MCS between both preference groups.

Statistical analysis

We performed all analyses separately for THR and TKR patients, as clinically important differences vary considerably between these patient groups.[20] We performed descriptive analyses of baseline patient characteristics. In order to predict which factors increased the probability of preference for electronic questionnaires, we performed multivariate mixed model logistic regression analyses. We considered the following potential predictors: age, gender, highest completed level of schooling category, BMI category, Charnley classification of comorbidity, indication for joint replacement and pre-operative PCS and MCS scores. In the mixed model regression analyses, patient preference was the dependent variable, all potential predictors were included as fixed effects and centre was included as a random effect. The explained variation was estimated using Nagelkerke’s generalised r2 and the discriminative ability was estimated using the area under the receiver operating characteristic (ROC) curve (AUC).[21] The extent of optimism in the r2 and AUC estimates was estimated using bootstrap resampling (n = 1000 bootstrap samples).[22-24] All analyses were performed using R v2.15.2 (R Development Core Team, Vienna, Austria).[25]

Results

Patient characteristics are shown in Table I. A total of 565 THR patients and 387 TKR patients completed the preference question. Pen-and-paper questionnaires were preferred by 462 THR patients (81.8% (95% confidence interval (CI) 78.4 to 84.7) and by 336 TKR patients (86.8% (95% CI 83.1 to 89.8)) (Table II).

Patient characteristics per preference group are shown in Table III and Table IV for THR and TKR patients, respectively. THR patients who preferred electronic questionnaires tended to be younger, more often male, more often obese, less comorbid, more often highly educated and had worse pre-operative physical health. Age, gender and highest completed level of education remained associated with mode of questionnaire preference while adjusting for age and gender (Table III). TKR patients who preferred electronic questionnaires were younger, more often male, less often morbidly obese, less often Charnley class B and more often Charnley class C, more often highly educated and had worse pre-operative physical health. Age and highest completed level of education remained associated with mode of questionnaire preference while adjusting for age and gender (Table IV).

Multivariate prediction of electronic questionnaire preference showed that lower age (p < 0.001), male gender (p < 0.001), higher completed level of schooling (p < 0.001) and higher BMI (p = 0.004) independently predicted preference of electronic questionnaires in THR patients (Table V). In TKR patients, multivariate prediction of electronic questionnaire preference showed that lower age (p < 0.001) and higher completed level of schooling (p < 0.001) independently predicted preference of electronic questionnaires (Table V). The prediction model for preference of electronic questionnaires in THR patients had an r2 of 0.31 with an optimism estimate of 0.04, yielding an optimism-corrected r2 estimate of 0.27. The AUC was 0.81, with an optimism estimate of -0.02, indicating absence of optimism. The prediction model for preference of electronic questionnaires in TKR patients had an r2 of 0.41 with an optimism estimate of -0.24, indicating absence of optimism. The AUC was 0.88, with an optimism estimate of -0.004, indicating absence of optimism.

Discussion

The vast majority of THR and TKR patients prefer pen-and-paper questionnaires. THR patients who prefer electronic questionnaires are younger, more often male, have completed higher levels of schooling and are more often obese. TKR patients who prefer electronic questionnaires are younger and have completed higher levels of schooling.

A limitation of our study is the mode of questionnaire used to capture the data. In this study, we invited patients to participate by conventional mail. Additionally, all questionnaires consisted of pen-and-paper questionnaires. Patients willing to participate in this study might be more inclined to prefer pen-and-paper questionnaires than THR and TKR patients in general, thus leading to an overestimation of the proportion of patients preferring pen-and-paper questionnaires. However, we consider it unlikely that the entire preference for pen-and-paper questionnaires is based on such selection bias. Additionally, the identified predictors for electronic questionnaire preference, such as age and completed level of schooling, are plausible, thereby indirectly validating our results.

Strengths of our study include the large sample size, which allows precise estimation and multivariate prediction of patient preference. Although the low r2 values indicate that not all variance is explained by the predictors, the high AUC values indicate that the prediction models have a high discriminatory ability. The limited extent of optimism in r2 and AUC estimates indicate that overfitting did not play a role in our study.[24] In other words, it is unlikely that the prediction models in this study have captured the peculiarities in this data set; conversely, it is likely that predictions, based on this data, will be generalisable to other, similar populations.

Unfortunately, we do not have any information on the availability of internet access of our patients. Although The Netherlands is rated as one of the most mature internet markets,[26] recent evidence suggests that non-users of the internet are more likely to be elderly,[27] which could explain the preference for pen-and-paper questionnaires. Practical advantages of electronic questionnaires are stressed in the current orthopaedic literature.[6,28] Patients are sometimes considered to prefer electronic questionnaires without any evidence supporting this claim.[28] Although electronic questionnaires certainly appear more efficient, our results reveal limitations in line with the findings of Rolfson et al.[10] Future studies, which only measure PROMs using electronic questionnaires, might suffer from limited generalisability, as elderly and less educated patients are less likely to participate. Moreover, selection bias might occur if the association of interest is related to age or social class.

When planning a study in which PROMs will be completed by THR and TKR patients at home, we recommend using pen-and-paper questionnaires, despite their logistic limitations. Such studies should at least provide the option of pen-and-paper questionnaires, in order to prevent selection bias by questionnaire mode.

Table I

Patient characteristics (THR, total hip replacement; TKR, total knee replacement)

Characteristic		THR (n = 565)		TKR (n = 387)
Mean (sd) age at THR/TKR (yrs)		65.9 (10.6)		68.9 (9.7)
Male (n, %)		198 (35.0)		126 (32.6)
Osteoarthritis (n, %)		486 (86.0)		346 (89.4)
Mean (sd) pre-operative Short-Form 36
Physical component summary		38.9 (9.61)		40.6 (9.53)
Mental component summary		51.8 (10.8)		51.5 (10.2)
Mean follow-up (yrs) (sd; range)		3.20 (1.13; 1.5 to 6.0)		3.14 (1.12; 1.3 to 6.0)
Body mass index at follow-up (n, %)
< 25 kg/m2		194 (34.3)		69 (17.8)
25 to 30 kg/m2		242 (42.9)		171 (44.1)
30 to 35 kg/m2		97 (17.1)		92 (23.8)
> 35 kg/m2		32 (5.7)		55 (14.2)
Charnley comorbidity classification (n, %)
A		123 (23.3)		54 (14.6)
B		75 (14.2)		39 (10.5)
C		331 (62.6)		278 (74.9)
Highest completed level of education (n, %)
University, higher vocational education and preparatory higher vocational & scientific education		115 (22.6)		52 (15.4)
Middle vocational education and preparatory middle vocational education		186 (36.6)		120 (35.5)
Lower vocational education, elementary schooling and no formal education		207 (40.7)		166 (49.1)

Table II

Proportion of patients who prefer pen-and-paper questionnaires to electronic questionnaires (THR, total hip replacement; TKR, total knee replacement; CI, confidence interval)

	THR (n = 565)			TKR (n = 387)
Preferred questionnaire	Patients (n)	Proportion (95% CI)		Patients (n)	Proportion (95% CI)
Pen-and-paper	462	81.8 (78.4 to 84.7)		336	86.8 (83.1 to 89.8)
Electronic	103	18.2		51	13.2

Table III

Comparison of total hip replacement (THR) patients preferring pen-and-paper versus electronic questionnaires (OR, odds ratio; CI, confidence interval)

Characteristic		Pen-and-paper	Electronic	Age- and gender- adjusted OR (95% CI)
Patients (n)		462	103
Mean (sd) age at THR (yrs)		67.5 (9.5)	58.5 (12.2)	0.93 (0.91 to 0.95)
Male (n, %)		140 (30.3)	58 (56.3)	0.35 (0.22 to 0.56)
Osteoarthritis (n, %)		406 (88.5)	77 (75.5)	0.75 (0.40 to 1.43)
Mean (sd) pre-operative Short-Form 36
Physical component summary		39.5 (8.9)	35.8 (11.9)	1.00 (0.97 to 1.03)
Mental component summary		51.2 (10.6)	54.8 (10.9)	1.01 (0.99 to 1.04)
Mean (sd) follow-up (yrs)		3.17 (1.13)	3.31 (1.11)	-
Body mass index at follow-up (n, %)
< 25 kg/m2		157 (35.4)	29 (29.3)	Reference
25 to 30 kg/m2		187 (42.1)	46 (46.5)	1.32 (0.75 to 2.32)
30 to 35 kg/m2		78 (17.6)	15 (15.2)	0.88 (0.42 to 1.83)
> 35 kg/m2		22 (5.0)	9 (9.1)	2.18 (0.84 to 5.69)
Charnley comorbidity classification (n, %)
A		100 (23.2)	23 (23.5)	Reference
B		57 (13.2)	18 (18.4)	1.28 (0.59 to 2.79)
C		274 (63.6)	57 (58.2)	1.10 (0.61 to 1.98)
Highest completed level of education (n, %)
University, higher vocational education and preparatory higher vocational & scientific education		78 (19.1)	37 (37.0)	Reference
Middle vocational education and preparatory middle vocational education		140 (34.3)	46 (46.0)	0.82 (0.47 to 1.45)
Lower vocational education, elementary schooling and no formal education		190 (46.6)	17 (17.0)	0.24 (0.12 to 0.47)

Table IV

Comparison of total knee replacement (TKR) patients preferring pen-and-paper versus electronic questionnaires (OR, odds ratio; CI, confidence interval)

Characteristic		Pen-and-paper	Electronic	Age- and gender- adjusted OR (95% CI)
Patients (n)		336	51
Mean (sd) age at TKR (yrs)		70.3 (8.9)	59.9 (9.7)	0.90 (0.86 to 0.93)
Male (n, %)		103 (30.7)	23 (45.1)	0.61 (0.31 to 1.18)
Osteoarthritis (n, %)		302 (90.4)	40 (81.6)	1.29 (0.48 to 3.45)
Mean (sd) pre-operative Short-Form 36
Physical component summary		41.1 (8.8)	36.7 (12.9)	1.02 (0.98 to 1.06)
Mental component summary		51.2 (10.4)	53.5 (8.9)	1.02 (0.99 to 1.06)
Mean (sd) follow-up (yrs)		3.10 (1.09)	3.44 (1.32)	-
Body mass index at follow-up (n, %)
< 25 kg/m2		55 (17.5)	10 (19.6)	Reference
25 to 30 kg/m2		136 (43.3)	25 (49.0)	0.86 (0.34 to 2.20)
30 to 35 kg/m2		75 (23.9)	12 (23.5)	0.65 (0.23 to 1.83)
> 35 kg/m2		48 (15.3)	4 (7.8)	0.41 (0.11 to 1.57)
Charnley comorbidity classification (n, %)
A		48 (14.9)	6 (12.2)	Reference
B		36 (11.2)	3 (6.1)	0.83 (0.18 to 3.83)
C		238 (73.9)	40 (81.6)	1.35 (0.52 to 3.52)
Highest completed level of education (n, %)
University, higher vocational education and preparatory higher vocational & scientific education		34 (11.7)	18 (37.5)	Reference
Middle vocational education and preparatory middle vocational education		96 (33.1)	24 (50.0)	0.55 (0.24 to 1.26)
Lower vocational education, elementary schooling and no formal education		160 (55.2)	6 (12.5)	0.08 (0.03 to 0.25)

Table V

Multivariate prediction for preference of electronic questionnaires in total hip and knee replacement patients. Odds ratios > 1 indicate higher odds of preferring an electronic questionnaire, per increasing predictor unit (CI, confidence interval; AUC, area under curve)

Characteristic*		Odds ratio (95% CI)	p-value
TOTAL HIP REPLACEMENT
r2 = 0.31; AUC = 0.81
Age at operation		0.93 (0.90 to 0.96)	< 0.001
Male vs female gender		0.31 (0.17 to 0.56)	< 0.001
Body mass index
25 to 30 kg/m2 vs < 25 kg/m2		2.06 (1.03 to 4.11)	0.04
30 to 35 kg/m2 vs < 25 kg/m2		1.17 (0.48 to 2.81)	0.73
> 35 kg/m2 vs < 25 kg/m2		5.49 (1.74 to 17.3)	0.004
Other indications vs osteoarthritis		0.59 (0.28 to 1.26)	0.17
Charnley comorbidity classification
A vs B		0.99 (0.40 to 2.42)	0.98
A vs C		0.87 (0.43 to 1.78)	0.70
Education
U+HVE+PHVSE vs MVE+PMVE		0.89 (0.45 to 1.77)	0.74
U+HVE+PHVSE vs LVE+ES+NFE		0.27 (0.12 to 0.59)	< 0.001
Short-Form 36 score
Physical component summary		1.00 (0.97 to 1.03)	0.80
Mental component summary		1.00 (0.98 to 1.03)	0.87
TOTAL KNEE REPLACEMENT
r2 = 0.41; AUC = 0.88
Age at operation		0.89 (0.84 to 0.94)	< 0.001
Male vs female gender		0.53 (0.21 to 1.34)	0.18
Body mass index
25 to 30 kg/m2 vs < 25 kg/m2		1.05 (0.26 to 4.28)	0.94
30 to 35 kg/m2 vs < 25 kg/m2		1.27 (0.30 to 5.38)	0.75
> 35 kg/m2 vs < 25 kg/m2		1.59 (0.28 to 8.91)	0.60
Other indications vs osteoarthritis		2.05 (0.53 to 7.89)	0.30
Charnley comorbidity classification
A vs B		1.40 (0.23 to 8.58)	0.72
A vs C		2.07 (0.58 to 7.31)	0.26
Education
U+HVE+PHVSE vs MVE+PMVE		0.33 (0.13 to 0.85)	0.02
U+HVE+PHVSE vs LVE+ES+NFE		0.04 (0.01 to 0.15)	< 0.001
Short-Form 36 score
Physical component summary		1.01 (0.96 to 1.06)	0.63
Mental component summary		0.99 (0.95 to 1.04)	0.75

* U+HVE+PHVSE, university, higher vocational education and preparatory higher vocational and scientific education; MVE+PMVE, middle vocational education and preparatory middle vocational education; LVE+ES+NFE, lower vocational education, elementary schooling and no formal education