The reliability of remembered pre-operative patient-rated wrist and hand evaluation (PRWHE) scores.

In clinical audits where pre-operative patient-rated wrist and hand evaluation (PRWHE) scores were not recorded, it would be useful if such scores could be recreated at the time of review. We recorded a PRWHE score during the last week before surgery for 143 patients. They were contacted after 21 months and asked to furnish a new PRWHE of the state they were in during the last week before surgery. 80 (56%) of the patients responded. The mean difference was 10 (SD: 20; SEM: 2) higher remembered pre-operative score. The limits of 95% agreement for individual scores were -29 and 50, while the 95% confidence interval of the mean was 6-15. If 10 is subtracted from the mean remembered preoperative score of a group of patients, the real pre-operative score will with 95% confidence be this score plus/minus 4. Remembered pre-operative PRWHE scores are far too inaccurate to be of value in individual patients. It may be possible to predict the mean real pre-operative PRWHE score in groups of patients with useful accuracy using the remembered pre-operative score.

Introduction

Patient rated outcome measures (PROMs) are often used when evaluating patients’ pain and function. The Patientrated wrist evaluation (PRWE) was introduced by MacDermid in 1996 as a selfadministered measure of pain and function in the wrist, particularly in relation to distal radial fractures.[1] It was validated also for other conditions in the hand in 2004 and became known as the patient-rated wrist/hand evaluation (PRWHE).[2] The questionnaire has two sections: one with five questions regarding pain and a functional section with ten questions. The questions are answered by using a numeric scale from 0 (No pain/no difficulty) to 10 (worst ever pain/unable to do). The PRWHE is scored so that the pain and function item are weighted equally. This is achieved by dividing the sum score for function by two before adding it to the sum of the pain score. The total PRWHE score grades from 0 (no pain/disability) to 100 (greatest pain/disability). Studies have found strong evidence for the reliability and validity of this tool,[3-5] and it has also been found to be easy and quick for the patient to complete.[3,6] It has been translated into many languages.[6-13]

Ideally, patients should evaluate their change in symptoms and function by completing the PRWHE both before treatment and at review. When a pre-operative score has not been obtained, the question arises whether patients are able to accurately recall their pre-treatment score in retrospect. Only a few studies have investigated patients’ ability to recreate their pre-operative PROM score months or years after surgery. Remembered Quick disability of the arm, shoulder and hand (quickDASH) has been studied.[14,15] These papers evaluated their results with correlation tests and concluded that remembered pre-operative scores were quite accurate. However, we feel that this method is inappropriate. It has been pointed out that such tests measure the relationship between two variables, not the agreement between them, and that data which seem to be in poor agreement can produce quite high correlations.[16,17] In a recent paper from our institution, we studied the accuracy of remembered pre-operative QuickDASH scores using Bland- Altman plots.[18] We found that both remembered pre-operative QuickDASH scores and VAS scores were far too inaccurate to be of use in individuals, but it seemed possible that they might be of use in groups of patients. To our knowledge, no study has investigated patients’ ability to recall their pre-operative PRWHE score months or years after treatment. The aim of the present study was to assess the accuracy of remembered pre-operative PRWHE scores in a consecutive series of patients.

Materials and Methods

All 170 patients scheduled for planned surgery of the wrist and hand at our hospital during the period February to June 2015 were asked on the day of surgery to complete the PRWEH questionnaire. We also included patients with compression of the ulnar nerve at the elbow as symptoms are mainly in the hand. Dupuytren’s disease was treated with collagenase injections. When only one answer in one or both of the two parts of the questionnaire had been omitted, the answer was imputed with the mean score of the answered questions in that section.[19] Responses with more than one missing answer in either section were discarded. We excluded patients with more than one surgery during the observation period. A total of 24 patients refused to participate. We were unable to find the correct address for one patient, another patient had moved abroad, and one patient had died. These 27 patients were excluded from the study. There were thus 143 patients for study of remembered pre-treatment PRWHE (Table 1). All received a letter with information about the study and two PRWHE questionnaires. One was to describe their current status, while the second asked them to answer as they remembered their condition during the week before surgery. The two questionnaires were clearly marked and printed on different colored paper to make it easy to distinguish between them. Letters that were returned because they were wrongly addressed were re-mailed to the correct address. Non-responders were sent new questionnaires after seven weeks. Patients who had not responded to the second mailing after 4 weeks were contacted by telephone and asked to return the questionnaire.

Table 1.

Main patient diagnoses.

	Mailed	No Reply	Incomplete PRWHE	Evaluated
Carpal tunnel syndrome	41	14	1	26
Thumb basal joint arthrosis	16	5	2	9
Hardware/foreign body removal	8	3	1	4
Dupuytren's contr. (collagenase)	9	1	0	8
Benign tumour	4	1	0	3
Ulnar nerve compression	10	6	0	4
Tendon/ligament surgery	7	3	0	4
TFCC injury - AS wrist	7	3	0	4
Ganglion	13	11	0	2
Finger amputation/arthrodesis	8	6	0	2
Distal radial fracture correction	5	5	0	0
Other, major	9	3	0	6
Other, minor	6	2	1	3
Total	143	63	5	75

Other minor: Cysts, trigger finger etc; Other major: Four corner arthrodesis, scaphoid non-union, metacarpophalangeal joint prosthesis etc.)

A total of 80 (56%) out of the 143 patients responded and were available for study. However, five of these patients had not returned a valid remembered pre-operative PRWHE and were excluded, leaving 75 patients for analysis. The mean age of these patients at the time of surgery was 52 (range 21-87) years. There were 43 (57%) women. The mean time between the real pre-operative score and the remembered preoperative score was 21 (18-25) months.

Statistics

Visual inspection of histograms of the scores were used to verify that the difference between remembered and real preoperative scores were normally distributed. The scores were shown in a scatter plot. A Bland-Altman plot of the scores was also constructed to visualize the difference between remembered and real pre-operative scores relative to their mean value.[20] The study was approved by the regional committee for medical and health research ethics for central Norway.

Results

All diagnoses groups had reported a higher median remembered score than the real pre-operative score (Table 2). The differences between remembered pre-operative scores and real pre-operative scores were normally distributed. The scatter plot (Figure 1) shows that the scores deviate considerably from the line y=x of perfect agreement while the Bland-Altman plot (Figure 2) shows that there is a systematic error between remembered and real preoperative scores. The remembered preoperative score was a mean score of 10 higher: (standard deviation (SD): 20; standard error. of the mean (SEM): 2) than the real pre-operative score. Lower and upper limits of 95% agreement for individual scores were -29 and 50, while the lower and upper limits of the 95% confidence interval of the mean were 6 and 15.

Table 2.

Median PRWHE scores for the most frequent diagnoses before treatment (real and remembered scores) and at review.

	Pre-operative		At review
	Real	Remembered
Carpal tunnel syndrome	47	50	11
Thumb basal joint arthrosis	64	66	24
Dupuytren's contr. (collagenase)	17	23	9
Other, major	26	51	29

Other major: Four corner arthrodesis, scaphoid non-union, metacarpophalangeal joint prosthesis etc.

Figure 1.

Scatter plot of the remembered against the real preoperative PRWHE score and the line of perfect agreement.

Figure 2.

Bland-Altman plot of the difference between the remembered and real pre-operative PRWHE scores plotted against the average of the two scores. The solid line indicates the mean difference. The dotted lines show the lower and upper limits of agreement.

Discussion and Conclusions

The response rate was above 50%, indicating a representative sample with little risk of response bias.[21] Patients were unaware that they would be asked to remember their pre-operative score. This reduces the risk of learning bias. The fact that there was approximately two years between obtaining the real and remembered pre-operative scores also reduces the risk of learning bias. This time period is also probably typical of the time before review in many follow-up studies. Our patients are an unselected, consecutive series of hand surgery patients. Patients had been treated for conditions where both high and low preoperative scores were to be expected. Nevertheless, the number of patients is limited and we cannot be sure that they represent all patients for whom the PRWHE is used.

In audits where pre-treatment PRWHE scores have not been recorded it still seems worthwhile to ask the patients to try to remember how they were before treatment. One must suppose that the difference between these scores and the one they give for their present status is an indication of how they themselves view the change in their condition. Many might consider this the most important measure of patient satisfaction. However, the convention is to compare the score at review with one obtained before treatment.

The lower and upper 95% limits of agreements of individual scores of -29 and 50 amply demonstrate that remembered pre-operative PRWHE scores are far too inaccurate to be of use in individual patients. The 95% confidence of the mean score, however, is much narrower than the limits for individual scores. If 10 is subtracted from the mean remembered preoperative score of a group of patients, the real pre-operative score will with 95% confidence be this score plus/minus 4 (SEM x 1.96), a total interval of 8.

Whether or not this inaccuracy is sufficiently small to make this calculated score useful, is a matter of judgement. Mean PRWHE scores are sometimes reported with one, or even two, decimal points, in effect, using a scale from 0 to 1000 or 10000.[3,10] This gives the impression that the scores are very precise. It is important to keep in mind that this is not the case. Anyone who has watched patients marking a PROM questionnaire will have observed the often haphazard way in which they do so. The reliability or stability of a questionnaire is calculated in a test-retest study where patients are asked to complete the questionnaire twice with an intervening period that is short enough that no change in the patients’ condition is to be expected to have occurred. It is usual to evaluate the results with the interclass correlation (ICC) test. An ICC of more than 0.75 is considered satisfactory.[22] The ICC for the PRW(H)E has variously been reported to lie between 0.88 and 0.99.[1,6,7,11,13,23] This may seem reassuring, however, Mehata et al.[4] reported on 50 distal radial fracture patients who completed the PRWHE with a two to three days interval. They found 95% limits of agreement of -11 and 24. Similarly, Schønnemann et al.[6] studied 29 patients with wrist fractures who completed the questionnaire with 3 days interval. The ICC was 0.88 and the 95% limits of agreement plus/minus 24.

Furthermore, it should be borne in mind that not all statistically significant differences in PRWHE scores are clinically relevant. The minimal clinically important differences (MCID) is the lowest score that corresponds with a simultaneous verbal statement from the patients that they are slightly worse or slightly better. The MCID has variously been reported to lie between 14 and 24.[24-26]

In the light of these inaccuracies and uncertainties inherent in the PRWHE itself, we conclude that it may be possible to predict the mean real pre-operative PRWHE score in groups of patients with useful accuracy using the remembered pre-operative score. In individual patients, the remembered pre-operative scores are far too inaccurate to be of value. However, we should like to emphasize that the real pre-operative scores should be used whenever possible.