Two-Year Recall Bias After ACL Reconstruction Is Affected by Clinical Result.

Recall bias is a systematic error caused by inaccuracy in reporting past health status and can be a substantial methodological flaw in the retrospective collection of data. Little is known about recall bias following anterior cruciate ligament reconstruction (ACLR). The purpose of this study was to evaluate patients' recall bias regarding preinjury knee function at 2 years after ACLR.
METHODS: Patients undergoing ACLR were enrolled in an institutional ACL registry. Preoperatively and at 2 years postoperatively, patients quantified their preinjury knee function on a scale of 0 to 10 (10 = best). Recall bias was quantified as the difference in the reported preinjury function between the 2 time points. The clinical result of ACLR was evaluated according to the International Knee Documentation Committee (IKDC) Subjective Knee Evaluation score. Patients meeting the minimal clinically important difference (MCID) in the IKDC score were considered to have had a good outcome, while patients who did not reach the MCID were considered to have had failure of treatment. Recall bias was compared between the 2 groups.
RESULTS: Of 2,109 patients enrolled in the registry, 1,219 were included in the study. Patients with a good outcome remembered their preinjury knee function on a 0-to-10 scale to be better than what they reported at baseline, by a mean difference of 0.40 points (95% confidence interval [CI], 0.22 to 0.58 points). The recall bias was stronger for patients with a poor outcome, who remembered their knee function to be worse than reported at baseline, by a mean difference of -0.81 (95% CI, -1.4 to -0.26). The mean difference in recall between the 2 groups was -1.21 (95% CI, -1.74 to -0.67) (p < 0.0001).
CONCLUSIONS: The recall bias of preinjury knee function following ACLR was small and not clinically meaningful for the majority of patients. However, patients with a poor outcome had a clinically relevant and significant recall bias. CLINICAL RELEVANCE: Our findings suggest that patients with a poor outcome have a substantial recall bias. This has clinical relevance when considering treatment effects of revision surgery, whereby the clinical benefit of the treatment might be affected by recall bias.

The goal of anterior cruciate ligament (ACL) reconstruction (ACLR) is to provide a stable knee and restore the patient’s preinjury level of health and function. To be able to assess the effect of treatment, preinjury health status must be measured. While ideally, data are collected prospectively, it is not possible to collect prospective data from patients who are not yet injured. Preinjury functional status can therefore only be collected retrospectively. This introduces a potential source of misclassification of the preinjury health state, known as recall bias. Recall bias is a systematic error caused by inaccuracy in reporting past experiences, which can influence the patients’ response regarding health data[1,2]. Recall bias can lead to over- or underestimation of the treatment effect, yet the effect of recall bias has received little focus in the orthopaedic literature and, to our knowledge, has not previously been evaluated for patients undergoing ACLR. Large national and institutional ACL registries have been established to evaluate the safety and efficacy of ACLR[3]. Most registries collect preinjury data retrospectively (after the ACL injury, but before surgery) and are subject to recall bias. It is therefore useful for the validation of the registries to quantify the recall bias in the ACLR population.

The potential effect of clinical results following ACLR on the recall of preinjury health status, to our knowledge, has not been previously evaluated. The purpose of this study was to evaluate patients’ recall of the preinjury knee status at 2 years following ACLR. We hypothesized that recall bias would be affected by the clinical outcome of ACLR.

Materials and Methods

Participants and Data Collection

All patients who underwent ACLR between June 2009 and September 2013 were enrolled in an institutional ACL registry. Demographic data were collected, and intraoperative information was provided by the surgeon at the time of surgery. All patients in the registry were asked to complete the International Knee Documentation Committee (IKDC) Subjective Knee Evaluation form preoperatively (baseline) and at 2 years postoperatively. The IKDC score ranges from 0 points (worst level of function) to 100 points (highest level of function). The IKDC score has been validated for patients with ACL injuries[4,5]. The patients responded to the questionnaire independently and in writing. Responses to the baseline questionnaire were collected on paper at inclusion, while the 2-year questionnaire was administered electronically. The questionnaire has been validated for electronic administration[6]. Patients with complete baseline and 2-year follow-up data were included in the present study. Use of data from the ACL registry was approved by the institutional review board. All patients provided informed written consent at the time of inclusion. The overall clinical results from the registry were previously described[7].

Recall Measurement

The following question was asked preoperatively and at the 2-year follow-up, providing the basis for the recall measurement: how would you rate the function of your knee prior to the ACL rupture on a scale of 0 to 10, with 10 being normal, excellent function and 0 being the inability to perform any of your usual daily activities, which may include sports?

This question is distributed with the IKDC questionnaire but is not included in the calculation of the IKDC score. The primary outcome of this study was the difference in the reported preinjury knee function as assessed using the 10-point scale at these 2 time points. Any difference represents a change in recall.

Clinical Results

The patient-reported functional outcome of ACLR was evaluated with use of the IKDC form at 2 years. The minimal clinically important difference (MCID) of the IKDC score was determined using a distribution-based method, where the MCID was equal to one-half of the standard deviation (SD) of the mean change in IKDC scores between the preoperative and 2-year postoperative time points[8]. By this method, the MCID of the IKDC score in our population was found to be 9.6.

Failure of treatment was defined as a case that did not achieve a positive MCID. The change in recall of preinjury knee function as reported on the 10-point-scale question was then compared between patients who achieved a positive MCID of the IKDC score compared with patients classified as having failure of treatment.

Statistical Analysis

Descriptive statistics were used to describe the cohort of ACLR patients. Groups who reached or did not reach the MCID were compared using the 2-sample independent t test. Multivariable linear regression analysis was performed to determine independent predictors for the magnitude of recall bias. A p value of <0.05 was considered significant. The analysis was performed using SPSS (version 25; IBM).

Results

A total of 2,109 patients were enrolled in the ACL registry between June 1, 2009, and September 6, 2013. Of these patients, 1,219 (58%) had baseline and 2-year data available and were included in the study. The mean age at the time of surgery (and SD) was 31 ± 12.1 years (Table I). The MCID of the IKDC score was reached by 1,068 (87.6%) of the included patients, while 151 (12.4%) of the patients did not reach the MCID. The baseline and 2-year IKDC results are presented in Table II.

Recall of preinjury knee function depended on the clinical outcome of ACLR. We found that 63% of the patients with a good outcome (those who met the MCID of the IKDC score) compared with 51% of the patients with a poor outcome (those who did not reach the MCID at 2 years) rated their preinjury knee function identically at baseline and 2 years (p < 0.003) (Table III). Patients with a good outcome remembered preinjury knee function to be better than what they reported at baseline, by a mean difference of 0.40 points (95% confidence interval [CI], 0.22 to 0.58 points). Patients with a poor outcome remembered their knee function to be worse than what they reported at baseline, by a mean difference in their recollection of knee function of −0.81 (95% CI, −1.4 to −0.26) (Table IV). The mean difference between the 2 groups in the recall of preinjury knee function was −1.21 (95% CI, −1.74 to −0.67) (p < 0.0001).

A multivariable linear regression analysis demonstrated that the baseline IKDC score significantly influenced recall. The better the preoperative IKDC score, the less recall bias regarding preinjury knee function. The regression analysis also confirmed that the change in IKDC score over the 2-year follow-up period influenced recall bias (Table V). Age and sex did not influence recall bias.

Discussion

The overall finding in this study was that there was a small but not clinically meaningful recall bias regarding preinjury knee function. Patients who did not benefit from ACLR had a significantly stronger recall bias regarding their preinjury knee function compared with patients with a better outcome. Patients who did not achieve the MCID of the IKDC score remembered their preinjury knee function to be worse than what they reported preoperatively.

Although the patients in this study demonstrated significant recall bias, the effect size was relatively small. For patients achieving the MCID of the IKDC score, the mean change in recall of preinjury knee function was only 0.40 points on a scale of 0 to 10. This does not seem to be clinically relevant. The IKDC questionnaire includes a question to rate current knee function, with the same wording and 0-to-10-point scale as the question regarding preinjury knee function. On the basis of the results from this question (which is not subject to recall bias), the MCID for the change in knee function on this scale was 0.9. For patients with a poor outcome after ACLR, the change in recall of preinjury knee function was −0.81, approaching clinical importance on an absolute scale. However, when comparing the 2 groups, the difference in recall of preinjury knee function was −1.21, which was both significant and clinically meaningful.

Recall has been shown to be affected by the age of the patient[9,10]. It may not be surprising that the effect of recall is more pronounced in the elderly. However, in our cohort of young ACL patients (mean age of 31 years), age did not influence the recall bias.

In our study, patients who underwent ACLR demonstrated significant recall bias regarding preinjury knee function, but with a relatively small effect size demonstrated for most patients who had a successful outcome of ACLR. This finding is in line with previous reports. Stepan et al. evaluated 140 patients with hand and elbow diseases and found that recall for the preoperative abbreviated version of the Disabilities of the Arm, Shoulder and Hand score (QuickDASH) was acceptable up to 2 years after surgery[11]. In contrast, in a recent study by Gotlin et al., patients undergoing arthroscopic rotator cuff repair demonstrated poor recall for the American Shoulder and Elbow Surgeons (ASES) score 1 year after surgery[10]. Likewise, patients undergoing lumbar surgery have demonstrated poor recall of their preoperative status 1 year after surgery, remembering their preoperative symptoms to be significantly more severe than what they reported at baseline[12]. These studies indicate that recall bias varies according to the type and nature of the medical condition for which patients are being treated. Young and active patients who experience a traumatic injury, such as ACL rupture, may have different expectations of the treatment than patients seeking interventions for chronic conditions such as lumbar pain or degenerative shoulder issues.

Our findings are similar to the negative recall effect described for patients who experience traumatic events or diagnoses, who recall more risk factors compared with healthy controls[13]. This is explained by a psychological need to understand and explain the event, provoking memories of potential risk experiences in the past that controls do not recall. The quality of life reported by patients with unsuccessful ACLR can be extremely poor[14]. ACL injury can be a career-ending and life-changing event. These patients are often young with high expectations of engaging in competitive sports and/or recreational activities. Chronic pain and instability from an unsuccessful ACLR can therefore have a detrimental psychological effect on the patient that may partly explain our results. Patients who do not improve despite investing time and effort in their own treatment and rehabilitation can develop cognitive dissonance to explain the poor outcome[15]. This is a form of rationalism, where the degree of effort invested in a task will influence the evaluation of the result[16]. More studies are needed to understand the psychological effect on young patients living with chronic pain and dysfunction from failure of ACLR.

Limitations

The main limitation of this study is that we measured recall bias regarding a single question only, which may not represent the full spectrum of potential recall bias that may exist. Future research should be designed to measure the recall bias of entire validated patient-reported outcome measure (PROM) scores. This study was conducted in a single institution, namely a high-volume tertiary care hospital, which may limit the external validity. However, our results are mirrored by others. For example, we found that 12% of the patients experienced failure of treatment, which is identical to the 12% reported by the Norwegian Knee Ligament Registry[14]. Furthermore, true preinjury knee function was not known (no data had been collected before ACL rupture), nor did we adjust for differences in time from injury to inclusion.

Conclusions

In summary, we found that recall bias regarding preinjury knee function following ACLR was small and not clinically meaningful for the majority of patients. This supports the current method of retrospective collection of preinjury knee function data by ACL registries. However, our findings suggest that, unlike patients who do well after ACLR, patients with poor outcomes have a recall bias that may be both clinically relevant and significant. This is important to acknowledge, especially when considering treatment effects of revision surgery, whereby a patient’s recall of preinjury health status might demonstrate considerable bias depending on the outcome of the previous surgery.

TABLE I

Patient Demographics*

	All Patients (N = 1,219)	Patients Reaching the MCID of IKDC at 2 Yr (N = 1,068 [87.6%])	Patients Not Reaching the MCID of IKDC at 2 Yr (N = 151 [12.4%])
Age at surgery† (yr)	31 ± 12.1	30.8 ± 12.1	32.2 ± 11.7
BMI at surgery† (kg/m2)	24.6 ± 3.7	24.5 ± 3.6	25.2 ± 4.40
Male (no. [%])	650 (53)	570 (53.3)	80 (53)
Mechanism of injury (no. [%])
Contact	171 (14)	150 (14)	21 (13.9)
Noncontact	814 (66.8)	721 (67.5)	93 (61.6)
Missing data	234 (19.2)	197 (18.4)	37 (24.5)
Graft choice (no. [%])
Autograft	755 (61.9)
BPTB	437 (35.8)	397 (37.2)	40 (26.5)
Contralateral BPTB	1 (0.1)	0 (0)	1 (0.7)
Hamstring semitendinosus	94 (7.7)	82 (7.7)	12 (8.0)
Hamstring semitendinosus and gracilis	216 (17.7)	187 (17.5)	29 (19.2)
Quadriceps-bone	7 (0.6)	3 (0.3)	4 (2.7)
Allograft	272 (22.3)
BPTB	2 (0.2)	2 (0.2)	0 (0)
Hamstring semitendinosus	15 (1.2)	14 (1.3)	1 (0.7)
Hamstring semitendinosus and gracilis	3 (0.2)	2 (0.2)	1 (0.7)
Quadriceps-bone	7 (0.6)	7 (0.7)	0 (0)
ITB	1 (0.1)	1 (0.09)	0 (0)
Achilles tendon	217 (17.8)	184 (17.2)	33 (21.9)
Tibialis anterior	20 (1.6)	17 (1.6)	3 (2.0)
Tibialis posterior	7 (0.6)	7 (0.7)	0 (0)
Missing data	192 (15.8)	165 (15.4)	27 (17.9)

BMI = body mass index, BPTB = bone-patellar tendon-bone graft, quadriceps-bone = quadriceps tendon bone block graft, ITB = iliotibial band, MCID = minimal clinically important difference, and IKDC = International Knee Documentation Committee.

The values are given as the mean and standard deviation.

TABLE II

IKDC Scores After Anterior Cruciate Ligament Reconstruction*

	Patients Reaching the MCID of IKDC Score at 2 Yr (N = 1,068 [87.6%])			Patients Not Reaching the MCID of IKDC Score at 2 Yr (N = 151 [12.4%])
	Baseline Score†	2-Yr Score†	Mean Change (95% CI)	Baseline Score†	2-Yr Score†	Mean Change (95% CI)
IKDC score	50.2 ± 14.7	85.8 ± 10.8	35.6 (34.7 to 36.5)	66.9 ± 16.5	64.6 ± 20	−2.2 (−4.1 to −0.3)

IKDC = International Knee Documentation Committee, MCID = minimal clinically important difference, and CI = confidence interval.

The values are given as the mean and standard deviation.

TABLE III

Difference in Recall of Preinjury Knee Function*

Difference in Recall of Preinjury Knee Function	All Patients (N = 1,219)		Patients Reaching the MCID of IKDC (N = 1,068 [87.6%])		Patients Not Reaching the MCID of IKDC (N = 151 [12.4%])
	No.	%	No.	%	No.	%
−10	8	0.7	6	0.6	2	1.3
−9	7	0.6	3	0.3	4	2.6
−8	10	0.8	9	0.8	1	0.7
−7	18	1.5	15	1.4	3	2.0
−6	18	1.5	13	1.2	5	3.3
−5	18	1.5	8	0.7	10	6.6
−4	13	1.1	9	0.8	4	2.6
−3	16	1.3	11	1.0	5	3.3
−2	38	3.1	33	3.1	5	3.3
−1	78	6.4	68	6.4	10	6.6
0	754	61.9	677	63.4	77	51.0
1	52	4.3	45	4.2	7	4.6
2	37	3.0	31	2.9	6	4.0
3	22	1.8	20	1.9	2	1.3
4	15	1.2	14	1.3	1	0.7
5	24	2.0	24	2.2	0	0
6	20	1.6	15	1.4	5	3.3
7	20	1.6	19	1.8	1	0.7
8	14	1.1	13	1.2	1	0.7
9	8	0.7	8	0.7	0	0
10	29	2.4	27	2.5	2	1.3

The difference in recall of preinjury knee function reported at 2 years and baseline on a scale of 0 to 10 (10 = best). A negative difference indicates that the patient recalled the preinjury knee function to be worse at 2 years than at baseline. A difference of 0 indicates perfect recall of preinjury knee function. MCID = minimal clinically important difference, and IKDC = International Knee Documentation Committee.

TABLE IV

Recollection of Preinjury Knee Function as Reported at Baseline and 2 Years After ACLR*

	Baseline	2-Yr	Mean Difference	95% CI	P Value
All patients (n = 1,219)	8.88	9.13	0.25	0.08 to 0.43	0.005
Reached MCID of IKDC score (n = 1,068)	8.85	9.25	0.40	0.22 to 0.58	<0.001
Did not reach MCID of IKDC score (n = 151)	9.09	8.28	−0.81	−1.4 to −0.26	0.004

On a scale of 0 to 10 (10 = best). ACLR = anterior cruciate ligament reconstruction, CI = confidence interval, MCID = minimal clinically important difference, and IKDC = International Knee Documentation Committee.

TABLE V

Linear Regression Analysis Demonstrating the Effect of Age, Sex, Preoperative IKDC Score, and Change in IKDC Score on the Change in Recollection of Preinjury Knee Score*

Variable	Univariable Analysis		Multivariable Analysis
	Coefficient (95% CI)	P Value	Coefficient (95% CI)	P Value
Age	−0.01 (−0.02 to 0.01)	0.31	−0.01 (−0.02 to 0.01)	0.24
Female sex	0.02 (−0.34 to 0.37)	0.92	−0.01 (−0.37 to 0.34)	0.95
Preop. IKDC score	−0.02 (−0.03 to −0.01)	0.007	−0.02 (−0.03 to −0.01)	0.006
Change in IKDC score	0.02 (0.01 to 0.03)	<0.001	0.02 (0.01 to 0.03)	0.001

Change in recollection of preinjury knee score from baseline to 2 years following anterior cruciate ligament reconstruction. IKDC = International Knee Documentation Committee, and CI = confidence interval.