Reliable Internal Consistency and Adequate Validity of the Forgotten Joint Score-12 after Primary Anterior Cruciate Ligament Repair.

PURPOSE: To validate the Forgotten Joint Score-12 (FJS-12) as a subjective outcome metric for patients undergoing arthroscopic primary anterior cruciate ligament (ACL) repair.
METHODS: In this retrospective study, all patients undergoing primary ACL repair for complete isolated proximal tears between 2008 and 2018 were eligible for inclusion. Patients were contacted and asked to complete the FJS-12, Lysholm Knee Score, modified Cincinnati score, Single Assessment Numeric Evaluation (SANE), Subjective International Knee Documentation Committee (IKDC), and Anterior Cruciate Ligament Return to Sport after Injury (ACL-RSI) scale at a minimum of 1 year of follow-up. Internal consistency, convergent validity, and ceiling effects were analyzed.
RESULTS: Ninety-six patients could be included (62%), with a mean FJS-12 score of 86.9 ± 14.0 (range, 31.3-100). The FJS-12 showed reliable internal consistency (Cronbach's alpha = 0.89). Construct validity was moderate to strong (r = 0. 621-0.702). Ceiling effect for the FJS-12 was 21.9%, which was lower than for the Lysholm (44.6%), modified Cincinnati (45.1%), equal to the IKDC subjective (21.9%), and SANE score (22.0%) but higher compared to the ACL-RSI (12.8%). Furthermore, internal consistency for the other scores was also good to excellent (range Cronbach's alpha = 0.627-0.953).
CONCLUSION: The FJS-12 shows high internal consistency and construct validity after primary ACL repair. Furthermore, this metric showed equal or less ceiling effect than most other scores, although still notable. This study suggests that the FJS-12 is an easy and validated outcome metric to evaluate subjective primary repair outcomes. LEVEL OF EVIDENCE: Level III, diagnostic study.

Arthroscopic primary repair of the anterior cruciate ligament (ACL) remains controversial but may be a promising and less morbid alternative to ACL reconstruction., Over the last decade, this procedure has gained popularity because of some potential inherent advantages. The main advantages of this technique are that the native ligament can be preserved, thus obviating the need for graft harvesting and an earlier return of range of motion. To evaluate and compare outcomes of ACL procedures, patient-reported outcomes measurements (PROMs) have been increasingly used over the last two decades.

With this increasing use of PROMs, additional scoring techniques have been designed to compare outcomes of ACL surgery. The Forgotten Joint-12 (FJS-12) questionnaire was initially designed for the evaluating of outcomes after arthroplasty surgery and is thought to allow improved discrimination in well-performing patients compared to other commonly used orthopaedic scores. A “forgotten joint” represents a well-functioning joint, because this requires absence of instability, pain, and stiffness, but it also integrates other variables, including patient’s expectations and psychological factors. Therefore this metric can be considered an important tool to assess subjective outcomes after surgery. More recently, this tool has also been introduced to evaluate clinical outcomes after ACL reconstruction. Although the FJS-12 has been shown to be a valid and responsive outcome metric after ACL reconstruction,, this questionnaire has not yet been validated to evaluate joint awareness after primary ACL repair. As repair procedures are increasingly advocated in the orthopaedic literature, it is important to validate this outcome metric.

The purpose of the present study was to validate the FJS-12 as a subjective outcome metric in patients undergoing arthroscopic primary ACL repair. The hypothesis was that the FJS-12 would show excellent internal consistency and strong construct validity compared with the current widely used ACL outcome metrics.

Methods

Patient Selection

This study includes data on previously reported patients.12 Institutional review board approval was obtained before study initiation (IRB 2017-0404-CR2). All patients treated with arthroscopic primary repair for isolated proximal ACL injuries between April 2008 and December 2018 by the senior author (G.S.D.) were screened for eligibility in this retrospective study. The same surgical intraoperative treatment algorithm was applied to all patients, which has been previously described more extensively., Patients were considered for inclusion if a minimum of 2 years’ follow-up was present and were excluded when failure of treatment had occurred (defined as side-to-side difference >3 mm, grade ≥2 Lachman or grade ≥2 pivot shift test result, or a subjective feeling of instability), or treated with primary repair for distal ACL tears. Patients with meniscal and chondral lesions were not excluded for this study.

Eligible patients were asked to complete several outcome questionnaires to assess patient-reported outcomes either in clinic or via mail. This questionnaire was only completed after surgery because primary ACL repair is generally performed in the acute setting. As a result, most patients do not experience a chronic instability situation with ACL deficiency.

Surgical Technique

Previously, a detailed description of the surgical technique for primary ACL repair using dual suture anchor fixation has been published., Although primary ACL repair is preferably performed in the acute setting, it should be noted that some chronic tears can be repaired when tissue quality and tissue length are sufficient. It should be noted, however, that a suture augmentation, consisting of an InternalBrace (Arthrex, Naples, FL), became recently clinically available and became standard of care in all patients, as also previously described. The suture tape augmentation is thought to protect the repaired ligament during the early phase of rehabilitation, which allows for earlier mobilization.

Postoperative Management

After surgery, the same rehabilitation protocol focusing on early mobilization combined with a knee brace worn for the first 4 postoperative weeks was followed for all patients. Passive range of motion exercises and swelling control were initiated within the first days after surgery. After 4 weeks, physical therapy was continued as standardized ACL reconstruction protocol prescribes. Running was allowed at 3 months, whereas return to pivoting sports was allowed at 6 to 12 months when patient’s confidence, range of motion, and quadriceps muscle strength (>90% of contralateral leg) were sufficient.

Forgotten Joint Score-12

The FJS-12 is composed of 12 questions with a 5-point Likert response developed to evaluate how much patients think about or are bothered by their affected knee joint during everyday life (Appendix Table 1). Item scores are summed and converted to a scale ranging from 0 to 100, with higher scores indicating a higher level of forgetting the operated joint during daily activities (i.e., a more favorable outcome).

Table 1

 Patient Demographics After Primary ACL Repair

	All Patients (n = 96)
	Mean ± SD	Range
Age (years)	35.3 ± 12.3	12-60
Male gender	52 (54.2%)
Right side	53 (55.2%)
Delay (days injury-surgery)∗	31	19-94
BMI (kg/m2)	25.1 ± 3.3	18.9-35.0
Preinjury Tegner	6.6 ± 1.6	3-10
Injury mechanism
Skiing	23 (24.0%)
Soccer	17 (17.7%)
Basketball	15 (15.6%)
Other	41 (42.7%)
FU (years)	2.8 ± 1.3	2.0 –10.6
Concomitant damage
Meniscus injury	49 (51.0%)
Chondral injury	26 (27.1%)
Repair + SA	57 (59.4%)

ACL, anterior cruciate ligament; BMI, body mass index; FU, follow-up; IQR, interquartile range; SA, suture augmentation.

Reported in median with IQR; N, number of patients.

Established PROMs

Besides the FJS-12, patients were also asked to complete the Lysholm Knee Score, modified Cincinnati Score, Single Assessment Numeric Evaluation (SANE), International Knee Documentation Committee (IKDC) subjective form, and Anterior Cruciate Ligament Return to Sport After Injury (ACL-RSI) scale (short-version). All scores have been widely used and validated after ACL surgery.

Statistical Analysis

All statistical analyses were performed using SPSS, version 25 (IBM Corporation, Armonk, NY). Continuous variables were compared between groups by performing Independent t-tests, and categorical variables were assessed by performing χ2 testing. To assess unidimensionality of the FJS-12, Cronbach’s alpha and item-total correlations were calculated. Excellent internal consistency was defined as Cronbach’s alpha greater than 0.9. Convergent validity between the FJS-12 and the established PROMs were assessed using Spearman correlation coefficients. This was defined as weak correlation (0.3 < r <0.1), moderate (0.7 < r <0.3), or strong (r ≥ 0.7). In addition, the item-level nonresponse rate was assessed. Finally, ceiling and floor effects were calculated by determining the number of patients with the lowest and highest possible score, and percentages greater than 15% were considered to indicate a pronounced ceiling effect. All comparative analyses were 2-sided, and P < .05 was used as the threshold for statistical significance.

Results

Patient Demographics

In total, 188 patients underwent primary ACL repair during the study period. Twenty-seven were excluded because of failure, and 5 were excluded because of treatment with primary repair for distal ACL tears, so 156 patients were deemed eligible for inclusion and were asked to complete outcome questionnaires. Sixty patients did not complete the questionnaires or could not be contacted. Therefore 96 patients with complete follow-up questionnaires were ultimately included in this study. The mean age was 35.3 ± 12.3 (range, 12-60) years, 54% were males, mean body mass index (BMI) was 25.1 ± 3.3, and 55% of patients underwent right-sided knee surgery. The median time from injury to surgery was 31 (interquartile range, 19-94) days, and mean follow-up time was 2.8 ± 1.3 years. In 57 patients (59.4%), an additional suture augmentation was added to the repaired ligament. Demographic data are further detailed in Table 1.

Internal Consistency

For all patients, the mean FJS-12 was 86.9 ± 14.0 (range, 31.3-100.0; interquartile range, 79.2-97.9). Internal consistency, measuring correlation among the 12 items, was reliable with a Cronbach’s alpha coefficient of 0.89. Inter-total correlations on the FJS-12 items ranged between 0.452 and 0.742. When stratified by treatment, Cronbach’s alpha was increased for all items in patients treated with versus without suture augmentation (0.923 vs. 0.802). Furthermore, internal consistency for the other scores was moderate to excellent (range Cronbach’s alpha = 0.627-0.953; Table 2).

Table 2

Cronbach’s alpha of all collected PROMs

Variables	All patients (n = 96)	Repair (n = 39)	Repair + SA (n = 57)
FJS-12	0.886	0.802	0.923
Lysholm∗	0.627	0.485	0.695
Cincinnati†	0.852	0.868	0.856
IKDC Subjective	0.892	0.881	0.903
ACL-RSI‡	0.953	0.907	0.974

ACL-RSI, Anterior Cruciate Ligament Return to Sport After Injury scale; FJS-12, Forgotten Joint Score-12; IKDC, International Knee Documentation Committee; PROMs, patient-reported outcome measurements; SA, suture augmentation; SANE, Single Assessment Numeric Evaluation.

Reported in 55 patients.

Reported in 54 patients.

Reported in 49 patients.

Correlation Coefficients

The mean Lysholm score was 93.8 ± 8.3, modified Cincinnati was 93.8 ± 8.5, SANE was 90.5 ± 10.2, IKDC Subjective was 91.3 ± 8.4, and ACL-RSI was 75.9 ± 23.7. Using Pearson’s correlation coefficient, the FJS-12 showed a strong correlation with the modified Cincinnati score (r = 0.702), whereas moderate correlation was found for the Lysholm score (r = 0.652), SANE score (r = 0.621), IKDC subjective score (r = 0.660), and ACL-RSI score (r = 0.666). Moreover, moderate to strong correlations were also found among the other scores (range r = 0.561-0.779). All correlations were statistically significant (all P < .001; Table 3).

Table 3

Spearman’ Correlation Coefficient of All Collected PROMs∗

Variables	FJS-12	Lysholm	Cincinnati	SANE	IKDC Subjective	ACL-RSI
FJS-12	—	0.652	0.702	0.621	0.660	0.666
Lysholm†	0.652	—	0.670	0.546	0.650	0.561
Cincinnati‡	0.702	0.670	—	0.599	0.779	0.629
SANE‡	0.621	0.556	0.599	—	0.569	0.645
IKDC Subjective	0.660	0.650	0.779	0.569	—	0.613
ACL-RSI§	0.666	0.561	0.629	0.645	0.613	—

ACL-RSI, Anterior Cruciate Ligament Return to Sport After Injury scale; FJS-12, Forgotten Joint Score-12; IKDC, International Knee Documentation Committee; PROMs, patient-reported outcome measurements; SANE, Single Assessment Numeric Evaluation.

All P values were significant at <.001.

Reported in 55 patients.

Reported in 54 patients.

Reported in 49 patients.

Nonresponse Rate and Ceiling Effects

There were no missing responses on the FJS-12 questionnaire and the other scores in this cohort. Twenty-one patients (21.9%) scored the highest possible FJS-12 score (Table 4). A more pronounced ceiling effect was found for the Lysholm score (44.6%), and modified Cincinnati (45.1%), whereas a similar ceiling effect was found for the IKDC subjective score (21.9%) and SANE score (22.0%). On the contrary, the ACL-RSI score (12.8%) showed a lower ceiling effect. No pronounced floor effect was found for any of the reviewed scores (all < 15%; Table 4). Box and whiskers plot distributions are presented in Figure 1.

Table 4

Ceiling and Floor Effects of the Collected PROMs

Variables	Ceiling Effect	Floor Effect
FJS-12	21 (21.9%)	0 (0.0%)
Lysholm∗	41 (44.6%)	0 (0.0%)
Cincinnati†	41 (45.1%)	0 (0.0%)
SANE†	20 (22.0%)	0 (0.0%)
IKDC Subjective	21 (21.9%)	0 (0.0%)
ACL-RSI‡	11 (12.8%)	2 (2.4%)

ACL-RSI, Anterior Cruciate Ligament Return to Sport After Injury scale; FJS-12, Forgotten Joint Score-12; IKDC, International Knee Documentation Committee; PROMs, patient-reported outcome measurements; SANE, Single Assessment Numeric Evaluation.

Reported in 55 patients.

Reported in 54 patients.

Reported in 49 patients.

Fig 1

Box plot distribution of all analyzed patient-reported outcomes measurements. In each box-plot, values are presented as median (line), interquartile range (box), and minimum (lower whisker) and maximum (upper whisker). Outliers are depicted by circles (small outliers) and stars (extreme outliers). (ACL-RSI, Anterior Cruciate Ligament Return to Sport After Injury Scale; FJS-12, Forgotten Joint Score-12; IKDC, International Knee Documentation Committee Subjective form; SANE, Single Assessment Numeric Evaluation.)

Subgroup Analysis

After primary repair, no statistical differences were noted in FJS-12 scores between patients younger and older than 35 years of age (84.7 ± 15.9 vs. 88.5 ± 12.3; P = .182), nor between male and female patients (86.1 ± 15.3 vs. 87.8 ± 12.4; P = .538). In addition, there were no differences in FJS-12 scores in patients with BMI < 25 compared to > 25 (86.0 ± 15.3 vs. 87.9 ± 12.4; P = .520), with and without meniscal injuries (85.7 ± 14.7 vs. 88.1 ± 13.3; P = .394), with and without chondral lesions (89.0 ± 11.2 vs. 86.1 ± 14.9; P = .361), and in those treated with and without suture augmentation (88.5 ± 14.8 vs. 84.5 ± 12.5; P = .163).

Discussion

The main finding of this study is that the FJS-12 has shown reliable internal consistency and adequate convergent validity after primary ACL repair as compared with the current widely used ACL outcome metrics. Although ceiling effect was notable, equal or a less pronounced effect and improved score distributions were found for the FJS-12 compared to the traditional PROMs including Lysholm, modified Cincinnati, IKDC subjective, and SANE scores. Therefore the FJS-12 seems a suitable and validated metric to evaluate subjective knee function after primary ACL repair.

In this validation study, the FJS-12 showed reliable internal consistency (Cronbach’s alpha of 0.89). Over the last decade, this score has been extensively validated to evaluate outcomes after different arthroplasty procedures and, more recently, also after ACL reconstruction.,, Behrend et al. showed high internal consistency (Cronbach’s alpha of 0.95) and considerably less ceiling effects than most Knee Injury and Osteoarthritis Outcome Scores and all Western Ontario and McMaster Universities Arthritis Index subscales. Similarly, Lee et al. found excellent internal consistency for the FJS-12 score after ACL reconstruction (Cronbach’s alpha of 0.9), whereas being normally distributed and showing lower ceiling effects compared to the Lysholm and Tegner scales. Because the FJS-12 may enable clinicians to assess and monitor functional outcomes over a prolonged period using the same metric, this validated tool seems useful to measure multiple facets of ACL surgery, including the potential development of posttraumatic osteoarthritis.

When reviewing ceiling effects in this study, it was noted that at least an equal, or a less pronounced ceiling effect was found for the FJS-12 (21.9%) as compared to the Lysholm (44.6%), modified Cincinnati (45.1%), IKDC subjective (21.9%), and SANE scores (22.0%). Furthermore, this study showed that FJS-12 has a greater scale distribution than most established PROMs. However, it should be noted that more than 15% of patients reached the highest possible FJS-12 score. Ceiling effects beyond this threshold are usually considered pronounced, although some studies have suggested that ceiling effects up to 30% are acceptable. This may potentially limit the discrimination power of the FJS-12 in well-performing repair patients. Nonetheless, the present study indicates that the FJS-12 had considerably less ceiling effect than the Lysholm, modified Cincinnati, and IKDC Subjective scores, which are currently one of the most widely used and validated outcome metrics in the ACL literature.

When specifically looking at previous studies validating the FJS-12 after ACL reconstruction, both studies showed less ceiling effect of the FJS-12 (range, 8.5%-15.5%) than that found in the present study (21.9%)., One of the potential explanations for this finding might be related to the invasiveness of both techniques. By preserving native tissue, we have previously shown that repair patients may have significantly less joint awareness than those treated with ACL reconstruction (85.3 vs. 74.5, respectively). Furthermore, more patients had no joint awareness at all (FJS-12 score of 100) after primary ACL repair than after ACL reconstruction in that study (22% vs. 9%, respectively). Therefore it seems that repair surgery may lead to closer restoration of native knee function than ACL reconstruction, although future prospective studies are certainly needed to test this hypothesis. In addition, the FJS score may also be a better outcome metric to assess subjective outcomes after both primary repair and ACL reconstruction surgery, because ceiling effects of other metrics are even higher in both the repair and reconstruction literature.

The FJS-12 showed a moderate to strong correlation with other PROMs in this study (r = 0.621-0.702), which were designed for and previously validated in the ACL literature.17, 18, 19, 20, More specifically, a strong correlation with the Cincinnati score was found, whereas other validated scores showed a moderate correlation with the FJS-12. A previous validation study for the FJS-12 in ACL reconstruction patients reported a weak correlation with Lysholm scores (Spearman’s correlation = 0.23) but did not assess other domains. It remains unclear why this PROM showed a smaller correlation, but the high observed ceiling effect of the Lysholm score may have affected this outcome. Taken these factors into account, the FJS-12 seems to provide a robust approach in evaluating overall knee function after ACL primary repair.

In the literature, although several outcomes metrics have been commonly used to assess and compare primary ACL repair outcomes, these metrics have not been validated for this procedure. Therefore the present study has also assessed the validity of other frequently used metrics, which all showed good to excellent internal consistency. Nevertheless, we suggest that the FJS-12, SANE, Subjective IKDC, and ACL-RSI should be used to assess functional outcomes after primary repair, although we would not rely solely on the Lysholm and modified Cincinnati scores because these scores are less sensitive to compare results in well-functioning patients.

Finally, this study showed that there were no statistical differences in joint awareness between patients treated with and without suture augmentation, nor in age, gender, with and without meniscal injury, and BMI. Future studies are needed to evaluate these parameters.

Limitations

There are limitations to this study. First, no repeated measurements were performed, and thus no test-retest reliability could be determined. Second, preoperative scores were not available, and future studies are therefore also needed to calculate the clinically meaningful difference. However, most patients treated with primary repair are treated in the acute setting, and no preoperative scores can be reliably assessed in this acute setting. In addition, the loss to follow-up was relatively high (38%). However, we believe that this does not affect the internal consistency and construct validity of the FJS-12 after ACL repair. Moreover, most patients were recreational athletes, which may limit the generalizability of this study. Finally, this validation study was on successful repair cases only. Patients with failed ACL repairs will undergo ACL reconstruction within weeks, and collecting outcomes on these patients does not provide valuable information; either you are only assessing the acute reruptured knee with hemarthrosis and pain, or they have already undergone ACL reconstruction.

Conclusion

The FJS-12 shows high internal consistency and construct validity after primary ACL repair. Furthermore, this metric showed equal or less ceiling effect than most other scores, although still notable. This study suggests that the FJS-12 is an easy and validated outcome metric to evaluate subjective primary repair outcomes.

Appendix Table 1

The Forgotten Joint Score-12

Are you aware of you knee joint …

1. … in bed at night?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

2. … when you are sitting on a chair for more than 1 hour?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

3. ... when you are walking for more than 15 min?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

4. ... when you are taking a bath/shower?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

5. ... when you are traveling in a car?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

6. ... when you are climbing stairs?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

7. ... when you are walking on uneven ground?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

8. ... when you are standing up from a low-sitting position?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

9. ... when you are standing for long periods of time?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

10. ... when you are doing housework or gardening?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

11. ... when you are taking a walk/hiking?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

12. ... when you are doing your favorite sport?

○ never ○ almost never ○ seldom ○ sometimes ○ mostly

Scoring: For scoring the Forgotten Joint Score-12(FJS-12), all responses are summed (never, 0 points; almost never, 1 point; seldom, 2 points; sometimes, 3 points; mostly, 4 points) and then divided into the number of completed items. This mean value is subsequently multiplied by 25 to obtain a total score range of 0 to 100. Finally, the score is subtracted from 100, to change the direction of the final score in a way that high scores indicate a high degree of “forgetting” the knee joint, that is, a low degree of awareness.

If more than 4 responses are missing, the total score should not be used.