The Forgotten Joint Score patient-acceptable symptom state following primary total hip arthroplasty.

AIMS: The Forgotten Joint Score-12 (FJS-12) is a validated patient-reported outcome measure (PROM) tool designed to assess artificial prosthesis awareness during daily activities following total hip arthroplasty (THA). The patient-acceptable symptom state (PASS) is the minimum cut-off value that corresponds to a patient's satisfactory state-of-health. Despite the validity and reliability of the FJS-12 having been previously demonstrated, the PASS has yet to be clearly defined. This study aims to define the PASS of the FJS-12 following primary THA.
METHODS: We retrospectively reviewed all patients who underwent primary elective THA from 2019 to 2020, and answered both the FJS-12 and the Hip Disability and Osteoarthritis Outcome Score, Joint Replacement (HOOS, JR) questionnaires one-year postoperatively. HOOS, JR score was used as the anchor to estimate the PASS of FJS-12. Two statistical methods were employed: the receiver operating characteristic (ROC) curve point, which maximized the Youden index; and 75th percentile of the cumulative percentage curve of patients who had the HOOS, JR score difference larger than the cut-off value.
RESULTS: This study included 780 patients. The mean one-year FJS-12 score was 65.42 (SD 28.59). The mean one-year HOOS, JR score was 82.70 (SD 16.57). A high positive correlation between FJS-12 and HOOS, JR was found (r = 0.74; p<0.001), making the HOOS, JR a valid external anchor. The threshold score of the FJS-12 that maximized the sensitivity and specificity for detecting a PASS was 66.68 (area under the curve = 0.8). The cut-off score value computed with the 75th percentile approach was 92.20.
CONCLUSION: The PASS threshold for the FJS-12 at one year following primary THA was 66.68 and 92.20 using the ROC curve and 75th percentile approaches, respectively. These values can be used to achieve consensus about meaningful postoperative improvement to maximize the utility of the FJS-12 to evaluate and counsel patients undergoing THA. Cite this article: Bone Jt Open 2022;3(4):307-313.

Introduction

Patient-reported outcome measures (PROMs) are routinely used to evaluate preoperative and postoperative symptom states of patients undergoing procedures, such as total hip arthroplasty (THA). PROMs are self-reported measures that are usually survey-based, and include subjective measures that can be quantified, such as pain, functional status, prosthesis awareness, satisfaction, and health-related quality of life. These questionnaires can be broken down into a dichotomy between generic factors, such as general health and specific factors which focus on disease and body function. Within arthroplasty, there is a focus on joint-specific PROMs; however, variation remains in how joint-related health is measured.

Most recently, the Forgotten Joint Score-12 (FJS-12) has become among one of the most commonly used hip-specific measures. First developed in 2012, the FJS-12 is a joint-specific questionnaire that focuses on the patients’ awareness of their prosthetic joint during daily activities, and has been shown to have a low ceiling effect compared to other PROMs. This enables it to better discern between patients achieving good and excellent results. Furthermore, the FJS-12 has since been validated in multiple languages.

With the number of THAs performed annually expected to continue rising, finding optimal methods to assess a patient’s postoperative outcomes is imperative. Although PROMs may be valuable in comparison of various surgical treatments and differences between distinct population groups, clinical interpretation of these differences can sometimes be misleading. For instance, a statistically significant mean change (p-value < 0.05) in scores may not necessarily translate into a considerable change for patients clinically. In order to improve the interpretability of PROMs and contextualize these scores, different clinically meaningful cut-off points have been previously established. One of the most commonly reported thresholds is the patient-acceptable symptom state (PASS). It is referred to the minimum cut-off value beyond which a patient considers their postoperative outcome satisfactory therefore it is most closely associated with patient satisfaction. PASS values can act as clinically significant benchmarks for procedures, as clinically relevant patient-centered outcomes for research, and as guides for surgeons to contextualize a patient’s postoperative symptom state.

Despite previous studies having demonstrated the validity and reliability of the FJS-12, the PASS value for this questionnaire remains relatively unexplored in the literature. This study aims to define the PASS value for the FJS-12 at one year following primary THA using the Hip disability and Osteoarthritis Outcome Score, Joint Replacement (HOOS, JR) as an anchor.

Methods

Using the arthroplasty registry of a tertiary academic orthopaedic speciality hospital, we retrospectively reviewed all patients who underwent primary elective THA from January 2019 to August 2020 and answered both the FJS-12 and the HOOS, JR questionnaires one-year postoperatively. THAs performed using both the posterior and anterior approaches were included. Patients undergoing bilateral or revision THA, as well as THA performed for non-elective or oncologic reasons, and patients under the age of 18 years were excluded. Institutional Review Board approval was obtained before initiating this analysis.

All 780 patients included in this study participated in our institutional-wide comprehensive total joint pathway programme, which encompasses uniform standardized protocols for all aspects of perioperative care. In addition, a standard institutional postoperative rehabilitation protocol, as well as a standard postoperative pain protocol, were followed for all patients.

As part of our institutional standard of care, at the time of surgical scheduling, patients were registered for an electronic patient rehabilitation application (EPRA; Force Therapeutics, USA) by clinical care coordinators. The EPRA is a mobile and web-based technology that wirelessly delivers digital PROM surveys to patients at pre-defined time intervals. This application was used to collect the FJS-12 and HOOS, JR scores at one-year follow-up. Demographic data were extracted from our institution’s electronic medical record system (version 15; Epic Caboodle, USA) using SQL Server Management Studio 2017 (Microsoft, USA), which included age, sex, race, BMI, American Society of Anesthesiology (ASA) classification, and Charlson Comorbidity Index (CCI).

Forgotten Joint Score (FJS-12)

The FJS-12 consists of twelve equally-weighted questions that are each answered on a five-level Likert scale aimed to measure patient satisfaction. The questionnaire was developed with the consideration that joint awareness is a very important and highly discriminative outcome parameter, especially in patients with good-to-excellent joint function. Answers to each question are individually scored and summed to create a raw composite score that is normalized to range from 0 (worst condition) to 100 points (best condition). In comparison to other satisfaction and pain scales, such as the Western Ontario and McMaster Universities Arthritis Index (WOMAC) and Hip disability and Osteoarthritis Outcome Score (HOOS), the FJS-12 has high test-retest consistency and a very low ceiling effect which provides it promising discriminatory ability between patients with good and excellent outcomes. Furthermore, there is strong evidence of good construct validity regarding the FJS-12 with respect to both total hip and total knee arthroplasty, with moderate evidence of good internal consistency.

Hip disability and Osteoarthritis Outcome Score, Joint Replacement (HOOS, JR)

The HOOS, JR is a six-question short-form alternative to the HOOS and WOMAC surveys for patients undergoing THA. Similar to the original HOOS domains, the HOOS, JR is scored on a 100-point scale, with zero representing total hip disability and 100 representing perfect hip health. Additionally, it provides a valid, reliable, and responsive measure of hip health. The floor and ceiling properties of the HOOS, JR are claimed to be similar or better than its predecessors, the HOOS and the WOMAC.

Statistical analysis

Demographic characteristics were reported as means and standard deviations for continuous variables, and counts and percentages for categorical variables. Floor and ceiling effects were defined as the proportion of patients reporting the worst and best possible scores.

All statistical analyses were performed using SPSS v25 (IBM, USA). The threshold for statistical significance was set at p < 0.05.

The PASS was derived using the anchor-based 80% specificity method, which has been previously shown to be the most reliable method of PASS derivation. Using this method, the interpretation of the PASS is the numerical FJS-12 value which 80% of dissatisfied patients are correctly identified. HOOS, JR scores at one-year follow-up were used as the anchor. We defined a HOOS, JR score of 70 as a clinically significant cut-off value based on previous literature. Therefore, those who reported a postoperative HOOS, JR < 70 indicated hip disability, and those who reported a HOOS, JR ≥ 70, indicated no disability. The minimal clinically important difference (MCID) used for HOOS, JR was 18 based on a previous study, obtained from the difference between one-year postoperative and preoperative scores. An external anchor was defined as valid if the correlation coefficient with FJS-12 was at least 0.3 with p < 0.05.

Receiver operating characteristic (ROC) curves were used to identify the change in FJS-12 with maximized sensitivity and specificity. The area under the curve (AUC) was used as a measure of diagnostic accuracy. AUC = 0.50 was equated with random assignment, while AUC = 1.0 was a perfectly accurate prediction. Values between 0.70 and 0.80 were considered acceptable discrimination, and AUC above 0.80 was considered excellent discrimination. To identify the cut-off, the Youden Index was used. The Youden method identifies the PASS as the coordinate on the ROC curve for which there is the highest combination of sensitivity and specificity. PASS thresholds of FJS-12 were calculated, using the 75th percentile of the cumulative percentage curve of patients who consider themselves in an acceptable state of symptoms, and the ROC curve point, which maximized the Youden index. The 75th percentile method defines the PASS as the numerical value on a PROM scale beyond which 75% of patients reported satisfaction with the outcome of their procedure.

Results

This study included 780 patients, of whom 339 (43.5%) were male and 441 (56.5%) were female, with a mean age of 63.58 years (standard deviation (SD) 10.78). The mean BMI was 28.63 kg/m2 (SD 5.78). With regards to race, 639 of the study population (81.9%) were caucasian, 63 (8.1%) were black, 13 (1.7%) were Asian, and 65 (8.3%) were of other races. The mean CCI was 3.80 (SD 2.26). There were 74 patients (9.5%) deemed ASA class I, 515 (66.0%) ASA class II, 187 (24.0%) ASA class III, and four (0.5%) ASA class IV. In all, 431 patients (55.3%) were non-smokers, 315 (40.4%) were former smokers, and 34 (4.4%) were current smokers (Table I).

Table I.

Demographics (n = 780).

Variable	Data
Sex, n (%)
Male	339 (43.5)
Female	441 (56.5)
Mean age, yrs (SD)	63.58 (10.78)
Smoking status, n (%)
Never Smoker	431 (55.3)
Former Smoker	315 (40.4)
Current Smoker	34 (4.4)
Race, n (%)
White	639 (81.9)
Black or African American	63 (8.1)
Asian	13 (1.7)
Other	65 (8.3)
ASA grade, n (%)
I	74 (9.5)
II	515 (66)
III	187 (24)
IV	4 (0.5)
Mean CCI (SD)	3.80 (2.26)
Mean BMI, kg/m2 (SD)	28.63 (5.78)

ASA, American Society of Anesthesiologists; CCI, Charlson Comorbidity Index; SD, standard deviation.

The mean one-year FJS-12 score was 65.42 (SD 28.59), ranging from a minimum of zero to a maximum of 100 (2% floor effect and 10% ceiling effect). The mean one-year HOOS, JR score was 82.70 (16.57), ranging from a minimum of 8.1 to a maximum of 100 (0.1% floor effect and 31% ceiling effect). A high positive correlation between FJS-12 and HOOS, JR at one-year follow-up was found (r = 0.74; p < 0.001), making the HOOS, JR a valid external anchor for PASS (Table II).

Table II.

Correlation between FJS-12 and HOOS, JR at one-year postoperatively.

Pearson correlation coefficient	p-value
0.74	< 0.001

FSJ-12, Forgotten Joint Score-12; HOOS, JR, Hip Disability and Osteoarthritis Outcome Score, Joint Replacement.

At one year, the ROC curve exhibited high AUC (0.8), demonstrating excellent discrimination for the FJS-12 between patients attaining and not attaining treatment success. The Youden’s Index found that the threshold score of the FJS-12 that maximized the sensitivity and specificity for detecting a PASS was 66.68 (Figure 1). The 75th percentile approach of the cumulative function of patients achieving treatment success found the threshold for FJS-12 at one year to be 92.20 (Table III).

Fig. 1

Receiver operating characteristic curve.

Table III.

Patient-acceptable symptom state (PASS) for FJS-12 at one-year postoperatively.

PASS computation method	Cut-off value (AUC)	Anchor
Receiver-operating characteristic	66.68 (0.8)	HOOS, JR
75th percentile	92.20	HOOS, JR

AUC, area under the curve; FSJ-12, Forgotten Joint Score-12; HOOS, JR, Hip Disability and Osteoarthritis Outcome Score, Joint Replacement.

Discussion

Many different methods and criteria exist to assess postoperative outcomes following THA. One of the primary goals of this procedure is to return the patient’s perception and function of their hip to its native pre-arthritic form. The FJS-12 was developed based on the concept of a patient’s ability to “forget” their artificial joint. It has since been validated and used in common practice to assess outcomes following arthroplasty. Nevertheless, the threshold defining a “forgotten joint” remains relatively inconclusive in the literature. Using data collected from 780 patients, we derived the PASS values of 66.68 and 92.20 for the FJS-12 at one-year follow-up after primary THA using the ROC and 75th percentile approaches, respectively.

Previous studies have reported that factors other than surgery, such as early postoperative movement, may influence PROMs and factors such as room conditions or inefficient communication with the nursing staff could decrease the overall satisfaction of the patients after THA. The heterogeneity of both the available PROMs, as well as their different applications complicates their interpretation. Therefore, changes in PROMs following surgical intervention could be statistically significant, but may not necessarily be considered clinically meaningful. The PASS can provide valuable insight into the interpretation of PROMs for clinicians and researchers alike by identifying the value on a PROM scale at which patients consider their symptom state to be satisfactory following a given procedure. Identifying the PASS value for the FJS-12 sets appropriate parameters for clinicians to define postoperative success in order to optimize treatment plans and appropriately counsel patients. The most commonly used approaches to calculate the PASS are the 75th percentile and the ROC approaches, reflecting the methodology used in the present study. The 75th percentile approach uses a cut-off point corresponding to the scores in patients who report a satisfactory health status by the anchoring question. The ROC approach finds the threshold that is the best compromise between sensitivity and specificity for each outcome criterion using Youden’s Index. The PASS threshold in our study was 66.68 and 92.20. The smaller threshold was established using the ROC method, whereas the 75th percentile approach resulted in the greater value. The high ROC-AUC (0.8) signifies that the FJS-12 had a high accuracy in discriminating between patients who attained high satisfaction, were pain-free, and with high-functional state from those who did not reach these benchmarks.

The PASS is likely to change within the first year of surgery as patients undergo rehabilitation. A combination of time points including those earlier than 1.5 years may better map across the typical recovery period for THA. A few recently published studies have reported the PASS of FJS-12 in patients who underwent THA. Galea et al reported the FJS-12 PASS at three months, one year, and two-year follow-up to be 59 (95% confidence interval (CI) 54 to 64), 68 (95% CI 61 to 75), and 69 (95% CI 62 to 75) points, respectively. Longo et al reported a FJS-12 PASS value range of 69.8 to 91.7 at six months following THA. Our one-year follow-up PASS findings for the FJS-12 are similar to the aforementioned studies; however, we established the 66.68 threshold value using the HOOS, JR as the anchor, unlike the previous two studies which used the Oxford Hip Score (OHS). Compared with the available literature, the most reasonable PASS value calculated was assessed by the ROC method. Furthermore, our threshold was established with a much larger sample size thus validating the findings of these previous studies.

Despite these previously conducted studies, consensus on the establishment of a PASS value for the FJS-12 following THA remains inconclusive in the literature. This is evidenced by the findings reported in a recently published study by Robinson et al, which defined the PASS threshold for the FJS-12 to be 29 at six months after THA for the UK population using ROC curve analysis. This discrepancy compared to other similar studies may be due to the recognized difference in postoperative FJS-12 scores between various populations following arthroplasty rendering these estimations to be non-universal. Additionally, variation in the anchor question has the potential to produce different results. Unlike our study which used the HOOS, JR as the anchor, Robinson et al used a five-point Likert scale (very satisfied, satisfied, neutral, dissatisfied, or very dissatisfied) to assess patient satisfaction. Furthermore, their results are limited to six months following surgery, while our study evaluates the PASS threshold at one year after THA.

Rosinsky et al determined a threshold of 73.96 and 69.79 at one and two years, respectively, for a successful outcome for the FJS-12 following THA. Although they used a composite score to assess outcomes as an external criterion due to a lack of a defined MCID and PASS, the PASS of 66.68 at one-year follow-up defined in the present study is fairly close to that of the aforementioned report. These similarities suggest that the PASS may be somewhat robust to variations in derivation methods and criteria. Giesinger et al conducted a study based on an online survey of individuals in the general USA population to determine normative FJS-12 scores. In their study, the mean FJS-12 was 70.6 (69.9 for females and 71.2 for males). In the current study, the average FJS-12 achieved at one-year was similar to the mean reported by Giesinger et al for the non-THA adult population. This implies that the average patient following THA achieves similar joint awareness as the general population with a native joint. Additionally, the PASS threshold detected in this study (66.68) was comparable to the general, non-THA population, attesting to the validity of this threshold as successful cut-off point to determine successful outcome following THA using the FJS-12.

Our study is not without limitations. Due to data constraints, we were adequately powered to establish the PASS for FJS-12 at only one-year follow-up and lacked postoperative scores at other variable time points. The retrospective nature of the study and the requirement to have both FJS-12 and HOOS, JR follow-up data at one year may introduce selection and loss to follow-up bias. The style of anchor question used to calculate the PASS threshold has been shown to yield variations in scores thus the values reported in the present study are anchor-dependent and can vary between studies on the same PROM depending on the methodology employed. In addition, our study population was sourced from a single, urban institution; therefore, the generalizability of our PASS values may be limited given that PROM results may vary across different regions across the world. However, the FJS-12 has been found to have comparable psychometric properties across various languages. The population of the present study included THA performed using both the anterior and posterior approaches, as well as robotic-assisted and fluoroscopic-guided component placement. Although the heterogeneity could influence the results, the goal was to determine a corresponding score for success, based on the patient’s perception in the postoperative course, regardless of which methods were used during surgery. Lastly, preoperative FJS-12 assessment was not conducted since FJS-12 was designed to evaluate an artificial joint. Although a general form applicable to native joints has been designed, it has not been implemented at our institution. Despite these limitations, our results enable orthopaedic surgeons to contextualize the FJS-12 scores of their patients, and provide researchers with a clinically relevant, patient-centered benchmark for the evaluation of surgical success.

In conclusion, the FJS-12 PASS threshold represents the value at which patients who underwent THA find their symptom state acceptable. The PASS for the FJS-12 at one year following primary THA ranges from scores of 66.68 to 92.20, with the former estimate obtained from the ROC method and the latter using the 75th percentile approach. Therefore, these values can be used as a marker for achieving acceptable patient satisfaction with regard to their artificial joint. Establishing consensus about standard definitions of meaningful postoperative improvement is necessary to maximize the utility of the FJS-12 in order to critically evaluate and counsel patients who undergo THA. Future studies should aim to derive PASS values at later time points in order to gauge whether age-related decline influences the PASS.

Take home message

- The patient-acceptable symptom state for the Forgotten Joint Score-12 at one year following primary total hip arthroplasty ranges from scores of 66.68 to 92.20, with the former estimate obtained from the receiver operating characteristic method and the latter using the 75th percentile approach.

- Therefore, these values can be used as a marker for achieving acceptable patient satisfaction with regard to their artificial joint.