Registry Studies Use Inconsistent Methods to Account for Patients Lost to Follow-up, and Rates of Patients LTFU Are High.

PURPOSE: To determine methods described in the literature to account for patients lost to follow-up (LTFU) in registry studies and whether rates of patient LTFU are within acceptable margins.
METHODS: A scoping review, where a literature search is conducted for studies from 9 arthroscopy registries, was performed on EMBASE, MEDLINE, and the annual reports of each registry. Inclusion criteria included studies with information on patient-reported outcome measures and being based on 9 national registries identified. Exclusion criteria included review articles, conference abstracts, studies not based on registry data, and studies from regional, claims-based, or multicenter registries. Studies were then divided into categories based on method of LTFU analysis used.
RESULTS: Thirty-six articles were identified for the final analysis. Categories for LTFU analysis included dropout analyses (n = 10), referencing validation studies (n = 12), contacting nonresponders (n = 4), and sensitivity analyses (n = 1). Referencing validation studies was the most common method (n = 12). Majority (n = 35) of the studies exceeded the recommended maximum rates for LTFU.
CONCLUSIONS: Registry studies use inconsistent methods to account for patient LTFU, and rates of patients LTFU are unacceptably high. CLINICAL RELEVANCE: The impact of patients LTFU in studies related to arthroscopic intervention is unknown. A universal method for accounting for patient follow-up is needed.

Recent advances over the last 2 decades have led to rapid progress in surgical techniques and innovation in implant technology. However, along with these advances there is a constant need to monitor the outcome of these surgical techniques and newer implants to ensure that adverse outcomes are identified early and intervened upon in the best interest of the patients.

The establishment of registries such as the Swedish Knee and Hip Arthroplasty registries (in 1977 and 1979, respectively),, the United Kingdom National Joint Registry in 2002 are methods used to achieve this level of monitoring. Registries are also useful in providing a good platform for research because of their low cost, large observational datasets, and the ease of analyzing data in comparison with other research methodologies. Outcome based registries in particular contain information on diagnosis, details of procedures, patient-reported outcome measures (PROMs), and long-term follow-up of patients. After the establishment of arthroplasty registries worldwide, arthroscopy registries have also been developed.

Many such registries use PROMs in the form of a standardized questionnaire to quantify various aspects of health including pain, quality of life, and function of the joint after the procedure. PROMs are collected at multiple time points in the follow-up of patients. Unfortunately, the rates of completion of PROMs or other surveys are often low because of poor response rates or patients “lost to follow-up” (LTFU). This means that studies based on these PROMs can be influenced by nonresponse bias if the patients lost to follow-up are nonrandom or “missing not at random.”

Despite this, many studies assume their missing data is “missing at random” and subsequently assume no nonresponse bias because there is no systematic reason for patients being LTFU. As a result, there is a possibility that these studies may overestimate the success of the interventions they are investigating. To avoid nonresponse bias, studies may incorporate missing data analysis to justify the validity of their data and results. Some methods for these “loss to follow-up analyses” include carrying out a comparison of baseline characteristics of patients,, referencing a validation study, that has been carried out on the registry or contacting nonresponders themselves.,

However, there is no universal agreement on the methodology to be used to account for LTFU nor on the acceptable follow-up rates for studies reported from arthroscopy registries. Traditionally, an 80% follow-up is seen as an acceptable compliance threshold for a survey study to be valid, which has been attributed to Sackett at al. by other studies., However, this number was chosen arbitrarily, and there was no statistical basis for it. Alternatively, the International Society of Arthroplasty Registries PROMs Working Group has recommended a minimum follow-up rate of 60%. This cutoff is also recommended by the Journal of the American Medical Association for survey-based research.

The purposes of this study were to determine methods described in the literature to account for patients LTFU in registry studies and whether rates of patient LTFU are within acceptable margins. We hypothesized that the rates of patient LTFU would be within the acceptable margins for survey studies and that there are methods used by authors to account for LTFU.

Methods

This scoping review was carried out using the methods outlined by Arksey and O’Malley to determine statistical methods used to account for and minimize the impact of patients LTFU.

Search strategy

A computer-based literature search of EMBASE, Medline, and a manual search of the latest annual reports of each arthroscopy registry was conducted on October 22, 2020. The eligibility criteria for the included articles were determined a priori by the authors. Inclusion criteria were that studies were from the PROM collecting arthroscopy registries as outlined by Ueland et al. and that the studies performed or referenced some form of LTFU analysis based either on their patient pool or on the patients in the registry the studies were based on. Studies were excluded if they did not meet these inclusion criteria, were conference abstracts or reviews, or did not have publication available in the English language.

The search strategy (Supplementary Table S1) consisted of the names of each of the included registries, (including variations of the names such as abbreviations), followed by a manual search of papers from the annual report of that registry. For example, to search for articles from the Swedish National Knee Ligament Registry (SNKLR), the search terms were “Swedish National Knee Ligament Regist∗,” “Swedish Knee Ligament Regist∗,” “Swedish National Anterior Cruciate Ligament Regist∗,” “Swedish Anterior Cruciate Ligament Regist∗,” “SNKLR,” “SKLR.” This was then followed up by a manual search of the 2019 annual report (latest report) of the SNKLR for additional studies.

A list of the registries included in the search are shown in Table 1. Further details on these registries can be found in Supplementary Table S2. One author carried out the search strategy, and article screening was performed by 2 authors, with any disagreements resolved by the senior author.

Table 1

List of Registries Included in the Literature Search

German Arthroscopy Registry (DART)

German Cartilage Registry (DGOU)

New Zealand Anterior Cruciate Ligament Registry (NZACLR)

Danish Hip Arthroscopy Registry (DHAR)

Danish Knee Ligament Reconstruction Registry (DKRR)

Norwegian Knee Ligament Registry (NKLR)

United Kingdom National Ligament Registry (NLR)

United Kingdom Non-Arthroplasty Hip Registry (NAHR)

Swedish National Knee Ligament Registry (SNKLR)

From the literature search, a screening was performed first of the titles and then of the abstracts. Following this, articles were selected for full-text review prior to final analysis. The references for each of the studies selected for final analysis were then scanned to identify any further relevant studies. A PRISMA (preferred reporting items for systematic reviews and meta-analyses) flowchart of the literature search is shown in Fig 1.

Fig 1

PRISMA (preferred reporting items for systematic reviews and meta-analyses) chart outlining studies returned during the literature search.

Variables included for articles that met the inclusion criteria were the size of patient sample, percentage of patients LTFU, type of LTFU analysis performed, and the registry the study was based on. The registry annual reports that were not available in English were translated using Google Translate. The extracted data were collated in Microsoft Excel. Statistical analyses focused on descriptive statistics.

Results

In total, 283 studies were identified in the original database search with an additional 173 studies included from the manual search of annual reports. After the removal of duplicates, 233 records were identified for the screening of titles and abstracts. Following this, 157 texts were identified for full-text analysis, resulting in 36 studies included for final analysis. One thousand ninety-two references from bibliographies of the included studies were then screened, leading to 70 further full-text articles being assessed. However, none of these additional 70 studies met the inclusion criteria, suggesting the search strategy was strong enough to capture all the required articles.

Included studies were divided into 5 categories (Table 2) depending on the type of LTFU analysis that was carried out. The main themes identified in the articles to account for loss to follow-up were (1) dropout analysis, (2) reference of a validation study, (3) contacting nonresponders, and (4) sensitivity analysis. No papers from the DGOU, DART, or NZACLR met the inclusion criteria to be eligible for this review. The reported rates of LTFU from each study was also noted. The details of the included studies, including the reported rates of LTFU, are summarized in Table 3. Following this, the maximum and minimum reported LTFU rates from the studies from each of the registries were compared to determine whether data from any 1 registry met any of the recognized follow-up percentage guidelines. These values are presented in Table 4.

Table 2

Categories of LTFU analyses Into Which the 36 Included Studies Were Divided

	Category	Description of Category	Number of Studies
1	Dropout analysis	Compared the baseline characteristics of responders and nonresponders	10
2	Referenced validation study	Used a validation study on the respective registry to validate the quality of data	12
1, 2	Both Dropout analysis and reference to a validation study		3
3	Contacted nonresponders	Contacted nonresponders for missing data	4
4	Sensitivity analysis	Performed sensitivity analysis to determine the quality of data	1
5	Miscellaneous/other		6

LTFU, lost to follow-up.

Table 3

Studies Included in This Review Stratified by the Category of LTFU Analysis That Was Performed

Publication	Registry	N	Subject Characteristics	PROMS	Significant Non-Responder Factors	Publication Noncompliance (% of Patients LTFU) (%)	Category
Poulsen et al.6	DHAR	2886	All registry patients	HAGOS, iHOT12, EQ-5D-3L, VAS, HSAS	Male, younger age	1 year: 42	1
Lind et al.26	DKRR	25,281	ACLR, MCLR	KOOS,Tegner Activity		Pre-op: 671 year: 82	2
Eysturoy et al.27	DKRR	17,204	ACLR	KOOS,Tegner Activity		Pre-op: 67Post-op :78	2
Nissen et al.30	DKRR	1619	ACLR	KOOS,Tegner Activity		1 year: 73	2
Lind et al.31	DKRR	22,401	ACLR, PCLR	KOOS,Tegner Activity		Pre-op: 651 year: 73	2
Bjerre et al.40	DKRR		ACLR	KOOS,Tegner Activity	Not invited to follow-up, data not sent to DKRR	1 year: 41	5
Fauno et al.32	DKRR	14,806	ACLR	KOOS,Tegner Activity		Pre-op: 611 year: 74	2
Rahr-Wagner et al.33	DKRR	13,647	ACLR	KOOS,Tegner Activity		Pre-op: 611 year: 74	2
Rahr-Wagner et al.34	DKRR	8375		KOOS,Tegner Activity		Pre-op: 631 year: 69	2
Rahr-Wagner et al.9	DKRR	14,500	ACLR	KOOS,Tegner Activity	Male, younger age, ACLR non-revision	Pre-op 671 year: 73	3
Lind et al.10	DKRR	12,193	ACLR	KOOS		Pre-op: 611 year: 72	3
Sandon et al.35	SNKLR	684	Soccer players	EQ-5D, KOOS			2
Kraus et al.36	SNKLR	26,014	ACLR	EQ-5D, KOOS		2 year: ∼50%	2
Balasingam et al.7	SNKLR	2229		KOOS	Male, younger age	Pre-op: 445 year: 5310 year: 63	1, 2
Hamrin Senorski et al.18	SNKLR	874	ACLR	KOOS	Male, younger age, articular injury, meniscal injury	10 year: 59	1
Hamrin Senorski et al.58	SNKLR	272	ACLR	Tegner Activity		1 year: 39	1
Hamrin Senorski et al.59	SNKLR	263	ACLR	Tegner Activity		1 year: 51	1
Hamrin Senorski et al.37	SNKLR	6889	Single bundle ACLR with hamstring tendons	KOOS		2 year: 55	2
Hamrin Senorski et al.42	SNKLR	343	ACL	KOOS			5
Reinholdsson et al. 8	SNKLR	1723	ACL surgery	EQ-5D, KOOS	Male, younger age, worse KOOS pain, worse KOOS quality of life	2 year: 48	3
Samuelsson et al.28	SNKLR	23,952	ACLR	KOOS		1 year: 702 year: 70	2
Hamrin Senorski et al.29	SNKLR	13,636	Single bundle ACLR	KOOS		Pre-op: 311 year: 472 year: 55	2
Barenius et al.44	SNKLR	8584	ACLR	KOOS	Male	2 year: 59	5
Ahlden et al.45	SNKLR	16,351	ACLR	KOOS		Pre-op: 361 year: 422 year: 515 year: 60	5
Ageberg et al.41	SNKLR	10,164	ACLR	EQ-5D,KOOS		Pre-Op: 48-561 year: 62-632 year: 76-79	5
Ulstein et al.19	SNKLR & NKLR	8470	ACLR	KOOS	Male, younger	5 year: 46	1, 2
Ulstein et al.20	SNKLR & NKLR	368	ACLR	KOOS	Male, younger, shorter time from injury to ACLR	Pre-op: 245 year: 43 (5 year figure is the percentage of patients who had pre-op KOOS)	1, 2
Rotterud et al.21	SNKLR & NKLR	357	ACLR and concomitant full-thickness cartilage lesion	KOOS	Male, younger, shorter time from injury to ACLR	Pre-op: 252-year: 45 (2 year figure is the percentage of patients who had pre-op KOOS)	1
Rotterud et al.22	SNKLR & NKLR	8476	ACLR	KOOS	Male, younger age	2 year: 46	1
Owesen et al.23	NKLR	252	PCLR	KOOS	Male, younger age	2 year: 32	1
Owesen et al.60	NKLR	5237	PCLR, ACLR	KOOS	Male	2 year: 47PCL: 35ACL 44	1
LaPrade et al.24	NKLR	4691	ACLR	KOOS	Male, younger age	2 year: 37	1
Grindem et al.43	NKLR	2690	ACLR	KOOS		2 year: 53	5
Granan et al.25	NKLR	5517	ACLR	KOOS	Male, younger, more ACLR revisions	2 year: 36	1
Holleyman et al.38	NAHR	630	Periacetabular osteotomy for DDH or FAI	EQ-5D, iHOT12	Younger age	iHOT-12:Pre-op: 106 month: 4812 month: 472 year: 85EQ-5D:Pre-op: 86 month: 4712 month: 462 year: 79	4
Maempel et al.11	NAHR	88	FAI	EQ-5D, iHOT12		1 year†: 19	3

DHAR, Danish Hip Arthroscopy Registry; DKRR, Danish Knee Ligament Reconstruction Registry; NKLR, Norwegian Knee Ligament Registry; NLR, United Kingdom National Ligament Registry; NAHR, United Kingdom Non-Arthroplasty Hip Registry; SNKLR, Swedish National Knee Ligament Registry; ACLR, anterior cruciate ligament reconstruction; MCLR, medial cruciate ligament reconstruction; PCLR, posterior cruciate ligament reconstruction; DDH, developmental dysplasia of the hip; FAI, femoroacetabular impingement; HAGOS, Copenhagen Hip and Groin Outcome Score; iHOT12, 12 item International Hip Outcome Tool; EQ-5D-3L, EuroQol-5D-3L; EQ-5D, EuroQol-5D; VAS, visual analog scale; HSAS, Hip Sports Activity Scale; KOOS, Knee Injury and Osteoarthritis Outcome Score.

Some studies referenced a validation study, but unless they were referencing a validation study on the register that the study itself is based on, they did not meet criteria for category 2 and thus did not meet the inclusion criteria.

Patients were contacted by survey if they did not reply to the 1-year follow-up survey. The median response time was 24.3 months

Table 4

The Maximum and Minimum Compliance Percentages (if Present) of Patient Follow-Up for the Studies That Met the Inclusion Criteria From Each Registry

Registry	LTFU Percentage for Study with the Highest LTFU (Lowest Compliance) %	LTFU Percentage for Study with the Lowest LTFU (Highest Compliance) %
DHAR∗	1 year: 42	1 year: 42
DKRR	Pre-op: 671 year: 82	Pre-op: 611 year: 41
SNKLR	Pre-op: 48-561 year: 62-632 year: 76-7910 year: 63	Pre-op: 311 year: 392 year: 4810 year: 59
NKLR	2 year: 53	2 year: 32
NAHR	1 year: 47	1 year: 19†

LTFU, lost to follow-up; DHAR, Danish Hip Arthroscopy Registry; DKRR, Danish Knee Ligament Reconstruction Registry; SNKLR, Swedish National Knee Ligament Registry; NKLR, Norwegian Knee Ligament Registry; NAHR, United Kingdom Non-Arthroplasty Hip Registry; NLR, United Kingdom National Ligament Registry; DGOU, German Cartilage Registry.

Multiple (>1) studies must have a follow-up percentage at a certain time point for that time point to be included in the table above. Studies were included only if reporting on a single registry and those that were based on multiple registries are not included in this table. No studies from the NLR or the DGOU were eligible for inclusion and therefore are not included in the table.

Only one study met the eligibility criteria from the DHAR, therefore the highest and lowest compliance is the same

Patients were contacted by survey if they did not reply to the 1-year follow-up survey. The median response time was 24.3 months

LTFU Analysis among the registries

From the 36 included articles, 5 categories of LTFU analyses were derived.

Dropout analysis

Dropout analyses were one of the most common methods of LTFU analysis with 10 studies reporting on this. This involved comparison of the baseline demographic details (e.g., age and sex) between patients who completed the PROMs and those who did not. The most common finding was that nonresponders tended to be of male sex and younger age.,,18, 19, 20, 21, 22, 23, 24, 25, 26, 27 However, some studies found worse outcomes in nonresponders such as having more ACLR revisions.

Referencing a validation study

Twelve papers28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39 referenced a validation study only, and an additional 3 articles,, referenced both a validation study and carried out a dropout analysis. Two validation studies were carried out on the dataset of 2 registries: the SNKLR and DKRR. These studies evaluated the completeness and the quality of the data in their respective registries to determine whether reliable conclusions could be drawn from them. This also included identifying whether there were any differences in demographic details or PROMs between responders and nonresponders in the registry.

Rahr-Wagner et al. carried out a validation study on the DKRR. The authors sent out 100 questionnaires to a sample of both, responders and nonresponders. Comparisons were then drawn between the responses from these. It was found that nonresponders tended to be male and of a younger age. For the SNKLR, a similar strategy was used by Reinholdsson et al. where all nonresponders (n = 1723) were contacted, of which 349 (21%) replied. Again, it was found that nonresponders tended to be male and younger, but they also tended to have a higher level of pain and lower quality of life.

Contacting Nonresponders

Four studies contacted nonresponders to increase the follow-up compliance percentage in their study. Lind et al. contacted 200 responders and 200 nonresponders, out of which 60% replied and showed that there was no difference in the Knee Injury and Osteoarthritis Outcome Score (KOOS) scores at the 1-year follow-up between the 2 groups. Maempel et al. attempted to contact each nonresponder a maximum of 5 times, after which they were able to increase their compliance to 81% (LTFU of 19%). The aforementioned validation studies by Reinholdsson et al. and Rahr-Wagner et al. are also included in this category.

Sensitivity Analysis

From the included studies, Holleyman et al. conducted a different sensitivity analysis to adjust for differences in demographics between responders and nonresponders. This was carried out by creating a cohort (“overall cohort”) with bootstrapping (random sampling with replacement) responders and nonresponders, followed by creating another cohort (“responder cohort”) via the same process of random sampling from all of the responders. Next, randomly selected responders from the “overall cohort” were transferred into the “responder cohort” and removed from the “overall cohort.” Nearest neighbor matching was used to match cases from the remaining patients in the bootstrapped overall cohort to those in the responder cohort based on age, sex, body mass index, and surgical diagnosis. This was repeated 1000 times. This did not aim to find differences between patients who were and were not LTFU but aimed to mitigate the bias that may have arisen as a result of the differences. Rahr-Wagner et al. also mention a sensitivity analysis very briefly but do not display the data or the methodology behind it and therefore is not included in this category.

Other

This category was formed of studies that did not meet the descriptions of the other categories but still performed some form of LTFU analysis. Of the six studies in this category, one was based on the DKRR and 5 on the SNKLR. The study from the DKRR looked at reasons for LTFU by focusing on the administrative failures of several hospitals when sending data to the DKRR. From the 5 studies from the SNKLR, 1 only looked at differences in response rates between male and female sexes and 4 stated that there were no differences between responders and nonresponders (suggesting a dropout analysis was performed) while not providing data as evidence for this.42, 43, 44, 45

Discussion

The main findings of this study were that there was no consistent method used to account for LTFU in arthroscopy registry studies. Many studies did not have any methods to account for loss to follow-up, and from those that did, there disagreement on how to tackle it and minimal acknowledgment of the potential bias it can introduce.

Despite a number of arthroscopic outcome studies acknowledging patients being LTFU as a limitation of their work, only 36 studies included some variant of a LTFU analysis in our review. Amongst these studies, there was no consensus on which was the best method of accounting for LTFU. The use of dropout analyses (n = 10), using validation studies (n = 12), or a mix of both (n = 3) were the most commonly used, whereas only 1 study used a sensitivity analysis.

Dropout analysis

Dropout analyses have the advantage of allowing a direct comparison of demographic details between patients who are lost to follow-up and those who are not. With such data, information on the likelihood of a patient from a particular demographic background (sex, age, preoperative clinical picture, and PROM scoring) can be used to predict whether the patient will be LTFU. From this, patients who at a higher risk of being nonresponders can then be focused upon with additional procedures of making contact such as trying to organize a phone call to ask them to be followed up, rather than an email.

Referencing a validation study

Both validation studies found that were only minimal differences in PROMs between responders and nonresponders. For the validation study on the DKRR, only 62% (62/100) of responders and 32% (32/100) of nonresponders replied to the questionnaire. Since only 32 participants represented the entire cohort of nonresponders from the registry, it would be incorrect to conclude whether factors did or did not differ between responders and nonresponders in the DKRR. For the validation study on the SNKLR, despite all nonresponders being contacted for follow-up, only 21% (359/1723) of nonresponders responded to the additional questionnaire. Because the majority of nonresponders still did not provide reasons for being LTFU or return their PROMs, there may have been other differences between the nonresponder and responder group in terms of outcomes and PROMs. The low response rate from nonresponders in both of these studies suggests that the noncompliance was not due to chance but due to an underlying reason. Significant factors for nonresponders included male sex and a younger in age in both studies. An increased level of pain and worse quality of life KOOS score in nonresponders was noted by Reinholdsson et al. Only 2 studies, of the 8 from the SNKLR that refenced the validation studies acknowledged the difference in these KOOS scores. The other studies from the SNKLR and the DKRR had simple statements stating there were no significant differences between responders and non-responders.

Many studies refer to these to validate their dataset despite failing to recognize the limitations of these two studies. However, without a higher sample size for both studies, it may be disingenuous to make this assertion.

Contacting Nonresponders

Contacting nonresponders has the advantage of increasing the sample size of the study data, and therefore limiting the effects of nonresponse bias that may have otherwise been present. However, not all nonresponders reply, meaning that there may still be elements of nonresponse bias. Additionally, nonresponders may be contacted at a date many months after the 1-year or 2-year follow-up timelines, and so there may either be missing data for the time points where there is no response, or there may be recall bias where these patients do not give accurate responses on how they were feeling at the time. For example, if a complication arose after when the 1-year follow-up time would have been, but the patient responded after being contacted at a later date, the data may bias the 1-year follow-up data.

Sensitivity Analysis

Sensitivity analyses play roles in the assessment of how robust the conclusions drawn from the primary analysis of data are by examining how the results are affected by changes in methods, models, assumptions, or missing data. When data are missing, the options of sensitivity analysis include either analyzing only complete cases (assume data is missing completely at random) or impute the missing data using imputation methods and redoing the analysis. Most studies should assume that data is missing not at random. There are several single and multiple imputation methods that can be chosen depending on the type of data being collected and depending on whether it is missing at random.

Just as it is common in clinical trials, it is of our opinion that all registry studies should incorporate a sensitivity analysis to mitigate the effects of nonresponse bias. However, only 1 study of the 36 included studies performed a sensitivity analysis.

As previously mentioned, the accepted rate of LTFU is disputed for survey studies, ranging from 20% to 40%. Only 1 study achieved a LTFU rate of less than 20%. Several studies quoted LTFU rates of greater than 50%, with 1 study from DKRR quoted a LTFU as high as 82% at 1 year. Although the recommended compliance rates differ between different guidelines, none of them quote a value of LTFU as high as seen in many of the included studies.

A simulation study by Kristman et al. observed that even follow-up rates of 80% (LTFU rate of 20%) were associated with a considerable bias if data were missing not at random. Zelle et al. further performed a simulation study to assess for an acceptable percentage of responders. The study was performed on a polytrauma database from which patient data was deleted randomly at an increasing percentage to simulate various levels of LTFU. It was found that in the 50 simulations performed, only nonresponder rates of 15% or less had no changes in significance on their studied outcome. On the other hand, a nonresponder rate of 20% changed significance in 14 of the 50 simulations they ran on the database.

There are other methods described to alleviate or account for LTFU in arthroscopy studies from the non-registry studies. One recent example includes the incorporation of machine learning into arthroscopy patient databases where the system can predict the patients that are most likely to not-respond. The study by Kunze et al. looked at 27 different preoperative variables including patient characteristics (such as age, sex, body mass index, race), patient behaviors (such as smoking, alcohol intake, drug use), orthopaedic history (such as previous surgery) and at PROMs to develop 3 models, each of which revealed patients most likely to be LTFU. The 3 models used were “cross-validation,” minimizing “Bayes information criteria,” and “adaptive selection.” Common variables that predicted LTFU among the 3 models were when patients who were of the male sex, nonwhite, smokers, not providing telephone numbers, and having a greater preoperative modified Harris Hip score and international Hip Outcome Tool 12-component questionnaire score (PROMs).

There are also varied conclusions on whether patients LTFU have different outcomes compared to those that are followed-up. An epidemiological study by Ekholm et al. suggested a positive correlation between socioeconomic status and response rates of patients to health interview surveys, although the authors did note there was no significant association between health and nonresponsiveness in their study. Furthermore, a validation study on a local registry was carried out by Lindman et al. reported a larger percentage of replies by nonresponders (76% out of 140 nonresponders) than the validation studies included in this review. Their study found that there was no significant difference in patient-reported hip function according to the PROMs carried out after hip arthroscopy. However, patients who were nonresponders were more likely to be less satisfied with their treatment, younger, and male sex. However, the authors noted that other studies on orthopaedic LTFU analysis did not find significant differences in satisfaction with treatment, and that studies on nonresponders after arthroplasty procedures reported worse outcomes in terms of knee function in nonresponders.,

A consensus must be reached regarding the handling of missing data and accounting for LTFU of patients included in arthroscopy registries. Because of the compliance rate of arthroscopy registries being lower than those of arthroplasty registries, these issues (despite being of importance) are less pressing for those registries. Although most studies use a dropout analysis or a reference to a validation study, there may be other methods that are more robust in alleviating for nonresponse bias. A follow-up rate of more than 85% may be needed to overcome effects of LTFU and have robust outcome data from registries.

The best form of minimizing nonresponse bias in a registry-based study is to avoid missing data; however, this is not always possible. Possible explanations for poor data collection may include length of questionnaires, method of contacting patients, patients not taking part in follow up due to their outcomes, or patient demographic factors. From the studies included (Table 3), common demographic factors for patients who are LTFU is that they are of male sex and often younger, and therefore more appropriate contact methods (other than questionnaires sent by post or email) may be beneficial. The SNKLR Annual Report from 2019 states that new data collection methods such as mobile applications or social media should be explored. The annual report also explores the idea of shortening the length of questionnaires to minimize the time needed to answer each questionnaire. The KOOS questionnaire for the knee for example contains 42 items in 5 sections, whereas the iHOT-12 only contains 12 items as it is a shortened version of the much longer iHOT-33 (a 33 item questionnaire)., Finally, as arthroscopy is a minimally invasive surgery it has shorter recovery times and fewer complications (such as infection) meaning patients may not feel the need to attend follow-up appointments or answer questionnaires in the years after the procedure.

As well as minimizing missing data, another possible next step after this review could be to perform a consensus study using experts in the field of registries to both determine an acceptable percentage of follow-up compliance and also to determine a universal method for accounting for patient LTFU. Guidelines may be formed from this consensus to guide future arthroscopy registry-based research.

Limitations

There are some limitations to this study. Firstly, only studies from PROM collecting national registries were included in the analysis. Studies from registries that are not nationally centralized may have had other forms of statistical analysis that are used to alleviate for patients who are LTFU. Examples of these include the Multicenter Orthopaedic Outcomes Network and the Kaiser Permanente Anterior Cruciate Ligament Reconstruction Registry, both of which have had numerous studies carried out using data from their patient cohorts. However, studies from such registries were not considered in the search for this review as they do not represent national arthroscopy registries. Second, annual reports for the DKRR and DGOU registries were not available in the English language and had to be translated by Google translate. Native speakers and readers of the languages (Danish and German) were not consulted for the accuracy of the translation. This meant that there was a reliance on the translation software to provide accurate translations. Finally, the search for LTFU analysis was limited to arthroscopy registries, which are still relatively new compared to the arthroplasty registries. However, we are not aware of any consensus on what LTFU analysis method should be used in arthroplasty registries, but this is not as great a concern because the rates of LTFU are lower in arthroplasty registries.

Conclusions

Registry studies use inconsistent methods to account for patient LTFU, and rates of patients LTFU are unacceptable.

Supplementary Table S1

Table Outlining Search Times Used on MEDLINE and EMBASE to Find Studies From Each of the Included Registries

Registry	Search Terms	Annual Report Year
DART	German Arthroscopy regist∗ OR German Registr∗ of Arthroscopy OR Deutschsprachiges Arthroskopieregister	Not found
DGOU	German Cartilage Regist∗ OR KnorpelRegister DGOU OR DGOU Cartilage Regist∗	2017
NZACLR	New Zealand Anterior Cruciate Ligament Regist∗ OR New Zealand ACL Regist∗ OR NZACL OR NZACLR	2020
DHAR	Danish Hip Arthroscopy Regist∗	2019
DKRR	Danish Knee Ligament Reconstruction Regist∗ OR DKRR OR Danish Cruciate Ligament Regist∗ OR DKKR	2019
NKLR	Norwegian Cruciate Ligament Regist∗ OR Norwegian Knee Ligament Regist∗ OR NKLR	2020
NLR	National Ligament Regist∗	2020
NAHR	Non-arthroplasty hip regist∗	2020
SNKLR	Swedish National Knee Ligament Regist∗ OR Swedish Knee Ligament Regist∗ OR Swedish Anterior Cruciate Ligament Regist∗ OR Swedish National Anterior Cruciate Ligament Regist∗ OR SNKLR OR SKLR	2019

DART, German Arthroscopy Registry; DGOU, German Cartilage Registry; NZACLR, New Zealand ACL Registry; DHAR, Danish Hip Arthroscopy Registry; DKRR, Danish Knee Ligament Reconstruction Registry; NKLR, Norwegian Knee Ligament Registry; NLR, United Kingdom National Ligament Registry; NAHR, United Kingdom Non-Arthroplasty Hip Registry; SNKLR, Swedish National Knee Ligament Registry.

Articles included in each of the latest annual reports of the registries were also screened for studies. Annual Reports were searched for online on the website of each respective registry.,61, 62, 63, 64, 65, 66, 67

Supplementary Table S2

Summary of the Included Registries From Which Studies Were Selected

Registry	Information Collected from	Year data Collection Initiated from	Entries	PROMs	Collection Time Points (years)	Registry Compliance Pre-op	Registry Compliance Post-op
DGOU	2017 Annual Report∗		5339
NZACLR	2019 Annual Report	2014	9849	Marx, KOOS	0.5, 1, 2, 5		6 months: >75%12 months: >70%24 months >70%60 months: >50%
Danish Hip Arthroscopy Registry (DHAR)	2019 Annual Report	2012	6214	HAGOS, iHOT-12, HSAS, VAS- hip function, NRS-rest, NRS-walk, EQ5D	1, 2, 5, 10	Pre-op: 57%	1 year: 59%2 year: 51%5 year: 45%
Danish Knee Ligament Reconstruction Registry (DKRR)	2019 Annual Report†	2005	35946	KOOS, Tegner Activity	1	40%	35%
Norwegian Knee Ligament Registry (NKLR)	2020 Annual Report	2004	31975	KOOS	2, 5, 10		2 year: 60.6%5 year: 56.7%10 year: 56.6%
United Kingdom National Ligament Registry (NLR)	2019 Annual Report	2013	12558	EQ5D, Tegner Activity, IKDC, KOOS	0.5, 1, 2	58%	1 year: 37%2 year: 32%
United Kingdom Non-Arthroplasty Hip Registry (NAHR)	2020 Annual Report	2012	12992	EQ5D, EQVAS, iHOT-12	0.5, 1, 2
Swedish National Knee Ligament Registry (SNKLR)	2019 Annual Report	2005	52816	EQ5D, EQVAS, KOOS	1, 2, 5, 10	67%	1 year: 55%2 year: 49%5 year: 45%10 year: 38% (EQ5D)

Pre-op, before surgery; Post-op, after surgery; DART, German Arthroscopy Registry; DGOU, German Cartilage Registry; NZACLR, New Zealand ACL Registry; DHAR, Danish Hip Arthroscopy Registry; DKRR, Danish Knee Ligament Reconstruction Registry; NKLR, Norwegian Knee Ligament Registry; NLR, United Kingdom National Ligament Registry; NAHR, United Kingdom Non-Arthroplasty Hip Registry; SNKLR, Swedish National Knee Ligament Registry; PROM, patient reported outcome measures; KOOS, Knee Injury and Osteoarthritis Outcome Score; HAGOS, Copenhagen Hip and Groin Outcome Score; iHOT-12, International Hip Outcome Tool; HSAS, Hip Sports Activity Scale; VAS, visual analog scale; NRS, Numerical Rating Scale; EQ5D, EuroQol Outcome Measure; IKDC, International Knee Documentation Committee Score; EQVAS, EuroQol Visual Analogue Scales.

Data were collected from the websites and the latest annual reports of each registry.,61, 62, 63, 64, 65, 66, 67 If no overall compliance is given, but multiple compliances are given for each individual PROM, the joint specific PROM is chosen. If there are multiple joint specific PROMs, the lowest scoring one has been included in the table. The annual report and website for the DGOU did not provide registry information more than what is displayed on the table. The annual report for the DART registry was not located online, and the website did not provide any details on the registry itself.

Translated from German using Google Translate.

Translated from Danish using Google Translate.