Screening questions for the diagnosis of restless legs syndrome in hemodialysis.

BACKGROUND: Restless legs syndrome (RLS) is common in end-stage renal disease and is associated with reduced health-related quality of life. Simple and accurate screening instruments are needed since RLS is underdiagnosed and treatable. We examined the operating characteristics of screening questions and a disease-specific measurement tool for the diagnosis of RLS in hemodialysis.
METHODS: We conducted a cohort study of prevalent adult hemodialysis patients in Hamilton, Canada. The diagnosis of RLS was made using the 2012 Revised International Restless Legs Syndrome Study Group (IRLSSG) criteria. All participants received three screening instruments: (i) a single screening question for RLS derived from a nondialysis population; (ii) a single question from the Edmonton Symptom Assessment System (ESAS); and (iii) the IRLSSG Rating Scale (IRLS). All instruments were compared with the reference standard using logistic regression from which receiver operating characteristics curves were generated. Cutoffs associated with maximum performance were identified.
RESULTS: We recruited 50 participants with a mean (SD) age of 64 (12.4) years, of whom 52% were male and 92% were on three times weekly hemodialysis. Using the reference standard, 14 (28%) had a diagnosis of RLS. The single screening question for RLS had an area under the receiver operating curve (AUROC) of 0.72 with a sensitivity of 85.7% and specificity of 58.3%. An ESAS cutoff of ≥1 had the highest AUROC at 0.65 with a sensitivity of 79% and specificity of 56%. An IRLS cutoff of ≥20 had the highest AUROC at 0.75 with a sensitivity of 71% and specificity of 81%.
CONCLUSION: IRLS had better specificity than the single question or ESAS for the diagnosis of RLS.

INTRODUCTION

Restless legs syndrome (RLS) is common in the setting of kidney disease, affecting 30% of individuals with end-stage renal disease (ESRD) [1]. It is associated with impaired health-related quality of life indicators [2-4] including anxiety, sleepiness, sexual dysfunction [5] and possibly mortality [4, 6, 7].

Screening is important to identify disease for which effective therapy exists that otherwise would be left undiagnosed and untreated resulting in adverse patient outcomes. Patient-reported outcome measures (PROMs) may be suitable screening instruments to help identify patients with RLS who are eligible for treatment. This is relevant given the recent recommendations to incorporate PROMs into clinical care [8] and the availability of potentially effective therapies to control RLS symptoms [9].

Given the prevalence and impact of RLS in the hemodialysis population, we undertook a study to determine how well PROMs compared with the gold standard of a clinical interview to diagnose RLS. Ideally, a single question or simple instrument that accurately identifies hemodialysis patients with RLS would be valuable in managing this important clinical problem.

MATERIALS AND METHODS

Study cohort

We performed a cross-sectional study of prevalent adult hemodialysis patients from two hemodialysis units (one tertiary, one community) in Hamilton, Canada. A consecutive sample of hemodialysis patients from morning and afternoon dialysis shifts were approached. Eligible participants were: (i) ≥18 years; (ii) receiving in-center hemodialysis at least two times weekly for at least the last 90 days; and (iii) provided informed consent. Patients were excluded if: (i) they had anemia defined by a hemoglobin <9 g/dL; (ii) they had relative iron deficiency defined by a transferrin saturation <21% or a ferritin <200 ng/mL; (iii) they were currently pregnant [(i), (ii) and (iii) are all secondary etiologies of RLS]; (iv) they had any change in RLS therapy in the last 4 weeks including dopamine, dopamine agonists, alpha 2 delta ligands and benzodiazepines suggestive of augmentation; and (v) they were unable to complete the study instruments due to cognitive impairment or an English language barrier.

Study procedures

Ethics approval was obtained by the Hamilton Integrated Research Ethics Board.

We collected participant demographics, comorbidities and laboratory results at baseline. Participants completed the following instruments during dialysis at the first study visit: the single question for the diagnosis of RLS, the Edmonton Symptom Assessment System (ESAS) single question modified for RLS (ESAS-RLS) and the International Restless Legs Syndrome Study Group (IRLSSG) Rating Scale (IRLS) (Supplementary Index). The single question for the diagnosis of RLS [10] has previously shown to have 100% sensitivity and 96.8% specificity in an outpatient neurology setting: ‘When you try to relax in the evening or sleep at night, do you ever have unpleasant, restless feelings in your legs that can be relieved by walking or movement?’ ESAS-RLS was based on the ESAS, which originates from palliative care [11], and has been extensively validated [12] in cancer settings and contains 10 symptoms measured on an 11-point scale anchored at 0 (no symptoms) and 10 (worst possible symptoms). It has been validated cross-sectionally [13] and longitudinally [14] in dialysis patients. IRLS is a valid, reliable and responsive RLS instrument that consists of 10 questions with response options ranging from 0 to 4 reflecting none, mild, moderate, severe and very severe. The scale reflects the subjective assessment of the primary features and diagnosis of RLS (Items 1–3 and 6), associated sleep problems (Items 4 and 5), intensity and frequency (Items 7 and 8) and mood and daily functioning (Items 9 and 10). It ranges from 0 to 40, with higher scores reflecting the severity of symptoms. It was validated in 405 patients (196 with RLS, 209 controls with other sleep disorder or from the general population) from 20 centers across 6 countries [15] and in clinical trial settings [16], but excluded any disease mimics. The diagnosis of RLS was assessed by a nephrologist (D.C.) with clinical expertise in RLS using the 2012 Revised IRLSSG criteria [17] and served as the reference standard.

Statistical analysis

Assuming a 30% prevalence of RLS [1], an area under the receiver operator curve (AUROC) of 0.75 and a two-sided α of 0.05, a sample size of 50 would have >80% power to detect a difference from an AUROC of 0.5 in screening instruments.

Participant characteristics are described using means (SDs) or medians and 25–75th percentile for continuous variables and frequencies (%) for categorical variables. Differences between those with RLS using the 2012 Revised IRLSSG diagnostic criteria were compared with those without RLS using a two-sample t-test for continuous variables and Chi-squared test for categorical variables.

The single question, ESAS-RLS and IRLS were compared with the gold standard of the 2012 Revised IRLSSG diagnostic criteria by calculating their sensitivity, specificity and AUROC. We explored cutoffs of ESAS-RLS and IRLS to maximize sensitivity and specificity using the Youden Index [18] and Liu method [19], respectively. A sensitivity analysis was performed removing the fifth criteria of ‘the occurrence of the above features are not solely accounted for as symptoms primary to another medical or behavioral condition (e.g. myalgia, venous status, leg edema, arthritis, leg cramps, positional discomfort, habitual foot tapping)’, allowing for ‘RLS mimics’, which are common in the dialysis population. The relationship between the ESAS-RLS and IRLS was assessed using the Pearson correlation coefficient. The AUROC for all three screening instruments was compared using a nonparametric approach [20].

All statistical tests were performed at a P < 0.05 level of significance. All analyses were performed using STATA (StataCorp. 2015 Stata Statistical Software: Release 14, StataCorp LP, College Station, TX, USA).

RESULTS

The study participants’ characteristics are provided in Table 1. In the study cohort, 14 (28%) had RLS by the 2012 Revised IRLSSG criteria and 19 (38%) had RLS when the fifth criteria accounting for RLS mimics were excluded. Patients with RLS were more likely to be female and have glomerulonephritis as the etiology of their ESRD. Only 3 of 14 patients (21.4%) with RLS were on pharmacotherapy with 0 (0%), 1 (7.1%), 2 (14.3%) of individuals on dopamine, dopamine agonist or alpha 2 delta ligand therapy, respectively.

Table 1.

Study cohort characteristics

	All (n = 50)	RLS (n = 14)	No RLS (n = 36)	P-value
Age, mean (SD), years	64 (12.4)	60.9 (12.0)	62.2 (12.5)	0.28
Gender, n (%)				0.007
Male	26 (52)	3 (21.4)	23 (63.9)
Female	24 (48)	11 (78.6)	13 (36.1)
Dialysis treatments, n (%)				0.74
Two times weekly	2 (4)	1 (7.1)	1 (2.8)
Three or more times weekly	48 (96)	13 (92.9)	35 (97.2)
Duration of HD, mean (SD), h	3.6 (0.4)	3.6 (0.48)	3.6 (0.43)	0.92
Vascular access, n (%)				0.30
Fistula	24 (48)	9 (64.3)	15 (41.7)
Graft	3 (6)	1 (7.1)	2 (5.6)
Catheter	23 (46)	4 (28.6)	19 (52.8)
URR, mean (SD), %	69.5 (6.2)	72.1 (5.2)	68.5 (6.3)	0.07
Etiology of ESRD, n (%)				0.019
DN	19 (38)	5 (35.7)	14 (38.9)
HTN	4 (8)	1 (7.1)	3 (8.3)
GN	12 (24)	5 (35.7)	7 (19.4)
Other	15 (30)	3 (21.4)	12 (33.3)
Comorbidities, n (%)
Diabetes	24 (48)	7 (50.0)	17 (47.2)	0.86
CAD	13 (26)	2 (14.3)	11 (30.6)	0.24
PVD	9 (18)	1 (7.1)	8 (22.2)	0.21
CVD	10 (20)	1 (7.1)	9 (25.0)	0.16
OSA	11 (22)	3 (21.4)	8 (22.2)	0.95
Dopamine, n (%)	1 (2)	0 (0)	1 (2.7)	0.53
Dopamine agonist, n (%)	2 (4)	1 (7.1)	1 (2.7)	0.48
Alpha 2 delta ligand, n (%)	9 (18)	2 (14.3)	7 (19.4)	0.18
Hemoglobin, mean (SD), g/dL	10.6 (0.99)	10.4 (1.0)	10.6 (0.99)	0.50
Serum iron, mean (SD), g/dL	11.0 (2.7)	10.3 (2.5)	11.2 (2.8)	0.28
TIBC, mean (SD), μmol/L	31.5 (6.2)	35.9 (6.1)	34.8 (6.3)	0.59
Transferrin saturation, mean (SD), %	32.2 (10.6)	29.7 (10.0)	33.2 (10.7)	0.30
Ferritin, mean (SD), μmol/L	724 (654)	959 (1083)	632 (362)	0.11
Calcium, mean (SD), mmol/L	2.28 (0.2)	2.33 (0.24)	2.26 (0.17)	0.26
Phosphate, mean (SD), mmol/L	1.72 (0.5)	1.82 (0.49)	1.69 (0.57)	0.46
Albumin, mean (SD), g/L	31.5 (2.8)	32 (2.3)	31 (3.0)	0.30
PTH, mean (SD), pmol/L	60.3 (42.2)	72.1 (51.7)	55.8 (37.8)	0.22

P-value is for the comparison of RLS versus no RLS.

HD, hemodialysis; URR, urea reduction ratio; DN, diabetic nephropathy; HTN, hypertension; GN, glomerulonephritis; CAD, coronary artery disease; PVD, peripheral vascular disease; CVD, cerebrovascular disease; OSA, obstructive sleep apnea; TIBC, total iron binding capacity; PTH, parathyroid hormone.

Using the single question to screen for RLS, the sensitivity was 85.7% and specificity was 58.3% with AUROC of 0.72 [95% confidence interval (CI) 0.59–0.85] (Figure 1). The mean (SD) ESAS-RLS was 3.57 (3.2) in those with RLS and 2.39 (3.4) in those without RLS (P = 0.27). Using ESAS-RLS to screen for RLS, the AUROC was 0.65 (95% CI 0.49–0.81) (Figure 2). The optimal screening cutoff was a score of ≥1 with a sensitivity of 79% and specificity of 56% with AUROC of 0.65. The mean (SD) IRLS was 19.2 (8.5) in those with RLS and 9.86 (10.6) in those without RLS (P = 0.0048). Each IRLS item (except Items 3 and 9) and the total IRLS were higher in those with RLS than those without RLS (P < 0.05, data not shown). Using IRLS to screen for RLS, the AUROC was 0.75 (95% CI 0.60–0.90) (Figure 3). The optimal screening cutoff was a score of ≥20 with a sensitivity of 71% and specificity of 81% with AUROC of 0.75. There was no statistically significant difference in discrimination between models (P = 0.29).

FIGURE 1

Receiver operating curve of the single question for the diagnosis of RLS from the general population. Sensitivity of 85.7%, specificity of 58.3%, AUROC of 0.72.

FIGURE 2

Receiver operating curve of ESAS-RLS for the diagnosis of RLS. ESAS-RLS cutoff of ≥1, sensitivity of 79%, specificity of 56%, AUROC of 0.65.

FIGURE 3

Receiver operating curve of IRLS for the diagnosis of RLS. IRLS cutoff of ≥20, sensitivity of 71%, specificity of 81%, AUROC of 0.75.

The sensitivity analysis excluding the fifth 2012 Revised IRLSSG criteria of RLS mimics did not improve the operating characteristics of any screening instrument (Supplementary data, Figures S1–S3). The correlation between the continuous scores of ESAS-RLS and IRLS was r = 0.70 (P = 0.0000) (Figure 4). Using an ESAS-RLS cutoff of 1, 3/14 (21.4%, 95% CI 4.6–50.8%) would have screened negative despite having RLS, and 16/36 (44.4%, 95% CI 27.9–61.9%) would have screened positive despite not having RLS. Using an IRLS cutoff of 20, 4/14 (28.6%, 95% CI 8.4–58.1%) would have screened negative despite having RLS, and 7/36 (19.4%, 95% CI 8.2–36.0%) would have screened positive despite not having RLS.

FIGURE 4

Scatterplot of ESAS-RLS and IRLS. r = 0.70, P = 0.0000. IRLS cutoff of ≥20 is labeled on the y-axis and ESAS-RLS cutoff of ≥1 is labeled on the x-axis.

DISCUSSION

In this observational study of 50 hemodialysis patients from tertiary and community units, we evaluated the use of three PROMs as RLS screening instruments. The single question and ESAS-RLS both had moderate sensitivity (85.7 and 79%) but poor specificity (58.3 and 56%). The specificity of the single question was less in our study compared with that previously resported [10], presumably due to interference by RLS mimics in the dialysis population that are not common in the outpatient neurology setting. However, when the reference standard was altered to allow for RLS mimics by excluding the fifth 2012 Revised IRLSSG diagnostic criteria, the performance of all screening instruments did not improve, suggesting that the poor specificity of the single question and ESAS-RLS is independent of any RLS mimics. ESAS-RLS has not been previously evaluated for screening purposes and given our results, can be recommended as a screening instrument either individually or as part of a global symptom assessment strategy. Given their similar ease of administration and operating characteristics, either the single question or ESAS-RLS could be used as an initial screening question for RLS in hemodialysis units given their ability to rule out RLS.

The use of IRLS as a screening instrument and not as a measurement tool is a novel aspect of this study. An IRLS cutoff of 20 showed reasonable discrimination with an AUROC of 0.76 and a sensitivity of 71% and specificity of 81% and may be considered as an acceptable screening instrument either alone or following the single question or ESAS-RLS, although it is more burdensome and time consuming to complete. ESAS-RLS did moderately correlate with IRLS, suggesting that it may be used to quantify RLS severity, but its other psychometric properties including validity, reliability and responsiveness in the dialysis population have yet to be studied.

Our study is limited by its relatively small size and homogeneous population from two hemodialysis units in Hamilton, Canada, but was powered sufficiently to detect a clinically relevant difference in AUROC. However, it was not powered to detect differences between screening instruments. We did not include chronic kidney disease, peritoneal dialysis or kidney transplant patients or those with many secondary etiologies of RLS, so it is unclear if our results can be applied to these settings. We also excluded participants with cognitive impairment, further limiting the generalizability of our findings. However, we had a broad approach for patient inclusion resulting in a population that is typical of a dialysis unit in North America. We included patients with a history of RLS as well as those undergoing treatment (even though only a limited number), which may artificially bias the accuracy of screening instruments. Lastly, it should be noted that participants in this study were informed of the construct of RLS during the consent process prior to completing screening instruments, which could further bias our results due to priming participants, which would not normally occur outside of a research setting.

We did not evaluate other screening instruments, such as the Hopkins telephone diagnostic interview for RLS [21], given its complexity and lack of applicability to the hemodialysis population, who are in regular contact with their healthcare providers. Similarly, we did not perform the immobilization test [22] or electromyography for periodic limb movements (PLM), given their complexity and poor operating characteristics despite the shared epidemiology of PLM with RLS.

The lack of simple, accurate screening questions to identify RLS in the hemodialysis population is a barrier to controlling RLS symptoms. IRLS is a suitable screening instrument (preceded by the single question or ESAS-RLS depending on its acceptability by patients), but ultimately screening cannot replace a detailed clinical assessment by a dialysis physician familiar with RLS and its mimics.

FUNDING

This study was funded by a grant from Cancer Care Ontario/Ontario Renal Network. D.C. is supported by a Kidney Research Scientist Core Education and National Training Program (KRESCENT) Post-doctoral Fellowship Award. M.W. is supported by a New Investigator Award from the Canadian Institutes of Health Research.

AUTHORS’ CONTRIBUTIONS

J.C.R., A.M., K.S., L.M., C.R. and K.S.B. were involved in conception or design. D.C. and M.W. contributed to analysis and interpretation of data, or conception or design, or both. D.C., J.C.R., A.M., K.S., L.M., C.R., K.S.B. and M.W. drafted and revised the article. D.C., J.C.R., A.M., K.S., L.M., C.R., K.S.B. and M.W. provided intellectual content of critical importance to the work described. D.C., J.C.R., A.M., K.S., L.M., C.R., K.S.B. and M.W. approved the final version of the manuscript to be published.

CONFLICT OF INTEREST STATEMENT

The results presented in this article have not been published previously in whole or part, except in abstract format.

Click here for additional data file.