Determination of the smallest detectable change (SDC) and the minimal important difference (MID) for the Small Fiber Neuropathy Screening List (SFNSL) in sarcoidosis.

Background: Previous studies found that 40-60% of the sarcoidosis patients suffer from small fiber neuropathy (SFN), substantially affecting quality of life. SFN is difficult to diagnose, as a gold standard is still lacking. The need for an easily administered screening instrument to identify sarcoidosis-associated SFN symptoms led to the development of the SFN Screening List (SFNSL). The usefulness of any questionnaire in clinical management and research trials depends on its interpretability. Obtaining a clinically relevant change score on a questionnaire requires that the smallest detectable change (SDC) and minimal important difference (MID) are known.
Objectives: The aim of this study was to determine the SDC and MID for the SFNSL in patients with sarcoidosis.
Methods: Patients with neurosarcoidosis and/or sarcoidosis-associated SFN symptoms (N=138) included in the online Dutch Neurosarcoidosis Registry participated in a prospective, longitudinal study. Anchor-based and distribution-based methods were used to estimate the MID and SDC, respectively.
Results: The SFNSL was completed both at baseline and at 6-months' follow-up by 89/138 patients. A marginal ROC curve (0.6) indicated cut-off values of 3.5 points, with 73% sensitivity and 49% specificity for change. The SDC was 11.8 points. Conclusions: The MID on the SFNSL is 3.5 points for a clinically relevant change over a 6-month period. The MID can be used in the follow-up and management of SFN-associated symptoms in patients with sarcoidosis, though with some caution as the SDC was found to be higher. (Sarcoidosis Vasc Diffuse Lung Dis 2018; 35: 333-341). Copyright:

Introduction

Small fiber neuropathy (SFN) is a disabling generalized sensory nerve disorder with structural and functional abnormalities of small fibers. It is characterized histopathologically by degeneration of small fiber nerve endings, selectively involving thin myelinated A∂- and unmyelinated C-fibers (1). Small fibers are associated with thermal and noci-ceptive sensations (somatic) and autonomic function, resulting in symptoms of neuropathic pain and autonomic dysfunction (2,3). The widespread spectrum of symptoms is associated with substantial negative effect on patients’ quality of life (QoL) (4,5).

To date, SFN has been difficult to diagnose, as a gold standard is still lacking. The diagnosis is based on the presence of characteristic clinical features in combination with abnormalities on neurophysiological tests and/or reduced numbers of small fibers in skin biopsy (3,6-8).

SFN can be an epiphenomenon in many diseases (9), including sarcoidosis. Sarcoidosis is a multisystem disorder that represents a burden on patients’ lives. The clinical manifestation, natural history, and prognosis of sarcoidosis are highly variable, and its course is often unpredictable (10). In addition to specific organ-related symptoms with functional impairments, sarcoidosis patients have disabling non-specific symptoms, including fatigue, physical impairments, everyday cognitive failure, and pain (5,11). Previous studies found that 40-60% of the sarcoidosis patients suffer from SFN (12-15). However, there is as yet insufficient awareness of this among clinicians responsible for the follow-up of sarcoidosis patients. There is a clear need for an easily administered small fiber neuropathy (SFN) screening instrument to identify sarcoidosis-associated SFN in general clinical practice and for research purposes. To meet this need, we have previously developed the Small Fiber Neuropathy Screening List (SFNSL) (16).

The quality of a (screening) questionnaire is determined by its validity, reliability and responsiveness (17,18) as well as interpretability (19). Interpretability is an important property of questionnaires that are regularly used in daily practice; it refers to what the SFNSL score means for the patient (19). Previously, Sun et al. showed the validity of the SFNSL in a different cohort, reporting a high correlation between intraepidermal nerve fiber density and the SFNSL, with a sensitivity of 94.1% and a specificity of 90.9% (20).

For evaluative purposes, interpretability is assessed by change in scores: it is important to know when it can be said that a patient has improved or when their situation has worsened. Interpreting change in score requires two benchmarks: the measurement error, expressed as the smallest detectable change (SDC), and the minimal important difference (MID). The SDC represents the minimal change that a patient must show on the scale to ensure that the observed change is real and not just measurement error (19). The MID is the smallest change score that a patient perceives to be important (21). Establishing the SDC and MID will help clinicians interpret the clinical meaning of changes in SFNSL scores over time at individual level. At group level, both measures can be used to perform methodologically sound research and for the clinical interpretation of research findings. Several previous studies have used the SFNSL scores as an outcome measure to determine therapeutic response, and have found significant differences between placebo versus study drugs in patients suffering from SFN (22-24). Although these differences were statistically significant, they might not be relevant to a patient, since no MID and SDC were known for the SFNSL at the time. Till now, no SDC or MID has been established for this questionnaire in patients with sarcoidosis. It is important to determine changes in the SFN-related complaints, and knowledge is therefore needed regarding the measurement properties of the SFNSL.

The present study aimed to determine the measurement error (SDC) and interpretability (MID) of the Small Fiber Neuropathy Screening List SFNSL in patients with sarcoidosis.

Methods

Study design and participants

A prospective cohort study was performed. From June 2014 until June 2017, a Dutch national sample of neurosarcoidosis and/or sarcoidosis patients with SFN-associated symptoms was recruited by inviting these patients to join an online neurosarcoidosis registry, the Dutch Neurosarcoidosis Registry (www.neurosarcoidose.nl). This registry includes patients with a recent diagnosis of neurosarcoidosis and/or SFN-associated symptoms as well as patients whose disease was of longer duration. The diagnosis of neurosarcoidosis and/or SFN-associated symptoms was confirmed by a neurologist for all patients. The diagnosis neurosarcoidosis was established using the Zajicek or Marangoni (modified Zajicek) criteria, labelling patients as possible, probable or definite neurosarcoidosis (25,26). SFN is considered a ‘paraneurosarcoidosis’, since it is not associated with the hallmark of sarcoidosis, granuloma formation, in the small fibers (3,27-29). The Dutch Neurosarcoidosis Registry is an ongoing registry recording demographic parameters, disease-related parameters and multiple electronic questionnaires.

All patients were >18 years old, had sufficient command of the Dutch language and had access to the internet. Patients included in the Dutch Neurosarcoidosis Registry agreed to participate in online research studies. The registry and study protocols have been approved by the Medical Ethics Committee of the St. Antonius Hospital Nieuwegein. Digital informed consent was obtained from all patients.

Procedure

Curavista (a certified eHealth platform) provided the online platform for this study and maintained the website. The data were collected online by Curavista and were exported to a database.

Invitation to complete demographic questions and questionnaires were sent by email via the online Dutch Neurosarcoidosis Registry to all patients every 6 months. The following demographic and medical characteristics of patients were collected (self-reported): sex, age, time since diagnosis of sarcoidosis, medications use, use of prednisone, use of azathioprine (Imuran), use of methotrexate, use of infliximab (Remicade), use of adalimumab (Humira), use of cyclophosphamide and use of other medication. Patients received usual care and medication for their sarcoidosis treatment. SFN-associated symptoms are likely to remain stable or progress in time, since no curative treatment is as yet available. At baseline, the whole cohort was asked to complete the Fatigue Assessment Scale (FAS) (30,31), King’s sarcoidosis questionnaire (KSQ) (32,33), World Health Organization Quality of Life (WHOQOL) Bref (34) and a visual analogue scale (VAS) for pain (35). Furthermore, they were asked to complete the SFNSL at baseline (t0) and at 6-month follow-up (t1). An anchor question was added at t1.

Outcome measures

Small Fiber Neuropathy Screening List

The SFNSL is a 21-item self-administered questionnaire to screen for symptoms related to SFN. The response scale is a five-point scale (0 never/not to 4 always/severe); scores on the SFNSL can range from 0 to 84. Change scores can range from -84 to 84 between t0 (baseline) and t1 (6-month follow-up). Change scores for the SFNSL were calculated by subtracting the baseline score from the 6-month score. A positive change on the SFNSL indicates an increase in SFN-related symptoms.

Cut-off values for diagnosing SFN using the SFNSL have been established after cross-validation in a second group based on temperature threshold testing (TTT) (16). The cut-off score for the SFNSL is 11: a score below 11 indicates few or no SFN-related symptoms (normal TTT), while a score of 11-48 indicates probable or highly likely SFN, and a score above 48 is indicative of SFN (abnormal TTT) (16).

The questionnaire is divided into two parts; part 1 assesses how often patients experience specific complaints, and part 2 assesses how severely patients experience specific complaints. Patients are instructed to answer each question, even if they do not have that specific complaint. No score was generated for patients who did not fully complete the questionnaire. The PDF and digital version of the English SFNSL can be found at http://www.wasog.org/education-research/questionnaires.html, as well as Danish, Dutch, French, German, and Japanese versions (©ild care foundation: www.ildcare.nl).

Anchor question

The anchor question asked at t1 was: Compared to six months ago, do you think that – regarding the questions about SFN complaints (SFNSL) – you are doing worse, the same or better?

Statistical analysis

Descriptive variables were calculated as means (± standard deviation, SD) or medians (25th-75th percentiles) for the continuous variables, depending on the data distribution, and as percentages for the dichotomous variables. Data of all subjects were checked for missing values, distribution (skewness and excess kurtosis), and outliers. Patients providing insufficient information on the SFNSL at t1 or who did not answer the anchor question were excluded from the analysis using case-wise deletion.

Following Crosby et al. (36) and Revicki et al. (37), anchor-based and distribution-based approaches were used to establish the MID and SDC, respectively. The anchor-based approach was used with an external criterion, or anchor, represented by individual patients’ perceived change over time. In a distribution-based approach, the distributional characteristics of our study sample were used to calculate the standard error of measurement (SEM), which was used to determine the SDC. Following De Vet et al. (38) and van Kampen et al. (19), the MID itself was not calculated using effect sizes or the SEM (i.e. data variability). Instead, the SDC was determined, thus taking into account the variation in the scores on the SFNSL scale due to measurement error. The MID was determined by the anchor-based method only. The MID is also known as the minimal important change (MIC) or the minimal clinically important difference (MCID), but strictly spoken these are not the same (39-41). The MID addressed in this paper implies the longitudinal within-person change in scores.

A minimum of 50 patients is considered sufficient for assessing measurement properties (42). Since the risk of loss to follow-up often increases after several months, we wanted to include at least 100 patients at baseline. Patients who only completed the t0 assessment were compared with patients who also completed the t1 assessment as regards sex, age, medications use, and the score on the SFNSL at t0, using independent t-tests and chi-square tests. At a minimal statistical power of 80%, p values below 0.05 were considered significant.

Smallest detectable change (measurement error)

Data from t0 were used to determine the measurement error. The change score distribution was tested for normality (skewness and excess kurtosis), as it was important that the change scores should be normally distributed and close to zero (43). Measurement error can be expressed as the standard error of measurement (SEM) or as SDC (19). The SEM represents the standard deviation of repeated measures in one patient, and was calculated using the intraclass correlation coefficient (ICC): SEM = SD * √(1-ICC agreement) (21). The ICC was calculated using a two-way mixed effects model for absolute agreement. The SDC, the smallest change in a score that that you can detect with the SFNSL above measurement error (19), is also known as the minimal detectable change when using its 95% confidence interval (MDC95), and was calculated using the SEM: 1.96 × √2 × SEM (44). These values were expressed in the unit of measurement of the SFNSL scale.

Minimal important change

In the anchor-based approach, a receiver operating characteristic (ROC) curve was used to estimate the MID (38), the smallest measured change in score that is perceived as being relevant by the patients. The anchor question was considered the gold standard to establish a cut-off value for the SFNSL. The anchor question firstly distinguished patients who had improved or worsened from patients who had remained stable, and secondly, distinguished improved patients from not improved (stable or worsened) patients in terms of the change in complaints they had perceived. The ROC curve was obtained by plotting the sensitivity against 1-specificity for each possible SFNSL change score. The ROC curve was used to estimate the anchor-based MID for the SFNSL. Most important for clinical practices is the MID for a change in score. For research purposes, the MID for improvement is preferable, therefore we have also added this MID. The area under the ROC curve represents the probability that the SFNSL will discriminate between two patient states, referred to as ‘important change (improved/worsened)’ and ‘no important change (i.e. the same)’. An area under the curve between 0.5-0.6 was regarded as ‘failed’, between 0.6-0.7 as ‘poor’, between 0.7-0.8 as ‘fair’ and between 0.8-0.9 as ‘good’. The optimal cut-off point was considered to be the lowest value for which the sum of percentages of true positive and true negative classifications was largest, assuming that sensitivity (ruling out important change and improvement on the SFNSL) and specificity (ruling in important change and improvement on the SFNSL) are equally important for this patient population. This optimal cut-off point was used to establish the overall MID.

Change scores on the SFNSL were calculated as each patient’s t1 score (6-month follow-up) minus their t0 score (baseline). This was done for each subscale, based on the anchor question. The MID for deterioration was defined as the mean change score in the subgroup that worsened (i.e. perceived their complaints as ‘worse’ after six months), while the MID for improvement was defined as the mean change score in the subgroup that improved (i.e. perceived their complaints as ‘better’ after six months).

Results

The flowchart of the study is displayed in Figure 1. A total of 150 patients subscribed to the online Dutch Neurosarcoidosis Registry study. Twelve of them never completed the SFNSL and were therefore excluded from the analysis. The response rate at t0 (baseline), with 138 respondents, was 92%. Of these 138 patients, 89 (64%) completed the SFNSL and the anchor question at t1, and could thus be analyzed.

Fig. 1.

Flowchart of study patients

Demographical and medical characteristics of the 89 patients are presented in Table 1. Neurosarcoidosis was established in 44 patients (49.4%) and SFN-associated complaints in 81 patients (91%). The organ involvement presented in table 1 was self-reported by completing the KSQ.

Table 1.

Demographic and medical characteristics of neurosarcoidosis patients and/or patients suffering from SFN symptoms (n=89)

Characteristics	Number (%) or median (25th-75th percentile) or mean±SD
Demographics
Sex: male	45 (51)
Age (yr)	53.2±10.5
Medical Variables
Time since diagnosis (yr)	6(2-12)
Age at diagnosis neurosarcoidosis (yr)	44 (49.4)
SFN-related complaints (SFNSL >11)	81 (91)
Medication use	62 (70)
• prednisone	32 (36)
• azathioprine (Imuran)	3 (3)
• methotrexate	30 (34)
• infliximab (Remicade)	12 (14)
• adalimumab (Humira)	4 (5)
• cyclophosphamide	0 (0)
• other medication	14 (16)
Psychological variables
FAS score	32.1±8.3
WHOQOL-BREF
Physical Health	10.5±2.7
Psychological Health	13.2± 2.7
Social Relationships	13.5±2.9
Environment	15.2±2.2
KSQ
General Health Status	55±19.2
Lung (N=60, %)	60.9±23.8
Skin (N=20)	66.7±22.5
Eyes (N=41)	54.3±22.6
Medication (N=62)	65.7±26.7

SFN=small fiber neuropathy, SFNSL=small fiber neuropathy screening list, FAS = fatigue assessment scale,

WHOQOL-BREF=World Health Organization Quality of Life BREF, KSQ=King’s sarcoidosis questionnaire

The mean scores on the FAS, SFNSL and VAS pain were 32.1, 34.9 and 4.4, respectively (see Tables 1 and 2). The VAS correlates well with the SFNSL score (R=0.542; p-value=<0.0001).

Table 2.

Small Fiber Neuropathy Screening List (SFNSL) and VAS pain scores (N=89)

	Mean±SD or number (%)
SFNSL t0 score (0-84)	34.9±18.7
SFNSL t0, definitely no SFN (<11)	8 (9)
SFNSL t0, probable SFN (11-48)	56 (63)
SFNSL t0, definitely SFN (>48)	25 (28)
VAS pain score	4.4±2.8

VAS=visual analogue scale

A statistically significantly higher age was found in the follow-up group compared to the age in the drop-out group (t(136)=2.75, p<0.01). No other differences were found in the variables mentioned in table 1 nor the SFNSL at t0 between the patients who dropped-out of the study before t1 (i.e. the 6-month follow-up) and the patients remaining in the study.

The t1 questionnaires were completed within an average of 5.9 months (SD=0.57) after t0, without skewness but with a leptokurtic distribution (z-score>3.29), implying a higher peak and thus more clustered data around the mean. The change scores of the SFNSL were normally distributed (no statistically significant skewness or excess kurtosis) and close to zero, with an average of -2.44 (SD=8.12). The intraclass correlation coefficient (ICC agreement) was 0.95 for the SFNSL, with a SEM of 4.26 as calculated by the abovementioned equation. The calculated value of SDC or MDC95 was 11.8.

Minimal important difference

The area under the ROC curve for importantly changed status was 0.6, so the ability of the SFNSL to distinguish patients who worsened or improved from patients who remained stable was just above change level. The cut-off value for a MID in the SFNSL score was 3.5 points, corresponding to a sensitivity of 73% and a specificity of 49%: 73% of the patients were correctly identified as changed and 49% of the patients were correctly identified as stable (Figure 2a).

Fig. 2.

ROC curves comparing Small Fiber Neuropathy Screening List (SFNSL) with the anchor question. Diagonal segments are produced by ties. a) Patients with an importantly changed status versus patients with a stable status; b) Patients with an importantly improved status versus patients with a non-improved status

The area under the ROC curve for importantly improved status was also 0.6. The cutoff value for a MID in the SFNSL score was 8.5 points, corresponding with a sensitivity of 56% and a specificity of 78%: 56% of the patients were correctly identified as improved and 78% of the patients were correctly identified as not improved (Figure 2b).

The mean change scores on the SFNSL per subgroup based on the anchor question are presented in Table 3. When using these data, a MID of 0.14 for deterioration and a MID of 7.94 for improvement was found.

Table 3.

Mean change scores on the SFNSL compared with those on the anchor question (n=89)

Anchor (perceived change) in complaints n (%)	Mean change score on SFNSL mean±SD
Worse 28 (31)	0.14±7.46
The same 45 (51)	-2.09±8.47
Better 16 (18)	-7.94±5.62

Discussion

The SFNSL is a validated questionnaire to assess the presence of SFN-related symptoms in sarcoidosis. The present study established the SDC and MID for the SFNSL in a group of sarcoidosis patients. A MID of 3.5 was found to distinguish between change of complaints and no change of complaints.

To avoid any misconception, the SFNSL was developed to assess symptoms which might be related to SFN. The results might offer a clinician a first suspicion of SFN and stimulate him or her to refer this patient to a neurologist for further evaluation of the problem related to or probably associated with the symptoms. Even when the symptoms are not related to SFN they cause a substantial burden for the patients and have impact on their QOL. It is always hard to measure and follow-up non organ related symptoms. Moreover, so far no gold standard for diagnosing SFN exists. Therefore, the MID was calculated just to evaluate whether from the patient’s perspective the symptoms changed or not. Hence, it is not meant to assess improvement of the underlying cause.

The MID is the smallest change score that patients perceive to be important (21). As such, it is the smallest change score that one wants to detect with the SFNSL. The SDC is the smallest change score that one can detect with the SFNSL in individual patients. The SDC we found was larger than the MID for detecting change of complaints and slightly larger than the MID for detecting improvement. This means that it is more difficult to distinguish a clinically important change from the measurement error with a large degree of certainty in individual cases, compared to patient-related outcomes (PROMs), where the SDC is smaller than the MID (19,45). In this case: if the individual change in score is 3.5 points or higher, there may be a clinical relevant change on the SFNSL. However, it may also be due to a measurement error on the scale. A change of 11.8 points or higher gives a high degree of certainty that it indicates a clinically relevant change in SFN related symptoms. Nevertheless, the similar results for both methods (MID for improvement 8.5 vs 8.0) might suggest that our estimation for MID is robust.

SFN is a major concern in sarcoidosis, leading to decreased QoL (4). SFN complaints are difficult to manage, since no curative treatment currently exists (3).The treatment of SFN is at present merely symptomatic (neuropathic pain medications) with poor to moderate responses. The validated SFNSL is a practical clinical screening tool that can guide clinicians in assessing whether patients are suffering from SFN-associated symptoms, but the interpretability of change scores (MID) over time was unclear. The MID is of great clinical importance, since it will help to manage these patients better. In clinical practice, a positive change in symptoms scores is not to be expected for many patients. This is why we decided to determine the MID, in order to enable doctors in clinical practice to distinguish changes in complaints.

For research purposes, we also determined the MID for improvement in SFNSL scores. The higher SDC is less relevant for use in clinical trials at group level than it is for use in individual patients. In clinical research, it is important to look at clinically relevant improvements of SFN complaints and not merely at a statistically significant change. The MID could therefore be used as a useful outcome measure in clinical trials. TNF-alpha inhibition and immunoglobulins have been reported to be effective treatment options for sarcoidosis-associated SFN (46-49). Whether these expensive treatments should be initiated for SFN is unclear. These treatment modalities, as well as new promising drugs such as Cibinetide® (ARA290) (24,50) also need to be evaluated using the SFNSL and MID, to assess whether there is a clinically relevant effect, rather than a mere statistically significant increase in the number of small fibers.

Our group has previously determined the MID for another validated questionnaire in sarcoidosis, the fatigue assessment scale (FAS) (31). Fatigue is also a major concern in sarcoidosis, and determining the MID for the FAS has made a significant contribution to clinical practice in terms of improving the clinical interpretation of fatigue scores. The present study used the same methods (anchor-based and distribution-based methods). The area under the ROC curve (AUC) for both analyses (improvement and change in complaints on the SFNSL) was 0.6, which is poor. The low AUC can be explained, amongst other things, by the heterogeneity of sarcoidosis and the wide organ- as well as non-organ related symptoms. For instance, the MCID of the FAS has an AUC of 0.6 and the MCID of the VAS pain was 0.7 (31,51). Therefore, an AUC of 0.6 cannot be considered a poor performance and may be used in clinical practice, especially since no other estimate is available’.

This study had some limitations. One limitation may be that the patients were recruited using the online neurosarcoidosis registry, which includes neurosarcoidosis patients and patients suffering from SFN. Online registries may generate biases. However, of all patients, additional medical information was provided by their caregivers, confirming the diagnosis sarcoidosis. In addition, we examined symptoms that can only be assessed by asking the patients themselves. Moreover, it could also be argued that only patients who are motivated to participate in research studies registered for our online neurosarcoidosis study. This may have caused selection bias. Nevertheless, in our experience, all sarcoidosis patients are highly motivated to participate in studies.

Furthermore, we have used one anchor question to examine the MID. This has several limitations, but is in line with many other studies (31,51). It is an option to examine more anchors, but the disadvantage is that it may yield a different result, which raises the question what result than represents the better information. Therefore, we have chosen one simple anchor. We would like to acknowledge that a simple anchor is also previously used in other studies regarding questionnaires (52). Another limitation could be a recall bias. Patients were asked the anchor question after 6 months. However, neurologic complaints usually persist for quite some time and often do not improve or recover. Moreover, these complaints are continuously bothering the patient, so in our experience patients are capable to tell whether or not the complaints have worsened or not, even after 6 months. Furthermore, in the literature 6 months is a very common period when establishing a MID (39).

In conclusion, the present study showed that a change of at least 3.5 points in the patients’ SFNSL score represents a clinically important difference, in that the SFN-related symptoms showed a relevant increase or decrease. Some caution is warranted for the use of the MID in individual cases in the clinic, as the SDC was found to be higher than the MID.