Diagnostic accuracy of the lumbar spinal stenosis-diagnosis support tool and the lumbar spinal stenosis-self-administered, self-reported history questionnaire.

Despite the applicability of the lumbar spinal stenosis (LSS)-diagnosis support tool (DST) and the LSS-self-administered, self-reported history questionnaire (SSHQ), their diagnostic accuracy has never been compared with that of the well-known North American Spine Society (NASS) clinical description of LSS. This study aimed to compare the diagnostic accuracy of the two diagnostic tools with that of the NASS guidelines' clinical description of LSS in a Japanese secondary care hospital setting. This multicenter cross-sectional study used data from the lumbar spinal stenosis diagnostic support tool (DISTO) project, which was conducted from December 1, 2011 to December 31, 2012. Japanese adults with low back pain (LBP) aged ≥20 years were consecutively included. The reference standard was LSS diagnosed by orthopedic physicians. The diagnostic accuracy of the two support tools was compared. Of 3,331 patients, 1,416 (42.5%) patients were diagnosed with LSS. The NASS clinical description of LSS had a sensitivity of 63.9% and specificity of 89.5%. The LSS-DST and LSS-SSHQ had sensitivities of 91.3% and 83.8% and specificities of 76.0% and 57.6%, respectively, with substantial improvements in sensitivity (P < 0.0001). Similar results were obtained when we limited included patients to those aged >60 years. These findings indicated that the LSS-DST and LSS-SSHQ were more sensitive in screening patients with LBP for a diagnosis of LSS than the NASS clinical description of LSS. This study strongly supports prioritizing the use of either of these two diagnostic support tools for screening.

1. Introduction

Lumbar spinal stenosis (LSS) is a common musculoskeletal disorder in the aging population, with a prevalence rate of approximately 11% in the general population [1]. An accurate diagnosis of LSS is challenging due to a lack of consensus concerning definitive diagnostic criteria and the requirement for consistency between physical manifestations and imaging features. Specifically, expert clinicians should diagnose LSS through careful physical examinations and consistent findings in imaging examinations, including roentgenography, computed tomography (CT), and magnetic resonance imaging (MRI). To facilitate this challenging diagnosis, numerous clinical definitions and diagnostic support tools for LSS have been developed [2-8]. Nonetheless, the diagnostic performance of these diagnostic aids has not yet been fully compared.

Two diagnostic support tools, the LSS-diagnosis support tool (LSS-DST) and the LSS-self-administered self-reported history questionnaire (LSS-SSHQ), have been developed in Japan to aid primary care physicians in accurately identifying patients with LSS and to provide appropriate care [3, 4]. The LSS-DST and LSS-SSHQ have been rated as having level II diagnostic evidence for LSS by the Degenerative LSS Work Group of the North American Spine Society (NASS) Evidence-Based Clinical Guideline Development Committee [2]. In addition, these support tools have shown good applicability according to the latest systematic review that used the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 assessment tool [9].

A new cut-off value for the diagnostic accuracy of the LSS-SSHQ in primary care settings has been reported [10]. However, diagnostic accuracy measures derived from research studies may not reflect real-world properties due to a lack of external validation and the possibility of a unideal diagnostic flow. Indeed, in that study, a definitive diagnosis of LSS was only partially guided by the LSS-DST and false negatives (i.e., missed diagnoses) may have occurred. The clinical description of LSS found in the NASS guidelines is the most common reference [2], and primary care physicians or orthopedic residents may utilize this clinical description when examining a patient with suspected LSS. However, it remains unclear how accurately this clinical description helps to identify LSS. Therefore, the superiority of the aforementioned two diagnostic support tools over the NASS diagnostic guidelines must be externally validated for situations in which a definitive diagnosis is made solely by an orthopedic surgeon.

This large-scale, multicenter, cross-sectional study aimed to compare the diagnostic test accuracies between the two support tools and the clinical description of LSS in the NASS diagnostic guidelines at secondary care hospitals in Japan. We hypothesized that the two diagnostic support tools for LSS would be more sensitive and more useful for screening than the clinical description of LSS in the NASS diagnostic guidelines.

2. Materials and methods

2.1 Study design and data collection

This multicenter cross-sectional study used data from the Lumbar Spinal Stenosis Diagnostic Support Tool (DISTO) project, which was conducted from December 1, 2011 to December 31, 2012. The DISTO project was implemented in 1657 medical institutions under the guidance of the Japanese Society for Spine Surgery and Related Research (JSSR) to verify awareness and the diagnostic accuracy of a lumbar spinal stenosis diagnostic support tool in order to contribute to early detection and treatment of LSS. Recruitment for study participation was announced on the JSSR website, and the study was conducted at facilities that expressed a willingness to participate. An LSS-DST checklist and the NASS clinical description of LSS were distributed to participating medical facilities. The physician-in-charge completed the checklist, in addition to providing usual medical care. Patients who agreed to participate in the study were asked to complete the LSS-SSHQ prior to their consultation. The DISTO project collected and analyzed the checklist and the diagnostic information provided by the physician concerning LSS, peripheral artery disease (PAD), and diabetes mellitus (DM). The target population included patients with low back pain (LBP) aged ≥20 years who had undergone a medical examination, irrespective of the reason for visiting secondary care hospitals with an orthopedic department. Patients were included only based on their LBP symptom, regardless of leg symptoms or the duration of LBP. Participants were consecutively recruited from December 1, 2011, to December 31, 2012. Exclusion criteria comprised patients with the following: heart failure, renal failure, respiratory failure, hepatic failure, a decreased level of consciousness, a history of psychiatric disorders (e.g., schizophrenia or personality disorders), and a history of spinal surgery. The ethics committees of Fukushima Medical University (No. 1136) and the Japanese Orthopaedic Association approved this study. The participants were informed that data from the study would be submitted for publication and they provided their written informed consent.

2.2 Reference standards for LSS

The reference standard for LSS was a final diagnosis of LSS by the orthopedic physicians in charge of the participants. Participants were carefully assessed based on their medical history, the results of a detailed physical examination, and radiological findings from modalities such as radiography, CT, and MRI. In the absence of universally acceptable diagnostic criteria for LSS, decision-making by a professional clinician was adopted to establish an accurate diagnosis.

2.3 Index tests

2.3.1 The LSS-DST

The LSS-DST is a brief clinical diagnostic tool that helps physicians precisely diagnose patients with LSS () [3]. It consists of 10 items that are grouped into three main categories, namely, medical history, symptoms, and physical examination. The LSS-DST can be scored by primary care physicians within their usual resources without the need for special equipment or imaging studies. There is a positivity cut-off point of 7, where the area under the receiver operating characteristic (ROC) curve is the highest. At this cut-off point of 7, the sensitivity and specificity of the LSS-DST have been reported to be 92.8% and 72.0%, respectively [3]. All participating orthopedic physicians consented to use the LSS-DST for each patient.

2.3.2 The LSS-SSHQ

The LSS-SSHQ was developed to evaluate the diagnostic value of the medical history of patients with LSS () [4]. This self-completed questionnaire comprises 10 items concerning subjective symptoms only. The LSS-SSHQ can be distributed to patients by primary care physicians unfamiliar with neurological physical examination. Scoring can be completed by the patients or their primary care physicians. One validation study reported a sensitivity and specificity of 84% and 78%, respectively, with an area under the ROC curve of 0.782 [4]. We adopted a new cut-off point for the LSS-SSHQ (LSS-SSHQ version 1.1; a total score of 3 on Q1–Q4 or a score of ≥1 on Q1–Q4 and ≥2 on Q5–Q10 indicated positivity), as this cut-off point had higher sensitivity and negative predictive value (NPV) than the original value used in primary care settings [10]. All patients completed the LSS-SSHQ, to which version 1.1 scoring was later applied. The written questionnaires were collected so that the attending physician could not refer to them.

2.4 Typical clinical descriptions of LSS according to the NASS

As one of the most universally recognized presentations of LSS, the following clinical descriptions written in the NASS guidelines were used in the present study [2]:

“1. Degenerative LSS describes a condition in which there is diminished space available for the neural and vascular elements in the lumbar spine secondary to degenerative changes in the spinal canal.

2. When symptomatic, this causes a variable clinical syndrome of gluteal and/or lower-extremity pain and/or fatigue, which may occur with or without back pain.

3. Provocative features include upright exercise, such as walking or positionally induced neurogenic claudication. Palliative features commonly include symptomatic relief with forward flexion, sitting, and/or recumbency.”

Given that the description in point 1 is a morphological feature and not testable in a clinical practice setting, we adopted the descriptions set out in points 2 and 3 for LSS and considered them to represent typical clinical presentations in this study. Attending physicians assessed patients based on these descriptions using a checklist.

2.5 Statistical analyses

Demographic characteristics, comorbidities, and outcomes were analyzed using descriptive statistics. To evaluate the diagnostic test accuracy of the clinical description of LSS in the NASS diagnostic guidelines, the LSS-DST, and the LSS-SSHQ, the sensitivity and specificity of each index test was examined. In addition, the sensitivities and specificities of the LSS-DST and LSS-SSHQ were compared with those of the clinical description of LSS in the NASS guidelines using the McNemar test [11]. Furthermore, the NPVs of the three tools were also calculated, as it is important to determine the number of false positives obtained by physicians who were unskilled in examining LSS when using these tools clinically. To examine the overall diagnostic accuracy of the three index tests, we also calculated the diagnostic odds ratio (DOR) according to the following equation: DOR = (sensitivity × specificity)/(1-sensitivity × 1-specificity) [12].

Several sensitivity analyses were performed. First, we included participants with a total score of >7 and those with a score of ≤3, despite lacking ankle brachial index (ABI) and other values, and these scores were regarded as being positive or negative for the LSS-DST, respectively. Second, we performed a sensitivity analysis limited to those aged ≥60 years. All statistical analyses were performed using SAS version 9.3 (SAS Institute Inc., Cary, NC, USA) software. A P-value <0.05 was considered to indicate statistical significance.

3. Results

3.1 Patient background

Overall, 10,669 patients with LBP participated in this study. After excluding 7,338 patients with missing or inappropriate data in relation to the LSS-DST and LSS-SSHQ, 3,331 participants were included in the primary analysis (). Numerous participants (n = 4,082) did not undergo an ABI assessment, as the ABI was usually only performed for patients with suspected PAD. presents the study participants’ characteristics. In total, 1,755 men and 1,564 women (12 cases of missing sex data) were examined by hospital-based orthopedists.

Flow chart of participant inclusion.

LBP, low back pain.

3.2 Outcome data

LSS was prevalent in 42.5% of the population. Test results obtained using the LSS-DST, LSS-SSHQ, and the NASS clinical description of LSS are shown in . Only 63.9% of patients with LSS met the NASS clinical description of LSS (sensitivity 63.9% [95% confidence interval (CI) 61.4%–66.4%]), while 89.5% of patients without LSS did not meet this condition (specificity 89.5% [95% CI 88.1%–90.9%]) ().

The sensitivity of the LSS-DST was superior to that of the NASS clinical description (91.3% [95% CI 89.9%–92.8%] vs. 63.9% [95% CI 61.4%–66.4%], P < 0.0001; ); however, its specificity was inferior to that of the NASS clinical description (76.0% [95% CI 74.1%–77.9%] vs. 89.5% [95% CI 88.1%–90.9%], P < 0.0001; ).

The LSS-SSHQ also exhibited superior sensitivity when compared with the NASS clinical description (83.8% [95% CI 81.8%–85.7%] vs. 63.9% [95% CI 61.4%–66.4%], P < 0.0001; ). However, the specificity of the LSS-SSHQ was inferior to that of the NASS clinical description (57.6% [95% CI 55.3%–59.8%] vs. 89.5% [95% CI 88.1%–90.9%], P < 0.0001; ).

The NPVs were 0.77 (95% CI 0.75–0.79) for the NASS clinical description, 0.92 (95% CI 0.91–0.94) for the LSS-DST, and 0.83 (95% CI 0.81–0.85) for the LSS-SSHQ (

The DORs for each index test were 15.1 (95% CI 12.6–18.1) for the NASS clinical description, 33.3 (95% CI 26.9–41.1) for the LSS-DST, and 7.0 (95% CI 5.9–8.3) for the LSS-SSHQ ().

Similar results were obtained when patients (n = 7,914) with >7 points on the LSS-DST without ABI data were treated as LSS-DST-positive and patients with <7 points without ABI data were treated as LSS-DST-negative (). When patients aged >60 years (n = 2,136) were included in the sensitivity analysis, the results were similar to those of the main analysis (

4. Discussion

In this study, which was conducted in secondary care hospital settings, the LSS-DST and LSS-SSHQ had significantly higher sensitivity for diagnosing LSS in patients with LBP than the clinical description of LSS in the NASS diagnostic guidelines.

Several diagnostic support tools for LSS have been developed, and each of these tools has been reported to have high sensitivity. The LSS-DST, which was the first diagnostic support tool used for LSS, was developed for patients aged >20 years with primary symptoms of pain or numbness in the legs in a Japanese hospital setting, including university hospitals, medical centers, and clinics affiliated with such hospitals. This 10-item tool (two items for medical history, three items for patients’ symptoms, and five items for physical examination) has been reported to have a sensitivity of 92.8% and a specificity of 72.0% [3]. The LSS-SSHQ was then later developed to assess the diagnostic value of a patient’s history in a hospital setting. The sensitivity and specificity of the LSS-SSHQ have been reported to be 84% and 78%, respectively [4]. The LSS-SSHQ was externally validated in a Japanese primary care setting, and its sensitivity has improved with the introduction of a new cut-off value (79.8% vs. 68.3%) [10]. However, no studies have compared the diagnostic accuracy of these diagnostic support tools among identical participants despite the influence of the patient spectrum (i.e., variations in the severity of the targeted disease and differential diagnosis) on sensitivity. To our knowledge, this is the first study to compare the diagnostic accuracy of these diagnostic support tools and the clinical description noted in the NASS guidelines for patients suspected of having LSS in the same patient spectrum.

We consider that the findings in this study may influence the activities of both physicians and epidemiological researchers for several reasons. First, the improved sensitivity of the two diagnostic support tools when compared with the clinical description of LSS in the NASS diagnostic guidelines provides evidence in support of their use in screening for the diagnosis of LSS. The high NPVs of the diagnostic support tools for LSS are important because of the clinical significance in terms of effectively limiting false-negative results, even for clinicians unfamiliar with diagnosing and treating patients with LSS. These two support tools can be useful, especially for primary care physicians and junior orthopedic surgeons, in conjunction with the universally recognized clinical description of LSS in the NASS diagnostic guidelines [2]. Further, these tools have been referred to in the management of patients with LBP and/or lower-extremity symptoms. A correct diagnosis of LSS is often difficult to achieve, particularly in the early stages, and extraspinal disorders such as PAD, diabetes-related peripheral neuropathy (DPN), and other musculoskeletal diseases can be misdiagnosed [13, 14]. The superior sensitivity of the two support tools can be explained by additional factors that are considered for differential diagnosis in actual practice, including age, the presence of DM, the ABI, and the results of a straight leg raising test. In addition, clinicians also consider lower extremity symptoms, intermittent claudication, and postural factors described in the NASS guidelines.

Second, the excellent sensitivity of these diagnostic support tools is likely to reduce unnecessary and costly imaging tests such as CT and MRI scans. The high sensitivity of these two tools and their low cost may also facilitate large-scale epidemiological studies on LSS. To date, data on the population-based epidemiology of LSS are relatively limited. This is due, in part, to the difficulty in diagnosing LSS [15] given the absence of objective diagnostic criteria, even with the use of imaging tests [16]. In this study, the LSS-DST had the highest DOR of the three tools, and the estimated DOR for the LSS-DST in detecting LSS was 33.3. This suggests that, when using the LSS-DST, the odds of positivity among patients with LSS is 33.3 times higher than the odds of positivity among patients without LSS. The DOR is a simple and statistically tractable indicator that can be used to assess diagnostic accuracy without the need for other indicators [12], and the results of this study indicated the LSS-DST was one of the best available screening methods for LSS. Moreover, this study verified the external validity of the diagnostic accuracy of the two diagnostic support tools, which have been considered to have good applicability according to the QUADAS-2 assessment tool [9].

This study had several strengths. First, the large-scale nationwide study design and inclusion of >3,000 participants ensured the generalizability of our findings regarding the usefulness of the two diagnostic support tools. Second, the use of diagnosis by orthopedic surgeons as a reference standard reflects the best current diagnostic practice. Although the large number of facilities participating in the study may make standardization of clinicians’ diagnostic procedures challenging, we consider that we were able to estimate the diagnostic accuracies of the screening tools at current standards of medical care. Third, similar results were obtained through sensitivity analysis conducted on two different populations, indicating that the detected results were robust.

Nevertheless, this study also had several limitations. First, no expert consensus has been reached regarding the reference standard for diagnosing LSS. According to a recent study, expert consensus building is recommended when conducting research on diagnostic accuracy for diseases for which a clear diagnostic definition has not been established [17]. However, it was impractical to build expert consensus on LSS for each participant, as this was a multicenter nationwide study involving thousands of patients. Second, some of our findings obtained in outpatient settings at orthopedic hospitals may not be applicable to primary care settings. As more patients are likely to have severe LSS in this setting than in the primary care setting, the sensitivity of the three index tests (LSS-DST, LSS-SSHQ, and the clinical description in the NASS guidelines) may be higher in secondary care settings than in primary care settings (i.e., spectrum bias). Further studies are warranted to confirm whether differences in diagnostic accuracy between the two diagnostic support tools and the NASS clinical descriptions estimated in this study apply to primary care settings. Third, as the physician who utilized the LSS-DST and the NASS description was also involved in the determination of the reference standard, the diagnosis of LSS may have been guided by the results of the LSS-DST [18]. Therefore, the sensitivities obtained in the current study may be higher than the actual sensitivities. Fourth, we did not have data on the severity of diseases that were used as exclusion criteria. It is possible that several patients with diseases such as DPN and PAD were excluded, and this should be considered when interpreting the results of this study. Fifth, there is the possibility of selection bias because many participants were not included in the main analysis due to missing values. Therefore, we compared participants included in the main analysis with those who had earlier been excluded (). Participants included in the main analysis were older, there was a higher proportion of men than women, and there was a higher presence of LSS. Therefore, this may suggest that selection bias may be introduced in the participants included in the study. However, we included participants with a total score of >7 and those with a score of ≤3, despite lacking ABI and other values in sensitivity analysis. The number of participants in the sensitivity analysis was increased to 7,914, and the results were similar to those of the main analysis. Therefore, we consider the results of our study were robust.

5. Conclusion

The LSS-DST and LSS-SSHQ were significantly more sensitive than the clinical description of LSS in the NASS diagnostic guidelines, based on our analysis of data from a large population of patients with LBP. Prioritizing the use of either of these two diagnostic support tools for screening should be emphasized in clinical practice.

Details of excluded cases because of missing data.

ABI, ankle brachial index; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire.

(DOCX)

Click here for additional data file.

In this analysis, participants with >7 points on the LSS-DST, despite missing ABI and other values, were treated as LSS-DST-positive, and participants with <7 points, despite missing ABI and other values, were treated as LSS-DST-negative (n = 7,914). ABI, ankle brachial index; CI, confidence interval; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire.

(DOCX)

Click here for additional data file.

DORs of the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ.

In this analysis, participants with >7 points on the LSS-DST, despite missing ABI and other values, were treated as LSS-DST-positive, and participants with <7 points, despite missing ABI and other values, were treated as LSS-DST-negative (n = 7,914). ABI, ankle brachial index; CI, confidence interval; DORs, diagnostic odds ratios; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire.

(DOCX)

Click here for additional data file.

Sensitivity and specificity of the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ in participants aged >60 years (n = 2,136).

(DOCX)

Click here for additional data file.

DORs of the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ in participants aged >60 years (n = 2,136).

(DOCX)

Click here for additional data file.

Comparison of the participants who were included and those who were excluded.

Each comparison was subjected to a χ-square test. LSS, lumbar spinal stenosis.

(DOCX)

Click here for additional data file.

28 Feb 2022

13 Apr 2022

Response to the Comments

We wish to express our gratitude to the Reviewers for their insightful comments that have helped us to significantly improve our paper. We have revised our manuscript and tables according to the feedback we received. Please check our revised files as well as our responses to your comments below.

Responses to Editor

Reviewer #1

Line 2. Suggest to include prevalence estimates from Jensen 2021 “Prevalence of lumbar spinal stenosis in general and clinical populations: a systematic review and meta-analysis” (10.1007/s00586-020-06339-1) as this is the most recent and comprehensive review of prevalence estimates for LSS.

Response: Thank you for pointing this out. Indeed, the review you suggested provides the most recent and comprehensive review prevalence estimates for LSS. We have therefore revised the relevant sentence as shown below. The references cited have also been revised accordingly.

“Lumbar spinal stenosis (LSS) is a common musculoskeletal disorder in the aging population, with a prevalence rate of approximately 11% in the general population [1].” [Page 5, Lines 56]

Lines 56-58. I question whether the diagnosis needs to be made by an orthopedic surgeon. There are many other medical specialties (and health professionals) that are primary contact for people with LSS. In fact, on line 76 you report that primary care physicians also diagnose LSS. At minimum, I would suggest a reference for the statement that LSS need be diagnosed by a well-trained orthopedic surgeon.

Response: Thank you for your meaningful remarks. In our previous studies, diagnosis by an orthopedic surgeon skilled in examining LSS has been used as the reference standard (i.e., the gold standard) to examine diagnostic accuracy [1-3]. However, as you point out, there are many medical specialties involved in LSS in clinical settings and, unlike in research, diagnosis by a skilled orthopedic surgeon is not essential for the diagnosis of LSS. Therefore, we have made the following modification in the relevant text.

“Specifically, expert clinicians should diagnose LSS through careful physical examinations and consistent findings in imaging examinations, including roentgenography, computed tomography (CT), and magnetic resonance imaging (MRI).” [Page 5, Lines 59]

1. Konno S, Hayashino Y, Fukuhara S, Kikuchi S, Kaneda K, Seichi A, et al. Development of a clinical diagnosis support tool to identify patients with lumbar spinal stenosis. Eur Spine J. 2007;16: 1951–1957. doi: 10.1007/s00586-007-0402-2.

2. Konno S, Kikuchi S, Tanaka Y, Yamazaki K, Shimada Y, Takei H, et al. A diagnostic support tool for lumbar spinal stenosis: a self-administered, self-reported history questionnaire. BMC Musculoskelet Disord. 2007;8: 102. doi: 10.1186/1471-2474-8-102.

3. Sugioka T, Hayashino Y, Konno S, Kikuchi S, Fukuhara S. Predictive value of self-reported patient information for the identification of lumbar spinal stenosis. Fam Pract. 2008;25: 237–244. doi: 10.1093/fampra/cmn031.

Line 66. Suggest to define NASS for readers here and remove from line 75.

Response: Thank you for this suggestion. We have defined NASS in the recommended sentence, and we have removed its definition from line 75.

“The LSS-DST and LSS-SSHQ have been rated as having level II diagnostic evidence for LSS by the Degenerative LSS Work Group of the North American Spine Society (NASS) Evidence-Based Clinical Guideline Development Committee.” (Page 6, Line 69-70)

Line 80. Suggest to add “diagnostic guidelines” after NASS.

Response: Thank you for the suggestion. We have added “diagnostic guidelines” after NASS, according to your suggestion.

“Therefore, the superiority of the aforementioned two diagnostic support tools over the NASS diagnostic guidelines must be externally validated for situations in which a definitive diagnosis is made solely by an orthopedic surgeon” (Page 6, Line 82)

Line 90. I would appreciate more clarity on how this study data was collected. 1,657 hospitals is a large recruitment procedure. Was this data collected specifically for this study, or integrated into the standard data collection across these centers? Perhaps a national registry?

Response: Thank you very much for your question. This study was conducted as part of the lumbar spinal stenosis Diagnosis Support Tool (DISTO) project under the supervision of the Japanese Society for Spine Surgery and Related Research, to investigate the awareness and diagnostic accuracy of the LSS diagnostic support tool. We have added the following details about the data collection in our revised manuscript.

“This multicenter cross-sectional study used data from the Lumbar Spinal Stenosis Diagnostic Support Tool (DISTO) project, which was conducted from December 1, 2011 to December 31, 2012. The DISTO project was implemented in 1657 medical institutions under the guidance of the Japanese Society for Spine Surgery and Related Research (JSSR) to verify awareness and the diagnostic accuracy of a lumbar spinal stenosis diagnostic support tool in order to contribute to early detection and treatment of LSS. Recruitment for study participation was announced on the JSSR website, and the study was conducted at facilities that expressed a willingness to participate. An LSS-DST checklist and the NASS clinical description of LSS were distributed to participating medical facilities. The physician-in-charge completed the checklist, in addition to providing usual medical care. Patients who agreed to participate in the study were asked to complete the LSS-SSHQ prior to their consultation. The DISTO project collected and analyzed the checklist and the diagnostic information provided by the physician concerning LSS, peripheral artery disease (PAD), and diabetes mellitus (DM)” (Page 7, Line 93-Page 8, Line 105)

Line 181. Were those not included different from those that were? Is the data available to check this? It is important to know how these individuals differed in clinical characteristics. A selection bias may have been introduced where people with more severe disease were included, therefore inflating the diagnostic accuracy of the tools. For example, it appears that the age, sex, and presence of LSS could be compared between those included and those with missing or inappropriate data (n=7,338).

Response: Thank you for the suggestion. As you point out, if patients with missing values have mild disease, omitting these patients may introduce a bias in the severity of the target population and potentially overestimate the sensitivity of the screening tool. As suggested by the Reviewer, a comparison was made between participants included in the study and those with missing or inadequate data.

The results showed that participants included in the main analysis were older, that there was a higher proportion of men than women, and that the participants had a higher presence of LSS. This may suggest selection bias was introduced in terms of the participants included in the study. However, we included participants with a total score of >7 and those with a score of ≤3, despite missing ankle brachial index (ABI) and other values in sensitivity analysis. The number of participants in our sensitivity analysis was increased to 7,914, but the results were similar to those of the main analysis. Therefore, we consider the results of our study were robust.

We have added selection bias as a limitation of the study, and we have added a table comparing participants who were included and participants who were excluded as supporting information (S6 Table).

“Fifth, there is the possibility of selection bias because many participants were not included in the main analysis due to missing values. Therefore, we compared participants included in the main analysis with those who had earlier been excluded. Participants included in the main analysis were older, there was a higher proportion of men than women, and there was a higher presence of LSS (S6). Therefore, this may suggest that selection bias may be introduced in the participants included in the study. However, we included participants with a total score of >7 and those with a score of ≤3, despite lacking ABI and other values in sensitivity analysis. The number of participants in the sensitivity analysis was increased to 7,914, and the results were similar to those of the main analysis. Therefore, we consider the results of our study were robust.” (Page 28, Line 361 to Page 29, Line 370)

“S6. Comparison of the participants who were included and those who were excluded

Characteristic Participants who were included Participants who were Excluded P-value

n(%) n(%)

Age (years) 0.0008

20-29 166(5.0) 420(5.7)

30-39 310(9.3) 780(10.6)

40-40 308(9.3) 792(10.8)

50-59 411(12.3) 976(13.3)

60-69 892(26.8) 1824(24.9)

70+ 1244(37.3) 2546(34.7)

Sex <0.0001

Male 1755(52.9) 3576(48.8)

Female 1564(47.1) 3756(51.2)

Missing data 12 6

Presence of LSS <0.0001

LSS(-) 1915(57.5) 5223(71.2)

LSS(+) 1416(42.5) 2155(28.8)

Each comparison was subjected to a χ-square test.

LSS, lumbar spinal stenosis

”

Figure 1. It is not clear how participants were excluded from the figure. I suggest that the “Excluded box” contain only those with missing data that were excluded. As written now, the numbers listed under exclusion sum to much greater than 7338. Please report missing data in a separate box. It would also be helpful to report how many patients were screened but declined to participate.

Response: Thank you for your suggestion. As you have suggested, we have divided Figure 1 into two parts and created a new figure (S1) with details of excluded cases because of missing data. Unfortunately, we did not collect data on patients who were screened but refused to participate, therefore we are not able to provide this information.

“Figure 1. Flow chart of participant inclusion

LBP, low back pain”

“S1. Details of excluded cases because of missing data

ABI, Ankle Brachial Index; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire”

Line 183. It would be helpful to have more baseline characteristic information on all participants, since the diagnostic accuracy of a tool is dependent on the study population. Perhaps there was no other data collected, but if so, I would highly recommend greater description of the study cohort.

Response: Thank you for your important remarks. As you correctly indicate, we have not been able to collect any data other than those described in this study.

We have added the following details in our revised manuscript to describe the study cohort:

“This multicenter cross-sectional study used data from the Lumbar Spinal Stenosis Diagnostic Support Tool (DISTO) project, which was conducted from December 1, 2011 to December 31, 2012. The DISTO project was implemented in 1657 medical institutions under the guidance of the Japanese Society for Spine Surgery and Related Research (JSSR) to verify awareness and the diagnostic accuracy of a lumbar spinal stenosis diagnostic support tool in order to contribute to early detection and treatment of LSS. Recruitment for study participation was announced on the JSSR website, and the study was conducted at facilities that expressed a willingness to participate. An LSS-DST checklist and the NASS clinical description of LSS were distributed to participating medical facilities. The physician-in-charge completed the checklist, in addition to providing usual medical care. Patients who agreed to participate in the study were asked to complete the LSS-SSHQ prior to their consultation. The DISTO project collected and analyzed the checklist and the diagnostic information provided by the physician concerning LSS, peripheral artery disease (PAD), and diabetes mellitus (DM)” (Page 7, Line 93-Page 8, Line 105)

Lines 284-286. I am not convinced that real world diagnosis should be considered a strong gold-standard/criterion measure. I do agree with the authors in that specialist diagnosis is the best that is currently available, but I would like to see some discussion around the fact that a clinician-based reference standard test can be problematic. In fact, the authors do later discuss the limitations of having no consensus diagnostic definition for LSS, so I would suggest that the language around “our findings on diagnostic accuracy can be considered valid” be softened and some additional uncertainty be presented. It would also be prudent to comment on the potential variation in clinician diagnosis standards across the large number of participating clinicians. It is probable that different clinicians would arrive at conflicting results on the reference standard for LSS, at least in a proportion of patients.

Response: Thank you very much for your insightful comments. As you pointed out, the reference standard for this study is diagnosis by clinicians, which on itself is problematic. In addition to the fact that there is no international consensus on the definition of LSS, this is a relatively large-scale study, and there is a possibility that diagnoses may vary among different diagnosing clinicians. In light of the points you raised, we have changed the wording as follows.

“Second, the use of diagnosis by orthopedic surgeons as a reference standard reflects the best current diagnostic practice. Although the large number of facilities participating in the study may make standardization of clinicians' diagnostic procedures challenging, we consider that we were able to estimate the diagnostic accuracies of the screening tools at current standards of medical care.” (Page 27, Lines 336-340)

Line 299-301. It is a bold claim that the care setting is unlikely to alter the comparison of the three index tests. I do, however, respect the right of the authors to make this claim, but suggest that some reasoning be provided. Why would spectrum bias not be a concern here?

Response: Thank you for your very important point. We strongly agree that sensitivity values can vary depending on the research settings, but we considered that the difference in sensitivity when two screenings are applied to the same population would not be compromised. However, as you have pointed out, there is a possibility that spectrum bias may occur depending on the care setting, which may affect the comparison of diagnostic accuracy. Therefore, we have amended the sentences as follows:

“Further studies are warranted to confirm whether differences in diagnostic accuracy between the two diagnostic support tools and the NASS clinical descriptions estimated in this study apply to primary care settings.” (Page 28, Lines 353-355)

Reviewer #2

Reviewer #2: This study was conducted to compare the sensitivity and specificity of lumbar spinal stenosis diagnosis utilizing the Lumbar Spinal Stenosis-Diagnosis Support Tool (LSS-DST), the Lumbar Spinal Stenosis - Self-administered, Self-reported History Questionnaire (LSS-SSHQ) and the clinical description of LSS from the NASS diagnostic guidelines in a secondary care hospital setting. The authors concluded that “the LSS-DST and LSS-SSHQ had significantly higher sensitivity for diagnosing LSS in patients with LBP than the clinical description of LSS from the NASS diagnostic guidelines.” This reviewer has some criticisms, which should be addressed by the authors.

These tools have been used to “support” the diagnosis of LSS. This reviewer thinks that the most important thing in the management patients with LSS does prevent mis diagnosis when physicians, who are not spine specialists, use these tools in clinical setting. Considering the clinical relevancy, this reviewer thinks that it is important to precise if the score was not involved in the setting, the diagnosis of patients, who examined by the tools, must be “not” LSS. Therefore, the negative predictive value of these tools is more important than the sensitivity and specificity. Negative predictive values should be shown and compared among three tools.

Response: Thank you for your thought-provoking points. As you pointed out, the negative predictive value is also an important indicator for physicians who are not familiar with LSS examinations. The negative predictive value in this study can be calculated as follows:

NASS clinical description, 0.77 (95% CI 0.75 – 0.79); LSS-DST, 0.92 (95% CI 0.91 – 0.94); and LSS-SSHQ, 0.83 (95% CI 0.81 – 0.85).

The results also indicated that the LSS-DST and LSS-SHQ may be useful in ruling out the possibility of LSS compared with the clinical description of NASS. Therefore, we first listed the abbreviation for negative predictive value (NPV) on Page 12, Lines 163-167.

“We adopted a new cut-off point for the LSS-SSHQ (LSS-SSHQ version 1.1; a total score of 3 on Q1–Q4 or a score of ≥1 on Q1–Q4 and ≥2 on Q5–Q10 indicated positivity), as this cut-off point had higher sensitivity and negative predictive value (NPV) than the original value used in primary care settings [10].”

Next, in the “MATERIALS and METHODS” 2.5 Statistical analyses” sub-section (Page 15, Lines 200-202), the following sentence was added:

“Furthermore, the NPVs of the three tools were also calculated, as it is important to determine the number of false positives obtained by physicians who were unskilled in examining LSS when using these tools clinically.”

In addition, the following information was added to the "RESULTS" section. (Page 21, Lines 258-264)

“The NPVs were 0.77 (95% CI 0.75–0.79) for the NASS clinical description, 0.92 (95% CI 0.91–0.94) for the LSS-DST, and 0.83 (95% CI 0.81–0.85) for the LSS-SSHQ (Table 5).”

Next, we added:

Table 5. NPVs for the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ

Index test NPVs

Point estimate 95% CI

1) NASS clinical description of LSS 0.77 0.75–0.79

2) LSS-DST 0.92 0.91–0.94

3) LSS-SSHQ 0.83 0.81–0.85

Since no universal analysis method has been established for NPV, no statistical tests for comparison have been performed.

Table 5 was changed to Table 6 (Page 22, Lines 270-274)

“Table 6. DORs of the NASS clinical description of LSS, the LSS-DST, and the LSS-SSHQ

Index test DOR

Point estimate 95% CI

1. NASS clinical description of LSS 15.1 12.6–18.1

2. LSS-DST 33.3 26.9–41.1

3. LSS-SSHQ 7.0 5.9–8.3

Finally, in the "DISCUSSION" section, we have added the following description on Page 25, Lines 307-309:

“The high NPVs of the diagnostic support tools for LSS are important because of the clinical significance in terms of effectively limiting false-negative results, even for clinicians unfamiliar with diagnosing and treating patients with LSS.”

Utilizing the typical clinical description of LSS according to the North American Spine Society (NASS), this reviewer does think that the description in point 1 is important items to diagnose the LSS. If this description was excluded, the NASS description to diagnose the LSS should not be used in the present study.

Response: As you have pointed out, the first of the NASS clinical descriptions, namely, degenerative LSS, describes a condition in which there is diminished space available for the neural and vascular elements in the lumbar spine secondary to degenerative changes in the spinal canal, is important for determining the morphological features of LSS. However, please note that the two diagnostic support tools in this study were designed to support the diagnosis of LSS based solely on clinical symptoms and physical findings, and do not include information obtained from radiological imaging studies. Therefore, like those tools, we treated the NASS as an index test and excluded the description on morphological features. Of course, when using the three descriptions of the NASS, it is clinically sensible to use it as a reference standard (i.e., the gold standard), not as an index test.

Reviewer #3:

This paper is well written. This study is a large-scale cross-sectional study and is noteworthy.

We agree that the two tools used in this study, LSS-DST and LSS-SSHQ can be used to improve the diagnostic accuracy of LSS.

Response: Thank you for your understanding of our research on the usefulness of LSS-DST and LSS-SSHQ.

Reviewer #4

Reviewer #4: Q1. Why do authors hypothesize that LSS-DST and LSS-SSHQ would be more sensitive and more useful for screening than the clinical description of LSS from the NASS diagnostic guidelines? (Line 85-86)

Response: Thank you for pointing out that the LSS-DST and SSHQ are diagnostic tools that have undergone an appropriate development process and derivation. The NASS guidelines state that "the above definition of lumbar stenosis was developed by consensus after a global review of the literature and definitive texts," but these guidelines do not explicitly state the criteria by which the definition was ultimately determined. Therefore, since there is no guarantee that the definition will be defined to maximize sensitivity, we assumed that well-derived screening tools would be superior in terms of sensitivity.

Q2. Did the author use the list or questionnaire to confirm the clinical description of LSS from the NASS diagnostic guidelines? Or did the authors confirm the clinical description of LSS from the NASS diagnostic guidelines from the items of LSS-DST and LSS-SSHQ? Because at least from the previous reports from your group, the reviewer could not find the description regarding “the clinical description of LSS from the NASS diagnostic guidelines”.

Response: Thank you for your question. The questionnaire from the NASS diagnostic guidelines was used to confirm the clinical description of LSS.

Comment:

1. Authors emphasized sensitivity or screening for LSS. But authors also discuss the specificity, the reviewer thinks.

Response: Thank you for your suggestion. The lower sensitivity of the LSS-DST and LSS-SSHQ compared with the NASS clinical descriptions may be due to the inclusion of non-specific questions that apply to conditions other than LSS. For example, the LSS-DST adds two points just for being aged ≥71 years. Abnormal Achilles tendon reflexes are also often present in patients without LSS, especially in older adult patients and those with diabetes mellitus. With the LSS-SSHQ, numbness in the lower extremities is often present not only in patients with LSS, but also in patients with PAD and diabetes mellitus, as well as lumbar disc herniation. Urination during walking is often present in patients with an overactive bladder. Please note that our diagnostic support tools were developed for primary care physicians to screen for LSS and these tools value sensitivity over specificity to reduce the need for confirmatory testing by a specialist with detailed examinations and additional imaging studies.

2. I clinical setting, how do primary care physician use LSS-DST, LSS-SSHQ and the clinical description of LSS from the NASS diagnostic guidelines. Authors should describe it.

Response: Thank you for your suggestion. We have created the following description of how to use LSS-DST and LSS-SSHQ and have added it to the MATERIALS and METHODS section in sub-sections 2.3.1 and 2.3.2, respectively.

“2.3 Index tests

2.3.1 The LSS-DST

The LSS-DST is a brief clinical diagnostic tool that helps physicians precisely diagnose patients with LSS (Table 1) [3]. It consists of 10 items that are grouped into three main categories, namely, medical history, symptoms, and physical examination. The LSS-DST can be scored by primary care physicians within their usual resources without the need for special equipment or imaging studies.” (Page 10, Lines 128-131)

“2.3.2 The LSS-SSHQ

The LSS-SSHQ was developed to evaluate the diagnostic value of the medical history of patients with LSS (Table 2) [4]. This self-completed questionnaire comprises 10 items concerning subjective symptoms only. The LSS-SSHQ can be distributed to patients by primary care physicians unfamiliar with neurological physical examination. Scoring can be completed by the patients or their primary care physicians.” (Page 12, Lines 160-163)

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

19 Apr 2022

Diagnostic accuracy of the Lumbar Spinal Stenosis-Diagnosis Support Tool and the Lumbar Spinal Stenosis-Self-administered, Self-reported History Questionnaire

PONE-D-21-38525R1

Dear Dr. Kurita,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Michael C Burger, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

26 Apr 2022

PONE-D-21-38525R1

Diagnostic accuracy of the Lumbar Spinal Stenosis-Diagnosis Support Tool and the Lumbar Spinal Stenosis-Self-administered, Self-reported History Questionnaire

Dear Dr. Kurita:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Michael C Burger

Academic Editor

PLOS ONE

Table 1

A clinical DST for identifying patients with LSS (LSS-DST).

Items	Score
History
Age (years)
60–70	+1
>70	+2
Absence of diabetes mellitus	+1
Symptom
Intermittent claudication, positive	+3
Exacerbation of symptoms when standing up	+2
Symptom improvement when bending forward	+3
Physical examination
Symptoms induced by having patients bend forward	-1
Symptoms induced by having patients bend backwards	+1
Ankle brachial index ≥0.9	+3
Abnormal Achilles tendon reflex	+1
SLR test result, positive	-2

If the total score is >7, there is a high possibility of LSS.

DST, diagnosis support tool; LSS, lumbar spinal stenosis; SLR, straight leg raising.

Table 2

A self-administered, self-reported history questionnaire for identifying patients with LSS (LSS-SSHQ).

Items
Q1: Numbness and/or pain in the thighs down to the calves and shins
Q2: Numbness and/or pain increase in intensity after walking for a while, but relieved through taking a rest
Q3: Standing for a while brings on numbness and/or pain in the thighs down to the calves and shins
Q4: Numbness and/or pain are reduced by bending forward
Q5: Numbness is present in both legs
Q6: Numbness is present in the soles of both feet
Q7: Numbness arises around the buttocks
Q8: Numbness is present, but pain is absent
Q9: A burning sensation arises around the buttocks
Q10: Walking nearly causes urination

A total score of 3 on Q1–Q4 or a score ≥1 on Q1–Q4 and a score of ≥2 on Q5–Q10 indicated the presence of LSS.

LSS, lumbar spinal stenosis; SSHQ, self-administered, self-reported history questionnaire.

Table 3

Patient characteristics.

Characteristic	n (%)	Missing data
Age (years)		-
20–29	166 (5.0)
30–39	310 (9.3)
40–49	308 (9.3)
50–59	411 (12.3)
60–69	892 (26.8)
70+	1244 (37.3)
Sex		12
Male	1755 (52.9)
Female	1564 (47.1)
Presence of LSS		-
(-)	1915 (57.5)
(+)	1416 (42.5)
Presence of DM		-
(-)	3012 (90.4)
(+)	319 (9.6)
Presence of PAD		-
(-)	2256 (67.7)
(+)	1075 (32.3)

DM, diabetes mellitus; LSS, lumbar spinal stenosis; PAD, peripheral arterial disease; (-), not present; (+), present.

Table 4

Sensitivity and specificity of the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ.

Index test	Sensitivity		P-value for heterogeneity	Specificity		P-value for heterogeneity
	Point estimate	95% CI		Point estimate	95% CI
1) NASS clinical description of LSS	63.9%	61.4%–66.4%		89.5%	88.1%–90.9%
2) LSS-DST	91.3%	89.9%–92.8%	2) vs. 1) <0.0001	76.0%	74.1%–77.9%	2) vs. 1) <0.0001
3) LSS-SSHQ	83.8%	81.8%–85.7%	3) vs. 1) <0.0001	57.6%	55.3%–59.8%	3) vs. 1) <0.0001

CI, confidence interval; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire.

Table 5

NPVs for the NASS clinical description of LSS, LSS-DST, and LSS-SSHQ.

Index test	NPVs
	Point estimate	95% CI
1) NASS clinical description of LSS	0.77	0.75–0.79
2) LSS-DST	0.92	0.91–0.94
3) LSS-SSHQ	0.83	0.81–0.85

CI, confidence interval; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; NPVs, negative predictive values; SSHQ, self-administered, self-reported history questionnaire.

Table 6

DORs of the NASS clinical description of LSS, the LSS-DST, and the LSS-SSHQ.

Index test	DOR
	Point estimate	95% CI
1. NASS clinical description of LSS	15.1	12.6–18.1
2. LSS-DST	33.3	26.9–41.1
3. LSS-SSHQ	7.0	5.9–8.3

CI, confidence interval; DORs, diagnostic odds ratios; DST, diagnosis support tool; LSS, lumbar spinal stenosis; NASS, North American Spine Society; SSHQ, self-administered, self-reported history questionnaire.