Diagnostic utility of the amyotrophic lateral sclerosis Functional Rating Scale-Revised to detect pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis.

OBJECTIVE: The ALS Functional Rating Scale-Revised (ALSFRS-R) is the most commonly utilized instrument to index bulbar function in both clinical and research settings. We therefore aimed to evaluate the diagnostic utility of the ALSFRS-R bulbar subscale and swallowing item to detect radiographically confirmed impairments in swallowing safety (penetration or aspiration) and global pharyngeal swallowing function in individuals with ALS.
METHODS: Two-hundred and one individuals with ALS completed the ALSFRS-R and the gold standard videofluoroscopic swallowing exam (VFSE). Validated outcomes including the Penetration-Aspiration Scale (PAS) and Dynamic Imaging Grade of Swallowing Toxicity (DIGEST) were assessed in duplicate by independent and blinded raters. Receiver operator characteristic curve analyses were performed to assess accuracy of the ALSFRS-R bulbar subscale and swallowing item to detect radiographically confirmed unsafe swallowing (PAS > 3) and global pharyngeal dysphagia (DIGEST >1).
RESULTS: Although below acceptable screening tool criterion, a score of ≤ 3 on the ALSFRS-R swallowing item optimized classification accuracy to detect global pharyngeal dysphagia (sensitivity: 68%, specificity: 64%, AUC: 0.68) and penetration/aspiration (sensitivity: 79%, specificity: 60%, AUC: 0.72). Depending on score selection, sensitivity and specificity of the ALSFRS-R bulbar subscale ranged between 34-94%. A score of < 9 optimized classification accuracy to detect global pharyngeal dysphagia (sensitivity: 68%, specificity: 68%, AUC: 0.76) and unsafe swallowing (sensitivity:78%, specificity:62%, AUC: 0.73).
CONCLUSIONS: The ALSFRS-R bulbar subscale or swallowing item did not demonstrate adequate diagnostic accuracy to detect radiographically confirmed swallowing impairment. These results suggest the need for alternate screens for dysphagia in ALS.

Introduction

Amyotrophic lateral sclerosis (ALS) is a progressive and fatal neurodegenerative disease affecting both upper and lower motor neurons within the cortex, brainstem and spinal cord [1]. Dysphagia, or swallowing impairment, occurs in a reported 85% of patients with ALS at some point during the disease process and is associated with malnutrition, weight loss, reduced quality of life, aspiration pneumonia and death [2-6]. Early detection and consistent monitoring of dysphagia provides the opportunity to mitigate associated risks and improve survival with timely interventions [7].

A universally accepted and validated clinical test battery to accurately assess and monitor bulbar disease progression is currently lacking [8]. A 2017 survey of Northeast ALS (NEALS) centers in the United States revealed highly variable practice patterns for the evaluation of bulbar function in patients with ALS [9]. Both clinical and instrumental swallow evaluations were found to be underutilized in multidisciplinary ALS clinics with less than 60% of respondents utilizing clinical swallow assessments and only 27% referring for the gold standard videofluoroscopic swallowing evaluation. Importantly, this survey revealed that the only clinical test routinely performed to evaluate bulbar function (>90% of sites) was the revised ALS Functional Rating Scale (ALSFRS-R).

The ALSFRS-R is a 12-item questionnaire with each question rated on a 5-point ordinal scale used to monitor progression of disability in patients with ALS. The scale currently represents the most widely used ALS outcome measure in Phase II and III clinical trials and longitudinal studies [10]. More recently, the psychometric properties of the ALSFRS-R have been evaluated with evidence suggesting multidimensionality, and the utilization of individual subscale scores rather than a total score has been recommended [11-13]. These bulbar, motor and respiratory subscores are intended to provide more precise prognostic information, as the individual’s domain scores have been demonstrated to be more clinically robust when reported as subscores rather than a combined score [14]. Specifically, individuals with bulbar-onset disease demonstrated slower rate of decline on the motor subscore and hastened decline on the bulbar subscore compared to those with spinal onset disease [14]. One study of 18 individuals with motor neuron disease (MND) investigated the relationship between the ALSFRS-R bulbar subscore and radiographically confirmed airway invasion; however, individuals who aspirated during swallowing were not included in the study cohort, limiting the generalization of the results [15]. The discriminant ability of the bulbar subscore and swallowing item score to detect radiographically confirmed pharyngeal dysphagia in ALS has not yet been determined. We therefore sought to evaluate the discriminant ability of the ALSFRS-R bulbar subscale and swallowing item scores to classify early radiographically confirmed pharyngeal dysphagia in patients with ALS. Given the that the scale is a five-point ordinal scale that lacks linearity, we hypothesized that the ALSFRS-R would not demonstrate adequate sensitivity to detect mild changes in pharyngeal swallowing function in individuals with ALS.

Materials and methods

Participants

All eligible ALS patients who attended the University ALS clinic were informed of the study and invited to participate, representing a convenience sample. Two-hundred and one individuals were enrolled in this study. Inclusion criteria were: 1) confirmed diagnosis of ALS (Revised El Escorial criteria) by a neuromuscular neurology specialist; 2) not pregnant, 3) no allergies to barium, and 4) still consuming some form of foods and liquids by mouth. This study was approved by the University of Florida Institutional Review Board and conducted in accordance with the Declaration of Helsinki. All participants provided informed written consent. Participants attended a single testing session which included completion of the ALSFRS-R (index test) and a standardized videofluoroscopic swallowing examination (VFSE, gold standard reference test).

Index test

The ALSFRS-R is a 12-item questionnaire validated to monitor functional disease progression across four subscales of activities of daily living that include bulbar, fine motor, gross motor and respiratory domains [10]. Three items assessing speech, salivation, and swallowing comprise the bulbar subscale with each item scored on a five-point ordinal scale (0–4) for a total of 12 points, with higher scores indicating better self-reported function (0 = total loss of function, 12 = normal functioning). A single question on swallowing is scored as follows 4—Normal eating habits, 3—Early eating problems; occasional choking, 2—Dietary consistency changes, 1—Needs supplemental tube feeding, 0—NPO (exclusively parenteral or enteral feeding) [10]. Participants completed the ALSFRS-R in interview fashion with an opportunity for input by their caregivers. All research personnel conducting these interviews completed training in the administration and scoring of the ALSFRS-R.

Reference standard

VFSE was completed by a trained research speech-language pathologist (SLP) with participants comfortably seated in an upright lateral viewing plane using a properly collimated Phillips BV Endura fluoroscopic C-arm unit (GE 9900 OEC Elite Digital Mobile C-Arm system type 718074). Fluoroscopic images and synced audio were digitally recorded at 30 frames per second using a high resolution (1024 x 1024) TIMS DICOM system (Version 3.2, TIMS Medical, TM, Chelmsford, MA) for subsequent analysis. A standardized bolus presentation was administered utilizing a cued instruction to swallow and included: three 5 mL thin liquid barium, one comfortable cup sip of thin liquid barium, three 5 mL thin honey barium, two 5 mL pudding consistency barium, and a ¼ graham cracker square coated with pudding consistency barium (Varibar®, Bracco Diagnostics, Inc., Monroe Township, NJ). If the patient was unable to self-feed due to upper extremity weakness, clinician assistance or alternative methods (i.e., straw) were employed, consistent with the individual’s feeding methods routinely utilized at home. SLPs enforced standardized bailout criteria requiring administration of thicker consistencies following two instances of aspiration and discontinuation if an additional aspiration event occurred during the exam. VFSE recordings were saved to a secure server and blinded for subsequent analysis.

VFSE outcome measures

Each VFSE was rated in duplicate by two trained, blinded and independent raters. Complete agreement (100%) was required for all ratings, with a discrepancy meeting utilized to finalize any inconsistent ratings between raters.

Swallowing safety

The Penetration-Aspiration Scale (PAS) was utilized to evaluate swallowing safety (Fig 1) [16]. This validated eight-point ordinal scale indexes the depth of contrast material entering the airway during swallowing events, the presence of a protective response, and if aspirate material was ejected from the airway [16]. All elicited swallows within a given bolus trial were ascribed a PAS score and the worst PAS score utilized for statistical analysis. Fig 1 denotes the PAS with established categorical levels of airway safety used.

Fig 1

Penetration Aspiration Scale (PAS) scores and corresponding binary classifications of swallowing safety status.

A) Representative videofluoroscopic images depicting safe swallowing with no contrast material entering the airway. B) Laryngeal penetration. C) Tracheal aspiration [16].

Global pharyngeal swallowing

The dynamic imaging grade of swallowing toxicity (DIGEST) is a validated five-point ordinal scale created to assess both efficiency and safety of bolus flow [17] and recently utilized in ALS [18]. The DIGEST (Fig 2) yields a global grade of pharyngeal dysphagia evaluated on bolus transport during the entirety of the videofluoroscopic swallow study to determine clinically relevant categories of overall pharyngeal dysphagia severity levels. DIGEST total scores are a composite of two subscores (scored 0–4) addressing: (i) swallowing efficiency based on degree of bolus clearance, and (ii) airway safety based on severity and frequency of PAS scores. DIGEST scores of zero indicate normal swallowing while total and subscore grades of 4 indicate life-threatening dysphagia.

Fig 2

Dynamic Imaging Grade of Swallowing Toxicity (DIGEST) outcome [17].

Statistical analysis

Descriptives were performed to summarize participant demographics and outcomes of interests. A receiver operator characteristic (ROC) curve analysis was then performed on the index test (ALSFRS-R bulbar subscale and swallowing item) to identify unsafe (PAS > 3) and global dysphagia (DIGEST > 1). Area under the curve (AUC) with bootstrapped 95% confidence intervals, sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) were calculated using JMP Pro Version 14.1.0 (SAS Institute Inc., Cary, NC). Optimal classification cutoffs for index test were determined by values that maximized both sensitivity and specificity.

Results

Participant demographics

Complete ALSFRS-R and VFSE data were missing in four participants resulting in 197 patients in the final analysis. Mean age was 62.9 (SD = 10.3) and average ALS disease duration was 26.6 months from symptom onset (SD = 23.6). Fifty-three percent were male (n = 106) and 58.1% presented with a spinal onset (n = 111). Mean ALSFRS-R score was 35.3 (SD = 7.4). Frequency data for the ALSFRS-R bulbar subscale and swallowing item scores are presented in histogram plots in Fig 3A and 3B respectively. Mean ALSFRS-R bulbar subscale score was 9.1 (SD = 2.4) and mean swallowing item score was 3.05 (SD = 0.79). Radiographically confirmed unsafe swallowing was identified in 38.9% of patients (n = 76, Fig 3C) and prevalence of global pharyngeal dysphagia was 58.9% (n = 116, Fig 3D).

Fig 3

Frequency distribution of outcomes of interest in 197 participants.

A) Amyotrophic Lateral Sclerosis Functional Rating Scale–Revised (ALSFRS-R) Bulbar Subscale scores. B) ALSFRS-R swallow item scores. C) Penetration aspiration scale scores (PAS). D) Dynamic Imaging Grade of Swallowing Toxicity scores (DIGEST).

Discriminant ability of the ALSFRS-R to detect swallowing impairment

Scatterplots depicting relationships between the ALSFRS-R and swallowing outcomes of interest are shown in Fig 4. ROC curve results for the ALSFRS-R bulbar subscale and ALSFRS-R swallowing items to detect radiographically confirmed penetrators/aspirators are presented in Table 1 and Fig 5A and 5B. Classification accuracy for both ALSFRS-R outcomes to detect global pharyngeal dysphagia are presented in Table 2 and Fig 5C and 5D. Optimized classification cutoff of ALSFRS-R swallowing score of ≤ 3 and ALSFRS-R bulbar score of ≤ 9 were found for both outcomes.

Fig 4

Scatterplots for the amyotrophic lateral sclerosis Functional Rating Scale–Revised (ALSFRS-R) and radiographic swallowing outcomes of interest (dynamic imaging grade of swallowing toxicity–DIGEST and penetration-aspiration scale–PAS).

Table 1

Summary of receiver operator characteristic curve results for the ALSFRS-R bulbar subscale and swallowing item to detect unsafe swallowing.

Unsafe Swallowing	Bulbar Subscale:					Swallowing Item:
	≤11	≤10	≤9	≤8	≤7	≤3	≤2	≤1
Sensitivity	92.1%	85.5%	77.6%	59.2%	43.4%	78.9%	26.3%	5.3%
(95% CI)	(86.0, 98.2)	(77.6, 93.4)	(68.3, 87.0)	(48.2, 70.3)	(32.3, 54.6)	(69.8, 88.1)	(16.4, 36.2)	(0.2, 10.3)
Specificity	29.8%	45.5%	62.0%	77.7%	90.1%	60.3%	92.6%	100%
(95% CI)	(21.6, 37.9)	(36.6, 54.3)	(53.3, 70.6)	(70.3, 85.1)	(84.8, 95.4)	(51.6, 69.0)	(87.9, 97.2)	(100, 100)
PPV	45.2%	49.6%	56.2%	62.5%	73.3%	55.6%	69.0%	100%
(95% CI)	(37.3, 53.0)	(41.1, 58.2)	(46.7, 65.7)	(51.3, 73.7)	(60.4, 86.3)	(46.2, 64.9)	(52.1, 85.8)	(100, 100)
NPV	85.7%	83.3%	81.5%	75.2%	71.7%	82.0%	66.7%	62.7%
(95% CI)	(75.1, 96.3)	(74.3, 92.3)	(73.6, 89.5)	(67.6, 82.8)	(64.6, 78.9)	(74.0, 90.0)	(59.5, 73.8)	(55.9, 69.5)
AUC:	0.76					0.72

Unsafe swallowing was defined as Penetration Aspiration Scale (PAS) scores ≥ 3. (CI, confidence interval, PPV, positive predictive value. NPV, negative predictive value, AUC, area under curve.)

Fig 5

Receiver operator characteristic curve results for the amyotrophic lateral sclerosis Functional Rating Scale–Revised (ALSFRS-R).

(A, B) ALSFRS-R bulbar subscale and swallowing item to detect unsafe swallowing. (C, D) Global pharyngeal dysphagia.

Table 2

Summary of receiver operator characteristic results for the ALSFRS-R bulbar subscale and swallowing item to detect global pharyngeal dysphagia.

Global Dysphagia	Bulbar Subscale:					Swallowing Item:
	≤11	≤10	≤9	≤8	≤7	≤3	≤2	≤1
Sensitivity	87.1%	76.7%	68.1%	50.9%	34.5%	68.1%	20.7%	3.4%
(95% CI)	(81.0, 93.2)	(69.0, 84.4)	(59.6, 76.6)	(41.8, 60.0)	(25.8, 43.1)	(59.6, 76.6)	(13.3, 28.1)	(0.1, 6.8)
Specificity	33.3%	48.1%	67.9%	84.0%	93.8%	64.2%	93.8%	100%
(95% CI)	(23.1, 43.6)	(37.3, 59.0)	(57.7, 78.1)	(76.0, 91.9)	(88.6, 99.1)	(53.8, 74.6)	(88.6, 99.1)	(100, 100)
PPV	65.2%	67.9%	75.2%	81.9%	88.9%	73.1%	82.2%	100%
(95% CI)	(57.7, 72.7)	(59.9, 75.9)	(67.0, 83.5)	(73.1, 90.8)	(79.7, 98.1)	(64.8, 81.5)	(69.0, 96.5)	(100, 100)
NPV	64.3%	59.1%	59.8%	54.4%	50.0%	58.4%	45.2%	42.0%
(95% CI)	(49.8, 78.8)	(47.2, 71.0)	(49.8, 69.8)	(45.7, 63.1)	(42.1, 57.9)	(48.2, 68.7)	(37.7, 52.8)	(35.0, 48.9)
AUC:	0.73					0.68

(CI, confidence interval, PPV, positive predictive value. NPV, negative predictive value, AUC, area under curve.)

Discussion

To our knowledge, this represents the first investigation to compare ALSFRS-R bulbar outcomes to those of the gold standard reference test for swallowing (VFSE). The ALSFRS-R bulbar subscale and swallowing item demonstrated poor to fair diagnostic accuracy to detect radiographically confirmed pharyngeal swallowing impairment in the 197 ALS patients examined (AUC: 0.68–0.76). No cut score emerged for ALSFRS-R outcome with an acceptable level of classification accuracy to distinguish normal versus disordered swallowing. Thus, the ALSFRS-R did not demonstrate adequate clinical utility as a screening tool to detect early pharyngeal dysphagia and demonstrated insufficient sensitivity as a marker of change in pharyngeal swallowing function for research clinical trials. These findings highlight the need for the development of sensitive tools to adequately screen relative risk of swallowing impairment for use in multidisciplinary ALS clinics and research settings alike.

Bulbar subscale

Classification accuracy of the ALSFRS-R bulbar subscale to detect global pharyngeal dysphagia was considered poor to fair when comparing our results to accepted screening tool criterion levels [19]. No clear score or threshold emerged that yielded an acceptable balance between sensitivity and specificity when examining ROC outcomes across bulbar subscale scores. An effective screening tool should accurately and quickly identify at risk individuals to triage for further comprehensive evaluation and ideally minimize false negatives (i.e., missing individuals with impairment) while at the same time avoiding over identification of individuals without the disorder being screened (i.e. false positives). While generally specificity is sacrificed at the cost of increased sensitivity; a screening tool with high sensitivity but very low specificity will create undue strain on health care workers, lead to overutilization of resources and unnecessary testing, and increase patient and caregiver burden. To this end, an ALSFRS-R bulbar subscale score of ≤11 correctly identified 87% of ALS patients with global pharyngeal dysphagia; however, misclassified two-thirds of patients as dysphagic who demonstrated normal swallowing on VFSE. Use of a lower cut-point of ≤10 decreased sensitivity to an unacceptable level without significant improvements in specificity, PPV or NPV. This cut point would miss one-quarter of patients with global dysphagia (i.e. false negatives) and would over-refer 52% of patients without dysphagia for additional testing. Finally, a bulbar subscale score of < 9 derived the most balanced degree of sensitivity and specificity of 68%, however would misclassify one-third of individuals with global pharyngeal dysphagia (false negatives).

Similarly, no cut score emerged for the ALSFRS-R bulbar subscale to detect penetration or aspiration. Although sensitivity of the bulbar subscale to detect unsafe swallowing was good-excellent for higher scores of 10 and 12 (> 86%) they were associated with low specificity (30% and 45%), high false positives, and ow PPVs. These findings are in agreement with observations the bulbar subscale is not sensitive to detect early speech impairment in ALS patients when compared to objective physiologic speech metrics [20].

Swallowing item

The ALSFRS-R swallowing item demonstrated poor overall screening accuracy to classify both global swallowing and safety status. Unlike the bulbar subscale, however, a clear score emerged to optimize obtained sensitivity and specificity. A swallowing item score of ≤ 3 accurately classified only 68% of individuals with confirmed global dysphagia missing approximately one-third of impaired individuals and representing a PPV that is not acceptable. Further, this score misclassified 36% of patients with confirmed normal swallowing as being dysphagic. Clearly this ‘optimal’ score is not acceptable for distinguishing global swallowing status in ALS.

Examination of the swallowing item’s classification accuracy to differentiate safe vs. unsafe ALS swallowers was noted to be higher, with a cut score of ≤3 yielding a sensitivity of 79% and a specificity of 60%. Although improved, diagnostic utility at this optimal score threshold remained suboptimal for a useful screening tool given that it would miss one in every five penetrator/aspirators and would over-refer 40% of patients without impairment for further evaluation. An important consideration when interpreting these data is the fact that individuals with ALS may not be fully aware of subtle dietary adaptations or modifications they may implement to compensate for a progressive decline in function [21-23]. This is highly relevant given that the ALSFRS-R is a patient report outcome that asks patients to select the descriptor for a function being queried.

Given that the ALSFRS-R is commonly used in research as a baseline stratification tool or outcome to measure change in function over time; researchers are advised to consider these findings for future clinical trials. Further, clinical adoption of these scores as a dysphagia screen could create unnecessary burden for patients and their caregivers and facilitate inappropriately timed referrals for instrumental swallowing evaluations.

Although no published screening tool exists for dysphagia in ALS, two reports have examined the clinical utility of another patient-reported outcome measure (PROM) and of voluntary cough testing to detect aspiration. The Eating Assessment Tool (EAT-10) is a validated 10-item swallowing specific PROM that is available in 13 languages [24]. A cut score of >8 on the EAT-10 demonstrated a sensitivity, specificity and likelihood ratio of 86%, 72% and 3.1, respectively to detect radiographically confirmed aspiration [25]. However, the discriminant ability of the EAT-10 to detect global pharyngeal dysphagia in ALS, has not yet been established. In addition to PROMs, voluntary cough function is noted to significantly differ in individuals with ALS compared to healthy age and gender matched controls, contributing to the impaired ability to effectively expel tracheal aspirate and manage secretions in this population [26]. Given that peak expiratory flow is noted to be reduced by 50% in ALS patients with unsafe swallowing [27], voluntary cough peak expiratory flow (commonly known as peak cough flow testing) has been suggested as a screen to index one’s physiologic airway defense capacity [28, 29]. Future work is needed to identify additional sensitive clinical markers in order to develop and validate a pragmatic and accurate dysphagia screening tool for use in ALS clinics [8, 9, 29].

While this work represents the first attempt to examine the discriminant ability and clinical utility of the ALSFRS-R for detecting radiographically confirmed dysphagia, limitations need to be acknowledged. First, following typical analytic methods used in dysphagia research [30, 31], the worst PAS score was utilized to determine swallowing safety status, which may have skewed outcomes towards impairment [26]. Given that we were interested in catching early impairment however, we feel that any potential bias was warranted. Further, the global pharyngeal dysphagia metric (DIGEST scale) incorporates both the frequency and severity of penetration and aspiration and therefore mitigated potential bias for this specific outcome [17]. Second, the global dysphagia outcome only examines pharyngeal phase swallowing impairments. Therefore, our exam was specific to pharyngeal phase deficits. It is possible that a patient may have rated the ALSFRS-R swallowing item to reflect or communicate perceived impairment in the oral phase that were not detected in this study with use of the DIGEST or PAS scales. Third, given practical and ethical considerations and constraints, our sample represented individuals with mild-moderate ALS severity and bulbar dysfunction with only one patient 100% dependent on non-oral nutrition. Therefore, this cohort may not represent the complete spectrum of ALS swallowing severities. Fourth, other important non-physiologic aspects related to dysphagia such as mealtime enjoyment, mealtime duration, caregiver burden and fatigue were not indexed in this study. Finally, although these data represent the largest VFSE dataset presented to date, further work in additional patients is warranted to validate these findings.

Conclusion

Early detection of dysphagia is paramount to guide timely clinical management decisions to mitigate or delay development of known sequalae. Given the widespread use of the ALSFRS-R to index bulbar and pharyngeal swallowing function, we aimed to determine the discriminant ability of the bulbar subscale and swallowing item to detect radiographically confirmed impairments in swallowing safety and global pharyngeal swallowing function using the gold standard VFSE. Overall accuracy of the ALSFRS-R was poor to diagnostic accuracy for swallowing safety and global pharyngeal swallow function did not meet acceptable standards across any score criteria. We therefore do not recommend use of the ALSFRS-R in isolation to screen for pharyngeal swallowing function and encourage the development of a disease specific screening tool that can accurately triage high-risk individuals for instrumental swallowing evaluation.

(XLSX)

Click here for additional data file.

16 Jun 2020

PONE-D-20-14426

Diagnostic utility of the Amyotrophic Lateral Sclerosis Functional Rating Scale -Revised to detect early pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis.

PLOS ONE

Dear Dr. Plowman,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 31 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Michelle Ciucci, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Additional Editor Comments (if provided):

Apologies that this took so long. It is difficult to find qualified reviewers for this work and a lot of people are declining to review at this time. We appreciate your patience!

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a well-written manuscript that evaluated the discriminant ability for the ALSFRS-R and the question regarding swallowing ability to predict abnormal penetration-aspiration scale or DIGEST scores. The authors report poor to fair classification ability for both the ALSFRS-R and swallowing question. Comments are below.

1. Recommend changing the title to remove “early”, as recruitment did not target patients in the early stage of ALS

Abstract

2. Considering videofluoroscopy to be a gold-standard is questionable, especially given that FEES may be more sensitive to small amounts of airway invasion (e.g., https://pubmed.ncbi.nlm.nih.gov/17906496)

3. Optimized classification accuracies are mentioned in the abstract but not in the methods or results section

Introduction

4. Introduction is thorough yet concise, but is lacking a hypothesis

Methods

5. What were the exclusion criteria?

6. Please clarify if participants were given a cue to swallow during the videofluoroscopy

7. Figure 1 is a nice depiction of the PAS, but it should be amended to include the same information for the DIGEST

8. Lines 161-163: were the videofluoroscopies rated in duplicate for DIGEST only or for PAS as well? As this statement is in the DIGEST sub-section, it is not clear

9. It is mentioned in the limitations that the worst PAS scores were used, but not in the methods

Results

10. The results would be improved with scatterplots of the ALSFRS-R and swallowing question vs. the PAS and DIGEST. This will help the reader identify the relationship/lack thereof between the index test and reference standard

11. It would be beneficial for the authors to explore the false negative and false positive cases. If a patient reported swallowing difficulty on the ALSFRS-R but did not have airway invasion or residue, what was causing the difficulty? There may be merit in using the ALSFRS-R in addition to PAS/DIGEST, but it is not clear what else it may capture This could be explored further as well in the Discussion

Discussion

12. This is briefly covered, but recommend exploring what might be captured by the ALSFRS-R but missed by the PAS/DIGEST

13. Also, recommend changing use of “swallowing function” to describe PAS/DIGEST, as this is only a small part of oro-pharyngeal-esophageal swallowing function

Reviewer #2: Thank you for the opportunity to review this paper looking at the diagnostic utility of the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised. Dysphagia is an important issue for people with ALS and this paper's focus on the bulbar subscale and swallow index is valid. The sample size is good and validated outcomes measures are used. The methods and data analysis are clinically relevant and accurate. Conclusions are clinically important to disseminate. This is a well written paper and the findings are important.

Introduction - well written, appropriate reference to literature, well argued rationale for study.

Methods - how were participants enrolled? Consecutively through a clinic or by invitation? Please clarify.

Were there other exclusion criteria? Those who were solely enteral tube fed? Those unable to swallow saliva???

Has the DIGEST been used in ALS previously? Please provide this information for the reader.

Results - well presented and relevant- nice use of charts and tables

Discussion - well written and relevant. Good discussion of alternative indicators of dysphagia risk available. It would be good to discuss whether DIGEST and aspiration provide ALL the relevant dysphagia burden indicators for those with ALS. Do DIGEST and aspiration represent other aspects of burden of importance? Such as carer burden, mealtime enjoyable / mealtime impact etc...

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Corinne A. Jones

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

23 Jun 2020

Dear Dr. Ciucci and Reviewers,

We sincerely thank you for the expert review of our manuscript and the constructive feedback. Please find below a response to each of the reviewers’ comments or questions.

Reviewer 1:

1. We have removed the word “early” from the title as suggested.

2. Abstract: Use of the word “gold standard examination” when referring to the videofluoroscopic swallowing exam is an established reference that is supported by many in our field [1]. Additionally, other researchers have utilized the VFSS as the ‘gold standard reference tool’ in other clinical validation studies [2-5]. Our laboratory utilizes both instrumental swallowing exam (ISE) techniques and this comment is interesting given we just received comments from another submission using FEES requesting that we state it is a limitation, as the VFSS is the gold standard and FEES is not. We see the utility in use of either ISE and acknowledge that different settings may have different needs. We prefer not to get into a debate regarding which ISE is better but believe the use of this term is acceptable and, in this paper, needed to validate its use as the reference standard. We respectfully therefore stand by our use of this term.

3. We have added a statement regarding optimization of classification accuracy in both the methods and results section as advised.

4. Introduction: We have added a hypothesis as suggested following the aims.

5. Methods: The exclusion criteria were provided in the original manuscript on page 5 in the Methods section under “Participants” we have expanded this in the revised document to read “Inclusion and exclusion criteria were: 1) confirmed diagnosis of ALS (Revised El Escorial criteria) by a neuromuscular neurology specialist; 2) not pregnant, 3) no allergies to barium, and 4) still consuming some form of foods and liquids by mouth.”

6. Methods: In response to your request to clarify if participants were given a cue to swallow during the videofluoroscopy, yes they were. Please see a reference to this in the original manuscript under “Reference Standard” – “A standardized bolus presentation was administered utilizing a cued instruction to swallow and included”

7. We have added an additional Figure to illustrate the DIGEST scale as suggested (Please see Figure 2).

8. Methods: Were the videofluoroscopies rated in duplicate for DIGEST only or for PAS as well? As this statement is in the DIGEST sub-section, it is not clear. Yes, both the PAS and DIGEST were rated in duplicate. We apologize that this was not clear and see how this may have been the case given that this description of duplicate ratings was in the DIGEST section. In the revised manuscript, we have moved this description of duplicate ratings to appear immediately after the subheading “Outcome measures” and prior to both outcomes.

9. Methods: Thank you for pointing out the omission for using worst PAS score in the methods. We have included this in the revised manuscript under ‘VFSE outcome measures’ and subsection of ‘Swallowing Safety’ of methods.

10. Results: We have added scatterplots as suggested.

11. Results: While we agree that it would be interesting to look at what might be was causing false negative or positives (i.e., your question - If a patient reported swallowing difficulty on the ALSFRS-R but did not have airway invasion or residue, what was causing the difficulty?”), patients did not elaborate on responses but rather completed the PRO in the validated fashion by merely circling a response (0 – 4) on the scale. The study design and the information obtained using the ALSFFRS-R – prohibits our ability to answer this question and this was outside the scope of this study.

12. Discussion: This is briefly covered, but recommend exploring what might be captured by the ALSFRS-R but missed by the PAS/DIGEST.

We have elaborated on this point in the revised manuscript to include a statement regarding the possibility that a patient rated the ALSFRS-R swallowing item to reflect or communicate perceived impairment solely in the oral phase that were not detected in this study with the DIGEST scale.

13. Discussion: Also, recommend changing use of “swallowing function” to describe PAS/DIGEST, as this is only a small part of oro-pharyngeal-esophageal swallowing function.

This point is well taken. We have re-worded text in the discussion that read as “swallowing function” to pharyngeal swallowing (or impairment etc) given that the safety and residue metrics using the DIGEST is a focused pharyngeal outcome that do not index oral or esophageal function.

We thank Reviewer One for their time and expertise in helping make this manuscript improved for publication.

Reviewer #2:

1. Methods - how were participants enrolled? Please clarify.

All eligible participants attending the ALS University multidisciplinary clinical were informed of this study and extended an invitation to participate. Those who agreed were enrolled. This therefore represents a convenience sample. We have provided more detailed information regarding this in the revised manuscript (please see Methods – Participants subsection).

2. Were there other exclusion criteria? Those who were solely enteral tube fed? Those unable to swallow saliva??? Inclusion and exclusion criteria were: 1) confirmed diagnosis of ALS (Revised El Escorial criteria) by a neuromuscular neurology specialist; 2) not pregnant, 3) no allergies to barium, and 4) still consuming some form of foods and liquids by mouth. This is in the revised manuscript under the Methods – Participant section.

3. Has the DIGEST been used in ALS previously? Please provide this information. Yes, it has been used previously in ALS, we have added these references in the methods section (please see Methods, Global Pharyngeal Swallowing subsection).

4. It would be good to discuss whether DIGEST and aspiration provide ALL the relevant dysphagia burden indicators for those with ALS. Do DIGEST and aspiration represent other aspects of burden of importance? Such as carer burden, mealtime impact etc. This is an excellent point. We have included in the revised manuscript, Limitations section the following new points to speak to this comment:

a. “Second, the global dysphagia outcome only examines pharyngeal phase swallowing impairments. Therefore, our exam was specific to pharyngeal phase deficits. It is possible that a patient may have rated the ALSFRS-R swallowing item to reflect or communicate perceived impairment in the oral phase that were not detected in this study with use of the DIGEST or PAS scales.”

b. Fourth, other important non-physiologic aspects related to dysphagia such as mealtime enjoyment, mealtime duration, caregiver burden and fatigue were not indexed in this study.

References:

1. Wirth R, Dziewas R, Beck AM, et al. Oropharyngeal dysphagia in older persons - from pathophysiology to adequate intervention: a review and summary of an international expert meeting. Clin Interv Aging. 2016;11:189-208. Published 2016 Feb 23. doi:10.2147/CIA.S97481

2. Leslie P, Drinnan MJ, Finn P, Ford GA, Wilson JA. Reliability and validity of cervical auscultation: a controlled comparison using videofluoroscopy. Dysphagia. 2004;19(4):231-240.

3. Costa MM. Videofluoroscopy: the gold standard exam for studying swallowing and its dysfunction. Arq Gastroenterol. 2010;47(4):327-328. doi:10.1590/s0004-28032010000400001

4. Szczesniak MM, Maclean J, Zhang T, et al. Inter-rater reliability and validity of automated impedance manometry analysis and fluoroscopy in dysphagic patients after head and neck cancer radiotherapy. Neurogastroenterol Motil. 2015;27(8):1183-1189. doi:10.1111/nmo.12610

5. Edmiaston J, Connor LT, Steger-May K, Ford AL. A simple bedside stroke dysphagia screen, validated against videofluoroscopy, detects dysphagia and aspiration with high sensitivity. J Stroke Cerebrovasc Dis. 2014;23(4):712-716.

Submitted filename: PLOS One Response Letter.docx

Click here for additional data file.

30 Jun 2020

PONE-D-20-14426R1

Diagnostic utility of the Amyotrophic Lateral Sclerosis Functional Rating Scale - Revised to detect pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis.

PLOS ONE

Dear Dr. Plowman,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 14 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Michelle Ciucci, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Thank you for the attention to the reviewer comments. There are two outstanding items. The first is the problematic term of 'gold standard.' While I agree with the authors that this is not the place for a semantics debate, when a new test outperforms an established test (that is the gold standard) the newer, better test is judged to be different than the gold standard (i.e. not as good). There are some articles (see the work of Susan Butler) that address this. Please use, 'most commonly used' or some other terminology. Second, (#11 from Reviewer 1), while it is understandable that the PRO can not provide information about false positives or negatives, this should be an important limitation that is discussed with regard to this study. There are a few more very minor comments from the second reviewers that should be addressed. Please include a note to me in the cover letter that I will address these minor changes editorially. Thank you for submitting this excellent work!

[Note: HTML markup is below. Please do not edit.]

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

3 Jul 2020

Per the email below - please refer to the cover letter, we have made both changes suggested by the reviewer and we were not sent any other edits. Thank you for your consideration.

Dear Dr. Plowman,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 14 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Michelle Ciucci, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Thank you for the attention to the reviewer comments. There are two outstanding items. The first is the problematic term of 'gold standard.' While I agree with the authors that this is not the place for a semantics debate, when a new test outperforms an established test (that is the gold standard) the newer, better test is judged to be different than the gold standard (i.e. not as good). There are some articles (see the work of Susan Butler) that address this. Please use, 'most commonly used' or some other terminology. Second, (#11 from Reviewer 1), while it is understandable that the PRO can not provide information about false positives or negatives, this should be an important limitation that is discussed with regard to this study. There are a few more very minor comments from the second reviewers that should be addressed. Please include a note to me in the cover letter that I will address these minor changes editorially. Thank you for submitting this excellent work!

Submitted filename: PLOS One Response Letter.docx

Click here for additional data file.

15 Jul 2020

Diagnostic utility of the Amyotrophic Lateral Sclerosis Functional Rating Scale - Revised to detect pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis.

PONE-D-20-14426R2

Dear Dr. Plowman,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Michelle Ciucci, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

3 Aug 2020

PONE-D-20-14426R2

Diagnostic utility of the Amyotrophic Lateral Sclerosis Functional Rating Scale – Revised to detect pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis.

Dear Dr. Plowman:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Michelle Ciucci

Academic Editor

PLOS ONE