Assessment of bulbar function in amyotrophic lateral sclerosis: validation of a self-report scale (Center for Neurologic Study Bulbar Function Scale).

BACKGROUND AND
PURPOSE: Impaired bulbar functions of speech and swallowing are among the most serious consequences of amyotrophic lateral sclerosis (ALS). Despite this, clinical trials in ALS have rarely emphasized bulbar function as an endpoint. The rater-administered Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) or various quality-of-life measures are commonly used to measure symptomatic benefit. Accordingly, we sought to evaluate the utility of measures specific to bulbar function in ALS.
METHODS: We assessed bulbar functions in 120 patients with ALS, with clinicians first making direct observations of the degree of speech, swallowing and salivation impairment in these subjects. Clinical diagnosis of bulbar impairment was then compared with ALSFRS-R scores, speech rate, time to swallow liquids and solids, and scores obtained when patients completed visual analog scales (VASs) and the newly-developed 21-question self-administered Center for Neurologic Study Bulbar Function Scale (CNS-BFS).
RESULTS: The CNS-BFS, ALSFRS-R, VAS and timed speech and swallowing were all concordant with clinician diagnosis. The self-report CNS-BFS and ALSFRS-R bulbar subscale best predicted clinician diagnosis with misclassification rates of 8% and 14% at the optimal cut-offs, respectively. In addition, the CNS-BFS speech and swallowing subscales outperformed both the bulbar component of the ALSFRS-R and speech and swallowing VASs when correlations were made between these scales and objective measures of timed reading and swallowing.
CONCLUSIONS: Based on these findings and its relative ease of administration, we conclude that the CNS-BFS is a useful metric for assessing bulbar function in patients with ALS.

Introduction

Impaired bulbar functions are among the most serious of the consequences of amyotrophic lateral sclerosis (ALS) and account for a disproportionate amount of the disability that accompanies the disease 1, 2. The brunt of the pathologic process is borne by large motor neurons in the spinal cord, brainstem, motor cortex and pyramidal tracts. As a result, patients experience inexorable paralysis of skeletal muscle, including the tongue and pharyngeal muscles. This results in varying degrees of impairment of bulbar functions, including speech and swallowing, which is dictated by the extent of lower and upper motor neuron involvement. Either directly or indirectly, bulbar dysfunction affects survival 3. For example, aspiration often occurs as a result of dysphagia, acutely causing respiratory arrest or leading to aspiration pneumonia. Additionally, weight loss is a consequence of impaired swallowing and is a negative predictive factor for survival 4; it may accelerate muscle wasting, which is a hallmark of ALS.

Almost all clinical trials in ALS have employed the Amyotrophic Lateral Sclerosis Functional Rating Scale‐Revised (ALSFRS‐R) as the primary outcome measure 5. It is a 12‐item rater‐administered ordinal scale that assesses function in four domains: bulbar function, fine motor function, gross motor function and respiration. The three items related to bulbar function query abnormalities in speech, swallowing and salivation. The ALSFRS‐R has been shown to correlate with changes in strength over time as well as quality‐of‐life measures and it can predict survival 6. However, the sensitivity of the bulbar function subscales in detecting modest changes or a specific effect of treatment has never been determined. Thus, a sensitive measure of bulbar dysfunction would be of great value, especially in the evaluation of a therapy aimed at impacting speech and/or swallowing.

Although not commonly employed in clinical trials, both the Norris and Appel scales have also been utilized to track disease progression in clinical and research settings 7, 8. More recently, researchers have begun to focus on other means, some of them more objective, for quantitating bulbar function. Instrumentation‐based approaches are being developed to analyze function across the four speech subsystems, with aerodynamic pressure‐flow and acoustic methods used to assess the respiratory, phonatory and resonatory subsystems, and three‐dimensional motion‐tracking techniques used to measure the articulatory subsystem 9. Electrical impedance myography, a technique involving the application of low‐intensity, high‐frequency current to a muscle, has been employed to longitudinally assess the extent of skeletal and tongue muscle function in patients with ALS 10, 11. Fiberoptic endoscopy, an alternative to videofluoroscopy 12, can be utilized to document impaired swallowing secondary to pharyngeal dysfunction. More recently, transcranial magnetic stimulation has been used to map the cortical representation of the swallowing musculature 13 and could potentially be used in patients with ALS to evaluate cortical changes in swallow representation and excitability following therapeutic interventions. Although promising, many of these newer techniques are difficult to implement in a clinical or research setting. It is relevant to this discussion, as noted by Green et al. in 2013 14, that “no standardized diagnostic procedure for assessing bulbar function in ALS exists” and “adequate markers of bulbar dysfunction have yet to be identified.”

Although a protocol for assessing bulbar function was recently published 9, we thought it important to determine the utility of this instrument in a research setting. Historically, ALS trials have emphasized survival. On a background of failed trials, Nuedexta has been shown to improve speech and swallowing in a double‐blind treatment trial 15. The endpoint of this Phase 2 study was a self‐report scale [Center for Neurologic Study Bulbar Function Scale (CNS‐BFS)] that proved to be more sensitive to a treatment effect than commonly used measures, such as timed speech and swallowing. This compelled us to critically review the tools available for the assessment of bulbar function, ultimately leading us to conclude that self‐report data are a critical adjunct to the diagnostic armamentarium in ALS clinical trials. This report summarizes the evidence that supports this conclusion, a realization that is consonant with new Food and Drug Administration guidelines on the value of patient report outcome measures in clinical trials. From the outset, this has been our bias as the CNS Lability Scale, a self‐report tool for assessing emotional lability that we previously developed 16, 17, 18, 19, has proven to be a robust endpoint in four clinical trials.

Methods

A total of 120 patients with ALS with bulbar impairment were assessed at seven participating research sites. Sixty of the subjects were representative of patients attending a general ALS clinic. The remaining subjects were participants in a double‐blind, Phase 2 crossover treatment trial to test the efficacy of Nuedexta in palliating impaired bulbar function (NCT01806857) 15. The demographic characteristics of the study population were typical of patients with ALS seen in the USA (i.e. average age of onset 57.9 ± 11.0 years; 64% male and 36% female; 95.8% Caucasian). The trial was approved by the institutional review boards (IRBs) of each site (Cleveland Clinic Foundation IRB, Providence Health and Services IRB, Hennepin County Medical Center IRB, Sutter Health IRB, Mercy Health IRB, Georgetown University IRB and the combined IRB of Bryan LGH Medical Center and Saint Elizabeth Regional Medical Center). All participants provided written informed consent.

A clinical observer made a diagnosis of impaired speech, swallowing and salivation through direct observations of the subjects. To evaluate speech, participants were instructed to read ‘The Rainbow Passage’, a commonly used paragraph employed by speech–language pathologists to objectively assess speech rate (words/min). Clinicians separately evaluated speech as normal or abnormal with respect to loudness, nasality and intelligibility. Speech was judged abnormal if any of these characteristics were abnormal.

Objective measures of swallowing performance were obtained. To evaluate the swallowing of liquids, participants were instructed to drink 30 mL of water from a cup; assistance with holding the cup was provided to those unable to hold it on their own. The total swallow duration [time between lip closure and swallow (in s)] was recorded for three trials. The average of the two shortest swallows served as an objective measure of liquid swallowing. To evaluate the swallowing of solids, participants were instructed to eat one tablespoon of cereal containing five Cheerios. The time to chew and swallow the cereal [from lip closure to swallow (in s)] from each of three trials was taken as an objective measure of solid swallowing. The inter‐ and intra‐assessor reliability of this measure is very high (intraclass correlation coefficient, 0.98; P < 0.001) 20. Clinicians separately rated swallowing as normal or abnormal with respect to choking, spillage and effort. Swallowing was judged abnormal if any of these three characteristics were abnormal.

Clinicians rated salivation as normal or abnormal by observing drooling and dabbing during unstimulated and stimulated conditions. ‘Unstimulated’ salivation was determined over 10–15 min during which time the subjects completed the self‐report scales. ‘Stimulated’ salivation was assessed while the subjects read the test paragraph and performed the timed swallowing studies. Salivation was judged as abnormal if either drooling or dabbing occurred in either the unstimulated or stimulated state. An overall clinician diagnosis of bulbar function was scored as abnormal if any of the speech, swallowing or salivation domains were judged to be impaired.

After completing the objective tests, subjects completed the self‐administered 21‐question CNS‐BFS and visual analog scales (VASs) for speech, swallowing and salivation. A trained evaluator scored the ALSFRS‐R.

The CNS‐BFS is structured to interrogate three domains of bulbar function: speech, swallowing and salivation. For each domain, subjects are asked to rate seven statements or questions on a scale of 1–5. (Subjects who are unable to speak are assigned a value of 6 for each item comprising the speech domain.) Scores can range from a low of 21 (no symptoms of bulbar dysfunction) to a high of 112. In a clinical setting, an evaluator provides instructions for completion of the form by reference to a sample question. In essence, the scale can be considered to be a quality‐of‐life measure. For example, six of the seven questions in the swallowing domain of the CNS‐BFS interrogate feeding behaviors. The complete instrument as presented to patients is illustrated in Fig. 1.

Figure 1

Center for Neurologic Study Bulbar Function Scale self‐report scale.

Relationships between clinician diagnosis of speech, swallowing and salivation impairment, and each associated measure (CNS‐BFS, VAS and ALSFRS‐R) were evaluated by logistic regression. The strength of association was summarized as area under the receiver‐operating characteristic curve (ROC AUC) and by five measures (sensitivity, specificity, positive and negative predictive value, and total accuracy), each evaluated at the threshold value that maximized the sum of sensitivity and specificity (equivalent to maximizing Youden's index). Relationships among the continuous measures were evaluated by Pearson correlations. Factor structure of the 21 questions comprising the CNS‐BFS was examined by confirmatory factor analysis of the inferred polychoric correlations for both a unidimensional scale and a three‐factor structure that associated each group of questions with its own domain. Goodness of fit was assessed based on root mean square error of approximation (RMSEA) and Bentler's Comparative Fit Index (CFI), with RMSEA < 0.06 and CFI > 0.90 considered evidence of adequate conformity to the tested factor structure. We estimated the natural rate of progression of the CNS‐BFS total score and alternative measures of bulbar function using a random‐slopes linear mixed model. The model included an intercept and a fixed effect of time and random participant‐specific intercepts and slopes with unstructured covariance. To estimate natural rate of progression unaffected by treatment, only data from observation times when a participant was not exposed to Nuedexta were included for analysis.

Results

Criterion validity

The CNS‐BFS total score and ALSFRS‐R bulbar subscale were highly predictive of clinician diagnosis of impaired bulbar function (ROC AUC, 0.95 and 0.92, respectively; P < 0.001). CNS‐BFS subscales also predicted corresponding clinician diagnoses of speech, swallowing and salivation impairment (ROC AUC, 0.83–0.95; P < 0.001) (Table 1.

Table 1

Criterion validity: prediction of clinician diagnosis of impaired function by individual measures of bulbar assessment

Measure 1	n	Full ROC curve				Cut‐off at maximum Youden's index
		AUC	Lower	Upper	P‐value	Value	Accuracy	PPV	NPV	Sensitivity	Specificity
CNS‐BFS total score	112	0.954	0.915	0.993	<0.001	≥39.0	0.920	0.955	0.870	0.913	0.930
CNS‐BFS speech	119	0.948	0.907	0.988	<0.001	≥14.0	0.891	0.873	0.917	0.939	0.830
CNS‐BFS swallowing	113	0.830	0.756	0.905	<0.001	≥16.0	0.770	0.690	0.855	0.833	0.723
CNS‐BFS sialorrhea	120	0.884	0.825	0.942	<0.001	≥11.0	0.775	0.469	0.986	0.958	0.729
ALSFRS‐R total score	111	0.603	0.496	0.710	0.059	≤32.0	0.559	0.732	0.457	0.441	0.744
ALSFRS‐R bulbar subscore	111	0.922	0.874	0.970	<0.001	≤9.0	0.856	0.919	0.776	0.838	0.884
ALSFRS‐R 1. Speech	118	0.916	0.872	0.960	<0.001	≤2.0	0.814	0.978	0.712	0.677	0.981
ALSFRS‐R 3. Swallowing	112	0.696	0.607	0.786	<0.001	≤3.0	0.652	0.559	0.795	0.809	0.538
ALSFRS‐R 2. Salivation	119	0.827	0.755	0.899	<0.001	≤3.0	0.597	0.324	1.000	1.000	0.500
VAS speech	117	0.862	0.792	0.933	<0.001	≤7.0	0.803	0.841	0.759	0.803	0.804
VAS swallowing	112	0.710	0.614	0.807	<0.001	≤8.0	0.679	0.620	0.726	0.646	0.703
VAS sialorrhea	118	0.854	0.786	0.921	<0.001	≤8.0	0.754	0.447	0.958	0.875	0.723
Timed reading (words/min)	117	0.917	0.867	0.968	<0.001	≤149.3	0.863	0.877	0.846	0.877	0.846
Timed swallowing: solids	113	0.762	0.674	0.851	<0.001	≥11.2	0.699	0.603	0.844	0.854	0.585
Timed swallowing: water	113	0.760	0.674	0.847	<0.001	≥5.0	0.717	0.682	0.739	0.625	0.785

ALSFRS‐R, Amyotrophic Lateral Sclerosis Functional Rating Scale‐Revised; AUC, area under the curve; CNS‐BFS, Center for Neurologic Study Bulbar Function Scale; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver‐operating characteristic; VAS, visual analog scale.

Cutting scores for the identification of impaired bulbar function were designated and categorizations based on these scores were compared with neurologists’ diagnoses for the purpose of examining criterion validity. For example, a CNS‐BFS score of 39 detected impaired bulbar function with a sensitivity of 91% and a specificity of 93%.

At a CNS‐BFS score of 43, the scale detected impaired bulbar function with a positive predictive value of 98% in our sample (Table 2). In a research setting, a score of 43 and above might be useful as an inclusion criterion to select subjects who are most likely to have the condition under treatment.

Table 2

Operating characteristics for prediction of any physician diagnosis of bulbar dysfunction by Center for Neurologic Study Bulbar Function Scale total score for specific cut‐off points

Threshold value	True positive	False positive	True negative	False negative	Sensitivity (%)	Specificity (%)	Total accuracy (%)	Positive predictive value (%)	Negative predictive value (%)
21	69	43	0	0	100.0	0.0	61.6	61.6
22	68	32	11	1	98.6	25.6	70.5	68.0	91.7
23	68	27	16	1	98.6	37.2	75.0	71.6	94.1
24	68	25	18	1	98.6	41.9	76.8	73.1	94.7
25	68	23	20	1	98.6	46.5	78.6	74.7	95.2
27	68	20	23	1	98.6	53.5	81.3	77.3	95.8
28	68	18	25	1	98.6	58.1	83.0	79.1	96.2
29	68	16	27	1	98.6	62.8	84.8	81.0	96.4
30	68	15	28	1	98.6	65.1	85.7	81.9	96.6
31	65	14	29	4	94.2	67.4	83.9	82.3	87.9
32	64	14	29	5	92.8	67.4	83.0	82.1	85.3
33	64	12	31	5	92.8	72.1	84.8	84.2	86.1
34	64	10	33	5	92.8	76.7	86.6	86.5	86.8
35	64	7	36	5	92.8	83.7	89.3	90.1	87.8
36	64	6	37	5	92.8	86.0	90.2	91.4	88.1
37	64	4	39	5	92.8	90.7	92.0	94.1	88.6
39	63	3	40	6	91.3	93.0	92.0	95.5	87.0
41	61	2	41	8	88.4	95.3	91.1	96.8	83.7
42	58	2	41	11	84.1	95.3	88.4	96.7	78.8
43	58	1	42	11	84.1	97.7	89.3	98.3	79.2
44	57	1	42	12	82.6	97.7	88.4	98.3	77.8
45	56	1	42	13	81.2	97.7	87.5	98.2	76.4
46	54	1	42	15	78.3	97.7	85.7	98.2	73.7
47	51	1	42	18	73.9	97.7	83.0	98.1	70.0
48	50	1	42	19	72.5	97.7	82.1	98.0	68.9
49	47	1	42	22	68.1	97.7	79.5	97.9	65.6
50	45	1	42	24	65.2	97.7	77.7	97.8	63.6
51	44	1	42	25	63.8	97.7	76.8	97.8	62.7
52	39	1	42	30	56.5	97.7	72.3	97.5	58.3
53	37	1	42	32	53.6	97.7	70.5	97.4	56.8
54	35	1	42	34	50.7	97.7	68.8	97.2	55.3
55	32	1	42	37	46.4	97.7	66.1	97.0	53.2
56	32	0	43	37	46.4	100.0	67.0	100.0	53.8
57	27	0	43	42	39.1	100.0	62.5	100.0	50.6
58	26	0	43	43	37.7	100.0	61.6	100.0	50.0
59	24	0	43	45	34.8	100.0	59.8	100.0	48.9
60	23	0	43	46	33.3	100.0	58.9	100.0	48.3
61	22	0	43	47	31.9	100.0	58.0	100.0	47.8
62	21	0	43	48	30.4	100.0	57.1	100.0	47.3
64	18	0	43	51	26.1	100.0	54.5	100.0	45.7
65	16	0	43	53	23.2	100.0	52.7	100.0	44.8
66	15	0	43	54	21.7	100.0	51.8	100.0	44.3
67	14	0	43	55	20.3	100.0	50.9	100.0	43.9
69	13	0	43	56	18.8	100.0	50.0	100.0	43.4
70	12	0	43	57	17.4	100.0	49.1	100.0	43.0
71	11	0	43	58	15.9	100.0	48.2	100.0	42.6
72	10	0	43	59	14.5	100.0	47.3	100.0	42.2
73	8	0	43	61	11.6	100.0	45.5	100.0	41.3
75	6	0	43	63	8.7	100.0	43.8	100.0	40.6
80	5	0	43	64	7.2	100.0	42.9	100.0	40.2
84	4	0	43	65	5.8	100.0	42.0	100.0	39.8
85	3	0	43	66	4.3	100.0	41.1	100.0	39.4
94	2	0	43	67	2.9	100.0	40.2	100.0	39.1
110	1	0	43	68	1.4	100.0	39.3	100.0	38.7

Construct validity

The self‐report CNS‐BFS total score was highly correlated with the bulbar subscale of the ALSFRS‐R (r = −0.90; P < 0.001). The CNS‐BFS subscales for speech, swallowing and salivation were better correlated with the corresponding VAS scores than were individual ALSFRS‐R bulbar function questions. Further, the CNS‐BFS speech and swallowing subscales were better correlated with speech rate and timed swallowing of liquids and solids than were the ALSFRS‐R or VAS speech and swallowing scores.

Correlations of longitudinal change in these measures among the subset of participants followed in the trial were weaker but largely paralleled the correlations observed at baseline, with significant correlation between changes in CNS‐BFS total score and ALSFRS‐R bulbar subscore (Table 3).

Table 3

Construct validity: correlations between individual measures of bulbar impairment at baseline and from baseline to visit 1

		Baseline only			Baseline to visit 1
Measure 1	Measure 2	n	Correlation	P‐value	n	Correlation	P‐value
CNS‐BFS total	ALSFRS‐R total score	119	−0.254	0.005	57	−0.400	0.002
CNS‐BFS total	ALSFRS‐R bulbar subscore	119	−0.896	<0.001	57	−0.388	0.002
CNS‐BFS total	ALSFRS‐R fine motor subscore	119	0.059	0.523	57	−0.192	0.149
CNS‐BFS total	ALSFRS‐R gross motor subscore	119	0.127	0.167	57	−0.083	0.535
CNS‐BFS total	ALSFRS‐R respiratory subscore	119	−0.117	0.202	57	−0.236	0.075
CNS‐BFS speech	ALSFRS‐R 1. Speech	119	−0.882	<0.001	57	−0.495	<0.001
CNS‐BFS swallowing	ALSFRS‐R 3. Swallowing	119	−0.762	<0.001	57	−0.182	0.173
CNS‐BFS sialorrhea	ALSFRS‐R 2. Salivation	119	−0.723	<0.001	57	−0.111	0.410
CNS‐BFS speech	VAS speech self‐assessment	118	−0.758	<0.001	58	−0.339	0.008
CNS‐BFS swallowing	VAS swallowing self‐assessment	118	−0.538	<0.001	58	−0.027	0.840
CNS‐BFS sialorrhea	VAS sialorrhea self‐assessment	118	−0.674	<0.001	58	0.153	0.247
ALSFRS‐R 1. Speech	VAS speech self‐assessment	117	0.730	<0.001	57	0.131	0.329
ALSFRS‐R 3. Swallowing	VAS swallowing self‐assessment	117	0.392	<0.001	57	−0.046	0.731
ALSFRS‐R 2. Salivation	VAS sialorrhea self‐assessment	117	0.567	<0.001	57	−0.112	0.404
CNS‐BFS speech	Timed reading	117	−0.762	<0.001	57	−0.269	0.041
CNS‐BFS swallowing	Timed swallowing: solids	120	0.519	<0.001	57	0.086	0.522
CNS‐BFS swallowing	Timed swallowing: water	120	0.519	<0.001	57	0.109	0.417
ALSFRS‐R 1. Speech	Timed reading	116	0.722	<0.001	56	0.411	0.001
ALSFRS‐R 3. Swallowing	Timed swallowing: solids	119	−0.376	<0.001	56	0.142	0.295
ALSFRS‐R 3. Swallowing	Timed swallowing: water	119	−0.481	<0.001	56	−0.195	0.146
VAS speech self‐assessment	Timed reading	115	0.627	<0.001	57	0.149	0.266
VAS swallowing self‐assessment	Timed swallowing: solids	118	−0.412	<0.001	57	−0.138	0.304
VAS swallowing self‐assessment	Timed swallowing: water	118	−0.309	<0.001	57	0.054	0.686

ALSFRS‐R, Amyotrophic Lateral Sclerosis Functional Rating Scale‐Revised; CNS‐BFS, Center for Neurologic Study Bulbar Function Scale; VAS, visual analog scale.

Factor structure

Confirmatory factor analysis suggested a weak fit to a unidimensional structure (RMSEA, 0.20; CFI, 0.63) or a three‐domain structure (RMSEA, 0.16; CFI, 0.77) as originally conceived. Although the presumed factor structure was weakly supported, Cronbach's α suggested strong internal consistency of the CNS‐BFS, with a coefficient of 0.939 for the speech subscale, 0.863 for the swallowing subscale, 0.862 for the sialorrhea subscale and 0.949 for the 21‐question total score.

This compared favorably with the ALSFRS‐R bulbar subscale (0.837). The ALSFRS‐R has historically been regarded as the gold standard for assessing function in ALS clinical trials.

Test–retest reliability

The CNS‐BFS total scores showed a test–retest reliability over the 2‐week screening interval of 0.86 [95% confidence interval (CI), 0.80–0.93].

Sensitivity to natural progression

Up to 10 weeks of untreated follow‐up, the ALSFSR‐R total score declined at an average rate of 1.1 points/month (95% CI, −1.6 to −0.6), consistent with previously published data from prior clinical trials in ALS (Table 4). The ALSFRS‐R bulbar subscore declined at an average rate of 0.38 points/month (95% CI, −0.59 to −0.17) out of a total 12‐point range. The CNS‐BFS total score increased at an average rate of 1.2 points/month (95% CI, 0.1–2.2). This was attributable to changes in speech (0.54; 95% CI, 0.05–1.03) and salivation (0.63; 95% CI, 0.11–1.16) with minimal change observed in swallowing (0.03; 95% CI, −0.49 to 0.55). Clarifying this further, the lack of change in swallowing was still evident when restricting evaluation to participants first randomized to placebo, with no demonstrable change in swallowing over the first 6‐week interval as measured by both the CNS‐BFS and ALSFRS‐R. Similarly, the recently developed self‐report EAT‐10 scale does not track well over time (https://f1000research.com/posters/5-2856).

Table 4

Estimates of natural rates of progression

Measure	Interval	Estimate	SE	95% confidence interval
				Lower	Upper
CNS‐BFS total score a	Baseline to visit 3	1.164	0.531	0.099	2.228
CNS‐BFS speech	Baseline to visit 3	0.541	0.245	0.050	1.033
CNS‐BFS swallowing	Baseline to visit 3	0.033	0.259	−0.485	0.552
CNS‐BFS sialorrhea	Baseline to visit 3	0.634	0.263	0.107	1.162
CNS‐LS total score	Baseline to visit 3	0.338	0.252	−0.167	0.843
ALSFRS‐R total score a	Baseline to visit 3	−1.124	0.239	−1.602	−0.646
ALSFRS‐R bulbar subscore a	Baseline to visit 3	−0.383	0.105	−0.593	−0.173
ALSFRS‐R fine motor subscore	Baseline to visit 3	−0.252	0.100	−0.452	−0.052
ALSFRS‐R gross motor subscore	Baseline to visit 3	−0.175	0.084	−0.343	−0.007
ALSFRS‐R respiratory subscore	Baseline to visit 3	−0.249	0.096	−0.441	−0.057
ALSFRS‐R 1. Speech	Baseline to visit 3	−0.159	0.041	−0.242	−0.076
ALSFRS‐R 2. Salivation	Baseline to visit 3	−0.173	0.075	−0.322	−0.023
ALSFRS‐R 3. Swallowing	Baseline to visit 3	−0.049	0.043	−0.135	0.037
VAS speech self‐assessment	Baseline to visit 3	−0.455	0.151	−0.759	−0.152
VAS swallowing self‐assessment	Baseline to visit 3	−0.327	0.185	−0.699	0.044
VAS sialorrhea self‐assessment	Baseline to visit 3	−0.312	0.185	−0.682	0.059
Timed reading test: words/min	Baseline to visit 3	−2.817	1.127	−5.075	−0.560
Timed swallowing test: solids	Baseline to visit 3	−0.583	0.639	−1.863	0.698
Timed swallowing test: water	Baseline to visit 3	1.004	0.673	−0.345	2.353

ALSFRS‐R, Amyotrophic Lateral Sclerosis Functional Rating Scale‐Revised; CNS‐BFS, Center for Neurologic Study Bulbar Function Scale; CNS‐LS, CNS Lability Scale; SE, standard error; VAS, visual analog scale.

The CNS‐BFS total score refers to the score for all 21 questions in the CNS‐BFS scale (seven questions each in the speech, swallowing, and sialorrhea domains); the ALSFRS‐R total score includes 12 questions, while the ALSFRS‐R bulbar subscore includes three questions addressing separate aspects of bulbar function (speech, swallowing and salivation).

Sensitivity to intervention

Although it may not be relevant for future studies involving other therapeutic interventions, we thought it noteworthy to determine which domains of the CNS‐BFS and ALSFRS‐R bulbar subscale were sensitive to change following treatment with Nuedexta. Both the speech domains of the CNS‐BFS and the ALSFRS‐R bulbar scale were sensitive measures of a treatment effect (P = 0.002 and <0.001, respectively). In contrast, the swallowing and salivation domains of the CNS‐BFS were both responsive to treatment (P = 0.007 and 0.005, respectively) whereas this was not the case in the instance of the swallowing or salivation questions of the ALSFRS‐R (P = 0.80 and P = 0.066, respectively), recognizing that P > 0.05 is not evidence of lack of an effect. Among all the other measures (timed speech and swallowing, and VAS for speech, swallowing, and salivation), only the VAS for speech was sensitive to treatment with Nuedexta (P = 0.005).

Discussion

The assessment of bulbar function presents a challenging conundrum in clinical and research settings. Unlike the evaluation of skeletal muscle, which is relatively straightforward, bulbar function assessment is more nuanced and not easily measured in the clinic. In both settings, a number of factors are at play: cost, convenience, reproducibility, subjective versus objective readouts, standardization and the issue of inter‐rater reliability. In an effort to shed light on this subject, we have undertaken an extensive evaluation of the utility of measures employed in clinical trials to assess bulbar function.

We initially compared each of these measures against the clinician diagnosis of impaired bulbar function. From our viewpoint, this is the best standard against which all other measures should rightfully be compared. It is reassuring that all of these measures (i.e. CNS‐BFS, ALSFRS‐R, VAS and timed speech and swallowing) compared favorably with clinician diagnosis. However, as observed, the self‐report CNS‐BFS and the ALSFRS‐R were better predictors of clinician diagnosis than any other measures (Table 1). Moreover, the recently developed CNS‐BFS outperforms both the bulbar component of the ALSFRS‐R and bulbar VASs when correlations are made between these scales and timed reading and swallowing. Based on these findings and its ease of administration, we feel that the CNS‐BFS is a useful metric for assessing bulbar function in patients with ALS in a research or clinical setting. Longer studies, perhaps extending over a year or more, need to be undertaken to fully understand the behavior of the CNS‐BFS over time, as well as more recently investigated measures, such as pause and speech rates 21. However, the 10‐week serial data reported in this study should be sufficient to provide guidance for an upcoming 12‐week treatment trial that will assess the effect of Nuedexta on bulbar impairment in ALS.

Is this relevant and useful information? One of the strengths of this study is that it affirms patients’ insights into their condition. For example, the swallowing domain of the CNS‐BFS addresses self‐reported feeding behavior and, as noted above, the CNS‐BFS was well correlated with clinician diagnosis of impaired swallowing and two timed measures of swallowing. Moreover, it is noteworthy that, in the instance of the recently completed Nuedexta treatment trial 15, both the CNS‐BFS and ALSFRS‐R were sensitive indicators of a treatment response, whereas traditionally applied measures, such as timed speech and swallowing, were less responsive. In essence, had this trial relied solely on timed measures, it would have failed due to a type 2 statistical error.

Historically, the Food and Drug Administration has been reluctant to place emphasis on patient self‐assessment scales. The patient's perception of their illness has not attained the stature that has been attributed to traditionally objective measures, such as timed speech, in the research community. Historical objections to this type of information have included the possibility of lengthy questionnaires not being completed and that patient attention could be impaired because of health issues and the concomitant use of medications. It is refreshing to note that recent Food and Drug Administration guidelines have given more credence to self‐report measures 22.

The above‐mentioned clinical trial also provided a unique opportunity to determine which measures best interrogated the various bulbar functions as determined by their response to therapy. The CNS‐BFS total score performed well in detecting treatment response to Nuedexta, but it is not certain that other drugs would have the same effect. We have not, as yet, confirmed that the psychometric properties of the CNS‐BFS are ideal. These properties might be optimized through application of modern item‐response theory with development of an abbreviated scale and one that better separates the underlying factors. A revision of the ALSFRS‐R has recently been recommended based on the use of similar clinimetric methodology 23.

Based on the imperfections inherent in the measurement of bulbar function, considerable effort is currently being devoted to the development of ‘objective’ measures of bulbar function. Examples of this include work demonstrating that speech and pause rates may be useful markers for monitoring longitudinal change 21. While these and additional objective measures are needed, our data suggest that patient self‐assessment scales are well suited for continued evaluation of the complex and nuanced features of bulbar function and thus useful in both clinical and research settings.

Disclosure of conflicts of interest

R.A.S., G.L.P. and E.P.P. report consulting and/or speaker's fees from Avanir Pharmaceuticals (parent company Otsuka), which is the manufacturer of Nuedexta. The Center for Neurologic Study provided a grant to E.A.M. to support biostatistical analysis. The other authors declare no financial or other conflicts of interest.