The Pooling-score (P-score): inter- and intra-rater reliability in endoscopic assessment of the severity of dysphagia.

This study evaluated the intra- and inter-rater reliability of the Pooling score (P-score) in clinical endoscopic evaluation of severity of swallowing disorder, considering excess residue in the pharynx and larynx. The score (minimum 4 - maximum 11) is obtained by the sum of the scores given to the site of the bolus, the amount and ability to control residue/bolus pooling, the latter assessed on the basis of cough, raclage, number of dry voluntary or reflex swallowing acts (< 2, 2-5, > 5). Four judges evaluated 30 short films of pharyngeal transit of 10 solid (1/4 of a cracker), 11 creamy (1 tablespoon of jam) and 9 liquid (1 tablespoon of 5 cc of water coloured with methlyene blue, 1 ml in 100 ml) boluses in 23 subjects (10 M/13 F, age from 31 to 76 yrs, mean age 58.56±11.76 years) with different pathologies. The films were randomly distributed on two CDs, which differed in terms of the sequence of the films, and were given to judges (after an explanatory session) at time 0, 24 hours later (time 1) and after 7 days (time 2). The inter- and intra-rater reliability of the P-score was calculated using the intra-class correlation coefficient (ICC; 3,k). The possibility that consistency of boluses could affect the scoring of the films was considered. The ICC for site, amount, management and the P-score total was found to be, respectively, 0.999, 0.997, 1.00 and 0.999. Clinical evaluation of a criterion of severity of a swallowing disorder remains a crucial point in the management of patients with pathologies that predispose to complications. The P-score, derived from static and dynamic parameters, yielded a very high correlation among the scores attributed by the four judges during observations carried out at different times. Bolus consistencies did not affect the outcome of the test: the analysis of variance, performed to verify if the scores attributed by the four judges to the parameters selected, might be influenced by the different consistencies of the boluses, was not significant. These initial data validate the clinical use of the P-score in the management of patients with deglutition disorders by a multidisciplinary team.

Introduction

In patients with dysphagia due to different pathologies, predisposing to complications, the clinical severity of a swallowing disorder needs to be established . Respiratory complications, related to false paths, taken by bolus or pooling of secretions, can arise in the short or long term . Clinical non-instrumental assessment (clinical swallowing evaluation – CSE) alone can fail to identify silent aspirations or micro-aspirations. In at-risk conditions for the patient, and to define physiopathology of the swallowing disorder, instrumental assessment is essential in order to plan treatment. Currently, there is no instrumental technique that clearly defines the risk of false pathways: video- fluoroscopic study of swallowing (VFSS) and fiberoptic endoscopic evaluation of swallowing (FEES) both yield false positives and false negatives . Choosing the instrumental method that best responds to clinical questions and the expertise of the clinician carrying out the procedure become crucial factors in clinical management of patients with deglutition disorders.

Endoscopic assessment provides an exhaustive definition of anatomical details of pharyngeal and laryngeal surfaces, including secretions or bolus residues that, in usual or unusual conditions, moisten the walls (coating) or occupy containment cavities (pooling, residue). Such a evaluation is considered a clinical criterion for severity that can be linked to the risk of respiratory complications . Although to date a standardised grading of pharyngeal residue is not available, this criterion is taken into account when planning treatment and management activities by a multidisciplinary team .

In previous publications -, pooling of materials is considered in the broadest sense as any material that is present in the containment cavities of the hypopharynx and larynx, before and/or after the act of swallowing. The severity criterion proposed (Pooling score, P-score) (Table I) takes into account different parameters: 1) site: identified by anatomical landmarks; 2) amount: determined in a semi-quantitative fashion by the amount of pooling materials (coating, more or less than 50% of cavity containment capacity); 3) management: the ability of the patient to clear the residue. The score refers to a specific type of consistency and volume of the bolus, changing according to these parameters. The score considers the most severe condition for each parameter, reached after a sequence of boluses of the same consistency (in our department a sequence of at least 3 boluses for each consistency).

Table I.

P-score and P-SCA score.

Pooling	Endoscopic landmarks		Bedside parameters
			Sensation	Collaboration	Age (years)
Site	Vallecula Marginal zone Pyriform sinus Vestibule/vocal cords Lower vocal cords	1 1 2 3 4	Presence = -1 Absence = +1	Presence = -1 Absence = +1	+1 (< 65) +2 (65-75) +3 (> 75)
Amount	Coating Minimum Maximum	1 2 3
Management	< 2 2-5 > 5	2 3 4
Score	P 4-11		P-SCA 3-16

In clinical practice, the P-score may be integrated with other parameters of clinical assessment (CSE) that are more easily determined: sensation of the pharynx, patient collaboration and age (P-SCA score). Both the scores express, as a numerical value, a continuum of severity that may be used in different ways, with correlations that still have to be verified. Therefore, a minimum score (P-score 4-5) may indicate the absence of endoscopic signs of dysphagia. A low score (P-score 6-7) may identify mild dysphagia, a medium score (P-score 8-9) moderate dysphagia and a high score (P-score 10-11) severe dysphagia.

This investigation considers the intra- and inter-rater reliability of the P-score among 4 judges with long-standing experience in the use of endoscopy. CSE parameters and the P-SCA score were not considered.

Materials and methods

Transits of 23 consecutive outpatients aged between 31 and 76 years (average age 58.56 ± 11.76), and referred to our department complaining of difficulties in swallowing, were enrolled. Sample characteristics and diseases are reported in Table II. Four judges with expertise in the FEES procedure were elected. Three judges have at least 10 years of experience, and one has 4 years of experience: they all routinely carry out at least 5 endoscopic assessments of swallowing per week.

Table II.

Patient characteristics and diseases.

Subject	Gender	Age	Pathology
1 PS	F	47	Globus
2 AA	F	68	Cortical ictus sequelae
3 GA	M	65	GERD
4 BA	F	62	COPD
5 CL	F	56	Dermatomyositis
6 RR	M	67	Cortical ictus sequelae
7 MM1	F	64	Laryngeal paralysis
8 XL	F	38	GERD
9 TS	F	64	Neurological degenerative
10 MM2	M	63	Cortical ictus sequelae
11 QG	M	42	Corea major
12 DA	F	71	Myasthenia
13 ME	M	52	H-N operated
14 DM	F	48	Cortical ictus sequelae
15 ME	F	46	Laryngeal paralysis
16 BG	M	62	Sjögren's syndrome
17 CC	F	76	Wallemberg sequaele
18 BF	F	73	Laryngeal paralysis
19 CR	M	51	Laryngeal paralysis
20 MP	M	71	COPD
21 RF	F	67	H-N operated
22 CL	M	31	Neurologial degenerative
23 SR	M	63	Cortical ictus sequelae

For some of the 23 subjects, the pharyngeal transit films of more than one bolus of a different consistency was recorded to obtain a total number of 30 films: 10 solid boluses (1/4 of a cracker), 11 creamy boluses (1 spoonful of jam) and 9 liquid boluses (1 tablespoon with 5 cc of water dyed with methylene blue, 1 cc in 100 ml, according to the procedure used in our department). The films are short sequences of acts of swallowing that are prolonged until the bolus has been completely swallowed, or at least 5 swallows. Film length varies from a few dozen seconds to no more than 180 seconds, so the complete session lasted no more than an hour, avoiding fatigue and a decrease in concentration among judges .

All the films were collected on two CD copies, respectively CD0 and CD1, that differed with respect to the sequence of the films. Before the session, the rationale and application of the score were explained to the judges. Each judge was given a CD-test containing 4 pharyngeal transits of boluses with different consistencies, in subjects who differed by the severity of dysphagia. The application of the P-score was explained personally by the first judge at an explanatory session lasting 30 min. Judges were asked to watch CD0 immediately thereafter (time 0), CD1 at 24 hours later (time 1) and CD0 after 7 days (time 2) in a different order than at time 0.

Data were collected on a predefined sheet, reporting separate findings for each parameter: pooling site, amount, management and total P-score. The different bolus consistencies used in our study between the pharyngeal transit films tested is a bias -. For this reason, we calculated the variance of scores attributed by judges to the parameters selected. Finally, the score can also be applied at the beginning of the observation . In the films considered, material pooling at the beginning of the observation was present in only a few cases, so it was decided not to apply the score at that time, but only after the transit of the bolus through the pharynx.

The analysis of data obtained was carried out using SAS statistical software and the inter- and intra-rater reliability was calculated with the intra-class correlation coefficient (ICC; 3,k) .

Results

The scores attributed by judges to the site, amount and management of the pooled material are shown in Tables IIIa, IIIb and IIIc. The site, apparently simpler, was modified by both the first and fourth judges over time, reaching agreement at the last observation [ICC (3, k) 0.999]. The parameter amount was changed many times during observations, with differences maintained among judges [ICC (3, k) 0.997]. The parameter management was replicated by all judges in all three observations [ICC (3, k) 1.000]. The total P-score attributed by the four judges confirmed that the criterion of severity established by the first judge was essentially shared by the other three judges [ICC (3, k) 0.999] (Table IV).

Table IIIa.

Site (anatomical landmarks): descriptive analysis.

Time		JUDGE
		1	2	3	4
		N	N	N	N
0	1	16	15	15	15
	2	4	5	5	5
	3	5	5	5	5
	4	5	5	5	5
1	1	15	15	15	15
	2	5	5	5	5
	3	5	5	5	6
	4	5	5	5	4
2	1	15	15	15	15
	2	5	5	5	5
	3	5	5	5	5
	4	5	5	5	5

P-score: site

ICC(3,k) 0.999

Time: time of observation

N: number of observation

Table IIIb.

Amount: descriptive analysis.

Time		JUDGE
		1	2	3	4
		N	N	N	N
0	1	18	19	19	18
	2	8	7	7	8
	3	4	4	4	4
1	1	18	19	19	18
	2	7	7	7	8
	3	5	4	4	4
2	1	19	19	19	19
	2	6	7	7	7
	3	5	4	4	4

P-score: amount

ICC(3,k) 0.997

Time: time of observation

N: number of observation

Table IIIc.

Management: descriptive analysis.

Time		JUDGE
		1	2	3	4
		N	N	N	N
0	2	8	8	8	8
	3	5	5	5	5
	4	17	17	17	17
1	2	8	8	8	8
	3	5	5	5	5
	4	17	17	17	17
2	2	8	8	8	8
	3	5	5	5	5
	4	17	17	17	17

P-score: management

ICC(3,k) 1.000

Time: time of observation

N: number of observation

Table IV.

Severity criteria: descriptive analysis.

Time		JUDGE
		1	2	3	4
		N	N	N	N
0	4	8	8	8	8
	5	3	3	3	3
	7	7	7	7	6
	8	4	4	4	5
	9	5	5	5	5
	10	2	2	2	2
	11	1	1	1	1
1	4	8	8	8	8
	5	3	3	3	3
	7	6	7	7	7
	8	4	4	4	4
	9	6	5	5	5
	10	2	2	2	2
	11	1	1	1	1
2	4	8	8	8	8
	5	3	3	3	3
	7	6	7	7	7
	8	5	4	4	4
	9	5	5	5	5
	10	2	2	2	2
	11	1	1	1	1

P-score: total

ICC(3,k) 0.999

Time: time of observation

N: number of observations

The analysis of variance, performed to verify if the scores attributed by the four judges to the parameters selected, might have been influenced by the different consistency of the boluses, was not significant (interaction consistency*judge = 1.000 at times 0, 1, 2, respectively, for the three parameters of the score and P-score total). Thus, the different types of consistencies did not influence the scores attributed by judges to the 30 films.

Discussion

The evaluation of secretions or bolus pooling in the pharynx end/or larynx represents an important step in the endoscopic examination of swallowing, as it is closely correlated with respiratory complications . As previously mentioned, to date, standardised grading for pharyngeal material pooling is not available: the P-score could be used in this clinical context.

From a physiopathological point of view, a swallowing disorder is the result of an imbalance between events that occur in the domain of space and time , domains in which vector forces guarantee the efficiency of defensive strategies, which protect the airways, or clear the containment cavities of the bolus passed through them . These events and forces can interact in different ways.

In the domain of space, where forces are acting, the Pscore identifies the pathway and flow of the bolus: the pathway is identified by the direction along the digestive or respiratory tracts, as well as false route (penetration or aspiration); the flow is indicated by the amount of bolus that does not cross the pharynx while swallowing. This amount (expressed in a semi-quantitative manner by the score) represents the difference between the total amount of the bolus taken and the amount of bolus swallowed. In the domain of time, the score identifies events that occur before or after swallowing, considering that material pooling after a previous swallow becomes a bolus for the next swallow, with a volume that is either increased or decreased. Thus, the subsequent swallowing act can clear the residue or push it below into the airway . The P-score considers the sequence of swallowing acts in the "management" parameter, evaluating the fate and final amount of a bolus that persists in the pharynx/larynx after 5 empty swallows, and also gives information about the reaction of the patient to material pooling or to airway invasion. The occurrence, or absence, of dry swallowing, cough or throat clearing, in response to the residue or penetration/ aspiration before, during or after swallowing, express the interaction between vectors and volumes. In this way, the number of dry swallows or clearing activities, related to the final amount of material pooling, can be assumed to be a parameter of efficiency of the entire sequence, closely linked to fatigue of muscular swallowing effectors.

These preliminary considerations can help us in the interpretation of the score attributed by the judges. The anatomical parameter (Table IIIa: site) was well identified by the 4 judges: disagreement may have been influenced by the amount and site of the residue, considering that the most severe condition has to be selected for scoring (the residue in a lower anatomical site may have been chosen with respect to a larger volume, both parameters indicating greater severity, or vice versa). In this case, site and amount are closely linked: in fact, "amount" (Table IIIb) created greater difficulties for the judges. It is worth remembering that the measurement of this parameter is semi-quantitative, and the four judges attributed a diversity of scores, which in some cases were modified during the three observations, but in other cases remained unchanged. A greater variability was seen when making a distinction between scores 2 and 3, while "coating" created fewer difficulties in score attribution.

The third parameter (management) considers the effectiveness of the manoeuvre carried out to clear the residue, regardless of whether it was spontaneous or performed upon request, and in many cases it solves the dilemma between site and volume mentioned above. This parameter (Table IIIc) was easily evaluated by all judges and in all observations, with no discrepancies. If we consider the total score (from 4 to 11), which marks the continuum of clinical severity (Table IV), it can be observed that extreme scores were replicated by the four judges in all observations, with good agreement. This was easy to identify, and maintain judgement, for subjects without pooling or patients with bulky residues. Differences emerged in the medium classes of severity where the attribution of a different partial score may affect the attribution of a different class of clinical severity. In our work, these differences, sometimes replicated at the third observation, never reached statistical significance and can be assumed to be casual.

Conclusions

One of the most difficult tasks for clinicians and management team is to determine the severity of the swallowing disorder. This criterion very often refers to the risk for the lower airways to be invaded by the bolus or by material pooling into pharyngeal or laryngeal cavities. The instrumental criterion of severity (endoscopic or radiological) needs to be contextualised according to a more general clinical criterion of severity, which should make reference to the patient, considering that the non-instrumental assessment (CSE) tends to underestimate the risk of aspiration, whereas instrumental assessment tends to overestimate it . "Severity", in this case, becomes a relative criterion, which is quantifiable by the parameters that define it. In clinical practice, aspiration is the most significant event that marks a swallowing disorder, although it is not the only one.

Endoscopic and radiological evaluations are complementary techniques , even if the former offers an optimal view of pharyngeal and laryngeal cavities, and particularly of material pooling . Parameters related to instrumental severity, devised for radiological evaluation, might not be the best for application in the clinical endoscopic field, though they maintain their effectiveness in terms of interand intra-rater reliability . Nonetheless, scores that can be easily applied in clinical practice are needed. There are several endoscopic scores reported in the literature, with severity criteria divided into 3, 4 or 5 levels. This division does not seem to interfere with the inter- and intra-rater reliability of a score . The P-score, which considers anatomical and functional parameters, evaluates the interaction between volumetric, vectorial and temporal events, disengaging the criterion of severity from the quantitative parameter alone.

The high inter- and intra-rater reliability of the P-score was verified for anatomical, semi-quantitative and temporal parameters. Variability among judges, which was not statistically significant, was expressed with respect to the "site" and "amount", whereas "management" seems more easily determinable.

Despite the high reliability of the P-score shown herein, one critical point could be represented by the unequal subdivision of clinical severity in the sample, taken from consecutive outpatients seen in our department. The patients with the most severe clinical conditions were not able to manage all consistencies or volumes tested by those less severely affected. In the former patients, for example, it was possible to administer only a few small amounts of the bolus with a creamy consistency. Nevertheless, the statistical analysis verifies the reliability of the judges in attributing a score for each parameter of the P-score, regardless of severity, which derives from the sum of the three parameters that determine it. This should not affect the clinical use of the score, which, in contrast, seems to be able to identify patients with small differences in severity.

Further research is being undertaken to check the effectiveness of the P-score in clinical management of patients with swallowing disorders that have a different aetiology, as well as its usefulness in indicating variations after specific treatment.