Reliability and validity of the neonatal feeding assessment scale (NFAS) for the early identification of dysphagia in moderate to late preterm neonates.

BACKGROUND: A clinical feeding assessment instrument to assist with early identification of oropharyngeal dysphagia (OPD) in neonates was developed.
OBJECTIVE: To investigate the validity and reliability of the Neonatal Feeding Assessment Scale (NFAS) in comparison to the modified barium swallow study (MBSS) as gold standard.
METHOD: A within-subject design was implemented. A group of 48 late premature neonates (mean gestational age 35.5 weeks) were sampled in the neonatal intensive care unit.
RESULTS: The NFAS consists of six subsections, including physiological stability, infant state, stress cues, screening of muscle tone and control, oral peripheral examination and feeding/swallowing assessment. 93% of participants (14/15) received confirmatory diagnosis of OPD on MBSS. The NFAS presented with high sensitivity (78.6%) and specificity (88.2%) scores. The positive predictive value was 78.6%. Subsequently the accuracy of the NFAS to identify the presence of OPD accurately was 85.4% when compared to MBSS. Inter-rater reliability was determined on 35% of the sample. The inter-rater agreement on overall instrument outcome was substantial beyond chance.
CONCLUSION: The NFAS may be of use to clinicians to support the early identification of OPD in this population, especially in resource constrained settings working without access to MBSS and to reach under served neonates.

Introduction

Neonatal dysphagia is a complex condition and is caused by a variety of underlying etiologies.1,2 The condition is associated with multiple medical problems such as bronchopulmonary dysplasia (BPD), congenital anomalies of the heart and gut, necrotizing enterocolitis, gastroesophageal reflux disease (GERD), prematurity, low birth weight (LBW) and small-for-gestational age (SGA).1 Clinicians should accordingly consider the complex interplay between various medical conditions along with associated risk factors and the evolving nature of dysphagia over time, in medically fragile neonates. An increase in the incidence of neonatal dysphagia in Africa and globally, could be attributed to a variety of factors such as improved survival rates of infants with medically complex conditions, improved identification of feeding and swallowing difficulties, expansion of the medical field of speech-language pathology within the neonatal intensive care unit (NICU), improved diagnostic ability with modified barium swallow studies (MBSS) and increase in skilled clinicians managing feeding difficulties in high-risk neonates.1–3 In the case of premature neonates, the immature digestive and respiratory systems of the neonate contribute to immature feeding skills, while essential medical management and comorbidities further contribute to the interruption of feeding development.1,4 Since it is possible to effectively bypass the oral feeding route in premature neonates by means of enteral and tube feeding, OPD may be a hidden problem and only receive attention once these neonates have difficulty transitioning to oral feeding when medically stable.

Neonates with OPD are at risk of a compromised nutritional status, slow weight gain, regulatory problems, later behavioural difficulties and developmental delays.6–9 OPD subsequently leads to increased healthcare costs and length of hospital stay.1,10 When OPD is not identified early via valid and effective assessment, it may be difficult to utilize available resources optimally in the presence of constraints in developing countries. Healthcare funding and physical as well as human resources should be utilized fully during assessment and treatment of neonatal dysphagia to ensure timely, cost-effective services. As such there is a need for valid and reliable assessments in the area of paediatric feeding difficulties.

In African countries where there may be limited access to MBSS or where a neonate is not medically stable to undergo instrumental assessment procedures, reliable clinical identification of OPD is required to provide effective and timely intervention. In turn, early identification and intervention may increase oral feeding opportunities, and decrease cost related to long-term medical and rehabilitation services. A valid instrument to address early identification of OPD remains unavailable for the neonatal population.3 Development of the Neonatal Feeding Assessment Scale (NFAS)16 began in response to the need for an efficient, objective, and clinically valid means, to reliably identify OPD in high-risk neonates.

Consequently, the research question for the current study was: ‘Is the NFAS a valid and reliable assessment instrument for the early identification of OPD in premature neonates?

Methods

Objectives

The objectives were a) to describe the feeding and swallowing assessment outcomes of the participants on the MBSS and the NFAS; b) to determine criterion validity regarding the psychometric properties of the NFAS, namely specificity, sensitivity, accuracy and predictive values; and, c) to determine inter-rater reliability of te NFAS.

Design

A comparative within-subject design18, where all participants were exposed to the same assessment procedures, was used to determine the psychometric properties of the NFAS on a group of high-risk neonates. The NFAS and MBSS results were compared concerning outcome for accurate identification of the presence of OPD.

Participants

Forty eight neonates admitted to a 29 bed NICU at a tertiary academic hospital in Gauteng, South Africa were purposely sampled. The inclusion criteria were: reported feeding difficulties, age range from 32 weeks gestational age (GA) to full term, medically stable for clinical and MBSS assessment as declared by the treating physician. Verbal or written informed consent was obtained from all the mothers. The information brochure and informed consent were available in three official languages of South Africa (Afrikaans, English and Setswana).

Table 1 indicated that the majority of participants presented with a >10 day duration of stay in the NICU (91.7%, n=44), LBW (85.4%, n=41) and late preterm birth (64.6%, n=31; mean GA of 35.58 weeks). Additional data from the case history and review of medical records highlighted numerous risk factors associated with neonatal feeding difficulties and dysphagia.1,14 These risks were: hyperbillirubinemia (62.5%, n=30), delayed introduction of oral feeding (60.4%, n=29), respiratory distress syndrome (RDS) (47.9%, n=23) and exposure to HIV in utero or during birth (10.4%, n=5).

Table 1

Participant description (n=48)

Participant characteristics	Mean	Median	Mode	Standard Deviation (SD)
Gestational age at birth (weeks)	35.58	35.0	34	3.06
Birth weight (grams)	2118	1960	1400	718.5
Corrected age at assessment (weeks)	26.96	36.85	35.00	2.92
Number of days in NICU	9.52	7.00	4	8.71

Materials

Medical records and parental interviews were used to obtain additional information. The NFAS16 and a MBSS checklist was developed for use in this study.19 The philosophy underlying the NFAS and a detailed description of the instrument is available in previously published articles.

NFAS sections and scoring

The NFAS consists of six sections to support the clinical assessment of neonatal feeding skills to identify the presence or absence of OPD.19 The six sections of the NFAS were scored using a binary system.19 The different items are clear descriptions of observable behaviours, thereby prompting the clinician about behaviours to evaluate - see Appendix A for examples of items included in the NFAS. The scoring instructions were provided in each section to reach a composite score when the NFAS was completed.19 The composite score indicated if OPD was present or absent.19 Clear administration guidelines are provided for all items.19 The sections of the NFAS is depicted in Figure 1.

Figure 1

Sections of the NFAS assessment instrument

MBSS material and apparatus

The MBSS checklist developed for this study allowed the raters to indicate which stage of swallowing -oral, pharyngeal, and/or oesophageal- was affected.26,30,37 The rater also indicated presence or absence of penetration or aspiration during the pharyngeal stage. In this study dysphagia was defined broader than only the presence of penetration or aspiration. A recent more comprehensive definition of dysphagia by Dodrill and Gasa43 was adopted for diagnosis of OPD in this study. The aforementioned authors defined dysphagia as “any disruption to the swallow sequence that result in a compromise of the safety, efficiency, or adequacy of nutritional intake” (p.24).43 The two raters evaluated the MBSS for the presence of signs of dysphagia according to provided criteria. In the oral stage the following signs were indicative of oral dysphagia: excessive anterior milk loss, disorganized lingual stripping, weak sucking and incoordination of the suck-swallow-breathe (SSB) sequence.26,20,37,40,42 During the pharyngeal stage the raters considered the presence of delayed elicitation of the pharyngeal swallow response, inadequate epiglottic inversion, laryngeal penetration, tracheal aspiration, cough in response to penetration/aspiration, resultant inadequate airway protection related to incoordinated suck-swallow-breathe (SSB) sequence, inadequate vocal fold adduction, pooling in the valleculae or/and pyriform sinuses, as well as nasopharyngeal reflux as signs of pharyngeal dysphagia.23,26,30,37,40,42–43 In the esophageal stage the presence of GERD indicated ED.1 The MBSS was performed using a fluoroscope (SYSCO 19” version Multi DiagnostEleva FD screening machine from Philips, Netherlands) with DVD recording capabilities.

Procedures

Before any research was conducted at the tertiary hospital, clearance was obtained from the Research Ethics Committees in the Faculties of Humanities and Medicine at the university and the tertiary academic hospital in Gauteng, South Africa. Informed consent was provided by all the mothers. A parental interview was completed, followed by a breast/bottle feeding assessment (NFAS), and lastly a MBSS. During the MBSS procedure, a solution of barium sulphate was reconstituted by mixing the powder (E-Z-HDTM) with the 50 ml of the mothers' expressed breast milk or recommended infant formula. The participants were fed by one of the blind raters. Fluoroscopy ran during the initial five to 10 serial swallows and when dysfunction was observed. During fluoroscopy the continuous mode with appropriate collimation was used to limit radiation exposure but still obtain the clearest view of the bolus procession.44–45 A frame capture rate of 30 frames per second was used.44–45 The maximum duration of radiation exposure was 3 minutes.44 A NUK MedicPro First choiceTM 120ml infant bottle with a MedicProTM disposable TPE Teat size 1 was used. Participants were positioned at a 45 degree upright angle with appropriate supported seating in a Tumble Forms 2 Feeder SeatTM (Jackson, MI). The MBSS was viewed in the lateral projection. The neonate's feeding and swallowing abilities were assessed with MBSS within seven days (mean=2.25) of the clinical assessment. Recorded studies were viewed and interpreted by two senior hospital speech-language pathologists blinded to the clinical outcome of the NFAS. The first view was in real time, followed by slow motion and frame-by-frame analysis after the MBSS was concluded.

Data analysis

The inter-rater reliability on the NFAS was determined using Cohen's Kappa coefficients and P Bar calculations.17 As in other clinical studies of instrument development, a Kappa value of greater than 0.41 was considered a minimal reliability criterion and a P Bar value of 0.50.17,46 Criterion validity of the NFAS outcome in comparison to the diagnosis obtained on MBSS was determined by calculating sensitivity, specificity, positive and negative predictive value indicators and accuracy scores.17

Results

NFAS results

OPD was identified in fifteen participants (31.3%) and 33 participants (68.7%) did not meet the criteria to be identified with OPD on the NFAS (Table 2). Signs and reported symptoms of oral and possible pharyngeal dysphagia could be documented on the NFAS, but pharyngeal and esophageal stage difficulties could not be confirmed without instrumental assessment.

Table 2

Comparative assessment results (n=48)

Assessment instruments	OPD present	OPD absent
1. NFAS	31.3% (n=15)	68.7% (n=33)
2. MBSS	29.2% (n=14)	70.8% (n=34)
Total agreement between assessment instruments	93.3%	97.1%

MBSS results

The MBSS results and the NFAS results are presented together in Table 2 to enable comparison between the results.

In the MBSS sample, 14 of the neonates presented with OPD (29.2%) and 25 presented with ED. Nine of the participants presented with no dysphagia. Co-occurrence of OPD and ED was present in 28.5% (n=4) of the participants. The total agreement between the NFAS and MBSS on accurate identification of OPD was 93.3%.

Comparative results of the NFAS and the MBSS Validity

Table 3 provides the data related to the criterion validity of the NFAS.

Table 3

Comparison between the gold standard (MBSS) and the NFAS (n=48)

		Outcome of MBSS (n=48)		Total neonates in which OPD is present/absent on NFAS
		OPD present	OPD absent
Outcome of NFAS		True Positive (TP)	False Positive (FP)
(n=48)	OPD present	11	4	15
	% NFAS	73.3%	26.7%	100%
	% MBSS	78.6%*	11.8%	31.3%
		False Negative (FN)	True Negative (TN)
	OPD absent	3	30	33
	% NFAS	9.1%	90.9%	100%
	% MBSS	21.4%	88.2%*	68.8%
Total neonates in which OPD is present/absent on MBSS	Count	14	34	48
	% NFAS	29.2%	70.8%	100%
	% MBSS	100%	100%	100%

Sensitivity and specificity are indicated in bold.

As evident from Table 3, a sensitivity score (true positive) of 78.6% was obtained with specificity (true negative) determined to be 88.2% for the NFAS. The data demonstrated that one false positive (11.8%) was rendered by the NFAS, which could possibly be ascribed to the set inclusion criteria. The predictive ability of the instrument incidentally achieved exact agreement with the sensitivity and specificity. The positive predictive value was 78.6% and the negative predictive value was 88.2%. The subsequent accuracy of the NFAS was 85.4% when compared to the MBSS outcome. The NFAS therefore presents with high sensitivity, specificity, good predictive ability and good accuracy for identification of OPD during clinical assessment.17

Reliability

Inter-rater reliability was determined for each section of the NFAS and for diagnosis for 35.0% of the sample, utilizing two raters. The results of each section and overall agreement on diagnostic outcome together with the asymptotic standard error (ASE) are depicted in Table 4. According to Table 4 results of three of the five sections on the NFAS reached a minimally acceptable level of agreement between two independent raters. However, four of the five sections had an acceptable P Bar level of agreement. Substantial agreement beyond chance (0.586 P Bar) was achieved between the two raters on the identification of OPD with the NFAS resulting in an acceptable ASE of 0.211.17

Table 4

Inter-rater reliability for each section and overall diagnostic outcome of the NFAS (n=17)

NFAS section	Kappa	Level of agreement	P Bar	Overall agreement between raters (%)	Asymptotic Standard Error (ASE)
A & B	0.062	Poor/chance agreement	0.764	76.4% agreement	0.044
A Physiological subsystem functioning
B State of alertness during feeding
C Stress cues during feeding	0.212	Slight agreement	0.176	17.6% agreement	0.141
D General movement & muscle tone screening	1.00	Perfect agreement	1.00	100% agreement	0.000
E Oral peripheral evaluation	0.628	Good agreement/Good agreement beyond chance	0.650	65% agreement	0.193
F Clinical feeding & swallowing evaluation	0.485	Fair agreement/Good agreement beyond chance	0.529	52.9% agreement	0.222
Total (Diagnostic outcome of NFAS)	0.598	Substantial agreement	0.586	58.6% agreement	0.211

Discussion

The purpose of the current study was to investigate the validity and reliability of the NFAS to determine if this instrument may be useful for the early identification of OPD in premature neonates. Such early identification may decrease the economic and social burden in lower and middle income countries such as South Africa to support the overwhelmed public health care system.54–55 Due to the possible life threatening nature of OPD in neonates, a valid clinical assessment instrument should be available to clinicians for use in the NICU.10

Validity and reliability of the NFAS in comparison to MBSs

The NFAS showed to be sensitive, specific, accurate and reliable to identify signs of OPD in the target population of this study. The diagnostic agreement between the NFAS and MBSS was very good17 (85.4%), indicating that the presence of OPD can be identified with the NFAS. Co-occurrence of OPD and ED is common in premature neonates due to the immature respiratory system, uncoordinated SSB sequence, and the high prevalence of gastro-oesophageal reflux all of which impact the different stages of swallowing.52

An unexpected result was that none of the participants demonstrated penetration or aspiration during the MBSS. This surprising finding could not be explained in light of other studies' findings where different prevalence rates of penetration and/or aspiration in preterm infants were reported. A wide range of penetration/aspiration prevalence rates are reported in various studies, ranging from 17.1% – 52.2%.53–56 The absence of penetration/aspiration in this sample does not rule out the presence of a continued risk of aspiration in future since the MBSS is a limited view of feeding performance at one point in time. Resilience of the airway's protective mechanism may already be evident in these late preterm neonates. Furthermore, the MBSS procedure is shorter than a typical feeding session therefore the impact of fatigue on SSB during the instrumental assessment could be limited.

A prevalence range of 25–35% for OPD in preterm and LBW neonates has been reported in some studies.50,57 The prevalence of OPD of 29.2% in this study concurs with previous research on this population.

Clinical use of the NFAs

The NFAS could be considered valid and reliable for clinical use in identifying the presence of OPD in late preterm neonates with risk factors such as HIV exposure, RDS, LBW and increased length of NICU stay. In a South African study by Pike et al., intrauterine growth restriction associated with SGA and an extended stay in the NICU was associated with OPD and ED in the same sample of participants.59 The results of the current study also found that physiologic immaturity is a contributing factor to neonatal dysphagia.

The NFAS is less invasive than MBSS and does not result in radiation exposure. The MBSS offers an observation of a discrete moment in time of the neonate's swallowing ability. Whereas the NFAS may be used more than once a day or in short succession to obtain a representative feeding profile. One of the main advantages of the NFAS is that it can be used in developing countries where less or no access to MBSS is available or while awaiting MBSS at another facility while the neonate is not medically stable to be transported. A notable feature of the NFAS is that assessment is guided by developmental supportive guidelines established for neonatal practice.10

Conclusion

The early assessment and timely management of OPD in preterm neonates is a priority since successful feeding with adequate weight gain is a discharge requirement from the NICU.1 The NFAS may offer valid early identification of OPD together with descriptive information that can support intervention planning in resource constrained settings. The NFAS enables clinicians to categorize the different signs of OPD in five categories, namely those related to physiologic instability, stress, state, level of alertness and structural and functional limitations impacting on feeding. The NFAS is likely to provide a more in-depth description of the neonate's feeding abilities than can be achieved with instrumental assessment alone. Despite the subjective nature of the NFAS, it offers a description of the signs of OPD and oral feeding competencies displayed by the neonate. Further independent research of other psychometric characteristics of the NFAS should be explored to determine test-retest reliability and responsiveness related to effect-size. This type of clinical instrument holds potential for providing a means for the early identification of OPD in settings without access to instrumental assessment, and may positively impact on service delivery to underserved high-risk neonates with OPD.

Table 5

Inter-rater reliability for each section and overall diagnostic outcome of the NFAS (n=17)