Face-to-face versus online training for the interpretation of findings in the fiberoptic endoscopic exam of the swallow procedure.

OBJECTIVE: The aim of this study was to evaluate the comparative effectiveness of an online, interdisciplinary, interactive course designed to increase the ability to accurately interpret the fiberoptic endoscopic exam of the swallow (FEES) procedure to traditional, face-to-face (F2F) lectures for both graduate medical education (GME) and graduate speech language pathology (GSLP) programs.
DESIGN: This was a prospective, quantitative, nonrandomized study. Participants were medical residents in physical medicine and rehabilitation from two affiliated programs and graduate students in speech language pathology from two instructional cohorts at a single institution. Group 1, traditional group (n=51), participated in F2F lectures using an audience response system, whereas Group 2, online group (n=57), participated in an online, interactive course. The main outcome measure was pre- and post-course FEES knowledge test scores.
RESULTS: For Group 1, the mean pre-course score was 26.94 (SD=3.24) and the post-course score was 34.96 (SD=2.51). Differences between pre- and post-course scores for Group 1 were significant (t=-16.38, P≤0.0001). For Group 2, the mean pre-course score was 27.05 (SD=2.74) and the post-course score was 34.05 (SD=2.84). Differences between pre- and post-course scores for Group 2 were significant (t=-13.5, P≤0.0001). The mean knowledge change score for Group 1 and Group 2 was 8.01 (SD=3.50) and 7.04 (SD=3.91), respectively (nonsignificant, t=1.372, P=0.173), suggesting groups made similar gains.
CONCLUSION: Incorporating technology into GME and GSLP programs yielded comparable gains to traditional lectures. Findings support the use of online education as a viable alternative to the traditional F2F classroom format for the instruction of the cognitive component of the FEES procedure.

Introduction

Swallowing disorders are a significant health care concern as these disorders may result in further debilitating or life-threatening conditions. An undiagnosed or misdiagnosed swallowing disorder may also result in prolonged hospitalization following a stroke or other related illness subsequently increasing health care costs. To effectively treat swallowing disorders, a health care team first needs to correctly diagnose the underlying cause of a disorder through the appropriate use and interpretation of diagnostic procedures.

Swallowing disorders frequently occur in patients undergoing physical rehabilitation following a stroke, brain injury, or other complex medical conditions.17 Clinicians providing direct treatment for dysphagia in the medical rehabilitation setting typically work under the guidance of physical medicine and rehabilitation (PMR) physicians. PMR physicians require a strong working knowledge of all rehabilitation modalities, and upon completion of the PMR medical residency program, they are expected to lead and guide allied health professionals in a variety of areas including clinicians working with patients with swallowing disorders.7

The current standard of care for the evaluation of swallow function includes the videofluoroscopic swallow study (VFSS) and the fiberoptic endoscopic exam of the swallow (FEES) procedure.5 The VFSS is more popular as it has been available since the 1950s,17 and incorporates the use of an X-ray along with barium to evaluate the physiology of the swallow function. The FEES procedure is newer and was first reported by Langmore et al.15 The FEES procedure is an instrumental assessment of the swallowing function that uses nasal endoscopy to provide a real-time view of both the structures of the pharynx and the swallowing mechanism. Both the VFSS and FEES are valuable procedures for evaluating dysphagia and demonstrate good agreement with diagnostic findings as related to aspiration, laryngeal penetration, pharyngeal residue, and diet recommendations.21 Even though the FEES has been available since 1988, it is less often used in clinical practice due to limited training opportunities and cost of the equipment to perform the procedure.3

Physician training

The Accreditation Council for Graduate Medical Education (ACGME) requires all medical residents to demonstrate competency in system-based practice. One of the core expectations of the system-based practice is the ability to work in interprofessional teams and to demonstrate team collaboration.11 The care of patients with dysphagia offers a medical resident the opportunity to work in a collaborative manner to promote the patient’s swallowing safety and maximize treatment outcomes. Therefore, it is important for new PMR physicians to receive appropriate training on a variety of swallowing diagnostic procedures such as the FEES during a residency program.

To address the educational needs to promote the system-based practice, the ACGME of the American Medical Association has introduced six domains of clinical competency for residency physician training. One of the domains of clinical competency for physician training and education includes practice-based learning and performance improvement.20 The provision of a theory-driven, evidence-based training program focusing on the diagnosis of dysphagia to future PMR physicians would assist residency programs to meet and exceed these educational guidelines.

Speech language pathology

Given the advanced skills and knowledge required to perform and interpret the FEES procedure, speech language pathologists (SLPs) have faced many challenges with FEES training procedures.3 Guidelines available from the American Speech-Language-Hearing Association (ASHA) suggest a three-step process involving observations, practice under direct supervision, and independent practice with indirect supervision.1 The training approach recommended by the ASHA is consistent with the surgical training approach of the master-apprentice model where a trainee learns how to perform the procedure under the supervision of a qualified surgeon. However, with advances in surgical training techniques, the master-apprentice model is no longer appropriate for the novice learner, as it may place a patient at too high of a risk to accommodate for a trainee’s learning curve.27 Novice learners need to acquire knowledge of the procedure itself, knowledge of the potential pitfalls of the procedure, and appropriate invention strategies when a problem presents itself.27 The teaching of these behaviors is complex, and an interactive, online training program may provide a novice learner with the knowledge necessary to effectively interpret the procedure.

Another challenge facing many SLPs is access to a facility equipped to perform the FEES procedure, which also provides time required to develop the skills and expertise with interpreting the procedure.3 The lack of available resources and reduced mentoring capabilities has been associated with reduced self-confidence with dysphagia management in general.13

Training needs

Like training with many medical procedures, within the context of the entire FEES training paradigm, observation of live FEES procedure is a common component of the training. Observation has been recognized as a key teaching strategy to learn motor skills; however, its effectiveness with learning and correctly interpreting the FEES procedure has not been previously investigated. According to Bandura (1986),28 learning through modeling supports the notion that observation provides a mental template for practice. Further, theories of acquisition of psychomotor skills forwarded by Adams (1986)29 are built upon Bandura’s construct to suggest observations lead learners to develop a mental blueprint allowing them to compare it to experience and performance.

A challenge faced by the observation method alone is using only patient exam, where a learner is physically present, does not always provide a learner the opportunity to observe more uncommon clinical presentations. In addition, the observation of live examinations does not always lend itself to the learning process, as urgent patient care issues take precedence over learning and observation may not provide a learner with an engaging learning process.16,30

It is proposed that an interactive interpretation model, such as the online educational program with immediate feedback, would facilitate an actively engaged educational experience. An online training program, developed to complement live observational training, will provide a consistent format of salient clinical examples that the examiner may encounter in clinical practice. Further, a computer-based online FEES training program offers the advantage of immediate feedback and enhanced accessibility to training materials.

The purpose of this study was to compare the effectiveness of an online, interdisciplinary, interactive course designed to increase the ability to accurately interpret the FEES procedure to traditional, face-to-face (F2F) lectures for graduate medical education (GME) and graduate speech language pathology (GSLP) programs.

Methods

Course development

The objective of the course was to develop the cognitive skills required for the correct interpretation of findings observed during the FEES procedure to accurately diagnose the swallowing disorder as viewed by endoscopy.1 The cognitive component required both the knowledge of the correct sequence of steps to perform the procedure and the appropriate decision-making process.26 This study focused on the cognitive skill area of training needed to successfully interpret a FEES procedure with the development of an online training course and F2F lectures. Both the online and F2F course used the same educational content and underlying theoretical learning concepts (slides and video clips) and allowed for interactive responses.

The FEES training program included three major components required to accurately interpret the FEES procedure.1 The first component included a section on the identification and assessment of the anatomy and physiology of the laryngeal and pharyngeal structures as observed during a FEES procedure. The second component included a section on the assessment of the accumulated oropharyngeal secretion levels observed during the FEES procedure and the importance of accurately identifying a secretion level. The final component included a section on the physiology of the swallow as observed during the FEES and appropriate interpretation and interventions for an examiner to use in different clinical scenarios.

The online educational intervention was developed using Adobe Presenter 10.0. This software allows for interactive, online content that can include audio overlays on slides and video clips. The interactive online FEES training program provided participants an opportunity to evaluate various video examples, and based upon participant response, receive immediate written and auditory feedback on the accuracy of the response. Participants with the online course could view and pause the video examples as many times as needed. Participants in the F2F course used an electronic audience response system as part of a PowerPoint plug-in software. The same interactions on the online system were in place for the F2F course. The instructor for the F2F course allowed an opportunity for the participants to request as many views of the videos as desired. Course slides and speaker notes were the same for both the online and F2F course.

Previous FEES exams and select video clip examples representing various disorders and abnormalities were captured to develop the comprehensive FEES training. The database of video clips from which content was drawn included hundreds of FEES exams completed at the sponsoring hospital over the last 15 years. Based upon the exams available in the FEES database, the researchers selected various video clip examples and edited them to illustrate the educational points of interpretation of the FEES. Administrative approval and permission was obtained from the hospital to use the endoscopic images from patient exams as part of the educational material. This study was deemed exempt as a Category 2 project by the Marianjoy Rehabilitation Healthcare/Marianjoy Medical Group Institutional Review Board. The study participants provided written informed consent to use the de-identified results from the pre- and post-course assessment measures.

Underlying theoretical concepts of the course

The social cognitive theory (SCT), as it relates to the development of an online dysphagia evaluation course for SLPs and medical residents in PMR, was the underlying theoretical framework for the development of the educational program.2 The perceived self-efficacy construct of the SCT was selected as the structure for the online course, which was designed to improve performance with the interpretation of the FEES procedure. Given the known risk of swallowing disorders and the scarcity of training programs available for the FEES procedure, this training approach provided a theory-based training on how to evaluate swallow dysfunction using the FEES procedure.3,8,22 This study used the SCT construct to increase the self-efficacy of health care providers to accurately interpret the FEES results to diagnose dysphagia.

Study design and outcome measures

This was a prospective, quantitative, nonrandomized study. Participants were medical residents in PMR at two residency programs and graduate students in speech language pathology at the same university but in two different cohorts. The two study groups included the following: participants who completed the traditional F2F lecture using the audience response system (Group 1) and participants who completed the online, interactive course (Group 2). The participants represented a convenience sample as assignment to each group was based upon the participant’s academic cohort/program and was not randomized.

The traditional F2F group lecture lasted 2 hours and was completed on two separate occasions. The F2F group attended an in-person, instructor-led lecture. The instructor used the same lecture notes that were included in the audio component of the online course; however, minor variation of the course content may have occurred with the in-person, instructor-led F2F group given the nature of the format. The online group participants completed the online course in its entirety while proctored by the researcher. The course completion time for the online group ranged from 1 hour and 30 minutes to 2 hours and 30 minutes with the majority of course participants completing the course at approximately 2 hours (including the pre- and post-course assessment). The online training sessions were held at multiple times across multiple locations within the hospital, and participants signed up to complete the online course at a time convenient to their schedule.

The main outcome measures included a pre- and a post-course assessment using the FEES knowledge test developed specifically for this study. The FEES knowledge test contained 40 forced-choice items divided into three major sections related to the major components of the training course and procedure interpretation. The first section of the FEES knowledge test included 14 items on the anatomy and physiology of the laryngeal and pharyngeal structures. The second section of the FEES knowledge test included 10 items to evaluate the participant’s ability to rate accumulated oropharyngeal secretion levels using a standardized secretion scale developed by Donzelli et al.9 The third section of the FEES knowledge test included 16 items to evaluate participants’ ability to judge bolus flow for laryngeal penetration, aspiration, and pharyngeal residue during the pre- and post-swallowing segments. The questions on the FEES knowledge test were accompanied by video clips and still endoscopic images from FEES exams, which required course participants to interpret the images to answer the question accurately. The same video clips and endoscopic images were used in both the pre- and post-course tests but were different from the video clips included in the online course. The pre- and post-course knowledge tests were administered immediately before and after participating in the course. The pre- and post-course FEES knowledge tests were identical exams.

To evaluate for changes in self-efficacy to interpret the FEES procedure, course participants completed a pre-and post-course questionnaire exploring the self-reported perceptions about self-efficacy with conducting the FEES procedure. The pre-course FEES self-efficacy questionnaire was completed after participants took the pre-course FEES knowledge test. The post-course FEES self-efficacy questionnaire was again completed after participants had taken the course and completed the post-course FEES knowledge test. The FEES self-efficacy questionnaire was developed for this study and asked participants to rate statements on a 5-point Likert scale. A rating of 1 on the FEES self-efficacy questionnaire indicated the respondent “strongly disagrees” with the statement, whereas a rating of a 5 indicated the respondent “strongly agrees” with the statement. There were five items on the FEES self-efficacy questionnaire, and the questionnaire was designed to measure only self-efficacy as related to the various aspects of performing the FEES.

Validation of course content and outcome measures

The validation of the educational intervention and outcome measures was a two-part process. First, video clips selected were independently evaluated by two experienced FEES providers in relation to the concept being evaluated (e.g., aspiration). If a disagreement was present between the expert raters and the targeted concept, then the video clip example was removed from the course. For the purpose of this study, an experienced/expert FEES provider was defined as a person who has routinely used FEES in clinical practice over the last 5 years and has demonstrated scholarly activity on FEES (e.g., publications or presentations). The second part of the validation process included a pilot of the intervention with three clinicians who met the inclusion criteria for this study. Feedback was obtained from these novice clinicians regarding clarity of the content of the educational course, the pre- and post-course knowledge test, and the self-efficacy FEES questionnaire.

Data analyses

The independent variable for this study was the course delivery model (defined as either online or F2F). The dependent variables were the change in knowledge scores from the pre-to the post-course test and the change in self-efficacy scores from pre- to post-course. The number of correct responses on the pre- and post-course FEES knowledge tests represents a continuous variable; therefore, paired t-tests were used to evaluate intragroup differences, and independent t-tests were used to evaluate intergroup differences. It was proposed the FEES self-efficacy questionnaire represented a Likert scale versus Likert-type items; therefore, parametric tests were used.4 For the intragroup comparisons related to self-efficacy, a paired t-test was completed. To reduce the potential of a Type I error, the alpha level was set at 0.01.10 Statistical analyses were completed using the Statistical Package for the Social Sciences (SPSS) software version 24.0.

Results

Course participants

A total of 108 individuals participated in the FEES training course across the two groups: Group 1, n=51, participated in the traditional F2F lecture and Group 2, n=57, participated in the online course. In Group 1, 92.2% had not previously viewed an FEES procedure before the course, and in Group 2, 82.5% had not previously viewed the procedure (χ2=2.245, P=0.134). Regardless of the study group assignment, all participants reported they were at the novice provider level for the FEES procedure.

Twenty-five of the participants were PMR medical residents, and 83 were graduate students in speech language pathology. Among the PMR medical residents, 40% participated in the F2F course and 60% participated in the online course. Among the graduate students in speech language pathology, 49% participated in the F2F course and 51% participated in the online course. The differences in participation rates between the specific discipline (i.e., PMR versus graduate students in speech language pathology) and the course format (i.e., F2F versus online) were not statistically significant (χ2=.681, P=0.409).

Knowledge change

Table 1 summarizes the pre- and post-course knowledge scores for each group. The FEES knowledge test consisted of 40 forced-choice items divided into three sections (i.e., anatomy, secretion levels, and bolus flow) reflecting the three major components of the training course and to allow for subtest comparisons. The overall intragroup differences observed between the pre- and post-course knowledge scores were significant in both groups suggesting that both groups demonstrated meaningful gains in knowledge. The mean knowledge change score for Group 1 (F2F) was 8.01 (±3.50) and for Group 2 (online) was 7.04 (±3.91). The intergroup differences between the mean knowledge change scores for the two groups were nonsignificant (t=1.381, P=0.17) suggesting the groups made similar knowledge gains. Table 2 summarizes the knowledge change scores between the two groups for subtest comparisons of the three major components of the course (i.e., anatomy, secretion level, and bolus flow). The differences observed in the change scores for the subtest were not statistically significant suggesting knowledge gains for all three subcomponents of the course were similar between the F2F and online participants.

Table 1

Pre- and post-course knowledge scores

Group	F2F			Online
	Mean	SD	SEM	Mean	SD	SEM
Pre-course knowledge score	26.94	3.24	0.45	27.05	2.74	0.36
Post-course knowledge score	34.96	2.51	0.35	34.05	2.84	0.38
Significance	t=−16.38, P≤0.0001			t=−13.5, P≤0.0001

Note: The maximum score is 40 points.

Abbreviation: F2F, face-to-face.

Table 2

Subtest knowledge change scores

Subtest	F2F			Online			Significance
	Mean	SD	SEM	Mean	SD	SEM
Anatomy	4.01	2.28	0.32	3.70	1.99	0.32	t=0.773, P=0.441
Secretions	1.96	1.24	0.17	1.49	2.07	0.27	t=1.44, P=0.153
Bolus flow	2.22	2.38	0.33	1.56	2.39	0.32	t=1.42, P=0.157

Abbreviation: F2F, face-to-face.

Self-efficacy change

Changes in the course participants’ self-efficacy to interpret the FEES procedure were evaluated with an FEES self-efficacy questionnaire using a 5-point Likert response scale. Table 3 presents the FEES self-efficacy questionnaire along with the pre- and post-course self-efficacy ratings for each item by study group and for the entire study sample. Overall and by study group, there were statistically significant changes from pre- to post-course with all the self-efficacy items.

Table 3

Self-efficacy changes

Item		Group	Pre-course mean score	Post-course mean score	Significance
1.	I feel comfortable evaluating swallowing disorders using the FEES procedure	Overall	1.58	3.48	P≤0.00001
		F2F	1.30	3.44	P≤0.00001
		Online	1.81	3.51	P≤0.00001
2.	I feel comfortable with identifying anatomical landmarks of the larynx during the FEES procedure	Overall	2.54	4.21	P≤0.00001
		F2F	2.32	4.13	P≤0.00001
		Online	2.74	4.28	P≤0.00001
3.	I feel comfortable with identifying accumulated oropharyngeal secretions levels during the FEES procedure	Overall	1.93	3.80	P≤0.00001
		F2F	1.62	3.89	P≤0.00001
		Online	2.21	3.77	P≤0.00001
4.	I feel comfortable with identifying the presence of laryngeal penetration during the FEES procedure	Overall	1.87	3.82	P≤0.00001
		F2F	1.60	3.88	P≤0.00001
		Online	2.11	3.77	P≤0.00001
5.	I feel comfortable with identifying the presence of tracheal aspiration during the FEES procedure	Overall	1.95	3.83	P≤0.00001
		F2F	1.61	3.83	P≤0.00001
		Online	2.25	3.83	P≤0.00001

Abbreviations: FEES, fiberoptic endoscopic exam of the swallow; F2F, face-to-face.

Discussion

The overall results of this study revealed, regardless of the course delivery method (F2F versus online), participants in both groups made statistically significant increases in knowledge gains and self-efficacy after completing the FEES training course. Gains in both areas are critical for the development of clinical competence with performing the FEES procedure.31 The development of professional self-efficacy, especially for novice providers, has been shown to be an essential component for the development of clinical competence.31 As this online course provided the opportunity for the participants to integrate theory, applied practice, and analysis into the development of a targeted clinical competence for interpreting the FEES procedure, this may have accounted for the improvements observed with perceived self-efficacy.

While both course delivery methods yielded similar gains, the online course offered some additional advantage over the F2F course. First, the online course participants could select a learning time that was convenient to their schedule as multiple training times were offered and the learner could select a time convenient to his or her schedule. Learner readiness is an important component to the learning process, and allowing the flexibility with GME and GSLP programs for the cognitive component of a skill may result in improved learning efficiencies. Second, the online participants could also control the media in the course and control the pace of the course versus having to ask the instructor to replay the video clip. Adult learners are generally more autonomous,14 and providing the learners in the GME and GSLP program the opportunity to review the various media clips at a self-paced rate in the online course may enhance the learning engagement to make it a more active learning environment.12 Per the US Department of Education report,19 online learning may be more effective when the learner is provided with control over the media content in the course. In addition, the online course offered the advantage of complete standardization of the course content. While attempts were made to standardize the F2F content, some minor variation in the exact content of the lecture was expected given the live classroom format.

Furthermore, an online course in conjunction with a flipped classroom format offers the advantage of using technology for instruction on the cognitive component of the FEES procedure while reserving the F2F training time for instruction on the motor component of the FEES procedure for GME.18 Even though this study did not specifically investigate the effectiveness of a flipped classroom format, the findings of this study may provide additional evidence to support the use of a blended learning approach in GME, such as the flipped classroom, when using technology with medical education as it may create a more efficient and effective learning environment.24 When using the flipped classroom approach for instruction with the FEES procedure, an additional component to the training could also include instruction on implementing swallowing interventions during the FEES procedure. This current study focused only on the diagnosis of the swallowing disorders and did not address treatment options and swallowing interventions.

Finally, the results of this study revealed both the PMR physician resident group and the SLP group demonstrated improvements in their ability to interpret the FEES procedure. The ACGME, which governs medical residency training, focuses on a competency system-based practice. One core expectation of system-based practice is the ability for the medical residents to work in interprofessional teams.11 This course was designed as an interprofessional educational learning activity intended for both PMR medical residents and SLPs. The interprofessional educational approach provides an opportunity to share knowledge through a common learning experience with the ultimate goal to improve the care provided to patients.6

One potential benefit of this course was the increased knowledge for the participants in both groups may help promote a collaborative working environment for the diagnosis of dysphagia when evaluated by the FEES procedure. Dysphagia is a complex medical issue, and as with many types of complex health issues, it requires more than one discipline to effectively address the health issues associated with it.6 While the design of this current study did not focus specifically on the effects of interprofessional education as related to the outcomes of direct patient care, this suggests one direction for future research.

A recognized limitation of this study is when providing educational interventions to high-level participants (e.g., medical residents and graduate students), there should be an expected benefit of knowledge gains merely by means of participation. Even though this may be a correct assumption, the use of various educational methods to develop the course regardless of the delivery method (F2F versus online) may also have contributed to the increased knowledge and self-efficacy demonstrated. For example, the validation process used to develop the course content and assessment tools provided an avenue to fully vet the course material prior to its implementation. Designing a course that used independent raters to validate the intended educational concepts of the video clips created a more robust process for the course content, regardless of the delivery method. Another example was both course delivery methods used technology to create an authentic environment for effective learning by using video clip excerpts from previously completed FEES exams and incorporating interactive technology to provide immediate feedback. This may have facilitated the learner to acquire new learning needed to successfully interpret the FEES procedure and is consistent with previous reports associated with the advantages of using technology.23 A third and final example that may account for the knowledge gains demonstrated by the participants in both course delivery methods may be the course design incorporating both a partial-task and a whole-task approach for the interpretation of the FEES procedure. The course was created to advance the learner from a simple, single component of the FEES procedure tasks to a higher, more complex whole-task approach to include the entire FEES procedure. By ordering the instruction in a hierarchy of partial tasks into the final task of a complete evaluation, the learner could develop a blueprint of the necessary components required for learning how to interpret the FEES procedure. Previous research using simulation-based training has demonstrated the effectiveness of using both partial-task and whole-task approaches to learning procedures with health care providers.26

The second limitation of this study was the self-efficacy items were developed for this study and were not a standardized outcome tool. However, given the specific nature of the FEES self-efficacy items, there was no previously published, standardized scale available for this study. Therefore, pilot participants were essential in the development of this online course as they provided an evaluation of the self-efficacy questionnaire within the context of the course development.

A third limitation identified was the study design did not provide any mechanism to evaluate if changes in self-efficacy resulted in actual improved clinical competence. That is, this study only measured the dimension of self-efficacy and knowledge gains as related to the FEES procedure. Adams and Ewen (2009)32 described the importance of educational interventions being designed to measure multiple dimensions of the activity including self-efficacy, knowledge gains, and actual performance. This finding would be an indication for future studies to investigate the relationship between self-efficacy, knowledge gains, and actual performance of the procedure including both the cognitive and motor components of the procedure. As reported by Stump et al,25 in addition to a learner having self-efficacy to perform a task, the learner must also be able to make an accurate estimation of their true ability or in other words correctly calibrate their perception of self-efficacy.

Conclusion

Incorporating technology into GME and GSLP programs yielded comparable gains as those of traditional F2F lectures. The findings of this study support including theory-driven and various educational design concepts into course development for interdisciplinary education endeavors. The use of technology in GME and GSLP programs can provide efficiencies for the learning process without compromising the effectiveness.