Effectiveness of a new interactive web teaching material for improving lung auscultation skills: randomized controlled trial for clinical nurses.

We developed a new interactive web-based teaching material to improve lung auscultation skills. Our objective was to investigate the effectiveness of the web-based teaching material on nurses with less than one-year work experience, using a prospective, open-label, stratified block randomized controlled trial. Of the 69 participants, 23, 22, and 24 participants were assigned to the web-based, paper-based, and control (with no intervention) groups, respectively. Using a simulator, a discrimination test on seven lung sounds, such as "normal," "wheeze," "rhonchi," "coarse crackles," "fine crackles," "left lung diminish," and "right lung absent," was conducted. Next, a post-test was conducted after one-week of training. Answers with formal names were considered "correct"; those with common names, misspellings, and without left and right parts were considered "insufficient"; and wrong answers were considered "incorrect." The control group showed no significant difference between the pre-test and post-test for any lung sounds. The paper-based group showed significant improvement in performance for "wheeze" (p=0.004) and "coarse crackles" (p=0.035). The web-based group showed a significant improvement in performance for "fine crackles" (p=0.026). The number of correct answers in the post-test was higher in the paper- and web-based groups than the control group (p=0.023). The web-based teaching materials that we had developed effectively improved the ability of new graduate nurses to auscultate lung sounds. Additionally, the results suggest that the combined use of web- and paper-based teaching materials may be more effective since the sounds that each method enhanced their ability to auscultate different lung sounds.

INTRODUCTION

Physical assessments are an integral part of nursing practice. In particular, auscultating “lung sounds” is something nurses perform daily. Accurately identifying lung sounds is a skill that they must acquire.[1] In basic nursing education in Japan, “normal lung sounds,” “asymmetry of lung sounds,” and “identification of abnormal lung sounds” are listed as minimum skills needed during the physical assessment of the respiratory system.[2] However, in clinical practice, the means of improving lung sounds auscultating ability depends on the experience and effort of individual nurses. The evaluation of the ability is subjective. Even active physicians are unable to properly auscultate all lung sounds.[3] It has been reported that the accuracy rate of identifying “fine crackles” is low.[3] A study conducted on home care nurses in Japan reported that they have problems in “discriminating between normal and abnormal lung sounds” and “determining the type of abnormal lung sounds”.[4] In addition, experienced nurses do not have significantly higher skills in auscultating lung sounds, compared to inexperienced nurses.[5] That is, clinical experience does not necessarily lead to the acquisition of lung auscultation skills. Therefore, nurses and nursing students need to be trained in lung auscultation skills. To train them properly, it is important to understand their current abilities in auscultating the lung.

Contact with patients is essential for nursing education. However, clinical settings offer limited opportunities to experience normal and abnormal lung sounds. For this reason, simulators are often used in nursing education. Nurses and nursing students practice using simulators to identify abnormal lung sounds. However, there are several limitations to their use, such as the cost of purchase and maintenance, the limited number of people who can use them at any one time, and the need to have skilled instructors.[6] Besides, the limited number of training sessions does not significantly improve the students’ lung auscultation skills.[7] One way to overcome this situation is to use web-based learning. It has already been integrated into various areas of medical education. However, it is not clear whether e-learning is any more effective than traditional learning in improving the skills of medical professionals.[8]

In general, it is possible to auscultate lung sounds online using web pages, web materials, and applications.[9-12] In Japan, there are some web pages with questions based on the playback of recorded lung sounds, but all of them use multiple-choice questions. There are no web applications in which learners can select a part of the lung and auscultate the lung sounds in that part. Moreover, these educational materials require the purchase of special equipment and are, therefore, inaccessible to many nursing students or new nurses. Other limitations of such application-based teaching materials are that they may simply involve auscultating played lung sounds, or even if they have a test function, it may be in a multiple-choice format. Learners of lung sound auscultation have various levels of auscultation ability, as they range from nursing students to clinical nurses. In order to teach this skill step by step, it is important to be able to properly assess individual learners’ abilities and create questions to match them. Therefore, we developed a new web-based teaching material. This tool is designed for nurses and nursing students, to provide training materials according to their learning stage. The lung sounds in the web materials are owned by co-author Yamauchi, CD sources, and sounds are from the simulator “Lung” (Kyoto-Kagaku in Japan).[13]

The study aimed to verify the effectiveness of the web-based materials in improving the auscultating lung sound ability of nurses and nursing students, targeting newly graduate nurses. This study also aimed to identify what types of lung sounds new graduate nurses are not good at auscultating, and why they identify them wrong. The results of this study will be useful for improving the quality of basic nursing education.

METHODS

Design

This study was a prospective, open-label, multicenter, three-arm parallel-group, stratified block randomized controlled trial. The intended users of this web-based teaching material are nursing students and nurses. However, because of possible skill differences depending on experience, our target population was new nurses with less than one year of experience. There were no other selection conditions. Snowball sampling was conducted to select target hospitals. Three hospitals consented to participate in the study. Participants were assigned to three groups: web-based, paper-based, and control. The allocation ratio was nearly 1:1:1. Participants were recruited from April 2019 to April 2020.

Web-based material

This web-based teaching material was developed to enable the users to learn and evaluate their lung auscultation skills at anytime and anywhere. To eliminate the limitations of location and time, we developed the material considering the use of smartphones. For learners to be able to acquire these skills, they must receive evaluations of them. Therefore, we developed a test that permits the learners to evaluate their respiratory auscultation skills. Compared to the conventional web-based teaching materials,[9-12] our teaching material has several advantages, such as the wide variety of tests that can be created. Questions can be created not only for beginners—in which lung sounds can be played back and answers are provided in response to the choices—but also for advanced learners—in which they can select an area for auscultation on an illustration of a patient’s upper body and provide the answer for each area. Additionally, since this web-based material contains multiple sound sources for a single lung sound, the instructor can evaluate the learner's auscultation skills using multiple sound sources for a single lung sound. This is important since lung sounds can vary between individuals. Furthermore, the system was developed so that it can be used on a smartphone, to meet the needs of the current learners, without any problems. This means that they can study without being restricted by location or time. Test results are stored on the learner’s account, so they can check their results. Learners can check the explanation and lung sounds by tapping “Review Question” for the questions they get wrong. There is also a function that can provide feedback in response to learner taps. However, since this material is intended to be used on smartphones, the explanations are simpler than those in paper-based materials, and are only displayed when learners tap to expand them.

Paper-based material

The paper-based material includes excerpts from the Lung Sounds Auscultation Guidebook[14] on the seven lung sounds to be auscultated pre- and post-test: “normal,” “wheeze,” “rhonchi,” “coarse crackles,” “fine crackles,” “left lung diminished,” and “right lung absent.” They use figures to explain the mechanisms of normal and abnormal lung sounds and how to auscultate them.

Data collection procedures

In the first phase, 72 nurses participated in the study (13 from Hospital A, 9 from Hospital B, and 50 from Hospital C) as shown in Fig. 1. All participants took a performance pre-test on the Physical Assessment Simulator, “Physiko” (Kyoto-Kagaku in Japan), in each hospital. They auscultated lung sounds on “Physiko” for one minute and they were given another minute to write the answer in the formal name of the lung sounds. They answered seven questions. To make assignments, the results of the pre-test were scored as follows. One point was given for answers that used the official name,[15] 0.8 points for those that contained common names and misspellings, and 0.5 points for those that did not mention whether it was the left or right lung. Incorrect answers were given 0 points. In each hospital, the participants were arranged in order of their pre-test scores, and groups of three individuals were formed according to the descending order of the scores. Next, as shown in Fig. 2, the groups were arranged according to the order of the highest score by hospital, and the pair numbers of Hospital A were assigned from [1] to [3], those of Hospital B from [4] to [7], and those of Hospital C from [8] to [23]. Members with a group number of 3n+1([1], [4], [7]…) were allocated to the web-based, paper-based, and control groups in the order of the highest score. The 3n+2 group([2], [5], [8]…) members were in the paper-based, control, and web-based groups in the order of the highest score. The 3n group([3], [6], [9]…) members were allocated to the control, web-based group, and paper-based groups in the order of the highest score. In Hospitals A and C, where the number of participants was not a multiple of three, one or two participants with median scores on the pre-test, respectively, were selected and assigned to the web-based, paper-based, and control groups again, based on the order of their scores. After the pre-test, members of the web-based group were given identifications and passwords for the web-based teaching material. Members of the paper-based group were given paper materials from a Japanese textbook.[14] A post-test was done one week after. After the post-test, members of the paper-based and control groups were given identifications and passwords. They were also allowed to use the web-based materials for one week after the post-test, like the members of the web-based group. When the posttest, participants asked learning time using each material.

Fig. 1

Participants’ flow

Fig. 2

Participants’ assignment

Pre-test and post-test

Pre-test and post-test consisted of seven questions. Seven kinds of lung sounds using “Physiko” were to be auscultated. The volume of “Physiko” was set to maximum. The sounds were “normal,” “wheeze,” “rhonchi,” “coarse crackles,” “fine crackles,” “left lung diminish,” and “right lung absent.” These seven kinds of lung sounds were played randomly, and each question required one auscultation, and the answer had to be written in the following one minute. We allowed them to auscultate again within one minute and allowed them to answer within one minute. However, rewriting after moving to the next question was prohibited.

Analysis

Answers with the formal names were considered “correct,” those with common names, misspellings, and without left and right parts were considered “insufficient,” and wrong answers were considered “incorrect.” Since each kanji has a different mean, we thought that typographical errors were insufficient for nurses. In clinical practice, errors in describing the left and right side of the lungs when making findings of lung sounds have been pointed out.[16] Therefore, no description on the left and right were insufficient for nurses.

Difficulty level by lung sound

Cross-tabulations along with chi-square tests were conducted to compare the difficulty level of each lung sound.

Change in results for each lung sound in the control, paper-based, and web-based groups

Cross-tabulations and Wilcoxon signed-rank tests were conducted to compare the results of the pre-test and post-test of the seven lung sounds in each of the three groups.

Comparison of the number of correct answers in the control, paper-based, and web-based groups

The Kruskal-Wallis test was used to compare the number of correct answers in the pre-test and post-test among the three groups. The Mann-Whitney U test was used to compare the control group with paper- and web-based groups, and Bonferroni’s Correction was applied. Also, Wilcoxon signed-rank test was used to compare the number of correct answers in the pre-test and post-test for each of the three groups.

The significance level for statistical analysis was set at 5%, and all tests were two-tailed. The IBM Statistical Package for the Social Sciences (SPSS) software version 27.0 was used for statistical analysis.

Ethical considerations

This study protocol was approved by the Nagoya University Medical School Ethical Review Board (Approval no.17-150). The participants were given a written explanation of the study, and they could choose to continue or withdraw from the study at any time.

RESULTS

Research participants

Fig. 1 shows the flow of the participants. There were 13, 9, and 50 participants from Hospitals A, B, and C, respectively. Of these, two participants from Hospital A (one from the web-based group and another from the paper-based group) and one participant from Hospital C (from the paper-based group) terminated their participation in the middle of the study due to personal reasons. Therefore, the final number of participants was 69—24, 22, and 23 in the control, paper-based, and web-based groups, respectively.

Difficulty level by lung sounds

Table 1 shows the cross-tabulation of pre-test by each lung sound. There was a significant difference in the results of the pre-test for the seven lung sounds (p<0.001). The percentages of incorrect answers in descending order were as follows: “left lung diminish” (98.6%), “right lung absent” (91.3%), “fine crackles” (87.0%), “coarse crackles” (69.6%), “rhonchi” (53.6%), “wheeze” (26.1%), and “normal” (10.1%).

Table 1

Results of pre-test by sound

Lung sounds	Incorrect	Insufficient	Correct
Normal	7	0	62
	10.1%	0.0%	89.9%
Left lung diminish	68	0	1
	98.6%	0.0%	1.4%
Right lung absent	63	6	0
	91.3%	8.7%	0.0%
Wheeze	18	47	4
	26.1%	68.1%	5.8%
Rhonchi	37	31	1
	53.6%	44.9%	1.4%
Coarse crackles	48	20	1
	69.6%	29.0%	1.4%
Fine crackles	60	9	0
	87.0%	13.0%	0.0%

Change in results by lung sounds in the control, paper-based, and web-based groups

Tables 2, 3, and 4 show the change in results by lung sounds in the control, paper-based, and web-based groups. Those is the control group who gave incorrect in the pre-test tended to provide the correct answers for the corresponding questions in the post-test. That is, they showed improvement. However, there was no significant difference between pre-test and post-test in any of the lung sounds. In the paper-based group, there was a significant improvement in performance for “wheeze” (p=0.004) and “coarse crackles” (p=0.035). In the web-based group, there was a significant improvement in performance for “fine crackles” (p=0.026).

Table 2

Cross-tabulation of changes between pre-test and post-test in the control group

Normal		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	1		3	0.705
	Insufficient
	Correct	4		16
Left lung diminish		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	21	3		0.083
	Insufficient
	Correct
Right lung absent		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	21	2		0.564
	Insufficient	1
	Correct
Wheeze		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	3	4		0.705
	Insufficient	3	13
	Correct			1
Rhonchi		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	8	5		0.739
	Insufficient	4	6
	Correct			1
Coarse crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	11	3	1	0.096
	Insufficient	1	6	1
	Correct			1
Fine crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	15	5		0.257
	Insufficient	2	2
	Correct

Table 3

Cross-tabulation of changes between pre-test and post-test in the paper-based group

Normal		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect			1	0.317
	Insufficient
	Correct			21
Left lung diminish		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	21			1.00
	Insufficient
	Correct			1
Right lung absent		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	16	2	1	0.48
	Insufficient	2	1
	Correct
Wheeze		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	2	3	1	0.004
	Insufficient		10	5
	Correct			1
Rhonchi		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	9	4	1	0.206
	Insufficient	2	6
	Correct
Coarse crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	10	5	1	0.035
	Insufficient	1	4	1
	Correct
Fine crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	13	3	2	0.053
	Insufficient	1	2	1
	Correct

Table 4

Cross-tabulation of changes between pre-test and post-test in the web-based group

Normal		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect			2	0.564
	Insufficient
	Correct	1		20
Left lung diminish		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	20	2	1	0.102
	Insufficient
	Correct
Right lung absent		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	20	1		1.00
	Insufficient	1	1
	Correct
Wheeze		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	1	3	1	0.186
	Insufficient	2	8	6
	Correct	1	1
Rhonchi		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	6	4		0.248
	Insufficient	4	5	4
	Correct
Coarse crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	8	4	5	0.074
	Insufficient	5	1
	Correct
Fine crackles		Posttest
		Incorrect	Insufficient	Correct	p
Pretest	Incorrect	15	3	4	0.026
	Insufficient	1
	Correct

Number of correct answers in the control, paper-based, and web-based groups

Table 5 shows the change in the number of correct answers in the control, paper-based, and web-based groups. There was no significant difference in the number of correct answers in the pre-test among the three groups (p=0.564). Comparing the pre-test and post-test, there was no significant increase in the control group (p=0.705), but there was one in the paper-based group (p=0.011) and the web-based group (p=0.042). The number of correct answers in the post-test was significantly different among the three groups (p=0.023). The paper-based group showed a significant increase in the number of correct answers compared to the control group (p=0.021). Although not significant, participants in the web-based group tended to have more correct answers than participants in the control group (p=0.05).

Table 5

Cross-tabulation of the changes in the number of correct answers in the control, paper-based, and web-based groups

Control group		Posttest
		0	1	2	3	4	5	6	Total	p
Pretest	0	1	2						3 12.5%	0.705
	1	3	16	1					20 83.3%
	2
	3					1			1 4.2%
	Total	4 16.7%	18 75.0%	1 4.2%		1 4.2%			24 100%
Paper group		Posttest
		0	1	2	3	4	5	6	Total	p
Pretest	0		1						1 4.5%	0.011
	1		14	3	1	1			19 86.4%
	2					1	1		2 9.1%
	Total		15 68.2%	3 16.1%	1 4.5%	2 9.1%	1 4.5%		22 100%
Web group		Posttest
		0	1	2	3	4	5	6	Total	p
Pretest	0		1				1		2 8.7%	0.042
	1	1	12	2		2	1	1	19 82.6%
	2		2						2 8.7%
	Total	1 4.3%	15 65.2%	2 8.7%		2 8.7%	2 8.7%	1 4.3%	23 100%

DISCUSSION

This study investigated the effectivity of a new interactive web teaching material using a randomized controlled trial among new nurses with less than one year of experience. The results showed that the traditional paper teaching material and our web teaching material were useful in improving skills to auscultate lung sounds.

There was a statistically significant difference in the answers to the seven lung sounds in the pre-test, suggesting that the degree of difficulty in auscultating each lung sound is different. For “left lung diminish” and “right lung absent,” more than 90% of the participants were incorrect. However, it does not imply that these two sounds are difficult to identify. In this study, we set a time limit for the auscultation. The pre-test and post-test were conducted in a setting that allowed the participants to write their answers when they knew the answers. A previous study showed that time limits make people reduce performance.[17] Moreover, a large number of incorrect answers of “normal” suggest that many participants filled in answers when they found “normal” lung sounds in the areas they were auscultating without auscultating both sides. It was considered necessary to understand lung structure and learn appropriate auscultation methods before acquiring the ability to identify adventitious sounds. The fact that more than half of the participants gave incorrect answers for “rhonchi” (53.6%), “coarse crackles” (69.6%), and “fine crackles” (87.0%) suggests that identification is difficult for these sounds. In particular, it has been reported that identifying “fine crackles” is difficult even for doctors and medical students.[3] Additionally, more than half of the nurses gave incorrect answers for five of the seven lung sounds, indicating that nurses with less than one year of clinical experience are unable to accurately auscultate lung sounds. This result was thought to be related to the system of the national nursing examination in Japan. The national nursing examination is knowledge-based and does not test practical skills. Therefore, many nursing students only study the theoretical part in preparation for the national examination. In the light of these facts and our results, it is necessary to develop effective educational tools to teach practical skills for newly-graduated nurses.

Interestingly, the paper-based material improved the ability to auscultate “wheeze” and “coarse crackles,” while the web-based material improved the ability to auscultate “fine crackles.” The paper-based material included an illustration of air being blown into a liquid with a straw to explain “coarse crackles.” This may have made it easier for the nurses to connect the illustration of the generation mechanism with the actual sound. The paper-based materials used the same illustrations of an airway constriction for “wheeze” and “rhonchi.” Although it might be helpful in understanding how the sound is generated, it resulted in many students answering “wheeze” instead of “rhonchi.” To explain “fine crackles,” an illustration of blowing into an old balloon was included. However, it was difficult to recall “fine crackles” from the illustration.[14]

Identifying the sound of “fine crackles” is difficult in itself, so it may be useful to use web-based teaching material where the participants can auscultate. The paper-based materials were helpful in understanding the mechanism of lung sounds, which may have helped the students master “wheeze” and “coarse crackles.” The web-based materials were helpful for auscultating actual lung sounds and evaluating their auscultation skills, which may have helped them master “fine crackles.” Previous studies showed that an interactive nursing skills mobile application had been significantly more effective than a non-interactive mobile application.[18] Although the paper-based material was not interactive, we did not set out to determine whether the paper-based material or the web-based material is more useful as an educational tool for developing lung auscultation skills. Our findings suggested that web-based material has the potential to supplement the weaknesses of paper-based material. In addition, in this web-based material, the instructor can check the performance of an individual or his or her group; based on the results, the instructor can create questions to address the learners’ areas of weakness. Instructors may also use the web-based materials together with their learners to help them improve their lung auscultation skills.

LIMITATIONS

Several limitations of this study should be acknowledged. First, we did not know the educational background of the research participants when they were students. Second, the content of the initial training the participants received at their hospitals could not be ascertained. Third, we did not provide learning methods for the web-based materials group. By showing effective ways to use the materials, the participants may have understood not only the sounds but also the mechanism of sound generation through the web-based materials, and therefore, their lung auscultation skills may have further improved. Fourth, to evaluate, in terms of convenience, the effectiveness of our material—which was designed to be used on smartphones—we should have included a group of participants who learned using conventional materials that focus on providing sound with limited location for use like a compact disc. Lastly, the content and presentation of the lung sound explanations differed between the web-based and paper-based materials. This difference may have affected the results.

CONCLUSION

The web-based teaching materials we developed effectively improved the ability of new graduate nurses in auscultating lung sounds. Additionally, the results suggest that the combined use of web- and paper-based teaching materials may be more effective since the sounds that were used in each method enhanced their ability to auscultate different lung sounds.

ACKNOWLEDGEMENT

We are grateful to the 72 nurses who willingly participated in this study and provided invaluable data. We are also grateful to the three hospitals for their cooperation.

DISCLOSURE STATEMENT

The authors of this manuscript have no conflicts of interest to disclose.