Assessment of final-year medical students' performance in diagnosing critical findings on chest X-ray.

PURPOSE: Due to the recently emerging shortage in medical staff during the novel corona virus pandemic, several countries have rushed their undergraduate medical students into the emergency department. The accuracy of diagnosing critical findings on X-rays by senior medical students is not well assessed. In this study, we aim to assess the knowledge and accuracy of undergraduate final-year medical students in diagnosing life-threatening emergency conditions on chest x-ray.
METHOD: This is a cross-sectional nationwide survey across all medical schools in Jordan. Through an electronic questionnaire, participants were sequentially shown a total of six abnormal X-rays and one normal. For each X-ray, participants were asked to choose the most likely diagnosis, and to grade the degree of self-confidence regarding the accuracy of their answer in a score from 0 (not confident) to 10 (very confident).
RESULTS: We included a total of 530 participants. All participants answered at least six out of seven questions correctly, out of them, 139 (26.2%) participants answered all questions correctly. Pneumoperitoneum was the highest correct answer (93.8%), whereas flail chest was the least correctly answered case with only 310 (58.5%) correct answers. Regarding self-confidence for each question, 338 participants (63.8%) reported very high overall self-confidence level. Answers related to tension pneumothorax had the highest confidence level.
CONCLUSION: Senior Jordanian medical students showed good knowledge with high confidence levels in diagnosing life-threatening conditions on chest x-rays, supporting their incorporation in the emergency department during pandemics and confirming the reliability of information they can extract.

Introduction

Chest X-ray imaging in the emergency department remains trivial in determining the pathology and triaging management, especially in high-load services. A previous report showed how accurate X-ray reporting in emergency department (ED) improved management of patients triage and reduced false-negative diagnoses [1]. While recent advances in machine learning technologies showed promises in accuracy of detecting pathologies and extracting data from different imaging modalities [2, 3], several challenges in implementing these automatic systems in clinical practice still exist [4, 5].

With the emergence of the novel coronavirus disease 2019 (COVID-19) outbreak, several countries have rushed their final-year medical students into the field to compensate for the shortage in medical doctors working in the emergency department [6]. In the UK as an example, students are expected to work in the field of their competency and willingness after their graduation; however, emergency department is the place most in need to these new graduates [7]. Assessing competency of medical students in recognizing life-threatening conditions is an important consideration when deciding to implicate medical students in practice. In this study, we aim to assess the knowledge of undergraduate final-year medical students in Jordan about the X-ray findings of life-threatening emergency conditions.

Materials and methods

Design and settings

This is a cross-sectional study conducted in all medical schools in Jordan. The study was approved by the Institutional Review Board (IRB) of The University of Jordan and was conducted in concordance with the latest declaration of Helsinki. The survey was conducted during March and April of 2020, where a corona virus outbreak was still ongoing in Jordan. Jordan has six universities teaching medicine: three in the central region, The University of Jordan, Hashemite University, and Al-Balqa Applied University; two in the northern region, Jordan University of Science and Technology and Al-Yarmouk University; and one in the southern region, Mut’ah University. All medical schools apply 6-year teaching programs until graduation.

Participants

We recruited sixth year medical students from all medical schools in Jordan, except from Al-Balqa Applied University, as it was recently established and the first batch will graduate in 2022.

We adopted the following inclusion criteria:

Sixth year medical student

Completed radiology rotation during their training

We used batch groups on social media, where all senior medical students are registered there, to distribute an electronic questionnaire using Google forms. All participants were informed about the nature of the survey and the privacy of their data.

Variables

Each participant was asked about general demographics (i.e., age, gender, university), their GPA, and their general self-confidence about self-competency if they are sent to the emergency department. After that, a total of six abnormal and one normal X-rays were shown sequentially. For each X-ray, the participant was asked to mention the abnormality and to grade their self-confidence about the accuracy of the answer in a score from 0 (not confident) to 10 (very confident).

After reviewing the most common life-threatening X-ray cases in our emergency department at our hospital, we chose the following six diagnoses: tension pneumothorax, pneumoperitoneum, cardiac tamponade, multiple rib fractures (flail chest), acute respiratory distress syndrome (ARDS), and aortic dissection, in addition to a normal X-ray. We obtained the images from our radiology department (Fig. 1). We sent the images to two radiologists, a senior radiology resident (L. M.) and a radiology specialist (N. A), who were blinded about the diagnosis, to answer and validate the images and to comment if any other major abnormality is present and may confuse the respondents about the required answer.

Fig. 1

An example of a normal chest X-ray rated by the students

Upon reviewing the results, describing the exact pathology or pointing to it in an informative way was considered a correct answer, as both cases will let the respondent to correctly triage the case to the appropriate management. For example, answering the ARDS case as “acute pulmonary edema” was considered correct. Answering aortic dissection as aortic aneurysm or widened mediastinum, answering cardiac tamponade as pericardial effusion, answering flail chest as multiple rib fractures, and answering pneumoperitoneum as perforated viscus or air under diaphragm were all considered correct answers. On the other hand, answering tension pneumothorax as pneumothorax alone was not considered true, as this might delay prompt life-saving intervention required in cases of tension pneumothorax.

Finally, we calculated the total score of correct answers by adding all correct answers together to get a score out of seven. We then calculated the total self-confidence score by adding self-confidence levels for all questions and dividing the total by seven to get an average score out of ten.

Statistical analysis

We used SPSS version 21.0 (Chicago, USA) in our analysis. We used mean (± standard deviation) to describe continuous variables (e.g., age). We used count (frequency) to describe other nominal variables (e.g., gender).

We used one-way ANOVA followed by Tukey post hoc test to compare GPA with total score. We used chi-square test to analyze the difference between gender and university with total score and total confidence levels, followed by post hoc Z-test for proportion. We tested the reliability of our questionnaire using reliability analysis of Cronbach’s alpha. All underlying assumptions were met, unless otherwise indicated. We adopted a p value of 0.05 as the significance threshold.

Results

We included a total of 530 participants in this study, 265 (50%) were men and 265 (50%) women, with a mean age of 23.23 (± 1.41) years. The mean GPA for the participants was 3.14 (± 0.45) out of 4. The responses showed almost balanced distribution among the included universities as follows: 134 (24.3%) participants were from Hashemite university, 127 (24.0%) from The University of Jordan, 120 (22.6%) from Jordan University of Science and Technology, 83 (15.7%) from Mut’ah University, and 66 (12.5%) from Yarmouk University. The confidence questionnaire had a high reliability score with a Cronbach’s alpha of 0.70.

Two hundred eighty-five (53.77%) participants answered at least six out of seven questions correctly, and out of them, 139 (26.2%) participants answered all questions correctly (Fig. 2).

Fig. 2

The number of questions and the percentage of students answering them correctly

The total score was significantly related to the university (< 0.001), where students from The University of Jordan had the highest number of participants with full score (Table 1). The total score was not related to GPA (0.291) or gender (p = 0.123).

Table 1

Comparison between numbers of images answered correctly out of seven images between different Jordanian universities

	University
	Hashemite University	Jordan University of Science and Technology	Mut’ah University	The University of Jordan	Yarmouk University
Total score	0	3 (75%)	1 (25%)	0	0	0
	1	2 (20%)	7 (70%)	0	1 (10%)	0
	2	4 (19%)	11 (52.4%)	0	5 (23.8%)	1 (4.8%)
	3	7 (22.6%)	13 (41.9%)	4 (12.9%)	5 (16.1%)	2 (6.5%)
	4	32 (49.2%)	12 (18.5%)	8 (12.3%)	10 (15.4%)	3 (4.6%)
	5	40 (35.1%)	30 (26.3%)	12 (10.5%)	19 (16.7%)	13 (11.4%)
	6	28 (19.2%)	25 (17.1%)	30 (20.5%)	39 (26.7%)	24 (16.4%)
	7	18 (12.9%)	21 (15.1%)	29 (20.9%)	48 (34.5%)	23 (16.5%)
Total		134 (25.3%)	120 (22.6%)	83 (15.7%)	127 (24.0%)	66 (12.5%)

Regarding each specific question, all questions had at least 50% correct answers. Pneumoperitoneum was the highest correct answer (93.8%), whereas flail chest was the least correctly answered case with only 310 (58.5%) correct answers. Figure 3 shows the percentages of students correctly diagnosed the pathology for each case.

Fig. 3

The percentage of students correctly diagnosing the pathology for each case

Regarding self-confidence for each question, 338 participants (63.8%) reported very high self-confidence level, 168 participants (31.7%) reported high confidence, and 24 (4.5%) reported moderate to low confidence level. Answers related to tension pneumothorax had the highest confidence level with 324 participants (61.1%) recorded very high confidence level, while it was the lowest for flail chest image, where only 259 participants (48.9%) recorded very high confidence (Table 2). No significant difference for GPA (p = 0.388), gender (p = 0.466), or university (p = 0.193) was found.

Table 2

Number and percentage for confidence level for each image

	Very low confidence	Low confidence	Moderate	High confidence	Very high confidence
ARDS	8 (1.5%)	32 (6%)	82 (15.5%)	143 (27%)	265 (50%)
Aortic dissection	12 (2.3%)	33 (6.2%)	84 (15.8%)	123 (23.2%)	278 (52.5%)
Normal	17 (3.2%)	26 (4.9%)	63 (11.9%)	102 (19.2%)	322 (60.8%)
Cardiac tamponade	2 (0.4%)	15 (2.8%)	75 (14.2%)	130 (24.5%)	308 (58.1%)
Flail chest	20 (3.8%)	51 (9.6%)	95 (17.9%)	105 (19.8%)	259 (48.9%)
Pneumoperitoneum	15 (2.8%)	30 (5.7%)	61 (11.5%)	107 (20.2%)	317 (59.8%)
Tension pneumothorax	15 (2.8%)	27 (5.1%)	66 (12.5%)	98 (18.5%)	324 (61.1%)
	Very low confidence	Low confidence	Moderate	High confidence	Very high confidence
ARDS	8 (1.5%)	32 (6%)	82 (15.5%)	143 (27%)	265 (50%)
Aortic dissection	12 (2.3%)	33 (6.2%)	84 (15.8%)	123 (23.2%)	278 (52.5%)
Normal	17 (3.2%)	26 (4.9%)	63 (11.9%)	102 (19.2%)	322 (60.8%)
Cardiac tamponade	2 (0.4%)	15 (2.8%)	75 (14.2%)	130 (24.5%)	308 (58.1%)
Flail chest	20 (3.8%)	51 (9.6%)	95 (17.9%)	105 (19.8%)	259 (48.9%)
Pneumoperitoneum	15 (2.8%)	30 (5.7%)	61 (11.5%)	107 (20.2%)	317 (59.8%)
Tension pneumothorax	15 (2.8%)	27 (5.1%)	66 (12.5%)	98 (18.5%)	324 (61.1%)

Discussion

Previous studies focused on assessing competency of reading chest X-rays among practicing physicians, including residents and fellows [8-12], and to a lesser extent medical students [13, 14]. Very few studies however have particularly focused on undergraduate medical students’ ability to identify acute life-threatening conditions [15]. Our national study found a reasonably high accuracy level among Jordanian senior medical students in identifying acute life-threatening pathologies manifesting on chest X-ray with high self-confidence scores, ranging from 58.5 to 93.8%.

A previous study compared competencies of medical students and residents in identifying critical chest X-ray findings and found a significant difference in overall scores between medical students and radiology residents, but not internal medicine residents, although residents generally had higher confidence than students [13]. Another study also did not find a significant difference in accuracy of X-ray interpretation between medical students and surgery residents [11]. In another study that assessed the ability to detect general chest X-ray findings, students and residents did not differ significantly in identifying pathologies [16]. A small study (n = 22) that was done on junior doctors to identify general pathologies on chest X-ray found a low overall accuracy and confidence (51 and 57.5%, respectively) (14). Another small study from Iran showed a low accuracy that reached only 8% for ARDS and 15% for normal chest X-ray among practitioners, pointing to the need for improving their emergency radiology curricula [9]. A recent study that included radiology residents showed that they generally have around 90.4 and 79.6% accuracy in identifying abnormal and normal chest X-rays, respectively, but only 52.7% in reaching the exact diagnosis, where they were tested on imagings of both life-threatening and non-life-threatening conditions [8].

Noteworthy, our study demonstrated significant difference in total scores between different universities which is mostly attributable to the curricula of radiology rotations taught during college years; this was also supported by a previous study that showed a significant improvement in accuracy of interpreting chest X-ray images after completing radiology rotation [15].

We believe our study has some limitations that need to be considered by future studies. The difference in total correct scores between universities points to the importance of considering a unified emergency imaging course to be implemented by all medical schools. Identifying life-threatening findings on chest X-ray is an important skill that needs to be well implemented in all radiology rotations; this is also the strategy adopted by program directors of several radiology and medical departments in the USA [17, 18]. On the other hand, we believe that our study has many strength points compared to previous similar studies as we included a representative balanced sample from medical schools all over Jordan, and we eliminated answering bias by not providing multiple choice questions, which can guide the respondent to the correct answer.

In conclusion, our national study showed high competency among senior medical students in identifying acute life-threatening findings on chest X-ray images with high self-confidence levels. The present study supports the possibility of deploying Jordanian senior medical students in emergency departments in crisis situations.