Evaluation of radiologist's knowledge about the Artificial Intelligence in diagnostic radiology: a survey-based study.

BACKGROUND: Advanced developments in diagnostic radiology have provided a rapid increase in the number of radiological investigations worldwide. Recently, Artificial Intelligence (AI) has been applied in diagnostic radiology. The purpose of developing such applications is to clinically validate and make them feasible for the current practice of diagnostic radiology, in which there is less time for diagnosis.
PURPOSE: To assess radiologists' knowledge about AI's role and establish a baseline to help in providing educational activities on AI in diagnostic radiology in Saudi Arabia.
MATERIAL AND METHODS: An online questionnaire was designed using QuestionPro software. The study was conducted in large hospitals located in different regions in Saudi Arabia. A total of 93 participants completed the questionnaire, of which 32 (34%) were trainee radiologists from year 1 to year 4 (R1-R4) of the residency programme, 33 (36%) were radiologists and fellows, and 28 (30%) were consultants.
RESULTS: The responses to the question related to the use of AI on a daily basis illustrated that 76 (82%) of the participants were not using any AI software at all during daily interpretation of diagnostic images. Only 17 (18%) reported that they used AI software for diagnostic radiology.
CONCLUSION: There is a significant lack of knowledge about AI in our residency programme and radiology departments at hospitals. Due to the rapid development of AI and its application in diagnostic radiology, there is an urgent need to enhance awareness about its role in different diagnostic fields. © The Foundation Acta Radiologica 2020.

Introduction

Advanced developments in diagnostic radiology, such as computed tomography (CT), magnetic resonance imaging (MRI), nuclear medicine, and ultrasound (US), have provided a rapid increase in the number of radiological investigations worldwide. Consequently, radiologists interpret a large number of diagnostic images daily. Interpretation involves reviewing multi-slice images, e.g. 256 images in CT, and providing a final report with an accurate diagnosis. Recently, Artificial Intelligence (AI) has been applied in diagnostic radiology. AI is defined as “the capability of a machine to imitate intelligent human behaviour” (1). Another term related to artificial intelligence is deep learning. It is considered a class of machine learning that mainly “concerns algorithms inspired by the structure and function of the brain” (2). Machine learning is defined as “algorithm-driven learning by computers such that the machine’s performance improves with greater experience” (3). It involves construction of algorithms based on learning from data and making predictions accordingly (3).

The application of AI in radiology includes robust machine learning algorithms to diagnose head and neck cancer on CT and MRI (4). MedyMatch utilises AI to diagnose stroke on CT and can be used in the radiology reporting room and the emergency department (5). Such applications are being used on a large amount of digital data produced by radiology imaging cases and in electronic medical reports. The main purpose of developing such applications is to clinically validate and make them feasible for the current practice of diagnostic radiology, in which there is less time for diagnosis (6,7).

The AI revolution raises concerns about the future of radiologists. Predictions about the dramatic alterations that may occur in a radiology career and the role of radiologists with this technology concern whether the AI revolution will take place in radiology and how soon it will occur. We hypothesized that the radiologists in our residency program have a lack of knowledge about the role of AI in diagnostic radiology and how to integrate it into the clinical practice. The aim of the present study was to assess radiologists’ knowledge about the role of AI and establish a baseline to help in providing educational activities on AI in diagnostic radiology.

Material and Methods

An online questionnaire was designed using QuestionPro software, licensed to Imam Abdulrahman bin Faisal University, to assess the knowledge of AI in diagnostic radiology of three groups: (i) first- to fourth-year trainees in radiology programs; (ii) radiologists and fellows; and (iii) consultants. The study was conducted in large hospitals located in different regions in Saudi Arabia, as follows: 12 hospitals in the western and south regions; two hospitals in the eastern region; and two hospitals in the middle region. A total of 150 hard copies of the questionnaire were distributed manually and collected for analysis. Emails containing a link to the questionnaire were also sent to the participants. Before the distribution, the survey was edited for typographic and formatting enhancements. The participation period lasted three months, from 1 September until the end of November 2019. Figure 1 demonstrates the inclusion and exclusion criteria of the participants.

Fig. 1.

Inclusion and exclusion criteria of the participants.

The questionnaire consisted of three sections and 15 questions; 11 questions were multiple choice and were free text. The first section requested the participant’s employment information, including the name and region of the hospital, the job description (e.g. trainee, radiologist, or consultant), and their subspecialty (e.g. neuro). No identifying information was requested. The second section referred to general demographic information, such as gender, age group, and years of experience in diagnostic radiology. The third section included questions referring to the participant’s knowledge of AI. All data were inserted and populated into an Excel spreadsheet (Microsoft Office) for analysis.

Statistical analysis

Descriptive statistics and graphical presentations of the survey data were expressed as frequencies and percentages. Comparisons of categorical data between groups were made using Pearson’s chi-squared analysis or an exact analysis when expected cell counts were < 5. Furthermore, Z statistic was used to find out the significant difference in the proportion of the radiologists’ responses towards categorical variables. In all analyses, P < 0.05 was significant. All statistical analyses were performed using SPSS statistical software, version 22.

Results

A total of 93 participants completed the questionnaire, of which 32 (34%) were trainee radiologists from year 1 to year 4 (R1–R4) of the residency program, 33 (36%) were radiologists and fellows, and 28 (30%) were consultants (Fig. 2). Twenty-eight participants (30%) were women and 65 (70%) were men. Fig. 3 shows the percentage distribution of the subspecialties of the second and third groups: radiologists and fellows; and consultants.

Fig. 2.

Percentage distribution of the three groups: trainees (R1–R4); radiologists and fellows; and consultants.

Fig. 3.

Percentage distribution of the subspecialties of radiologists, fellows and consultants.

For the question on dealing with large data, the answers were correlated to the job positions. The consultants had the highest percentage (50%), then the radiologists (45%). The trainees’ response showed the lowest percentage (34%). The number of trained radiologists who could analyze large data was very low for all participants (≤9%) (Table 1).

Table 1.

Radiology trainees (years R1–R4), radiologists and fellows, and consultants’ responses to a survey on artificial intelligence in diagnostic radiology in Saudi Arabia.

Question/Response	Total	R1–R4	Radiologists and Fellows	Consultants	χ2, P value*
Q1: Are you able to deal with large data analytics?					2.091, 0.719
No	46 (49.5)	19 (60)	15 (45.5)	12 (43)
Yes	40 (43)	11 (34)	15 (45.5)	14 (50)
Yes, and trained	7 (7.5)	2 (6)	3 (9)	2 (7)
Q2: Do you know about Artificial Intelligence?					0.882, 0.644
No	22 (24)	9 (28)	8 (24)	5 (18)
Yes	71 (76)	23 (72)	25 (76)	23 (82)
Q3: Would you learn about Artificial Intelligence application as it related to your job?					0.087, 0.958
No	16 (17)	5 (18)	6 (8)	5 (8)
Yes	77 (83)	27 (82)	27 (82)	27 (82)
Q4: As a radiologist, do you use Artificial Intelligence software on daily basis?					3.010, 0.222
No	76 (82)	27 (84)	24 (73)	25 (89)
Yes	17 (18)	5 (16)	9 (27)	3 (11)
Q5: Are you willing to help in creating Artificial Intelligence software/application to do the tasks for the radiologist?					0.552, 0.759
No	27 (29)	8 (25)	11 (33)	8 (29)
Yes	66 (71)	24 (75)	22 (67)	20 (71)
Q6: How will Artificial Intelligence affect your job in the next 10–20 years?					1.556, 0.817
No / to minimum effect	23 (25)	8 (25)	7 (21)	8 (29)
The job will be changed dramatically	68 (73)	25 (75)	25 (76)	19 (68)
The job will be obsolete	2 (2)	0 (0)	1 (3)	1 (3)
Q7: Will your current knowledge about Artificial Intelligence change your decision to continue your career as radiologist?					0.226, 0.893
No	63 (68)	22 (69)	23 (70)	18 (64)
Yes	30 (32)	10 (31)	10 (30)	10 (36)

Values are given as n (%).

*Pearson chi-square test.

Overall, the trainees, radiologists, and consultants were familiar with the term AI, with responses of 72%, 74%, and 82%, respectively. However, 22 (23%) participants reported that they had no general knowledge about AI or if it was used in diagnostic radiology (Table 1).

The responses to the question related to the use of AI on a daily basis illustrated that 76 (82%) of the participants were not using any AI software at all during daily interpretation of diagnostic images. Only 17 (18%) reported that they used AI software for diagnostic radiology (Table 1). They stated that they used computer-aided detection (CAD), voice-to-text converters, and dictation software for daily tasks in diagnostic radiology departments. The Z statistic showed that there is a significant difference in the proportion of radiologists using AI software on daily basis for diagnostic radiology (P < 0.05).

Most of the participants (approximately 71%) in the present study were willing to contribute to developing AI software to help in the diagnostic radiology process. Trainee radiologists had the highest percentage response compared with the other two groups (Table 1).

For the question related to the future of AI in diagnostic radiology in the next 20 years, the respondents believed that AI would have a major effect on diagnostic radiology tasks. Only 2% of the respondents thought that no change would occur in the diagnostic field. A total of 63% of the participants had no intention to change their specialty based on their knowledge about AI. Overall, 23 (25%) participants reported having read more than four articles on the topic of AI in diagnostic radiology during the past 12 months. However, most of the participants (approximately 62%) reported not having read scientific medical articles related to the same topic for the same period. The statistical analysis of all the survey questions showed no significant differences between the three groups (Table 1).

Discussion

The present study was designed to measure radiologists’ basic knowledge about the potential application of AI in diagnostic radiology. There has been a significant development of AI in diagnostic radiology globally (8). However, lack of knowledge about AI was found among the radiologists who participated in the present study.

Most of the answers to the question on the name of the AI software in daily use were CAD. According to Wong et al. (2), CAD is a tool based on machine learning that uses data classification, feature analysis, and image processing for diagnostic purposes (3). It is also used to distinguish between interstitial lung and lung nodule diseases (9). However, it has no level of intelligence that could be made by deep learning.

Although most responses to Q2, which asked whether the respondents knew what AI was, were “yes,” that they understood the term “artificial intelligence,” this was from a general perspective and not linked to diagnostic radiology. This implies that radiologists have a vague understanding of AI, caused by low exposure to recent scientific articles about this subject, because most of these articles have not been published in major radiology journals (10). A search on PubMed with the date range August 2016 to July 2017 and the search terms “radiology” and “artificial intelligence” returned 85 articles (10). These articles included the use of AI in bone radiology (11), oncology radiology (12), chest radiography (13,14), head and neck radiology (15), and breast imaging (16). Eight articles out of 85 were published in major radiology journals, such as the Journal of Nuclear Medicine, Radiology, the JACR, and the American Journal of Roentgenology (10). Another reason for the lack of knowledge among radiologists is that there is no subject related to AI in radiology taught in our residency programs yet. This is due to the advances of this topic in relation to diagnostic radiology.

Chockley and Emanuel (3) have suggested that radiology faces three fatal threats: payment reform; care moving out of hospitals; and machine learning; they emphasize that the last threat is the ultimate. They have predicted that computers will replace radiologists within the next five years (3,17). However, most of the radiologists in the present study felt that they were not threatened by AI in their work. According to radiology experts, AI is not expected to be adopted in radiology in the near future. This is due to the high cost of the labor-intensive process that are required for training some of these tools independently for each disease and patient condition, the actual integration of AI into clinical practice, and the regulatory processes required by the U. S. Food and Drug Administration (18). Despite these facts, most experts imply that radiologists must be aware about the development of AI in radiology, as it may become a serious risk to their jobs due to commoditization and pressure (18). Unlike the models of traditional medical research that need clinician supervision, computer scientists will likely lead the investigation of artificial imaging. As a result, the research input of radiologists may become restricted (2).

Based on our results, there is a strong need to enhance the awareness of AI among our residency program (trainee radiologists) and radiologists in hospitals in Saudi Arabia. This can be achieved by providing recent scientific literature about the subject, creating learning resources, and making them available in national radiology organizations and by developing a series of lectures related to the applications of AI in diagnostic radiology that will provoke trainee radiologists and radiologists to obtain good knowledge about the appropriate role of AI in different diagnostic areas.

In conclusion, there is a significant lack of knowledge about AI in our residency program and radiology departments at hospitals. Due to the rapid development of AI and its application in diagnostic radiology, there is an urgent need to enhance awareness about its role in different diagnostic fields.