Artificial intelligence in radiology: Are Saudi residents ready, prepared, and knowledgeable?

OBJECTIVES: To assess the knowledge and perception of artificial intelligence (AI) among radiology residents across Saudi Arabia and assess their interest in learning about AI.
METHODS: An observational cross-sectional study carried out among radiology residents enrolled in the Saudi Board of Radiology, Saudi Arabia. An anonymized, self-administered questionnaire was distributed in April 2020 and responses were collected until July 2020.
RESULTS: A total of 154 residents filled the questionnaire. The top 3 aspects of AI participants wanted to learn were: clinical use of AI applications, advantages and limitations of AI applications, and technical methods. Approximately 43.5% of participants did not expect AI to affect job positions, while 42% anticipated that job positions will decrease. Approximately 53% expected a reduction in reporting workload, while 28% expected an increase in workload.
CONCLUSION: Currently, the exposure of radiologists to the use of AI is inadequate. It is imperative that AI is introduced to radiology trainees and that radiologists stay updated with advances in AI to be more knowledgeable on how to benefit from it. Copyright: © Saudi Medical Journal.

Research on the use of artificial intelligence (AI) has been gaining popularity in the field of medicine. Recently, various kinds of AI programs have been developed based on the concept of “big data”. This concept can be defined as “extremely large datasets characterized by the large volume, high velocity of generation, variety, and veracity or credibility of the data”. With access to big data, machine learning (ML), a branch of AI, analyzes a vast amount of information in a rapid, accurate, and efficient manner through the use of complex computing and statistical algorithms. Currently, the development and utilization of AI programs in the medical field are entering the stage of commercialization.

Several medical specialties are benefiting from the emerging advances in AI. In medical physics and radiation oncology, AI aids in auto-segmentation, prognostic prediction, and decision support. In diagnostic radiology, there are multiple examples of the use of AI, including screening, staging and restaging of cancer, support for structured reporting, detection of incidental findings, and imaging protocol optimization.

As the concept of AI emerged in the field of radiology, scientists and doctors initially predicted that AI would replace radiologists. Obermeyer et al, argued that the digitized images used by radiologists can be turned over to algorithms using AI. However, experts say that radiologists would not be replaced by AI, in 2019, Langlotz, found that the roles of radiologists may change and that “radiologists who use AI will replace radiologists who do not”. He argued that despite the initial claims on their oncoming redundancy, radiologists will master new technologies with each emerging advance in their field. They will be better equipped to diagnose uncommon or rare entities that fall under the “long tail” of disease distribution compared to AI. Langlotz, also compared these advancements to the autopilot option for human pilots and found that AI could assist the radiologist’s judgement with accurate detection and measurement, making him an important component of precision healthcare. Unfortunately, the initial response of radiologists to AI has been persistent with a lot of misconceptions. Several studies were carried out to gauge the opinions and perceptions of medical students, residents, and radiologists. Medical students were queried to understand if current developments in AI would deter them from choosing radiology as a specialty. In Canada and the United Kingdom, researchers found that one-sixth and almost one-half of students would not choose radiology as a specialty because of the predicted impact of AI. Interestingly, a study by Dos Santos et al, surveying medical students in 3 German universities, found that a third of the respondents were somewhat concerned on the recent developments and the majority did not think that AI would replace radiologists. Furthermore, over half of the surveyed participants said that AI should be a part of medical training.

Several studies showed that radiology residents and radiologists do not think that AI will replace them and that these advancements would not change their choice of career. Most participants showed an active interest in learning and getting involved with AI and ML. In a national multi-center study carried out in Singapore, most respondents (84%) said that AI should be taught in residency. A study surveying members of the European Society of Radiology showed that opinions were nearly equally divided on whether AI will increase or decrease job opportunities and workload. The overall perception on these innovations leading to more time for completing sub-specialization or working with other specialties and patients was positive. Interestingly, all participants believed radiologists will play a role in developing and validating AI applications to medical imaging. Also, 64.3% believed radiologists should supervise all development stages of an AI system applied to radiology.

This study aimed to measure the knowledge and perception of AI among radiology residents in Saudi Arabia and understand their level of interest in learning on AI.

Methods

An observational cross-sectional study carried out on residents enrolled in the diagnostic radiology training program in Saudi Arabia. The program is developed, supervised, and approved by the Saudi Commission for Health Specialties and is the only approved diagnostic radiology program in Saudi Arabia. It is a 4-year program divided into 2 junior and 2 senior years. Most residents rotate in several hospitals which provide different levels of care. These include: community hospitals, tertiary hospitals, stroke, trauma, and oncology centers. In addition to core radiology rotations, the curriculum includes courses and workshops covering imaging basics such as imaging informatics and advanced visualization, ultrasound, and medical physics. However, these courses are not mandatory. There is no formal education related to AI.

An email with a link to an online survey was sent to all residents by the program administrative assistant. The minimum required sample size was calculated as 585 (total resident population across the 4 years). The sample size calculator tool was used (Raosoft, Inc., Seattle, WA, USA). A 5% margin of error and a 95% confidence level were set, which yielded a required sample of 233.

Participation was voluntary and electronic consent was collected before data collection. The participants were informed regarding the aim, objective, and process of the study, and there were no additional benefits for involvement other than participants’ reflection on their understanding of AI.

An anonymous self-administered questionnaire collection from letrutre reviews and was created on Google forms (Googleplex, Mountain View, CA, USA). A link to the survey was sent by email and WhatsApp messages (Facebook, Inc., Menlo Park, CA, USA) in April 2020 and the responses were collected in July 2020. Responses were tabulated on Google spreadsheets.

The questionnaire consists of 5 parts. The first part inquired demographic information and training levels. The second assessed exposure to AI. The third inquired applications of AI in radiology. The fourth assessed their knowledge of AI/ML, and the fifth part evaluated the effect of AI on radiology and medicine.

The study was approved by the Research Committee of the Unit of Biomedical Ethics of King Abdulaziz University-Faculty of Medicine, Jeddah, Saudi Arabia.

Statistical analysis

Data was carried out using Microsoft Excel 2016 and Statistical Package for the Social Sciences, version 25 (IBM, Armonk, NY, USA). Statistical differences between the qualitative variables were carried out using the Chi-squared test and Fisher’s exact test. P-values of <0.05 and 95% confidence intervals were considered significant.

Results

A total of 154 radiology residents answered the questionnaire, with an overall response rate of 26.3%. A total of 85 (55.2%) respondents were males and 69 (44.8%) were females. A total of 75 (48.7%) respondents were from the Central region, while the rest are from the remaining 4 regions. A total of 40 (25.9%) residents were in the first year of training, 34 (22.1%) were in the second year, 52 (33.8%) were in the third year, and 28 (18.2%) were in the fourth year.

A total of 64 (41.6%) residents reported being familiar with AI. Upon further inquiry of this subgroup, 55 respondents reported reading articles or searching websites related to radiology, while 10 had taken AI and ML courses. A total of 6 participants carried out experiments in computer science involving AI.

When asked regarding “which radiological subspecialties would be more influenced by AI in the next 5-10 years?” the top 3 responses were breast imaging (99; 64.3%), followed by molecular/nuclear imaging (56; 36.4%) then both neuroradiology and thoracic imaging (54; 35.1%). In regards to the most important fields of AI applications in the next 5-10 years, 91 (59.1%) selected mammography, followed by positron emission tomography/nuclear imaging (46.8%) then computed tomography (44.8%). When asked regarding “what AI would bring to the profession of radiology?”; 82 (53.2%) participants believed it would help in the detection of asymptomatic patients (screening), 74 (48.1%) believed it would help in detecting incidental findings, and 73 (47.4%) believed it would help in image post-processing.

On the expectation of AI’s impact on the number of job positions in the next 5-10 years; 67 (43.5%, p=0.742) participants anticipated that there will be no effect, 64 (41.6%, p=0.919) anticipated that job positions will be reduced, while 23 (14.9%, p=0.869) expected job positions to increase. In regards to AI’s impact on the total reporting workload in the next 5-10 years; 29 (18.8%, p=0.440) participants responded it will have no effect, 43 (27.9%, p=0.192) responded that the workload will increase, and 82 (53.2%, p=0.905) responded that the workload will decrease. Moreover, in the next 5-10 years, 79 (51.3%, p=0.810) participants expected that the use of AI-based applications will make the duties of radiologists more technical and clinical, and 36 (23.4%, p=0.065) participants believed that the rate of dedication to subspecialties will remain unchanged. A total of 79 (51.3%, p=0.489) participants believed that most patients would not accept a report from AI applications without the supervision and approval of a physician. When asked regarding the legal responsibility of AI-systems’ output, 105 (68.2%) participants believed that radiologists would take the legal responsibility, while 79 (51.3%) believed AI application developers should be responsible. A total of 35 (22.7%) believed it is the insurance companies responsibility and 9 (5.8%) participants believed the responsibility to be on other physicians. A total of 97 (63%) participants considered the role of radiologists in the development/validation of AI applications for medical imaging to be important and that their supervision is required to develop AI-based applications. Table 1 elaborates the responses in details.

Table 1

- Exposure assessment to AI in radiology corresponding to gender, training level and familiar with big data.

Questions/category	n (%)	P-value
		Gender	Training level	Big data
Which radiological subspecialties do you foresee will be more influenced by AI in the next 5-10 years? (choose up to 3)
Breast	99 (64.3)
Molecular/nuclear imaging	56 (36.4)
Neuroradiology	54 (35.1)
Thoracic	54 (35.1)
Emergency	32 (20.8)
Musculoskeletal	24 (15.6)
Cardiovascular	22 (14.3)
General	22 (14.3)
Gastrointestinal/abdominal	20 (13)
Interventional	17 (11)
Oncologic imaging	16 (10.4)
Head and neck	11 (7.1)
Urogenital	3 (1.9)
Pediatric	2 (1.3)
Which techniques do you foresee will be the most important fields of AI applications in the next 5-10 years? (choose up to 3)
Mammography	91 (59.1)
PET/nuclear	72 (46.8)
CT	69 (44.8)
Radiography	61 (39.6)
MRI	46 (29.9)
DXA	37 (24)
Angiography/fluoroscopy	18 (11.7)
Hybrid imaging	9 (5.8)
Ultrasound	8 (5.2)
Experimental imaging (animal models)	6 (3.9)
Optical imaging	4 (2.6)
Which of the following AI applications do you think are more relevant as aids to the radiological profession? (choose up to 3)
Detection in asymptomatic subjects (screening)	82 (53.2)
Detection of incidental findings	74 (48.1)
Image post-processing	73 (47.4)
Imaging protocol optimization	54 (35.1)
Support to structured reporting	44 (28.6)
Lesion characterization/diagnosis in symptomatic subjects	43 (27.9)
Staging/restaging in oncology	43 (27.9)
Quantitative measure of imaging biomarkers	31 (20.1)
Prognosis	12 (7.8)
Do you foresee an impact of AI on the professional life of radiologists in terms of the number of job positions in the next 5-10 years?
No	67 (43.5)
Yes, job positions will be reduced	64 (41.6)	0.742	0.919‡	0.869
Yes, job positions will increase	23 (14.9)
Do you foresee an impact of AI on the professional life of radiologist in terms of total reporting workload in the next 5-10 years?
No	29 (18.8)
Yes, it will increase	43 (27.9)	0.44	0.192	0.905
Yes, it will be reduced	82 (53.2)
In the next 5-10 years, the use of AI-based applications will make radiologists’ duties
More technical	28 (18.2)
More clinical	38 (24.7)
Unchanged	9 (5.8)	0.566‡	0.269‡	0.244‡
More technical and clinical	79 (51.3)
Do you think that in the next 5-10 years, the use of AI-based applications will help to reduce the need for subspecializing?
No, radiologists will be more focused on radiology subspecialties	102 (66.2)
Yes, radiologists will be less focused on radiology subspecialties	16 (10.4)	0.685	0.033	0.065
The rate of dedication to subspecialties will remain unchanged	36 (23.4)
In the next 5-10 years, who will take the legal responsibility of AI-system output?
Radiologists	105 (68.2)
Other physicians (namely, clinicians asking for the imaging study)	9 (5.8)
Developers of AI applications	79 (51.3)
Insurance companies	35 (22.7)
In the next 5-10 years, will patients accept a report from AI applications without supervision and approval by a physician?
Yes	17 (11)
No	79 (51.3)	0.381	0.489‡	0.847
Difficult to estimate at present	58 (37.7)
What will be the role of radiologists in the development/validation of AI applications to medical imaging? (choose at most 3)
Supervise all stages needed to develop an AI based application	97 (63)	0.320	0.169	0.687
Help in task definition	67 (43.5)	0.255	0.663	0.381
Develop AI-based applications	59 (38.3)	0.175	0.070	0.742
Provide labelled images	49 (31.8)	0.114	0.268	0.762
None	7 (4.5)	1.00‡	0.452‡	1.00‡

PET: positron emitted tomography, CT: computed tomography, MRI: magnetic resonance imaging, DXA: dual-energy x-ray absorptiometry, AI: artificial intelligence, ‡Fisher’s exact test

A total of 120 (77.9%) respondents were willing to learn and train on ML algorithm so it could perform some of the tasks they do as radiologists. When asked if they were involved in research projects on AI-based development, only 15 (6.5%) participants confirmed that they were involved. The aspects of AI that respondents considered important to learn were as follow: clinical applications of AI (76%), advantages and limitations of AI applications (74.7%), and technical methods such as ML or deep learning algorithms (40.9%). Top responses regarding the advantages of using AI were: speeding up processes in healthcare (79.2%; p=0.257) and helping reduce medical errors (47.4%; p=0.298). Regarding concerns on AI applications in medicine; 59 (38.3%; p=0.193) respondents were concerned that it cannot be used to provide opinions in unpredicted situations because of inadequate information. A total of 53 (34.4%; p=0.198) respondents were concerned that AI was not flexible enough to be applied to every patient. Detailed results are shown in Table 2.

Table 2

- Artificial intelligence applications in radiology corresponding to gender, training level and familiar with big data.

Questions/answers	n (%)	P-value
		Gender	Training level	Big data
Should radiologists be educated on (choose at most 3):
Clinical use of AI applications	117 (76)
Advantages and limitations of AI applications	115 (74.7)
Technical methods, such as machine/deep learning algorithm	63 (40.9)
How to get into the driver seat in using AI	62 (40.3)
How to survive the AI revolution	29 (18.8)
How to avoid the use of AI applications	11 (7.1)
What are the advantages of using AI? (choose at most 2)
AI can speed up processes in health care	122 (79.2)	0.257	0.468	0.748
AI can help reduce medical errors	73 (47.4)	0.298	0.734	0.784
AI has no emotional exhaustion nor physical limitation	43 (27.9)	1.00	0.190	0.618
AI can deliver vast amounts of clinically relevant high-quality data in real time	27 (17.5)	1.00	0.440‡	0.582
AI has no space-time constraint	16 (10.4)	0.860	0.118‡	1.00
What are you concerned about regarding the application of AI in medicine?
It cannot be used to provide opinions in unpredicted situations due to inadequate information	59 (38.3)
It is not flexible enough to be applied to every patient	53 (34.4)
It is difficult to apply to controversial subjects	21 (13.6)	0.193‡	0.198‡	0.968‡
The low ability to sympathize and consider the emotional well-being of the patient	11 (7.1)
It was developed by a less experienced medical clinician	10 (6.5)
Are you involved in any research project based on AI-based application development
Yes, testing	5 (3.2)
Yes, developing	10 (6.5)	0.485‡	0.774‡	0.027‡
No, but planning to be involved	44 (28.6)
No	95 (61.7)
Would you be willing to help in learning about ML and training a ML algorithm so that it can imitate some of the tasks you perform as a radiologist?
Yes	120 (77.9)	0.096	0.049	0.026
No	34 (22.1)

AI: artificial intelligence, ML: machine learning, ‡Fisher’s exact test

Approximately 63.4% of participants agreed that AI will augment the capabilities of radiologists, thereby making them more efficient. In addition, 111 (72.1%) participants agreed that these developments will make radiology more exciting. Interestingly, 117 (76%) participants wanted AI to be a part of residency training, while 49 (31.8%) participants were frightened by the development of AI. Furthermore, 126 (81.8%) participants agreed that AI would improve medicine in general and 135 (87.7%) participants wanted AI to be used in evaluating radiological images. In contrast, 81 (52.6%) responses did not believe that AI alone could be used to evaluate radiological images even if it achieved high diagnostic accuracy (Tables 3&4).

Table 3

- Evaluation of AI effect on radiology and medicine.

Question	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
	n (%)
Artificial intelligence will augment capability of radiologists and make radiologists more efficient	5 (3.2)	10 (6.5)	40 (26)	61 (39.6)	38 (24.7)
Radiologists should embrace artificial intelligence, and work with the IT industry for its application	0 (0)	4 (2.6)	34 (22.1)	63 (40.9)	53 (34.4)
You expect a significant acceleration of your work from new technologies (AI)	0 (0)	6 (3.9)	33 (21.4)	73 (47.4)	42 (27.3)
If artificial intelligence achieves high diagnostic accuracy, it should be used to evaluate radiological images alone	31 (20.1)	50 (32.5)	43 (27.9)	22 (14.3)	8 (5.2)
Artificial intelligence should be used as a support for evaluating radiological images	2 (1.3)	2 (1.3)	15 (9.7)	84 (54.5)	51 (33.1)

AI: artificial intelligence, IT: information technology

Table 4

- Perception of radiologist on AI in radiology.

Question	N/A	Disagree entirely	Rather disagree	Rather agree	Agree entirely
	n (%)
A potential application for AI in radiology (automated detection of pathologies in imaging examinations)	26 (16.9)	2 (1.3)	10 (6.5)	79 (51.3)	37 (24)
Artificial intelligence will improve medicine in general	14 (9.1)	3 (1.9)	11 (7.1)	77 (50)	49 (31.8)
These developments frighten me	23 (14.9)	41 (26.6)	41 (26.6)	38 (24.7)	11 (7.1)
These developments make radiology more exciting to me	18 (11.7)	13 (8.4)	12 (7.8)	66 (42.9)	45 (29.2)
Artificial intelligence should be part of residency training	17 (11)	9 (5.8)	11 (7.1)	67 (43.5)	50 (32.5)

AI: artificial intelligence, N/A: no answer

Discussion

The new field of AI received a great deal of interest from radiologists over the last few years and the medical imaging filed have been buzzing with talks and discussions regarding AI. Despite that, less than half of our participants were familiar with the concept of AI and big data. This is an important finding that might reflect scarcity of AI related dialogue in the local imaging community in Saudi Arabia, especially in residency training and day-to-day work. Reporting that journal articles, courses and research projects related to AI as the main source of information regarding AI confirms lack of formal training in the subject and can help direct efforts to the deficient areas.

Several participants in our study believed that AI would influence different radiological subspecialties, including breast imaging, molecular/nuclear imaging, thoracic imaging, and neuroradiology. These results are supported by a study conducted in European countries. Breast imaging, as a cancer screening tool, is performed frequently and to a large number of patients. Hence, it provides an enormous source of data, which is frequently needed to train AI algorithms. The large volume of cases, at the same time, is a burden that many radiologists would not mind having a helping hand in. Neuroradiology being among the top 3 responses was not surprising because of the high demand for accurate and early detection of critical clinical situations such as in acute stroke care or in tricky inconspicuous diagnosis such as cancer recurrence vs post-radiation treatment changes.

Screening, incidental-finding detection and post-processing were the top 3 expected applications of AI. These findings indicate a need for specific aids in repetitive and routine work, and the need for a safety net to make sure nothing is missed. The large volume of normal exams (in the case of screening), the lack of targeted attentive search, the fear of missing an important abnormality (in the case of incidental findings), and the tedious repetitive work (in the case of post processing) appear to demonstrate the areas were participants needed most help to achieve a quick and accurate diagnosis.

The advent of AI in radiology raises questions on how it might affect the jobs of radiologists. Interestingly, our results show a strife. Approximately 40% of our participants believed that there will be no effect, while another 40% percent anticipated that job positions will be reduced. The distance seen between AI developers lead by big corporate and the common practicing radiologist impose a fear of the unknown on the later group. Afterall, hearing regarding an impending disruptive technology without actually having a role in its development brings in ambiguity and misunderstanding. Added to that, is the limited applications of AI in everyday radiology workflow at present in most training centers. Such divided opinions identifies an important area for trainee education based on published evidence and opinions of field experts. A little more than half of the participants looked forward to a reduction in workload. Artificial intelligence techniques have been applied specifically to visual tasks such as automatic segmentation of regions of importance in imaging or analysis of images, which may reduce human errors, decrease cost, help in repetitive tedious tasks, and consequentially, reducing the workload. A similar percentage was reported in the literature in a study carried out among members of the European Society of Radiology.

Approximately half of our participants believed that most patients will not accept a report from AI applications without the supervision of a physician and would require their approval. The literature supports this. Despite the reported accuracy of AI in diagnosis, it has been proven that many situations require a physician’s knowledge, examination skills, and experience for interpretation and discussion of the diagnosis with the patient.

More than two-thirds of participants believed that the radiologists would take legal responsibility of the AI system. However, guidelines for the use of AI in healthcare are still under development. This is a highly important topic and opens up interesting ethical and legal discussions, as an example of how to protect patients’ privacy, where the institutions should make certain that their large datasets are properly utilized.

The top 3 areas our participants recommended education on were: clinical applications of AI, advantages, limitations of AI applications, and technical methods used in AI. It is not surprising that our participants chose these areas since radiologists must consider the pitfalls, vulnerabilities, and possible errors that can arise when an AI product conducts a pattern recognition. Although AI algorithms are efficient, they are often fragile and can offer inappropriate answers when confronted with images outside their knowledge set. This can include images with technical issues such as motion or beam hardening, or images obtained with poor techniques.

Approximately 64% of our participants agreed that AI would augment the capabilities of radiologists and make them more efficient and approximately 70% agreed that developments in AI make radiology more exciting. These positive views are consistent with previous studies. Artificial intelligence has the ability to detect early findings with higher accuracy, providing an early diagnosis that can lead to better patient’s outcome. Overall, image enhancement, quicker and more accurate reading with higher impact, easier reporting, and a decreased workload understandably makes the radiologists’ day more exciting.

Approximately 72% wanted to include AI as a part of their residency training. An early exposure to AI can make radiologists aware of its importance and its usefulness. Moreover, this early introduction can help in the development of new AI programs and open the way for more future applications in the field of Radiology. This is one important take-home point out of this survey.

Study limitations

The sample size and response rate were lower than expected which could be a result of the questionnaire distribution during the peak of COVID-19 in Saudi Arabia. In addition, we have not included open ended questions that could allow our participants to specify their exact concerns and issues with AI.

In conclusion, radiology residents are inadequately exposed to existing resources on the science of AI and showed interest in learning more on it. Supplementing training curriculums with resources to promote knowledge and facilitate better implementation of AI applications in diagnostic radiology is an opportunity, if implemented, in promoting the development and use of AI in patient care. In addition, there are concerns regarding the impact of AI on radiology job opportunities while the nature of work is expected to change. Artificial intelligence is expected to reduce workload by speeding up processes of reporting, reducing medical errors, and attaining efficiency in repetitive and tedious tasks. These findings can help guide better decisions by radiology training programs, radiological societies, decision makers, AI developers, and researchers.