The landscape of digital resources in radiation oncology.

In recent years, the number of educational medical resources accessible to residents and practicing radiation oncologists online has grown exponentially to include discussion boards, wikis, videos, podcasts, journal clubs, online communities, and interactive experiences to augment medical education. In this review, we identify, catalog, and critically evaluate educational websites, smartphone applications, web-based multimedia, and podcasts for radiation oncologists. Literature searches were conducted over a 2-month period (April to May 2022) using OVID-MEDLINE and PubMed with a combination of relevant search terms. Websites of relevant radiation oncology societies were reviewed for e-learning resources. Internet searches including the Google search engine, application stores, and podcast-publisher websites were conducted to identify digital resources for radiation oncology education. To ensure credibility, resources were assessed by two independent reviewers utilizing the criteria of authority, accuracy, objectivity, currency, depth, and appearance per suggested formats for evaluating digital resources in medical literature. Literature searches using OVID-MEDLINE and PubMed yielded 425 citations. Those pertinent to radiation oncology provide examples of resource development, integrations into curriculum, interactive modules, case studies, and learner experiences. The multilevel search identified 47 free digital education resources including online radiation oncology tutorials, podcasts, videos, slide sets, applications, and other interactive resources, some requiring membership or a fee for full access. The myriad online educational tools available to radiation oncology residents represent excellent resources for continuing education. This review represents the first comprehensive summary of available online education resources for radiation oncologists to guide clinicians who are increasingly reliant on digital resources, especially during the COVID-19 pandemic.

Introduction

The education provided by residency programs is considered the keystone in the development of clinical competency, skills required for practice, and preparation for board examination. Various methods of teaching exist in a radiation oncology curriculum [1]. These include physical classes, didactic lectures, group discussion, webinars, and conferences. The revolution of the internet and ease of accessibility with increasing smartphone usage in the past decade has broadened the availability of resources that one can access for study [2]. Furthermore, the COVID-19 pandemic shifted paradigms in clinical education globally with restrictions imposed in the form of social distancing and curtailment of hands on procedures [3], [4]. Face-to-face classes and case presentations saw a drastic decrease in attendance, thus limiting trainees' learning opportunities [5]. Online tools and platforms such as Zoom, WebEx, and Microsoft Teams have been increasingly utilized for knowledge sharing throughout radiation oncology departments [6]. Moreover, the decreased volume of radiation therapy has impacted hands-on training opportunities while decreasing educational opportunities via direct patient care [7].

Amidst this changing landscape of educational opportunities, clinical education has become increasingly reliant on online educational resources [8]. Online tools, such as radiation oncology-focused websites, podcasts, smartphone applications (apps), and videos, provide useful sources of information and study material for the budding resident and practicing clinician that have become essential [9], [10]. These resources vary in their accessibility, with some content existent behind paywalls, while others information at no cost to the user. With the vast number of open access websites, apps, and varied options available online, it becomes essential to ensure only reputable sources with high-yield content, regular updates, and proper functionality are utilized.

The purpose of this review is to identify, catalog, and critically evaluate knowledge-providing websites, smartphone apps, videos, and podcasts tailored to radiation oncology.

Materials and methods

To identify and evaluate web-based resources for radiation oncology education, a multilevel search was conducted over a 2-month period (April to May 2022). Literature searches were conducted using OVID-MEDLINE and PubMed with the terms “web-based learning,” “computer-assisted instruction,” “e-learning,” “radiation oncology,” “tutorials,” “continuing education,” “virtual learning,” “mobile application,” and “web-based education.”.

We identified and searched the websites of relevant and reputable radiation oncology societies (American Society of Radiation Oncology [ASTRO], European Society of Radiation Oncology [ESTRO], Radiation Oncology Education Collaborative Study Group, American College of Radiation Oncology, Association of Residents in Radiation Oncology [ARRO], Radiation Oncology Institute [ROI], and Rayos Contra Cancer) for the availability of, or reference to, web-based resources. We performed a Google search using the terms “e-learning,” “tutorials,” “continuing education,” “virtual learning,” “radiation oncology,” “radiation,” “radonc,” “rad onc,” “resident,” “residency,” “education,” “curriculum,” “radiobiology,” “bootcamp,” “online,”, “radbio,” “physics,” “smartphone application,” “app,” “web-based education,” and “video.” The smartphone apps were searched in the Google Play Store and the Apple App Store for Android and iOS devices, respectively. The apps were searched with a focus on their content, functionality, and tools available for academics or radiotherapy planning. Podcasts were searched in Apple Podcasts, Google Play Music, and Spotify. YouTube was also searched for channels dedicated to academic radiation oncology. Channels with fewer than two thousand views were excluded. Contents of these YouTube channels were scrutinized based on the intended audience. Online-video series and podcasts under a common theme or name were categorized as a single digital resource.

To ensure the credibility of digital resources, all items were assessed by two independent reviewers according to authority, accuracy, objectivity, currency, coverage, and appearance per criteria for evaluating websites and digital media in the medical literature developed by University of North Carolina librarians and outlined in Table 1 [11].

Table 1

Criteria for evaluating digital resources.*

Authority	Can the legitimacy of the organization, group, company or individual be verified?Is there any indication of the author’s qualifications for writing on a particular topic?Is the information from sources known to be reliable?
Accuracy	Are the sources for factual information clearly listed and verifiable in another source?
Objectivity	Is the information objective or subjective?Is the information biased?If there is any advertising on the page; if so, is it clearly differentiated from the information content?
Currency	Is the information current?How current are the sources or links?
Coverage	Does the information contain the breadth and depth needed?Is the information helpful?Is the information in a form that is useful, such as words, pictures, charts, sounds or videos?
Appearance	Does the site look well organized?Do the links work?Does the site appear well maintained?

Adapted from University of North Carolina Health Sciences Library [11].

Resources were evaluated on the basis of the six categories outlined and those not meeting all six were not retained. A variety of reasons deemed a resource non-credible, including unclear authorship, biased content, out-of-date content (resources not updated for 5 years or more), and uncertain use of references. These resources were not chosen for inclusion in this review. Traditional resources such as society guidelines, digital textbooks, conference-specific smartphone apps, and peer-reviewed journals were not included. Also excluded were paid courses or those requiring scheduling, question banks, social media content (i.e., Facebook, Twitter, Instagram), industry-sponsored resources, and calculators or nomograms lacking radiation oncology-focused educational materials. This review covers resources unique to general radiation oncology education and does not include additional non-specialty education, such as diagnostic radiology.

Results

Literature searches using OVID-MEDLINE and PubMEd resulted in 425 records identified, which were analyzed for references to digital resources and relevance to radiation oncology. The published studies on the use of digital resources for medical education focused on the general use of web-based learning in medical education, the growth of e-learning options, learners’ perception of and satisfaction with online education, and trainee experiences with novel technologies.

Those articles pertaining to radiation oncology covered a broad range of topics such as radiation oncology curriculum development [12], surveys reflecting current resource utilization and needs [13], interventions using digital resources to improve knowledge and treatment planning [14], how to use contour-specific resources [15], and development of critical skills when using novel technologies such as virtual reality [16]. Only a few studies directly related to identification of online resources unique to radiation oncology [17], [18].

Review of the websites of relevant radiation oncology societies revealed the existence of a substantial number of digital resources. The Radiation Oncology Education Collaborative Study Group (ROECSG) website [19] provides links to various external and internal educational resources including medical physics lectures, statistics seminars, content series geared toward medical students, board study review, and an introductory radiation oncology curriculum (IROC) designed for new resident orientation. It additionally hosts its presentations from conferences and slide sets with video. A site map outlines available resources and topics [20], [21].

ASTRO’s website [22] provides a wide variety of information from patient-focused resources to clinical guidelines for practitioners. Clinician-focused information includes links to external sites of interest (i.e., eContour), video courses on radiobiology and medical physics, and educational seminars often requiring membership and additional fees for full access. Included within ASTRO’s website is a page dedicated to the ARRO, an organization that provides webinars, monthly newsletters, resident e-learning resources, and collaborative opportunities. ASTRO additionally provides links to ROhub [23], discussion forum that requires membership for access, as well as RTAnswers [24], a patient-focused website addressing common concerns for patients undergoing radiation therapy.

ROI’s website [25] provides links to their mobile application, RO Toolbox, which is available through the Apple App Store and Google Play Store. Additionally, they provide links to previously identified resources from ASTRO including the ARRO del Regato Webinars, which provide in-depth instruction on contouring, treatment planning, and other disease site-specific instruction.

The ESTRO website [26] primarily offers scheduled online and in-person courses for in-depth training, such as FALCON, an ESTRO project to improve homogeneity in counting among radiation oncologists. No freely available resources are directly linked through ESTRO, outside of the scheduled courses offered, and reference links to external content are not available.

The ACRO’s website [27] offers a monthly podcast, webinars (membership required), and online videos published through YouTube with disease site-specific educational materials as well as conference slides and presentations.

Rayos Contra Cancer’s website [28] offers unique content that is internally produced and distributed through a Youtube channel. This content encompasses various topics from a brachytherapy planning curriculum to guides on contouring head-and-neck target volumes. They also provide links to useful educational resources for contouring, medical physics, study materials, and anatomy review.

Online digital resources identified from this review that included education and teaching resources on a variety of topics pertinent to radiation oncology are listed in Table 2. In total, 344 education and teaching resources were reviewed and critiqued to formulate the 47 resources identified. An extensive description of each resource is beyond the scope of this review; however, direct links and categorical tags are provided for readers’ further review.

Table 2

Categorized list of digital radiation oncology resources (see reference list for links).

Resource Name	Last Update	Material Covered (Tags separated by,)	Web	iOS - app	Android - app	Online-Video	Youtube Channel	Podcast	File Format (i.e.,.pdf,.ppt,.csv)	Free/Paid	Membership or Account Required
Contouring and Radiation Planning
Applied Virtual Anatomy for Radiation Oncology [32]	2020	Contouring and Radiation Planning	Y	N	N	Y	N	N	Y	Free	Y
ARRO del Regato Webinars [33]	2022	Contouring and Radiation Planning, General Reference	Y	N	N	Y	N	N	N	Free	N
Dosepedia [34]	2021	Contouring and Radiation Planning	Y	N	N	Y	Y	N	N	Free	N
eContour [35]	2022	Contouring and Radiation Planning	Y	N	N	N	N	N	N	Free	Y
EduCase [36]	2020	Contouring and Radiation Planning	Y	N	N	N	N	N	N	Free/Paid	Y
I Treat Safely [37]	2020	General reference, Contouring	Y	N	N	Y	N	N	Y	Free	Y
IGRT Online [38]	2021	Contouring and Radiation Planning	Y	N	N	N	N	N	N	Free	Y
Michigan Medicine GI Contouring Atlas [39]	2022	Contouring and Radiation Planning, Case Vignettes	Y	N	N	N	N	N	Y	Free	N
NRG / RTOG Oncology Contouring Atlases [40]	2022	Contouring and Radiation Planning	Y	N	N	N	N	N	Y	Free	N*
ProKnow Contouring Accuracy [41]	2020	Contouring and Radiation Planning	Y	N	N	Y	N	N	Y	Free	Y
Prostatdoodle [42]	2022	Contouring and Radiation Planning	Y	N	N	N	N	N	Y	Free	N
Quality Assurance Review Center / Imaging and Radiation Oncology Core [43]	2022	Contouring and Radiation Planning	Y	N	N	N	N	N	Y	Free	N
Rayos Contra Cancer YouTube [44]	2022	Contouring and Radiation Planning, Brachytherapy, Physics, General Reference, Online video	Y	N	N	Y	Y	N	N	Free	N
Radiobiology / Medical Physics
Applied Sciences of Oncology (ASO) distance-learning course [45]	2021	Radiobiology, Medical Physics	Y	N	N	Y	N	N	Y	Free	Y
ASTRO Radiation Physics [46]	2020	Medical Physics, Online Video	Y	N	N	Y	N	N	Y	Free	N
ASTRO Radiobiology Lectures [47]	2022	Radiobiology, Online video	Y	N	N	Y		N	N	Free	N
EMITEL e-Encyclopaedia of Medical Physics and Multilingual Dictionary of Terms [48]	2022	Medical Physics	Y	N	N	N	N	N	N	Free	N
HiPhy Physics [49]	2022	Radiation Oncology Physics, Online Video	N	N	N	Y	Y	N	N	Free	N
Routledge Textbooks Radiation Oncology Physics [50]	2022	Radiation Oncology Physics	Y	N	N	Y		N	N	Free	N
RSNA Physics Modules [51]	2022	Medical Physics	Y	N	N	N	N	N	N	Free	Y
General Radiation Oncology Resources
Accelerated Education Program [52]	2019	General Reference, Discussion, Contouring	Y	N	N	Y	Y	N	N	Free	N
ARROCase [53]	2022	Case Vignettes	Y	N	N	N	N	N	Y	Free	N
ASTRO [22]	2022	General reference	Y	N	N	Y	N	N	Y	Free/Paid	N*
Introductory Radiation Oncology Curriculum (IROC) [54]	2019	General Reference, Online video	Y	N	N	Y	Y	N	Y	Free	N
Learn Oncology [55]	2022	General Reference	Y	N	N	Y	N	Y	N	Free	N
National Cancer Grid [56]	2021	General Reference, Radiobiology, Contouring and Radiation Planning, Medical Physics	Y	N	N	Y	N	N	Y	Free	Y
OncologyPro [57]	2022	General Reference	Y	N	N	Y	N	N	N	Free	Y
QuadShot News [58]	2022	General Reference, Discussion	Y	N	N	N	N	N	N	Free	N
Radiation Oncology Education Collaborative Study Group [19]	2022	General Reference, Statistics, Medical Physics, Radiobiology, Introductory curriculum	Y	N	N	Y	Y	N	N	Free	N
Radiation Oncology Virtual Education Rotation [59]	2022	General reference, Case Vignettes	Y	N	N	Y	N	N	N	Free	N
RSNA Case Collection [60]	2022	Case Vignettes	Y	N	N	N	N	N	N	Free	Y
RadOnc Calc [61]	2021	General Reference, Calculator	N	Y	Y	N	N	N	N	Free	Y
RadOnc Tables [62]	2022	General Reference, Literature review	Y	Y	Y	N	N	N	Y	Free	N*
RO Toolbox [63]	2021	General Reference, Toxicity Scales, Calculator	N	Y	Y	N	N	N	N	Free	Y
The Med Net [64]	2022	Case Vignettes, Forum	Y	N	N	N	N	N	N	Free	Y
University of Cincinnati Cancer Center Statistics Seminar Series [65]	2020	Statistics, Online video	Y	N	N	Y	Y	N	N	Free	N
WikiBooks [66]	2022	General reference, Literature review	Y	N	N	N	N	N	N	Free	N
Podcasts
Advances in Radiation Oncology Podcasts [67]	2022	Podcast - General	Y	N	N	N	N	Y	N	Free	N
American College of Radiation Oncology Podcasts [68]	2022	Webinars, Podcast - General	Y	N	N	Y	Y	Y	N	Free	N*
Hormesis [69]	2020	Podcast - Medical Physics	Y	N	N	N	N	Y	N	Free	N
Practical Radiation Oncology Podcast [70]	2022	Podcast - General	Y	N	N	N	N	Y	N	Free	N
Rad-Cast [71]	2020	Podcast - General	Y	N	N	N	N	Y	N	Free	N
Radioactive Podcast [72]	2021	Podcast - General	Y	N	N	N	N	Y	N	Free	N
Red Journal Podcasts [73]	2022	Podcast - General	Y	N	N	N	N	Y	N	Free	N
The Accelerators Podcast [74]	2022	Podcast - General	Y	N	N	N	N	Y	N	Free	N
The Fraction [75]	2019	Podcast - General	N	N	N	N	N	Y	N	Free	N

Indicates that an account is not mandatory, but full access is dependent on account creation.

Discussion

The COVID-19 pandemic introduced an unexpected dilemma in the field of medical education, forcing educators and trainees to adapt in-person teaching to digital platforms. Alternative modes of education and the use of online resources for on-demand access and knowledge sharing continue to become increasingly adopted. Previously published literature focuses on the integration of e-learning resources into trainee education [12], [14], [29] or smartphone specific resources [17], [30], [31]; however, no prior publications have provided information on the availability of e-learning resources. This report represents the first comprehensive compilation and analysis of reputable resources available to radiation oncology trainees and instructors.

This review identified 47 free digital resources for radiation oncology education that are currently available online. As outlined in Table 2, a variety of online resources exist. Many specific aspects of radiation oncology education, such as contouring tutorials, disease site-specific lessons, case vignettes, introductory courses, medical physics lectures, and radiobiology videos were available at the time of this analysis. Internet searches and a review of the websites of specialty societies were the most useful sources for the identification of numerous digital resources, including introductory courses, podcasts, webinars, slide sets, and mobile apps.

Although a comprehensive identification strategy was used, the results of this study are not exhaustive of all available online resources. While the internet provides a wealth of instructional resources, locating reliable sources for learning can be daunting. During data gathering, synthesis, and compilation of this review, spanning a 2-month duration, several websites were found to no longer be active, had broken redirect links, and lacked functional account creation abilities. These findings highlight another limitation of these dynamic resources. Furthermore, an additional inherent weakness in the systematic evaluation of open access resources is the wide variety of rubrics proposed to assess the quality of these modern educational resources and, thus, the potential for inconsistency and weakness in critiquing their educational utility [1]. In this analysis, the evaluation criteria published by UNC Health Sciences Library were utilized due to generalizable criteria that could be universally applied across various multimedia resources. Additionally, some of the resources require registration or society membership to access; however, many of the educational resources are free of charge, highlighting the benefit of open access in facilitating on-demand education. Despite uniform evaluation of all resources, concerns exist regarding the content and validity of the information shared. It is upon the end user to ascertain whether the information provided is credible or not, for which we advise the use of criteria like that outlined in Table 1. The compilation of resources presented herein represent a curated and comprehensive list of available digital resources produced by publishers with strong reputations for accuracy and expertise.

Conclusion

This review represents the first known comprehensive review of the vast number of digital educational resources for radiation oncologists: both practicing clinicians and trainees. Digital resources exist in a variety of e-learning formats, including tutorials, self-directed learning modules, slide sets, podcasts, and videos. As identified by the literature review performed in this study, e-learning is being actively integrated into radiation oncology training programs. Web-based and digital resources serve as adjunct knowledge tools for educators and trainees. Awareness of available web-based education resources will enhance radiation oncology residency education and clinical competence, and likely represent the future of medical education as they allow for self-directed and supplementary education.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.