Impact of High-Dose-Rate Brachytherapy Training via Telehealth in Low- and Middle-Income Countries.

PURPOSE: Our objective was to demonstrate the efficacy of a telehealth training course on high-dose-rate (HDR) brachytherapy for gynecologic cancer treatment for clinicians in low- and middle-income countries (LMICs).
METHODS: A 12-week course consisting of 16 live video sessions was offered to 10 cancer centers in the Middle East, Africa, and Nepal. A total of 46 participants joined the course, and 22 participants, on average, attended each session. Radiation oncologists and medical physicists from 11 US and international institutions prepared and provided lectures for each topic covered in the course. Confidence surveys of 15 practical competencies were administered to participants before and after the course. Competencies focused on HDR commissioning, shielding, treatment planning, radiobiology, and applicators. Pre- and post-program surveys of provider confidence, measured by 5-point Likert scale, were administered and compared.
RESULTS: Forty-six participants, including seven chief medical physicists, 16 senior medical physicists, five radiation oncologists, and three dosimetrists, representing nine countries attended education sessions. Reported confidence scores, both aggregate and paired, demonstrated increases in confidence in all 15 competencies. Post-curriculum score improvement was statistically significant (P < .05) for paired respondents in 11 of 15 domains. Absolute improvements were largest for confidence in applicator commissioning (2.3 to 3.8, P = .009), treatment planning system commissioning (2.2 to 3.9, P = .0055), and commissioning an HDR machine (2.2 to 4.0, P = .0031). Overall confidence in providing HDR brachytherapy services safely and teaching other providers increased from 3.1 to 3.8 and 3.0 to 3.5, respectively.
CONCLUSION: A 12-week, low-cost telehealth training program on HDR brachytherapy improved confidence in treatment delivery and teaching for clinicians in 10 participating LMICs.

INTRODUCTION

Cancer death disproportionately burdens low- and middle-income countries (LMICs), which account for 70% of all cancer deaths worldwide.[1-4] Limited capacity to provide necessary cancer therapy in LMICs is multifactorial and, beyond material resource constraints, is caused by a lack of provider training and infrastructure development.[5] For instance, locally advanced cervical cancer, a leading cause of death in women in LMICs,[6-8] is often curable with high-dose-rate (HDR) brachytherapy.[9-12] HDR brachytherapy involves the delivery of a radiation treatment by temporarily placing a high-activity radioactive source inside or near a target volume. Widespread implementation of HDR brachytherapy in LMICs, where more than 230,000 women die annually as a result of advanced cervical cancer,[7,13] could affect survival for hundreds of thousands of patients each year.

CONTEXT

Key Objective

Can advanced high-dose-rate (HDR) brachytherapy training be delivered to medical physicists and other related radiation oncology professionals in an international setting using video teleconferencing to create increased provider confidence?

Knowledge Generated

Confidence scores of attending participants increased significantly in all 16 domains of HDR brachytherapy in the telehealth curriculum, in addition to two general confidence domains. Consistent attendance and positive qualitative feedback indicated a positive experience for both attendees and educators, with many notable changes in clinical practice resulting from this initiative.

Relevance

Provider training via expert-led videoconferencing education curricula may be a viable, efficient vehicle to expand access to advanced cancer therapy in low-resource areas globally.

The age-standardized incidence of invasive cervical cancer in African LMICs is more than three times that of the United States or Europe[14] and, on the basis of current available data, only 19 out of 52 African nations offer brachytherapy.[15] In Africa, there is now a strong political push for more centers to offer brachytherapy. However, the lack of clinician training opportunities is a significant barrier.[16-18] Similar struggles are taking place in LMICs in the Middle East and in some Asian countries.[18] Telehealth education may be a valuable vehicle for offering provider training in proven life-saving therapies to reduce the existing cancer-related mortality rate.

Rayos Contra Cancer (RCC) is a nonprofit organization that seeks to provide needed education and training to radiation oncology clinicians in LMICs using the Project ECHO (Extension for Community Healthcare Outcomes) telehealth model (ECHO Institute, Albuquerque, NM). This growing global training and mentorship initiative has been shown to enhance provider knowledge and treatment outcomes in numerous medical fields.[19-21] Using this model, RCC seeks to enhance provider education in LMICs to promote the development of regional capacity to deliver high-quality, timely, and affordable cancer treatments.

To this effect, RCC conducted a novel pilot study that administered HDR brachytherapy training via videoconferencing to radiation oncology clinicians in Africa, the Middle East, and East Asia. The aims of the telehealth curriculum were to expand provider confidence in HDR brachytherapy delivery and teaching. This program was implemented with the expectation that the knowledge and confidence gained by the trainees would promote the successful growth of advanced radiotherapy treatments in these regions. The purpose of this study was to evaluate the feasibility of a radiotherapy curriculum via telehealth in an international LMIC setting and to explore the reported experiences and learning outcomes of both educators and participants.

METHODS

Site Recruitment

Radiation oncology departments from 10 cancer centers in Egypt, Ghana, Iraq, Jordan, Nepal, Nigeria, Mozambique, Qatar, and Zambia were selected for training. Selection was based on interest in an educational partnership and an urgent need for training. All 10 centers had recently acquired an HDR afterloader or were in the planning phases of receiving one within the next year. Medical physicists from each center, as well as radiation oncologists, dosimetrists, and other clinical staff involved in HDR brachytherapy patient care, were invited to participate. Clinic-specific data were collected from each center, including the number of patients treated weekly and annually.

Curriculum Development

A team of volunteer faculty with expertise in the clinical and technical aspects of HDR brachytherapy was assembled to design and implement a 3-month training program remotely (Fig 1). The faculty included five radiation oncologists and 12 medical physicists from the United States and Australia with past experience in global health. Additional undergraduate and medical student volunteers provided coordination and administrative support throughout the length of the course. Relying on Kern’s model for curriculum development and a needs assessment conducted through informal interviews and electronic REDCap surveys of liaisons from each partner center, the academic faculty created a curriculum tailored primarily to medical physicists. This HDR brachytherapy curriculum developed by RCC was oriented predominantly toward medical physicists, and it included HDR commissioning, treatment applicator modalities, dose prescription, treatment planning, and treatment delivery (Table 1).

FIG 1

High-dose-rate brachytherapy program development model.

TABLE 1

HDR Brachytherapy Curriculum

Curriculum Implementation

Sixteen live video conference sessions were scheduled once or twice per week from June 30, 2019, through October 15, 2019. Each session was 1 hour long and was conducted remotely via Zoom Web conferencing (Zoom Video Communications, San Jose, CA) in English. Live questions and/or comments from the audience were encouraged via microphone and messaging board and were answered with the support of a session moderator. Each session was recorded, and presentation material was made available to the participants and the end of a session for offline review and local dissemination.

Outcome Measures

Participation.

Participant attendance was recorded at each session, both for the participating centers and for the individual participants. A certificate of completion was awarded to participants who attended at least 70% of the sessions. The total number of participant-hours was calculated, and participant and center-specific attendance rates were plotted in Microsoft (Redmond, WA) Excel (2011) as a function of program progress.

Participant confidence.

Surveys were designed to explore participant experiences and to generate hypotheses to guide future work. All participants were asked to complete both a pre- and a post-curriculum 17-item (15 practical domains that were based on sessions and two general confidence questions) survey measuring confidence on a 1-5 Likert scale. Surveys items were developed based on a review of previous telehealth educational programs[19,22,23] and were adapted for use in an international radiation oncology setting with input from academic radiation oncologists in the United States and Middle East and medical physicists in a 10-person roundtable discussion. All surveys were pretested and discussed for relevance with one to two center liaisons before being distributed in an anonymous electronic format using the Research Electronic Data Capture (REDCap) database (Vanderbilt University; Nashville, TN; Data Supplement).

REDCap. End points included confidence in 15 practical domains of HDR commissioning, treatment applicator modalities, dose prescription, treatment planning, and treatment delivery. Using branching logic in the survey form, seven of these domains were assessed only in respondents who self-identified as medical physicists. In addition, each participant was asked to rate their confidence in (1) providing HDR brachytherapy overall and (2) teaching HDR brachytherapy overall before and after completion of the program on a 1 to 5 scale, with 1 indicating no ability and 5 indicating expert ability. Surveys were recorded through a REDCap.

The mean and standard deviations for pre- and post-program confidence scores were calculated using Microsoft Excel (2011). An a priori decision was made to conduct a subgroup analysis of all participants who submitted both a pre- and a post-program survey, to analyze learning outcomes, and change-in-score calculations included only paired pre- and post-survey responses. For paired data, nonparametric Wilcoxon signed rank tests (significance level of < .05) were conducted using R Statistical Software to evaluate whether confidence scores improved significantly after the curriculum.

Educator experience.

On the basis of qualitative feedback from educators in other international RCC programs, a survey was generated in REDCap to assess educator experience, perceptions, and interest in future related opportunities. At the conclusion of the curriculum, this survey was administered anonymously via REDCap online format to all volunteer educators to assess their experience with the HDR brachytherapy telehealth curriculum. The descriptive survey results were compiled and exported using the REDCap data reporting and analysis features.

Qualitative Participant Feedback

Furthermore, descriptive feedback from clinics and educators was recorded via conference calls throughout and at the conclusion of the course. The sessions can be publicly accessed[24] (recorded from live sessions and released after conclusion of the curriculum). Qualitative feedback was solicited from participants and site liaisons via e-mail, WhatsApp (Facebook; Menlo Park, CA), and text throughout the duration of the program, and they gave permission for de-identified comments to be stored and analyzed for quality improvement and research purposes. Feedback related to changes in practice, clinical capabilities, and learning outcomes was stored and reviewed informally by study personnel, and a basic coding system was developed on the basis of the common themes noted in these responses. This schema was applied to classify recorded comments by thematic category.

This investigation was performed according to Declaration of Helsinki principles. No institutional review board approval was required because of the nature of this research.

RESULTS

Attendance

A total of 326 attendance hours was recorded during the curriculum. Representing 10 cancer centers in nine countries, 46 clinicians with a mean of 5.6 years of clinical HDR brachytherapy experience participated virtually in the course. A total of 31 medical physicists, five radiation oncologists, and seven other clinical staff provided demographic information, as listed in Tables 2 and 3. The average cumulative attendance per participant was 44%. Participating centers were represented by one or more attendee in 80% of the sessions on average, ranging from 10 (63%) to 16 (100%) sessions. Aggregated participant and center attendance per session is shown in Figure 2.

TABLE 2

Participant Demographic Information (n = 43)

TABLE 3

Participating Center Demographic Information (n = 10)

FIG 2

Session attendance over time. HDR, high-dose-rate.

Participant Confidence

Pre- and post-curriculum surveys were completed by 38 and 17 participants, respectively, yielding 12 paired responses. Aggregate confidence scores increased across all 15 curriculum topics (Table 4). Increases in scores for paired responses were statistically significant (P ≤ .05) for 11 of 15 topics, as summarized in Table 5. Survey respondents also reported their preprogram and post-program confidence in two general domains: (1) providing HDR brachytherapy services and (2) teaching HDR brachytherapy. For confidence in running a safe HDR brachytherapy program, paired post-curriculum survey scores improved from 3.1 to 3.8 out of 5 (P = .077). For confidence in teaching other providers to run an HDR brachytherapy program, the post-curriculum survey scores improved from 3.0 to 3.5 out of 5 (P = .0159), as seen in Table 5.

TABLE 4

Pre- and Post-Curriculum Confidence Scores

TABLE 5

Paired Pre- and Post-Curriculum Confidence Scores

Educator Experience

Seven of 17 educators completed post-curriculum feedback surveys. Complete questions and responses are listed in Table 6. Notably, 56% had prior experience in teaching a virtual telehealth lecture. One hundred percent of surveyed educators strongly agreed that the experience was meaningful, and all strongly agreed that they would be willing to lead another similar session in the future. All educators strongly agreed that the ECHO model has the potential to meaningfully educate future clinicians, and all strongly agreed that they would recommend this style of program to a colleague.

TABLE 6

Telehealth Educator Experience at Conclusion of Curriculum (n = 7 respondents)

Ten qualitative comments were submitted as formal feedback to RCC liaisons or study personnel by participants (n = 2) and site liaisons (n = 8). An informal analysis of these 10 comments revealed four common themes, which are listed in Table 7, including (1) general positive feedback (80%); (2) requests for more information, scholarly clarification, or skills training (40%); (3) reported changes in clinical practice on the basis of program session or sessions (70%); and (4) reported increase in confidence in a treatment modality already offered (50%). No negative general feedback was received.

TABLE 7

Qualitative Participant Feedback (n = 10 comments analyzed)

DISCUSSION

This program evaluated the implementation of a novel telehealth curriculum for clinician training in HDR brachytherapy. Participating sites were cancer centers in LMICs in Africa, the Middle East, and East Asia selected because of the regional burden of advanced cervical cancer and limited brachytherapy availability. Telehealth-based solutions hold the promise of addressing the shortage of radiotherapy training opportunities, particularly in LMICs where non-governmental organization and governmental efforts have accelerated the procurement of modern radiotherapy technology.[18,22]

One aim of this study was to evaluate the program’s effectiveness in engaging and retaining learners. Attendance was highest for the lead correspondent for each center, with lower attendance for other participants who were not our primary contacts for each center. Notably, our reported data underrepresents the true attendance of non-lead participants, because several participants reported “sharing a screen” with others in their center rather than individually logging in. Although the number of participants per session decreased slightly (approximately one less per session), attendance by center showed no decline over the program's duration. In all, these data suggest mild participant attrition and overall effective retention of participating centers throughout the program’s duration.

Another aim was to generate increases in confidence related to HDR brachytherapy services and teaching. Our goals were to improve the centers’ confidence in providing HDR brachytherapy services across 15 different curriculum domains, in addition to increasing clinicians’ general confidence in both providing services and training other providers. In general, the mean reported changes in confidence were encouraging. The mean increase in participants’ topic-based confidence rating was statistically significant (P < .05) in 11 of the 15 key HDR brachytherapy domains, suggesting that, overall, participation in the curriculum significantly enhanced clinicians’ confidence in their knowledge.

To further address site-specific training needs, educators have been paired with the individual sites and in-person training has been arranged for more than one half of the sites. At the time this article was drafted, our partner educators had made site visits for the purpose of needs-based training to our partner centers in both Nepal and Zambia. Both completed visits were received with an overwhelmingly positive response from both the visiting educators and the site liaisons. Because of COVID-19–related travel restrictions, visits are pending to partner centers in Nigeria, Jordan, Iraq, and Egypt.

Although our study findings generally support the model of telehealth education via expert-led videoconferencing and provide insight into future directions, the study had inherent limitations. The study size and the number of participants were relatively small. Our analysis did not examine the relationship between the changes in outcomes and numerous potentially confounding contributing factors (prior expertise, center, title, and so on). The same survey was provided to all providers, but the clinical significance of the surveyed changes in confidence was, in reality, more nuanced and dependent on the provider role. This curriculum was oriented to predominately benefit medical physicists, and positive findings in this study may not apply to all participants equally and may not be generalizable to all clinicians in other LMIC settings. The study was offered only in English, without available translators to enhance learning in non-English languages. This may limit the expansion of this model in limited-English settings. Many of the expert educators were from the United States or Australia, and it may have been beneficial to have a more diverse representation in the educators, particularly content experts from the Middle East and Africa who are more likely aware of region-specific challenges and nuances in HDR brachytherapy service needs and infrastructure. Finally, the used assessment tools likely do not capture the extent of the impacts, positive or negative, of this program. Using self-reported confidence carries inherent bias and is a nonobjective measure of true learning, and it should be acknowledged that improvements in self-reported confidence represent Kirkpatrick level 1 assessment of learning evaluation (reaction only).[25] This type of data is highly subject to self-report bias, expectation bias, and hindsight bias. Although informal quizzes and tests of knowledge were incorporated sporadically in the curriculum, which would represent level 3a outcomes of continuing education on the basis of Moore’s Seven-Level Outcomes Model, these were not analyzed formally.[26] Qualitative feedback from participants included numerous examples of direct changes in behavior (Kirkpatrick level 3 evaluation), but the process of measuring and evaluating these changes was informal and lacked the rigor needed to draw firm conclusions.

Despite these limitations, this pilot study has highlighted the potential value of videoconferencing telehealth education as a modality to bring needed radiotherapy training to emerging cancer centers in LMICs in Africa, the Middle East, and East Asia. The RCC program development model may continue after program completion with iterative curriculum revision and scaling, facilitating both curriculum improvement and scaling to reach more centers in need at a low cost. Additional investigation with a larger, more robust sample of surveys including both qualitative and quantitative measures of learning could validate the reported learning outcomes observed. Regarding program evaluation, for future iterations of this and similar programs, more effort could be given to selecting tools that capture more nuanced perspectives on participants’ learning, growth, and overall confidence (eg, metrics that capture finer changes in the continuum of confidence, knowledge, and performance). Finally, although knowledge confidence gains seemed significant, it must be stressed that new knowledge gained from such a rigorous curriculum requires time to digest and implement in the local clinical setting. Increased confidence in a trainee's own expertise and the ability to pass it on by teaching comes largely through repeated application and practice over time. On-site follow-up visits that are based on a synergistic hands-on teaching agenda by volunteer educators after curriculum completion are desirable for solidifying concepts learned by program attendees. Future investigation could look specifically at the practical changes in behavior tied to each unique session, and a long-term monitoring plan for implementation would likely benefit from results-level analysis of the impact on patient care, including quantification of the impact on treatment delivery, number of patients treated, clinic efficiency, and ultimately, patient outcomes.

This intervention applied the ECHO model of international education partnership to deliver a 16-session HDR brachytherapy curriculum to 46 clinicians representing 10 cancer centers in LMICs throughout North Africa, the Middle East, and East Asia. The investigators were able to successfully build a multinational panel of expert educators, develop a curriculum relevant for nascent HDR brachytherapy centers, and deliver this curriculum via videoconferencing technology. Participants of varied clinical backgrounds demonstrated consistent attendance and reported positive learning outcomes, with significant P values for mean confidence score increase in all selected brachytherapy topics, running a brachytherapy program, and the ability to educate other brachytherapy providers. Ultimately, the authors believe this study provides support for the use of expert-led videoconferencing in global telehealth education initiatives. The ECHO model used in this intervention could pave the way for increased connectivity and knowledge dissemination as radiation oncology programs seek to expand their global impact, placing an emphasis on both education capacity and regional partnership. We believe that those completing the program will be better poised to serve as regional leaders in HDR brachytherapy education and expansion. This same educational capacity development model could be applied to any cancer treatment modality, both in the field of radiation oncology and beyond.