A multi-institutional assessment of COVID-19-related risk in radiation oncology.

PURPOSE: The COVID-19 pandemic has presented challenges to delivering safe and timely care for cancer patients. The oncology community has undertaken substantial workflow adaptations to reduce transmission risk for patients and providers. While various control measureshave been proposed and implemented, little is known about their impact on safety of the radiation oncology workflow and potential for transmission. The objective of this study was to assess potential safety impacts of control measures employed during the COVID-19 pandemic.
METHODS: A multi-institutional study was undertaken to assess the risks of pandemic-associated workflow adaptations using failure mode and effects analysis (FMEA). Failure modes were identified and scored using FMEA formalism. FMEA scores were used to identify highest-risk aspects of the radiation therapy process. The impact of control measures on overall risk was quantified. Agreement among institutions was evaluated.
RESULTS: Thirty three failure modes and 22 control measures were identified. Control measures resulted in risk score reductions for 22 of the failure modes, with the largest reductions from screening of patients and staff, requiring use of masks, and regular cleaning of patient areas. The median risk score for all failure modes was reduced from 280 to 168. There was high institutional agreement for 90.3% of failure modes but only 47% of control measures.
CONCLUSIONS: COVID-related risks are similar across oncology practices in this study. While control measures can reducerisk, their use varied. The effectiveness of control measures on risk may guide selection of the highest-impact workflow adaptions to ensure safe care in oncology.

The COVID-19 pandemic has posed serious challenges for the delivery of quality care in healthcare systems. This is acutely felt in the oncology setting, where patients are a high-risk population often being older or challenged with significant comorbidities and having systematic immunosuppression due to treatments. Cancer patients and those with a cancer history have been found to be at serious risk for complications and death from this disease [1], [2], [3], and also may be more vulnerable to contracting the disease, with as much as a two-fold elevated risk [4]. This small-number study has been confirmed in a survey of 17 million adults in the UK suggesting that a history of cancer is a significant risk factor for increased mortality in patients hospitalized for COVID-19 [2]. Nosocomial transmission (i.e. originating in a hospital setting) is a particular concern with emerging viruses, and has been described [5]. Radiation oncology is of particular interest because patients often undergo extended treatments and are in close contact with staff for positioning, heightening the potential for nosocomial transmission. Due to the concerns of poor outcome and inherent susceptibility of cancer patients to COVID-19, bringing cancer patients to the hospital for daily radiation therapy treatments must be done in a carefully controlled environment that limits the potential for viral transmission.

Given the potential impact on oncology patients, it is especially important to understand and control the risks associated with COVID-19. One challenge to this is the rapid evolution of the pandemic. In January 2020, the World Health Organization classified COVID-19 as a Public Health Emergency of International Concern. By March 11, the disease was classified as a pandemic. In the US, the first reported case of COVID-19 was on January 19th in Washington state. As of this writing, in June 2020, the number of confirmed cases in the US approaches 2 million and number of deaths has exceeded 100,000 (Fig. 1 ) [6]. This rapid evolution makes it difficult to navigate a response and provide quality care for cancer patients. The medical community has responded quickly with guidance and sharing of experiences, including sharing institutional [7], [8], [9], [10], [11], [12] or national experiences [13], [14], [15], pandemic-focused oncology treatment recommendations [16], [17], [18], [19], [20], [21], [22], [23], recommendations for prevention of nosocomial transmission in oncology clinics [24], [25], and an exploration of the impact of the pandemic on radiation oncology practices [26], [27], [28]. While the literature on this topic is extensive and growing, to our knowledge there has not been a formal evaluation to assess risk or to measure the potential impact of existing recommendations in the radiation oncology practice. This is a natural place for radiation oncology to contribute, as formal risk assessment tools like failure mode and effects analysis (FMEA) are often used in this specialty and radiation oncology professionals are experienced with these techniques [29], [30], [31].

Fig. 1

Number of confirmed coronavirus cases and deaths in the US over time. Relevant infection control measures at institution A of this study were implemented at the various times shown.

The purpose of this study is to perform a multi-institutional assessment of risks related to COVID-19 workflow adaptations in radiation oncology clinics and to evaluate the potential impact of associated control measures on safety of the radiation therapy process as well as the potential for nosocomial transmission. The goal is to quantify the effectiveness of the various control measures and workflow adaptations associated with the COVID-19 pandemic and also to describe a framework for adapting these measures to new situations or for evaluating potential new measures.

Methods

Failure Mode and Effects Analysis (FMEA)

FMEA is a risk-assessment tool that can be used to prospectively identify weaknesses in a process. FMEA is used in manufacturing and systems engineering and has been widely used in healthcare [32], [33], [34]. Its use for radiation oncology has been outlined in the American Association of Physicists in Medicine (AAPM’s) Task Group 100 report [31]. FMEA provides a systematic way to evaluate the steps of a process, identify the magnitude of risk associated with each step, and develop interventions to improve the process.

An FMEA is performed by identifying a process map and visualizing components of the process with a multidisciplinary team. At each step in the process map, challenges that may lead to errors or potential unsafe conditions are identified. Each of the ways that the workflow can fail is called a failure mode. A list of these failure modes is compiled, along with their possible causes and effects. For each failure mode, the team provides scores for each of three criteria: Occurrence, Severity, and Detectability. Each variable is scored on a scale from 1 to 10, with 10 being the most likely to occur (O), the most severe (S), or the least detectable (D), respectively. The product of O, S, and D produces a Risk Priority Number (RPN) which assigns a number to the perceived risk of each failure mode. Failure modes with the highest RPN represent steps in the process where interventions or control measures should be prioritized to reduce risk.

In this study, an FMEA was used to identify the ways in which radiation oncology departments may respond to the COVID-19 pandemic. In particular, it assessed the potential effectiveness of new procedures and viral control measures utilized in several radiation therapy departments across the United States. Six institutions participated in this study, representing diverse geographical regions of the US and areas of different COVID-19 case numbers during the pandemic (supplementary table A). Each group participated in the FMEA processes and assessment of control measures.

Control measures

Throughout the pandemic, many changes to the hospital-wide and radiation therapy specific processes were made to help reduce the spread of the virus (Fig. 1 from one center in this study). Such changes included the implementation of telehealth visits, requirements for face masks for all staff and visitors, and patient screening. In this study, new processes (e.g. universal masking) or novel adaptations of existing processes (e.g., telehealth visits) are considered “control measures.” A list of control measures was identified by the participating institutions, which was cross-checked against control measures discussed in a subsequent ASTRO survey in late April 2020 [26]. All of the control measures identified in that survey were also found in the present study.

Failure Mode Identification and Scoring

In order to assess the safety impact of each new control measure, an FMEA was undertaken using the methodology described previously. A process map for a patient undergoing radiation therapy treatment was developed with an interprofessional team at the Department of Radiation Oncology at institution A of this study, including nurses, physicians, physicists, and therapists.

Potential failure modes were identified from this process map. The severity, occurrence, and detectability of each failure mode were determined by institutional consensus. The scales for scoring each failure mode are shown in Table 1 . This follows a ten-point scale commonly used in FMEA [31] but with changes made to reflect the COVID-related practices considered in this investigation. The risks focused on were impact on quality of cancer care and the potential for viral transmission. The RPN for each failure mode was calculated by multiplying the individual scores. RPNs were then used to rank each failure mode.

Table 1

Description of the 10-point scales used to generate risk priority numbers for each failure mode.

	Severity (S)	Occurrence (O)	Detectability (D)
1–2	Minimal effect on patient or staff	Has not been seen during the pandemic, but is possible	Easily detected, >90% chance of being caught
3–4	Unlikely to cause infection or reduced care quality	Seen at least once during pandemic	Easy to detect with multiple checks
5	Likely increased chance of infection or reduced care quality	Seen several times during the pandemic	5% chance of being caught
6–8	Highly likely to cause infection or reduced care quality	Seen approximately once a week during the pandemic	Very difficult to detect
9–10	Certain to cause infection or reduced care quality	Has happened with almost all patients	Almost impossible to detect, 0.1% chance of being caught

The failure modes and control measures were initially developed at one center and were then evaluated by the five other institutions participating in this study. Each institution evaluated which failure modes were relevant in their practice and which control measures were in place. Additions to the initial lists were also made. Additionally, each institution provided feedback on each failure mode as well as their own scoring of each failure mode. An institutional consensus score was generated for each item. This score ranges from 1 to 6, where a 1 indicates that this is relevant only at one institution and a 6 means that all collaborators had a similar failure mode or control measure.

To quantify the impact of implementing control measures on overall risk, the failure modes were scored twice using the FMEA methodology: first as if no preventative actions were in place and second with all of the control measures in place. For each control measure, the number of failure modes it impacted was assessed. In addition, to determine the impact of each control measure on the overall risk, a relative risk reduction score was defined. This was motivated by the fact that the number of failure modes that are affected with each control measure does not alone indicate that control measure’s potential impact on risk. For example, a particular control measure may affect many failure modes, but those failure modes may be relatively low-risk so the impact may be minor. On the other hand, a particular control measure may affect only a few failure modes but those failure modes may be high-risk so the expected impact is large. We, therefore, also report a relative risk reduction score. The relative risk reduction score was calculated as the sum of RPNs for each affected failure mode divided by the total risk budget (defined as the sum of RPNs for all possible failure modes). A higher relative risk reduction score indicates that the control measure has a higher potential impact on the overall risk. A score of one would mean that the control measure affects all potential failure modes.

Results

A list of 33 failure modes were identified in the radiation therapy workflow for patients during the COVID-19 pandemic. Table 2 shows the ten failure modes with the highest RPN scores. The full list of 33 failure modes is included as supplementary table B. Failures modes were found to fall into two categories: infection spread (n = 22) and changes to quality of care (n = 11). Those with the highest RPNs related to failures in infection control. Examples included viral transmission from asymptomatic staff and patients present in the department, lack of proper cleaning of common areas, and patient and staff non-disclosure of symptoms. Fig. 2 illustrates the range of RPN scores for all failure modes listed in Table 1, with reduction in risk score demonstrated for 22/33 control measures. RPN values both before control measures and procedures were in place as well as after are shown. For most failure modes, the control measures and interventions reduced the risk score. The median RPN changed from 280 (interquartile range: 54–560) to 168 (interquartile range: 50–224). However, for the lowest-scoring failure modes, the RPN tended to stay the same or increase slightly. For example, implementing telehealth visits can have negative effects on care quality or accessibility, as in failure modes 21 and 22.

Table 2

Failure modes identified for the radiation therapy process during COVID-19. Risk scores are S, severity, O, Occurrence, and D, detectability for an overall risk priority number (RPN). Institutional consensus is the number of institutions (out of 6) at which each failure mode applies. Failure modes are arranged ordinally and correspond to Fig. 2 from highest to lowest RPN. Starred rows indicate failure modes whose control measure implementation increased or did not impact RPN.

Failure Mode Rank	Failure Mode	Cause	Effect	S	O	D	RPN	Institutional Consensus
1	Infectious person not caught at initial screening	Person is asymptomatic	Patient or staff exposure from infectious person	7	8	8	448	6
2	Infectious person not caught at initial screening	Person is unclear about symptoms or mistakes it for a chronic condition, treatment effect, etc.	Patient or staff exposure from infectious person	7	6	8	336	6
3	Waiting/changing room not cleaned routinely	No policy is in place, Lack of cleaning supplies or staff	Patient or staff exposure from infectious patient	6	6	9	324	6
4	Infectious person arrives at department with unknown symptom status	Patient or staff is unclear about what constitutes exposure to a positive person	Patient or staff exposure from infectious patient	7	5	8	280	6
5	Infectious person arrives at department with unknown symptom status	Visual marker for screened status not used (piece of paper, sticker)	Patient or staff exposure from infectious patient	7	4	9	252	2
6	Infectious person not caught at initial screening	Initial hospital and department entry screening not effective, person enters from unsecured entrance	Patient or staff exposure from infectious person	7	4	8	224	6
7	Infectious person not caught at initial screening	Guidelines for symptoms were unclear and changing	Patient or staff exposure from infectious person	7	4	8	224	6
8	Infectious patient arrives at department with unknown symptom status	Arrival instructions (e.g., letter from patient care coordinator) given to patient are not read	Patient or staff exposure from infectious patient	7	4	8	224	3
9	Infectious staff member arrives at department	Staff screening not implemented, not effective	Patient or staff exposure from infectious staff	7	4	8	224	6
10	Treatment or exam room is not disinfected after patient is treated	No procedure in place, clinical staff is not compliant	Patient or staff exposure from infectious patient	8	3	9	216	6

Fig. 2

Failure mode risk scores (RPN, risk priority number) before and after control measures were implemented.

There was high agreement across all six institutions for 30 of 33 failure modes, meaning that similar issues arise in different clinical environments. The 3 failure modes with a lower institutional consensus score were related to issues of patient communication. For example, some institutions relied on the patient to carry a piece of paper with their screening status or to read all informational material given to them, whereas other institutions did not require these behaviors of their patients.

As shown in Table 3, 21 control measures in response to the pandemic were identified. 38% of the changes in policies and procedures applied to all patients within the department. In Fig. 2, the effects of all control measures on the RPN scores are illustrated. To determine which control measures represent the greatest risk reduction, the relative risk reduction score is shown in Table 3 . The largest risk reduction was from the following measures: initial screening of patients, regular cleaning of patient areas, and requiring masks.

Table 3

Control measures in response to COVID-19. The relative risk reduction assesses the impact of each control measure on the overall risk. Institutional consensus represents the number of institutions (out of 6) in which each control measure is in use. Asterisks indicate control measures with ≥90% compliance in a survey from late April 2020 of 222 radiation oncology leaders in the US [22].

Label	Applies to	Measure	# of failure modes affected	Relative Risk Reduction	Institutional Consensus
A1	All patients	Screening at clinic front desk	8	0.52	5*
A2	All patients	Screening at hospital entrance, limit access points	7	0.46	6
A3	All patients	Pre-RT COVID testing	7	0.40	2
A4	All patients	Sanitizer on entry/exit of treatment room	6	0.38	5
A5	All patients	Masks required in all areas of the hospital	6	0.36	4
A6	All patients	Social distancing, waiting rooms rearranged and decluttered	6	0.31	6*
A7	All patients	Screening call from nurse prior to arrival	4	0.25	4
A8	All patients	Telehealth visits offered	4	0.02	2
A9	All patients	Visitors limited to 1 person	2	0.13	2*
S1	Staff	Staff in surgical masks in patient areas	9	0.57	6*
S2	Staff	Telework for staff highly encouraged	4	0.18	4
S3	Staff	Staff screening, self-assessment, and testing if indicated	5	0.09	6*
S4	Staff	Staff training on how to don/doff PPE with droplet precautions	2	0.06	6
O1	Operations	Common areas and changing rooms are cleaned frequently	9	0.56	6*
O2	Operations	Aerosolizing procedures are avoided (e.g. intubation, anesthesia)	8	0.50	5
O3	Operations	Eliminate use of ABC breathing device	8	0.50	1
O4	Operations	All immobilization devices stored in single use plastic bags	7	0.47	5
C1	Symptomatic or COVID + patients	If aerosolizing procedure must be used then airborne precautions are observed. Room is kept clear for 6 air exchanges post treatment	2	0.10	4
C2	Symptomatic or COVID + patients	Staff observe droplet precautions during treatment (gown, surgical mask, face shield, gloves)	2	0.08	5
C3	Symptomatic or COVID + patients	Patients treated on one machine at the end of the day. Cleaning afterwards.	2	0.06	5
C4	Symptomatic or COVID + patients	R&V system has a pop-up alert for COVID + patients	2	0.06	4
C5	Symptomatic or COVID + patients	Patients get a mask when they come in and are treated with it on. Patients are escorted in by nurse and kept separate.	1	0.03	6
C6	Symptomatic or COVID + patients	Taking a COVID + patient off precautions requires two negative tests at least 24 hour apart	1	0.01	4

Each institution was asked to score the control measure on relevance to their site of practice. The measures were stratified into 3 levels of agreement across clinics. Those that applied to <2 clinics were considered to have low institutional consensus. A medium institutional consensus score corresponds to 3–4 sites, and high is 5–6 sites. The control measures with the highest institutional agreement also had the highest relative risk reduction (Fig. 3 ).

Fig. 3

Institutional consensus for the control measures listed in Table 2.

Discussion

Given that cancer patients appear to be at increased risk for contracting COVID [4] and for experiencing negative outcomes from the disease [2], it is important to control the clinical environment in such a way as to reduce nosocomial viral transmission and maintain the quality of care. Experience in China demonstrated that the institution of control measures prevented staff and patient infection despite high community prevalence [35]. The present report provides a framework for further understanding this. It suggests that there is good agreement among institutions regarding which processes have the greatest risk of nosocomial transmission. It also suggests that the control measures currently in place can substantially reduce risk, though the magnitude of risk reduction depends on what measures are employed and how reliable they are.

There are several interesting and unexpected findings in this report. First, some of the most effective measures that reduce risk were those that limited the spread of the virus from asymptomatic patients or staff. Examples include requiring the use of masks for all staff and patients, regular cleaning of rooms, and having patients use hand sanitizer before entering rooms and after leaving. These measures are important because asymptomatic carriers, unlike those experiencing symptoms, are difficult to identify. For symptomatic patients or staff there are many other “levers to pull” to reduce risk because the risk is more apparent. With symptomatic patients, treatment can occur in a dedicated room and/or a dedicated time of day and staff can employ isolation precautions. It is also interesting to recognize that control measures were implemented in a rolling fashion as the pandemic evolved (Fig. 1). Some of the most effective measures were only implemented relatively later (e.g. requiring staff to wear masks). This underscores the importance of having a framework, such as FMEA, to inform the use of control measures, as the earliest control measures were not necessarily the most impactful.

For the highest-scoring failure modes, control measures and interventions reduced the risk score (Fig. 2). However, in some cases, the RPN stayed the same or increased slightly. Many of these (5/11) were related to telehealth visits. As telehealth is a new technology, it is not surprising that we found that this control measure could increase risk. We have not, as a medical community, had time to fully and effectively implement this technology. In the future, telehealth visits can be improved by providing more support to patients around the technology, coordinating with staff and improving quality at the physician’s end. These improvements may include more rigorous training on how to use telehealth apps effectively, ensuring that all relevant health information is collected, and addressing state licensure barriers. Telehealth technology and access also present issues with regard to disparities of care due to differing socioeconomic status of patient. The other control measures which did not reduce RPN were related to decreased on-site staffing and failure to treat patients for a greater public health concern. The rapid pandemic evolution resulted in staff members working remotely without much time for preparation, and this has been a learning process for all departments engaging in this. Overall there appears to be some benefit to remote work [36] and it is notable that these failure modes were given low RPN scores. Although the holding of treatment for COVID-19 testing may not benefit the individual patient quality of care, the benefits to staff and other patients are substantial.

Unlike the data on failure mode, the results of control measures showed variability between institutions. There was high institutional agreement (greater than 4 institutions) for only 47% of control measures. This may be a reflection of the diverse geographical locations of these practices and the fact that different regions experienced different COVID-19 case numbers during the pandemic. Of note, most of the control measures which had high compliance in the April 2020 ASTRO survey [22] (asterisks in Table 2) also had high institutional agreement in the present study, with the exception of limiting visitors. Interestingly, we found that the control measures with the highest institutional agreement also had the highest relative risk reduction (Fig. 3). This indicates that widespread actions taken by many departments were perceived to be the most impactful, although some low consensus actions did have a high impact. One example was one center’s policy on limiting the use of the active breathing coordinator as a motion management strategy. While this had a high impact score at one institution, similar devices were not in use at the other institutions. This lack of consensus seems to be due to an equipment difference rather than a philosophical difference. For the six institutions involved, large changes to practice such as increased hypofractionation schedules [23], [37], [38] or adaptations to patient QA [39] were not made. These changes were considered at one institution if the typical standard of care could not be met (e.g. multiple staff members unable to work, physicians called to assist in other parts of the hospital) but were avoided, as sudden changes in operational procedures are frequently sources of error.

The findings of this study will continue to be important as the pandemic evolves over what is expected to be many months. Work patterns will likely continue to change and new control measures will be introduced. The results of this study as well as the methodological framework may provide a rational way to assess the potential impact on care delivery.

There are several limitations of the study. First, participants included a relatively small number of institutions and all were from the United States. These results, therefore, may not be applicable to all practices. The practices involved in this FMEA all had different community viral prevalence, as well as different equipment, so not all control measures were felt to be necessary at all sites. For instance, pre-RT COVID-19 testing as a control measure had poor consensus, and this could be due to baseline community viral prevalence, as well as regional variances in the availability of testing materials. Additionally, the control measures practices of these institutions was a snap shot in time, and given the exponential growth of the virus in hard hit communities, practice patterns can change quickly. We sampled all practices within a one week period to minimize the impact of this, although this pandemic has been marked by rapid informational change. For instance, since the start of this project, it is now felt that viral shedding can continue weeks after infection, but may not pose a transmission risk [40]. As such, the practice of waiting for 2 negative nasopharyngeal swabs has been phased out at many locations. Second, as a risk assessment methodology FMEA is known to have limitations in terms of bias and variability which means that results are only semi-quantitative. Finally, this study was performed at a point in time when the response to COVID-19 is still evolving and the results reflect the status of the pandemic at present. If, for example, the stress on the healthcare system were to increase and the system become overwhelmed the risk equation may be different. This study examined risks associated with COVID-19 in the oncology setting. Common control measures were found to reduce the risk associated with unintended viral transmission as well as potential gaps introduced in the quality of care. However, the use of these control measures varied across institutions due to the prevalence of the virus and individualized responses in different practice environments. The failures modes identified in this analysis and the effectiveness of current control measures can be used as a framework for assessing new policies and procedures as oncology clinics continue to adapt to the pandemic.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of Interest Statement

There are no conflicts of interest to disclose.