A practical implementation of risk management for the clinical introduction of online adaptive Magnetic Resonance-guided radiotherapy.

BACKGROUND AND
PURPOSE: The clinical introduction of on-table adaptive radiotherapy with Magnetic Resonance (MR)-guided linear accelerators (Linacs) yields new challenges and potential risks. Since the adapted plan is created within a highly interdisciplinary workflow with the patient in treatment position, time pressure or erroneous communication may lead to various possibly hazardous situations. To identify risks and implement a safe workflow, a proactive risk analysis has been conducted.
MATERIALS AND METHODS: A process failure mode, effects and criticality analysis (P-FMECA) was performed within a group of radiation therapy technologists, physicians and physicists together with an external moderator. The workflow for on-table adaptive MR-guided treatments was defined and for each step potentially hazardous situations were identified. The risks were evaluated within the team in order to homogenize risk assessment. The team elaborated and discussed possible mitigation strategies and carried out their implementation.
RESULTS: In total, 89 risks were identified for the entire MR-guided online adaptive workflow. After mitigation, all risks could be minimized to an acceptable level. Overall, the need for a standardized workflow, clear-defined protocols together with the need for checklists to ensure protocol adherence were identified among the most important mitigation measures. Moreover, additional quality assurance processes and automated plan checks were developed.
CONCLUSIONS: Despite additional workload and beyond the fulfilment of legal requirements, execution of the P-FMECA within an interdisciplinary team helped all involved occupational groups to develop and foster an open culture of safety and to ensure a consensus for an efficient and safe online adaptive radiotherapy workflow.

Introduction

In 2016, preventable medical mistakes have been the third leading cause of death in the US behind cardiac diseases and cancer [1]. Subsequent patient safety efforts have been established in all medical disciplines. Regarding radiation therapy, several guidelines and requirements have been published within the last decade [2], [3], [4]. While in some countries, it is a regulatory requirement to conduct a prospective risk analysis prior to the clinical introduction of any new or modified treatment procedure, this still cannot be considered common practice in the radiotherapy community [5].

Online adaptive radiotherapy (ART) is a technique that uses daily patient imaging to adapt the initial treatment plan to the anatomical and biological variations of the patient during the course of a treatment [6], [7]. A new treatment plan is created while the patient lies in treatment position on the treatment couch (also called on-table adaption). The specific workflow for adaptive radiotherapy strongly depends on the technology applied. In the last years, a rapid improvement in imaging, delivery techniques and software assistance took place. This is accompanied by an increasing need of multidisciplinary cooperation. To manage the growing load of complexity, the development of new ART procedures must go hand in hand with a careful consideration of safety measures.

Magnetic Resonance (MR)-guided radiotherapy with hybrid devices combining MR-imaging (MRI)and linear accelerators (Linacs) allows MRI directly before and during treatment [8]. In the last years, commercial systems have become increasingly available, and a growing number of systems are being installed globally, enabling more and more facilities to perform ART with an integrative system. The imaging, recontouring, reoptimization and QA steps necessary for the adaptive treatment need to be performed in a time- and resource-effective way. Multiple disciplines work side by side to create a new treatment plan in a short amount of time. This requires a clearly defined operation procedure and a high standard of automatization. A prospective, institution-specific risk analysis conducted by a multidisciplinary team of experts is highly recommended in order to achieve this. For example, Cai et al. used a failure mode and effect analysis (FMEA) to implement a quality assurance (QA) program for ART that included different checks at vital steps of the process, plan quality and integrity checks, dose verification by secondary Monte Carlo dose calculation as well as delivery monitoring [9].

FMEA is a commonly used prospective risk management tool, which detects potential failure modes in a system, product or process and their causes and effects. It was first applied in radiotherapy to evaluate for the need of QA-tests and the frequency they should be applied [4], [10], [11], [12]. Further work focused on the application of FMEA to more advanced radiotherapy techniques like stereotactic body radiation therapy (SBRT) [13], [14], [15]. Also, in the development of ART workflows, FMEA quality management was used to improve processes [9], [16]. It was shown that high quality and safety can only be guaranteed if proactive measures are applied and workflows are properly defined before implementation.

In this work, we present a practical implementation of a process-based FMEA, which was performed prior to the clinical implementation of ART with a commercially available hybrid MR-Linac at our institution, we describe the results of the FMEA and present mitigation strategies.

Materials and methods

A process failure mode and effects and criticality analysis (P-FMECA) was performed for the clinical introduction of MR-guided ART with a ViewRay MRIdian Linac (ViewRay Inc, Oakwood Village, USA) at our institution. The workflow for adaptive treatments with this MR-Linac system [17], [18] remained largely identical to the workflow for the predecessor system using 60Co, which has been described previously by other authors [19], [20]. In the treatment software of the MRIdian Linac, some of such QA tools are included in a so-called “adaptive QA-tool” (AQA). AQA performs a secondary Monte Carlo dose calculation and produces a configurable report which can be defined to include gamma analysis of re-calculated and originally calculated dose, a plan comparison with the original treatment plan, beam fluence comparison and statistics of contour-specific dose metrics [21].

In general, FMEA is a systematic method of reviewing as many workflow steps as possible to identify potential hazardous situations, perform the cause and effects analysis, followed by identifying prevention- and/or detection-methods. FMECA, in addition to the FMEA process steps, evaluates the criticality of each hazardous situation using either a quantitative or a qualitative approach. In a quantitative approach, the failure mode criticality (CM) for a system or product can be calculated as the product of the failure mode probability (β), the failure mode ratio (α), the failure ratio (λ) and the operating time (t) of the system or product:

Here, following the German joint recommendation on risk analyses for radiotherapeutic practices [22], instead of calculating criticalities or risk priority numbers (RPN), a qualitative approach using a risk matrix was chosen. A two-dimensional risk matrix was defined by the severity that a hazardous situation could yield in a worst-case scenario and the probability of occurrence of such hazardous situations. For the sake of simplicity, the probability of detection of a hazardous situation was omitted. Both dimensions had a five-level grading scale, ranging from “negligible” to “catastrophic” for the qualitative description of the severity and from “hardly ever” to “frequently” for the description of the probability of occurrence. Within the risk matrix, the risk classes were defined reflecting the institution’s specific risk strategy on online adaptive radiotherapy using a hybrid MR-Linac. The risk class definition in this case reflected a conservative approach ensuring a high level of patient safety. Risk class I considered the criticality of a potential hazardous situation as generally acceptable. Risk class II grouped potential risks as acceptable under the “as low as reasonably achievable” (ALARA) principle while requiring continuous observations e.g. by means of critical incidents and reporting system (CIRS) analysis. Risk class III identified not acceptable risks requiring implementation of risk control measures. Any hazardous situation which could yield a catastrophic severity, defined as immediate or near-term lethal consequence, was deemed not acceptable regardless of its probability of occurrence (see supplementary material for a tabular depiction of the risk matrix).

The P-FMECA was conducted by a group of experts consisting of two physicians, two medical physicists and two radiation technologists who met weekly starting four months before the introduction of the new procedure. First, the workflow of on-table adaptive MR-guided treatments was defined in a process tree. For the sake of completeness, the whole treatment process for MR-guided treatments, starting from MR-simulation and ending with the last treatment fraction, was described and used for the following risk analysis. For each process step, the potentially hazardous situations were identified. The risks were then evaluated with respect to severity (S) and probability (P) within the entire team to homogenize the assessment of all situations. Risk levels (D) were determined from S and P with the risk matrix. Based on the risk levels, the team elaborated and discussed mitigation strategies, which were implemented subsequently.

Results

Fig. 1 shows the process tree with the main process steps for MR-guided treatments. Except for simulation and treatment planning, all steps were repeated at every treatment fraction. The FMECA conducted for the entire treatment process including online treatment plan adaption at the MR-Linac revealed 89 risks for the institution-specific workflow. Eleven of these failures (12%) were assigned to the low-risk class, 19 (21%) to the medium-risk class and 59 failures (66%) to the high-risk class III (see Table 1). Of the 89 risks, 30 (34%) were related to MR-specific processes and device-specific workflows, 41 (46%) directly concerned online treatment plan adaption, and 18 (20%) were of general nature, i.e. not related to the specific workflow at the MR-Linac or to online adaption. For the full FMEA results, the reader is referred to the supplementary material.

Fig. 1

Process tree for the entire treatment process at the MR-Linac, including online treatment plan adaption. Except for simulation and treatment planning, which are only performed once at the start of a treatment course, all steps are repeated at every treatment fraction.

Table 1

Risk classification prior to mitigation.

Risk classification showing the numbers of risks in each category as assessed by the interdisciplinary team, prior to implementation of mitigation strategies.

Mitigation strategies were elaborated within the multidisciplinary FMEA-team. By those, hazardous situations could be reduced to 61 (68%) in the medium risk class and 28 (31%) of low-risk. Risks in the high-risk class could be mitigated in their entirety (see Table 2).

Table 2

Risk classification after mitigation.

Risk matrix after implementation of risk mitigation strategies and re-assessment of classifications. No risks in class III (high risk) remained.

A detailed list of all implemented mitigations can be found in the supplementary material. Briefly, the standard operation procedures (SOP) were complemented, and specific checklists were developed (also shown in the supplementary material) guiding through the adaptive workflow. For critical workflow steps, a four-eye principle was implemented, such that those steps need to be double verified by two members of the adaptive treatment team. Furthermore, a general need for additional training beyond the usual level of system-specific training applied in our institution was identified for all professional groups involved in the process of online adaptive radiotherapy.

Since the adapted plan cannot be measured before application, an automated plan check tool was developed to get an additional safety level for the integrity of the adaptive plan. The tool checks if the correct treatment plan was selected and evaluates the recontoured structures (targets and organs at risk) in terms of integrity, evaluating the volume and unusual gaps within the structures. It also checks the consistency of the treatment plan quality in terms of segment size, total number of Monitor Units and modulation factor. In addition, machine-QA measures were also adjusted. An estimation of machine-specific delivery uncertainties was carried out, and a comprehensive program for routine QA was set up with a special focus on daily machine stability. Upon introduction of online adapted treatments, it was decided to dosimetrically verify every treatment plan prior to the first fraction (regardless if this plan would ever be applied or not) as well as every adapted treatment plan retrospectively via measurement, until sufficient data would be available in order to verify the stable performance of the machine for adaptive treatments.

The most critical risks detected were related to an unknown existence of a MRI-incompatible cardiac pacemaker (ICD) (S = 5/P = 3), a wrong recontouring of the GTV by the physician of the day (S = 4/P = 5) and considerable errors in the assignment of the electron density for dose calculation (S = 4/P = 5). As described above, those were mitigated via definition of additional workflow steps in the SOP and in the checklists, a four-eye principle for critical workflow steps and the automated plan check tool.

Discussion

We present a practical implementation of a risk management process in radiation therapy, in our case conducted prior to the clinical introduction of online adaptive MR-guided therapy. While the presented data shows the results at the time of clinical introduction, the risk management process should be pursued continuously, in order to include newly identified risks as well. In our case, we continue to conduct less frequent meetings, where newly identified risks are discussed and mitigation strategies are defined.

It is important to note that any risk assessment will always be institution-specific, as it will depend on the institution’s technical equipment, specific processes, level of training and also risk strategy, i.e. perception of acceptable risks. This also means that individual risk analyses are never fully transferable to different institutions. Since specific tools or guidelines are still rare, every institution depends on developing its own risk management system and process description. To establish a common understanding, individual results, which will be different for every institution, need to be compared to learn through an exchange of experience. Such comparison of multiple possible approaches for risk management in radiation therapy will also help to elaborate consensus guidelines. The application of FMEA should not be seen as mere fulfilment of legal requirements, but rather as an effective tool to increase safety in radiation therapy and should find a broader use. Regarding clinical introduction of new techniques or treatment procedures, it not only increases safety, but also strengthens the involvement of the whole team and helps everyone to relate their role with the entire context. In our case, the interdisciplinary meetings enhanced the commitment of the whole team and helped with the long-existing overall aim to establish an open culture of safety.

While several authors have reported on FMEA for different procedures in therapy using RPN defined as the product of severity (S) and probability (P) [15], [16], [23], we have chosen a more qualitative approach using a risk matrix. Initially, this was perceived as a lower obstacle for the interdisciplinary team with no prior experience with risk management tools. Also, from our point of view, RPN calculations are not ideal for radiation therapy processes, since two different risks yielding the same RPN should not generally receive the same prioritization for mitigation. A risk with critical severity and very rare occurrence is not equal to a risk with small severity and occasional occurrence.

In our case, dealing with online adaptive MR-guided radiotherapy, a lot of the risk mitigation strategies focused on workflow definitions and methods to ensure adherence to those workflows. Based on an already comprehensive QA program for the machine with for example extensive checks of geometry and Multileaf-Collimator performance, only a few remaining QA-related risks were brought up during the group meetings and had to be mitigated subsequently. This shows on the one hand that also the starting point of any risk analysis is institution-specific and somehow arbitrary. No risk analysis in radiation therapy will start from scratch, meaning that well established workflow practices, for example common to every patient treatment at a specific institution, will not always need to be fully analyzed. On the other hand, this demonstrates that QA for online adaptive radiotherapy to a certain degree means control of workflows and processes. In our analysis, human errors were ranked consistently higher than expected catastrophic machine errors.

This does of course not diminish the need for strict QA tests in online-adaptive radiotherapy. In that respect, there is broad agreement in the community that secondary dose calculation should be performed if the treatment plan cannot be verified by measurement prior to treatment [9], [24], [25]. Also, evaluation of the machine performance via logfile analysis is warranted [9], [25], [26]. Other authors also report on software tools for automated integrity-checks of online-adapted treatment plans [9], [27]. In case of the MR-Linac system used in this study, the vendor supplies a kind of all-in-one software tool for that purpose, covering parts of the above described. While this tool is used for all our adaptive treatments, the FMEA has revealed additional risks not mitigated by it. Considering the individual nature of process-based risks as described above, this is not surprising and will likely be the case for other institutions as well. Vendors should therefore rather make those tools more accessible and configurable, so that every institution can choose and configure their most important parts. Especially regarding logfile analysis, the availability of interfaces and access to all data needed is important and needs to be further enhanced.

In conclusion, we have demonstrated a practical way of conducting an interdisciplinary risk analysis prior to clinical introduction of MR-guided adaptive radiotherapy. Beyond the fulfilment of legal requirements, this increased the confidence in the adaptive process and improved patient safety. While the mitigation measures, implemented as result of the risk analysis, complemented the existing QA program, online adaptive radiation therapy requires specific and comprehensive quality assurance methods, which need to be further advanced.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: SK has received speaker fees and travel reimbursement from ViewRay Inc. outside the submitted work. JHR received speaker fees and travel reimbursement from ViewRay Inc., as well as travel reimbursement form IntraOP Medical and Elekta Instrument AB outside the submitted work. JD received grants from CRI – The Clinical Research Institute GmbH, ViewRay Inc., Accuray International, Accuray Incorporated, RaySearch Laboratories AB, Vision RT limited, Astellas Pharma GmbH, Merck Serono GmbH, Astra Zeneca GmbH, Solution Akademie GmbH, Ergomed PLC Surrey Research Park, Siemens Healthcare GmbH, Quintiles GmbH, Pharmaceutical Research Associates GmbH, Boehringer Ingelheim Pharma GmbH Co, PTW-Freiburg Dr. Pychlau GmbH, Nanobiotix A.A. as well as IntraOP Medical, all outside the submitted work.