Prevalence of software alerts in radiotherapy.

Radiotherapy software messages (sometimes called alerts, pop-up windows, alarms, or error messages) to the user appear continuously on computer screens. These software messages sometimes require decisions to be made as to the next appropriate action. However, mainly these messages are for information only. Dealing with software messages is a well-recognized problem in healthcare and has contributed to catastrophic events both outside and within radiotherapy. The purpose of this work is to highlight the prevalence and raise awareness within the radiotherapy community of such software messages related to external beam radiation therapy procedures at the linear accelerator. Radiation Therapists (RTTs) were asked to record the type and frequency of software message over 50 fractions and for 50 different patients. The data was collected at 6 institutions in the Netherlands using linear accelerators from Elekta, Ltd. and Varian Medical Systems, Inc. Results show that linear accelerator software messages (including record and verify) occur at a rate of about 8.9 messages per patient fraction. This number of software messages is potentially impacting on patient safety as these messages range in level of importance. The impact and potential reduction of these software messages should be the focus of future research and improved implementation.

Introduction

Software messages (referred to generically as ‘alerts’ in this paper) affect everyone using software applications in the department. Alert fatigue and alert confusion are well recognized patient safety problems in healthcare generally [1], [2], [3] but have not been widely studied in radiotherapy [4]. Alerts were identified as contributing factors in two major accidents in radiotherapy. In the 1980′s, the indecipherable “Malfunction 54” alert contributed to the Therac-25 accidents [5] and in 2005 the misunderstanding of the alert ending in “Do you want to save your changes before application aborts?” (shown in Fig. 1d) contributed to the IMRT accident in New York when a patient received 39 Gy in three fractions with an open field [6]. Alert fatigue can be defined as a sensory overload condition where RTTs are exposed to so many messages of little or no consequence that they become desensitized. The consequence of alert fatigue is that important, potentially lifesaving, information may be missed by healthcare professionals. Another aspect of this phenomena is known as alert confusion where software messages are observed but the meaning or next appropriate action is not understood.

Fig. 1

Examples of alerts: (a) an alert that is considered informational only, (b) an alert that is considered to require an explicit decision by the RTT, (c) an alert that may lead to an error but requires only an acknowledgment by the RTT, (d) Suspected alert that contributed to the IMRT accident in New York.

While alerts affect everyone in the department, RTTs are particular impacted because they frequently experience alerts at the linear accelerator when a patient is on the treatment couch. This creates a time-pressured situation where they need to make an immediate decision on the next best action.

The purpose of this work is to highlight the prevalence of alerts and to raise awareness within the radiotherapy community of the problems experienced by the RTTs at the linear accelerator when carrying out external beam radiation therapy procedures. This work is not meant as a definitive audit of alerts on radiotherapy software, rather it is intended to initiate discussion within the radiotherapy community and to present data that could be used to compare with local data.

Materials and methods

The study was conducted at 6 institutions in The Netherlands: (MAASTRO (Maastricht), Catharina Hospital (Eindhoven), Erasmus Medical Centre (Rotterdam), University Medical Centre Utrecht (Utrecht), ZRTI (Vlissingen) and Verbeeten Institute (Tilburg)). All of the institutions are members of the PRISMA-RT collaboration (www.prisma-rt.nl). Data was collected by the RTTs during their daily treatment schedule at the linear accelerator. The RTTs recorded the number of alerts that were issued during a single treatment fraction for 50 patients. RTTs were asked to record any type of alert (informational and decision alerts). Informational alerts can simply indicate that the RTT is going to proceed with the intended action (see Fig. 1a). Alerts that require a decision to be made by the RTT may indicate a potential deviation from the intended treatment plan (see Fig. 1b) or a potential error even if the alert does not require a decision (see Fig. 1c).

Data was acquired across 8 linear accelerators over the course of 1 day. The departments that contributed to the data collection treated a wide variety of sites and techniques including IMRT, VMAT, with both standard fractionation and hypo-fractionation regimes. As this was an exploratory study, no attempt was made to control or stratify detailed results up front by linear accelerator model or treatment type (see Fig. 2).

Fig. 2

Visualization of the informational and decision alerts pro institute for 50 patients.

In addition to the data collection on the actual alerts, two questions were answered by the RTTs after the data collection. Question 1, “How burdensome (e.g., disruptive to workflow) do you feel that alerts at the linear accelerator are (from not burdensome at all (0) to very burdensome (100))?” Question 2, “What do you perceive as the risk associated with acknowledging alerts at the linear accelerator (from no risk (0) to very risky (100))?” These questionnaires were filled out by the RTTs who recoded the alerts.

Results

The results of the alert data are shown in Table 1. The equipment on which the study was carried out included four Varian linear accelerators and four Elekta linear accelerators with approximately the same number of patient fractions evaluated from each vendor: 50 patient fractions on 4 units (2 Varian and 2 Elekta) and 25 patient fractions on a further 4 units also 2 Varian and 2 Elekta.

Table 1

Overview of alert data.

Number of linacs• Varian True beam, software 2.5 MR 2 • Varian True beam, software 2.0.33.4 • Varian True beam, software 2.0.33.4 • Varian Trilogy software 13.0 • Elekta Synergy agility, software Integrity 3.2 • Elekta Precise MLCi2, software 1.2 • Elekta Synergy platform, software Integrity 1.2 • Elekta Axesse agility, software Integrity 3.2	8
Number of patient fractions	300
Total number of alerts• Informative alerts • Decision alerts	2675 (100%)• 2479 (93%) • 196 (7%)
Average number of alerts/fraction	8.9
Maximum number of alerts/fraction	42
Minimum number of alerts/fraction	2

The range of 2–42 alerts is substantial and resulted from the instructions which required the inclusion of all software messages that occurred during the treatment of a patient. In some instances, this also include messages relating to technical failures and/or equipment repositioning issues.

The results of the survey data are shown in Table 2. Given the limitations of the study and the variation of responses, no firm conclusion can be drawn from this data about the relationship between how the RTTs feel about the burdensome nature of all the alerts and the degree that the alerts are perceived as risky to patient safety. However, it highlights the importance of carrying out further studies and forms a basis on which to move forward.

Table 2

Survey data collected in this study.

	Question 1	Question 2
Number of RTTs surveyed	30	30
Mean response	41.7%	45.3%
Standard deviation of responses	30.0%	28.7%

Discussion

In healthcare, we have seen humans can become conditioned very quickly when it comes to the appearance of software alarms, and if they appear to have no impact on their work, they commonly acknowledge them and continue with treatment delivery without too much consideration. The survey results indicate that the number of alerts is only considered a moderate risk and not too burdensome (i.e., less than 50%).

The vendors also have a role in reducing or minimizing alarm fatigue. Alarms in their hardware and software systems provide the user with notification that the system has detected anomalous behavior and may require attention. However, it may be beneficial to have a balance between notification of anomalous behavior and a well-defined action plan when an alarm is detected by the RTT. This balance may not always be optimal in current linear accelerators.

Institutions can introduce specific policies on how to deal with different types of alerts. This is something that is typically not routine practice. If there are too many different types of alerts, then it is unmanageable to overcome every one by having clear manuals and procedures. The RTT needs user centered, clear, actionable, specific, concise and clinical reviewed information [4]. With an average of 8.9 alerts per patient fraction, this project demonstrates that alerts may be a problem in the safe delivery of radiotherapy and should be the subject of additional research to further characterize the alerts for different linear accelerators and treatment types. In the future, it would also be interesting to look at treatment incidents that have occurred in the clinic and that have either been picked up by these alerts or accidentally mistreated due to this “automated conditioning”,

As this was a generative study, definitive conclusions cannot be drawn from the results. Future work should be designed to determine the RTTs perceived risks and benefits with alerts at the linear accelerator. Industry standards should be established for the number and type of alerts during each treatment appointment.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.