Clinical implementation and validation of an automated adaptive workflow for proton therapy.

Background and purpose: Treatment quality of proton therapy can be monitored by repeat-computed tomography scans (reCTs). However, manual re-delineation of target contours can be time-consuming. To improve the workflow, we implemented an automated reCT evaluation, and assessed if automatic target contour propagation would lead to the same clinical decision for plan adaptation as the manual workflow. Materials and methods: This study included 79 consecutive patients with a total of 250 reCTs which had been manually evaluated. To assess the feasibility of automated reCT evaluation, we propagated the clinical target volumes (CTVs) deformably from the planning-CT to the reCTs in a commercial treatment planning system. The dose-volume-histogram parameters were extracted for manually re-delineated (CTVmanual) and deformably mapped target contours (CTVauto). It was compared if CTVmanual and CTVauto both satisfied/failed the clinical constraints. Duration of the reCT workflows was also recorded.
Results: In 92% (N = 229) of the reCTs correct flagging was obtained. Only 4% (N = 9) of the reCTs presented with false negatives (i.e., at least one clinical constraint failed for CTVmanual, but all constraints were satisfied for CTVauto), while 5% (N = 12) of the reCTs led to a false positive. Only for one false negative reCT a plan adaption was made in clinical practice, i.e., only one adaptation would have been missed, suggesting that automated reCT evaluation was possible. Clinical introduction hereof led to a time reduction of 49 h (from 65 to 16 h).
Conclusion: Deformable target contour propagation was clinically acceptable. A script-based automatic reCT evaluation workflow has been introduced in routine clinical practice.

Introduction

Proton therapy (PT) is very sensitive to density changes in the beam path. The proton dose distribution can also be distorted due to minor changes in patient anatomy. Therefore, higher adaptation rates are often seen for proton treatments compared to standard photon treatments [1]. A full dosimetric evaluation, including re-delineation of the anatomical structures, on a repeat-computed tomography scan (reCT) is therefore often required to ensure that the target coverage is still satisfied as the treatment progresses.

Often, patients treated with PT get weekly reCTs to monitor if the plan still fulfills the dose constraints. Plan re-evaluation is very time-consuming and several professionals are involved; including radiation oncologists (ROs) and radiotherapy technicians (RTTs). Especially the hand-over time between professionals increases the overall time of the reCT evaluation. The whole process from reCT acquisition to the decision on whether or not a plan needs to be adapted typically takes several days. And additional time is needed if an adapted plan needs to be made. This sometimes leads to the (clinical) decision that in case a plan adaptation is needed this re-planning is postponed until the next weekly reCT, which is acquired a few days later. In the meantime, if within clinically acceptable criteria, the patient is treated with the non-adapted plan, however this is suboptimal for the patient.

To increase efficiency, we investigated the possibility to automate several steps in the proton plan re-evaluation workflow. Our goal was to decrease the workflow time from on average three days to below 24 h. One of the main time-consuming steps in the reCT evaluation workflow was the manual target re-contouring. We therefore wanted to assess if this step could be replaced by deformable image registration (DIR) between the treatment planning-CT (pCT) and the following reCTs. DIR has previously been shown to provide good results for reCTs of head-and-neck (HN) patients [2] and 4DCTs for lung patients [3]. In this study, we included a mixed cohort of patients (brain, HN, breast, lung and lymphoma) treated with proton therapy. The DIR-based target contour propagation was assessed both geometrically and dosimetrically and it was investigated if the conclusion of the clinical evaluation was similar to the one for the manual workflow. The DIR contour propagation was automated using scripting, along with several other steps in the reCT workflow including the robust dose calculation and evaluation.

The aim of this study was to assess if a script-based reCT evaluation workflow would lead to the same level of plan adaptation as the manual workflow, as well as to evaluate the reduction of the workflow timing.

Materials and methods

Patient cohort

All patients treated with proton therapy at our center during the first year of clinical operation were evaluated. In total, 79 patients (12 brain, 14 HN, 35 breast, 14 lung, and 4 lymphoma patients) were included. Each patient had weekly reCTs, and in total 250 reCT evaluations were performed. This study was approved by the Maastro institutional review board (approval number W201000039).

Treatment planning and evaluation strategy

The treatment planning strategy, beam arrangement, optimization objectives, and dose constraints differed per treatment site and depended on the anatomy of the individual patient. However, the plan evaluation workflow applied during reCT evaluation followed the same structure across the different patient groups.

Proton plans were created in RayStation (RaySearch Laboratories, Stockholm, Sweden), and the patients were treated with a Mevion S250i Hyperscan system (Mevion Medical Systems, Littleton, MA, USA). All proton plans were robustly optimized, applying 3D robust optimization, with a range uncertainty of 3%. The setup uncertainty depended on the treatment site: 1 mm for brain, 3 mm for HN, and 5 mm for all other indications. All plans were 3D robustly evaluated, computing the voxel-wise minimum and maximum dose distribution (VWmin/VWmax) as described by Korevaar et al. [4].

For all patients, the clinical treatment plan was re-evaluated on the weekly reCTs to assess if the target coverage was still satisfying the clinical constraints. A 3D robust dose evaluation was performed on the reCTs, in the same way as on the pCT. Here, the same range uncertainty, 3%, as for the pCT was used, since range uncertainty is mainly caused by CT conversion inaccuracy and therefore systematic and not diminished by the re-evaluation on reCTs. The setup uncertainty used for the robust reCT evaluation, on the other hand, was set to 1 mm for all indications included in this study, assuming that the image guidance procedure applied for patient setup during treatment accounted for setup errors. The main criterion during reCT evaluation was target coverage, and plan adaptations were primarily triggered by an inadequate target coverage.

Workflow for reCT evaluation

The original manual workflow for reCT evaluation is outlined in the top part of Fig. 1. The reCT of the patient was imported to the treatment planning system (TPS) and registered to the pCT, rigidly and deformably. The OAR contours were rigidly or deformably mapped from the pCT to the reCT. These first steps were performed by RTTs, and hereafter an RO delineated the target(s) and checked the DIR. Then an RTT performed the dose re-computation and robust re-evaluation, before an RO finally evaluated if a plan adaptation was needed. Due to the many hand-overs, the full process typically took around three days – the individual steps were not that time-consuming, the main throughput time was spent on waiting for the right person to be available to do the next step in the process.

Fig. 1

Schematic workflow for the reCT evaluation leading to the decision whether the plan was still acceptable (i.e., satisfying the target coverage constraints) on the reCT. The top row shows the previous manual workflow which typically took a couple of days to complete. In the third step, the RTT created the image registration, here the * indicates that both a rigid and a deformable image registration (DIR) was created (since a rigid registration was needed to create a DIR), and then the OARs were propagated using the DIR. In the bottom row, the new workflow is shown. In this workflow the largest work-burdens were replaced by scripting. A single script was created to perform all the scripted steps.

To increase efficiency, we proposed a workflow with fewer hand-overs between different professionals, by introducing a higher level of automation (lower part of Fig. 1). The largest change introduced with this new workflow was the use of DIR also for target contour propagation instead of manual RO delineation. In this study, we therefore evaluated whether the manual target delineations and the deformably propagated target contours would result in the same level of acceptability of target coverage.

Evaluation of contour propagation

In this study, we propagated the clinical target volume (CTV) structures from the pCT to each of the following reCTs. If a plan adaptation had been performed on one of the reCTs, the CTV contours manually delineated on this reCT were propagated to the following reCTs. The DIR was created in RayStation using the hybrid method, which is based on the ANACONDA model [5], and the default settings, which includes a grid size of 0.25 × 0.25/0.30 × 0.25 mm, and no focus or controlling regions-of-interest (ROIs) were used.

For the lung and lymphoma patients, the doses were computed on an average CT (CTavg) based on a 4DCT scan, for both the pCT and the reCTs. The targets and OARs were delineated on the 50% expiration phase (CT50ex) and rigidly copied to the CTavg [3]. In the automated workflow, the target contours were therefore also deformably propagated from the pCT50ex to the reCT50ex and then rigidly copied to the reCTavg. Moreover, for the lung patients, the manually delineated target structures were the gross tumor volumes (GTVs) and the CTVs were created as a 5 mm expansion with adjustments to bones and major vessels. Therefore, for lung patients we mainly propagated the GTV contours instead of the CTV contours, and then created the CTVs as expansions of the deformably propagated GTVs. However, no adjustments were done to anatomical structures as this could not easily be done automatically. Instead, if the CTV had been edited by the RO on the pCT (this was judged by checking if the CTV differed from an isotropic expansion of the GTV), this CTV was deformed to the reCTs.

In the rest of the paper, we will denote the propagated CTVs as CTVauto, and the original manually delineated CTVs on the reCTs as CTVmanual.

As a pre-assessment of the DIR quality for the contour propagation, we computed the volume differences between the CTVauto and CTVmanual, as well as the DICE score between them on each reCT.

Impact of contour propagation on the dosimetric evaluation

The dose-volume-histogram (DVH) parameters for CTVmanual and CTVauto were extracted from the dose distributions (either nominal, VWmin, or VWmax, following clinical criteria) computed on the reCTs. The evaluated DVH parameters for each treatment site can be seen in Table S1 in the Supplementary Material. It was evaluated if CTVmanual and CTVauto would lead to differences in the final clinical decision, that is, if the clinical constraints were satisfied for one contour but not for the other. As our main aim was to ensure that a potential adaptation would be caught if using CTVauto, we defined the following four outcomes:

True negative: All clinical constraints were satisfied for both CTVmanual and CTVauto

True positive: At least one clinical constraint was failing for both CTVmanual and CTVauto

False positive: All clinical constraints were satisfied for CTVmanual, but at least one constraint failed for CTVauto

False negative: At least one clinical constraint failed for CTVmanual, but all constraints were satisfied for CTVauto

True positives and true negatives would mean that the two contour sets (manual and deformed) would lead to the same clinical decision (though not necessarily exactly the same DVH values). False positives would mean that the automated workflow failed, but this would be less problematic as this would flag a failing target dosage, whereby the RO would have a look at the plan and the contours, and then most likely would find the propagated contour to be wrong, and then adjust the contour. That is, a false positive would lead to extra work for the RO, but a potential adaptation would not be overlooked. False negatives were the most problematic outcome for the automated workflow as they could cause potential adaptations to be missed. The false negatives were therefore our focus.

Script-based automatic reCT evaluation workflow

A script was developed in the TPS (RayStation) to perform most of the steps in the reCT evaluation workflow (lower part of Fig. 1), to decrease the manual workload on the RTTs. The script created a DIR between the pCT and the reCT, propagated the contours, rigidly or deformably (user choice), and robustly re-computed the dose on the reCT. A detailed description of the steps in the script is given in the Supplementary Material. The script is available upon request.

The time to perform the reCT evaluation before and after the introduction of this script was compared. Here we extracted the full time from importing the CT to the TPS until noting in the patient electronic journal whether or not the plan needed to be adapted. Equal-sized subgroups of reCT evaluations were evaluated before and after the introduction of the script; 100/35/150 manual and 100/35/150 script-based reCT evaluations were timed for brain, HN, and lung patients, respectively. These reCT evaluations were chosen sequentially among the first and the last performed evaluations, respectively. Breast and lymphoma patients were not included due to a very low number of script-based evaluations.

Results

Geometric evaluation of contour propagation

The volume difference between CTVmanual and CTVauto is seen in Fig. 2. In general, the volumes were quite similar for the two sets of contours, 90% of contours had an absolute volume difference below 14 cm3 (11%), and for 8% the difference was 0 cm3. The DICE value between CTVmanual and CTVauto was above 0.9 for 71% of the contours, and the 5th percentile was 0.76. For the brain patients, the lowest DICE value was 0.94 and the 5th percentile was 0.98. The spread in the DICE values for the breast patients was much larger, here the lowest value was 0.31 and the 5th percentile was 0.74. The breast patients were the biggest group of patients (Table 1), and each patient had several target contours to evaluate for each reCT, therefore the results for the breast patients had a large influence on the results for the total group.

Fig. 2

(Left) Volume difference between the manually delineated and the deformably propagated target contours. (Right) DICE value for the overlap between the manually delineated target contours on the reCTs and the deformed contours propagated from the pCT to the following reCTs.

Table 1

Number of reCTs presented with each result; the numbers in the parentheses are the rates relative to the total number of reCTs.

	Total	Brain	Head-and-neck	Breast	Lung	Lymphoma
# Patients	79	12	14	35	14	4
# reCTs evaluated	250	59	63	69	50	9
# Contours evaluated	643	58	153	303	110	19
# True positives (rate)	118 (47%)	35 (59%)	34 (54%)	22 (32%)	19 (38%)	8 (89%)
# True negatives (rate)	111 (44%)	24 (41%)	19 (30%)	39 (57%)	28 (56%)	1 (11%)
# False positives (rate)	12 (5%)	0 (0%)	5 (8%)	5 (7%)	2 (4%)	0 (0%)
# False negatives (rate)	9 (4%)	0 (0%)	5 (8%)	3 (4%)	1 (2%)	0 (0%)

In total, 92% of the evaluated reCTs were correctly classified (44% true negatives and 47% true positives) when using deformable contour propagation (Table 1). Only nine (4%) of all reCTs presented with a false negative, i.e., a clinical goal being violated by CTVmanual but all goals being satisfied by CTVauto. Details on these nine false negative reCTs are given in Table 2, and in the Supplementary Material case descriptions are given along with figures showing the relevant contours.

Table 2

Details on false negatives. Second column specifies the reCT number (no.), (x2) means that two target dose constraints were violated for CTVmanual but not for CTVauto on the given reCT. Fourth column states how much the target dose-volume-histogram (DVH) constraint was violated for CTVmanual, computed as , where is the DVH parameter extracted for CTVmanual. The DVH parameter difference listed in column five is given as . If and are both close to 0%, the constraint was just barely violated for CTVmanual and just barely fulfilled for CTVauto, meaning that the two DVH parameters were almost the same. The second reCT for the third HN patient is marked in boldface, since a plan adaptation was made on this reCT.

Site (no.)	reCT no.	Contour volumes(CTVmanual vs CTVauto; cm3)		CTVmanual failure (F; %)		Goal value difference (Δ G; %)
HN (1)	reCT 5	23.1 vs 26.0		−0.6		1.8
HN (2)	reCT 4 (x2)	135.7 vs 126.8		−0.04	−0.05	−0.1	−0.1
HN (3)	reCT 1 (x2)	32.6 vs 23.6		−0.1	−0.2	−0.5	−0.6
	reCT 2	33.0 vs 26.3		−2.2		3.1
HN (4)	reCT 2	232.9 vs 234.1		−0.5		1.1
Breast (5)	reCT 1	17.3 vs 16.7		−0.9		2.6
Breast (6)	reCT 1 (x2)	960.7 vs 941.8	886.1 vs 866.5	−0.4	−0.6	0.7	0.7
Breast (7)	reCT 2	510.9 vs 499.1		−0.2		0.6
Lung (8)	reCT 2	100.3 vs 67.9		−0.1		4.4

For eight of the false negative reCTs, the small target under-dosage was clinically accepted and did not lead to a plan adaptation. Only for one false negative reCT, an adaptation was made in clinical practice that was not caught with the automated workflow. This patient had an under-dosage of 2.2%, corresponding to a V94% of 96%, while the clinical constraint was 98%. The location of an air cavity changed between the pCT and the reCT (Fig. 3). The resulting deformation vector field caused CTVauto to shrink compared to the CTV delineated on the pCT, whereby the under-dosed region of CTVmanual was not included in the smaller CTVauto, resulting in this false negative.

Fig. 3

Example of false negative repeat-CT (reCT), for HN patient 3 who had a plan adaption on this reCT (reCT 2; see Table 2). (Left) Planning-CT (pCT), here an air cavity, representing the right piriform sinus, is seen (dashed orange circle and arrow) which is not present in the same location on the reCT (middle), likely due to radiation-induced edema. The blue contour is the delineated CTV on the pCT (it has been rigidly copied to the reCT for visualization purposes), the yellow contour is the delineated GTV, the purple contour is the delineated CTV on the reCT, and the red contour is the deformed CTV on the reCT propagated from the pCT. (Right) Deformation vector field between the pCT and the reCT; the colors of the vectors indicate their length. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Clinical implementation of the script-based automated workflow and achieved time reduction

The script-based automated workflow has been clinically implemented for all indications. Several iterations of the development and testing of the script were performed, to ensure that the script met the needs of the end-users, i.e., the RTTs. After implementation, the time for the full reCT evaluation workflow has been reduced by around 75%, from on average 65 h to 16 h (Table 3). The time reduction was similar in the three patient groups evaluated.

Table 3

Average duration of the reCT evaluation workflow before and after implementing the script-based workflow. The number in parenthesis after the tumor site indicates how many reCTs were included in each subgroup (before and after).

Tumor site	Timing of reCT evaluation workflow (h)		Time reduction (h/%)
	Manual workflow	Applying script
Brain (100)	68.4	15.0	53.4 (78%)
HN (35)	66.4	19.2	47.2 (71%)
Lung (150)	63.2	16.1	47.1 (75%)
Total (285)	65.4	16.1	49.3 (75%)

Discussion

The aim of this study was to assess if a script-based automated reCT evaluation strategy would lead to the same clinical conclusion for treatment adaptation as our manual workflow, and furthermore if it would lead to a decreased workload and throughput time. Only the target contours were included in this study and we did not investigate the effect of deformable OAR propagation, since rigid or deformable OAR propagation was often used already in the manual workflow (Fig. 1, top part). In most cases, CTVmanual and CTVauto were comparable in terms of volume and overlap (Fig. 2). The target coverage constraints were also generally similarly fulfilled, and the false negative rate was only 4% (nine reCTs; Table 1). Furthermore, for most of these nine reCTs, the clinical target constraint failure was very small (Table 2), and only in one out of the nine, a plan adaptation was clinically performed to counteract the under-dosage seen for CTVmanual.

The reasons for the false negative reCTs mainly fell into two categories. For HN patients, the main reason was the department guideline not to change the original GTV volume throughout the treatment. This could not always be guaranteed by applying deformable mapping of the CTV. The main reason for the breast and lung patients was inter-observer variability between the delineation on the pCT and the reCT. To account for the first category, the script could be modified to allow for isotropic expansion of the GTV, if this structure is selected for propagation. For the latter category, the script would actually be beneficial, since it could reduce the intra-patient delineation variability if the contours on the reCTs are not manually delineated from scratch, but assisted by the propagated contours from the pCT [6], [7].

The results on the clinical accuracy led to the introduction of this script-based procedure in clinical routine. In the clinical implementation of this script-based reCT evaluation, the risk of missing an adaptation is mitigated by having the RO check the deformed target contour and adapt if needed, which still leads to time-saving compared to a full re-delineation. The time-saving was evaluated several months after clinical introduction, by comparing the workflow timing before and after introduction. The automated script-based workflow led to a reduction of around 49 h, resulting in an average time of 16 h, whereby we reached our original goal of a time less than 24 h.

The time-saving was partly obtained by replacing manual contouring with DIR target contour propagation. However, only the overall time-saving was assessed in this study. Vaassen et al. evaluated the time-saving of automated OAR contouring including manual editing for lung cancer patients; they found a time-saving of 40% for atlas-based and 71% for deep-learning contouring [8].

The target contour propagation used in this study was based on simple DIR, but still fairly high DICE scores were obtained (Fig. 2). Deep-learning is increasingly used for automatic contouring, and implemented in a commercial platform [9], [10], [11]. Rhee et al. investigated deep-learning auto-contouring for the target and OAR structures in cervix cancer patients and found a DICE of 0.91 for the nodal CTV [9]. DIR-based target contour propagation was also investigated for five lung cancer patients by Nenoff et al. [12]. They used six different DIR algorithms, including the ANACONDA algorithm used in this study. They also found that the DVH parameters for the CTV were very similar between the manually delineated and the propagated contours.

The RayStation implementation of DIR has also been compared to ground-truth deformation vector fields between pCT and reCTs for HN patients by Pukala et al. [13]. They found this DIR to generally provide good results of similar quality as other commercially available DIR algorithms, based on target registration errors computed for several OARs. Also for target contour propagation between pCT and daily cone-beam CT in prostate patients, it was found that the DIR-propagated target contours aligned well with RO manually delineated contours [14]. Similar conclusions for DIR-propagated target contours were also found for HN patients based on applying other commercially available DIR algorithms, resulting in DICE scores of around 0.8–0.9 for the planning target volume [15].

In a recent survey in the United Kingdom, it was found that the most common use of DIR in the radiotherapy workflow was contour propagation [16]. In general, the use of DIR for contour propagation of targets is receiving more interest from the community, also with new online adaptive solutions being introduced [17], [18]. There has also been a great interest in online adaptive PT, and one of the promising approaches to online adaptation proton PT is dose restoration [19], [20]. The aim of dose restoration is to quickly adapt the dose distribution, to regain the plan quality seen on the pCT, with the least amount of manual interference. Another way to speed up the plan adaptation, though typically in an offline workflow, is to perform automated re-planning, again with the aim of reducing the manual workload. Several automated treatment planning strategies have been suggested [21], [22], [23]. In general, we strive to increase efficiency by automating many of the clinical processes, in order to be able to treat as many patients as possible without compromising the plan quality for any patient. The introduction of the automated reCT evaluation workflow described in this study, can be seen as a first step on the way to more automation of the entire workflow.

In conclusion, deformable image registration of (clinical) target volumes from the pCT to the reCTs was found clinically usable. A script-based automatic workflow for reCT evaluation, including contour (targets and OARs) propagation and robust dose re-evaluation was therefore clinically implemented. This led to an average time saving of 49 h for the reCT evaluation workflow.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.