Developments in deep learning based corrections of cone beam computed tomography to enable dose calculations for adaptive radiotherapy.

Adaptive radiotherapy (ART) planning and delivery strategies are often a prerequisite for accurate dose delivery for several tumour sites influenced by inter-fractional anatomical changes. This includes tumours in a number of sites such as the head-and-neck, as this anatomical region can experience nasal cavity filling and weight loss [1]; in the lung, where tumour baseline shifts and atelectasis can occur [2]; and in the cervix, where different levels of bladder or rectum filling can impact the target coverage [3].

ART relies heavily on the accessibility and quality of in-room imaging. Different modalities have been suggested for use in adaptive workflows, including magnetic resonance imaging (MRI) [4], computed tomography (CT) [5], and cone beam CT (CBCT) [3]. The advantage of using CBCT for this purpose is its wide availability; many treatment rooms for both photon and proton therapy are equipped with CBCT scanners, and daily or weekly CBCT scans are often acquired for verification of patient positioning. These verification images are in general used to visibly assess if anatomical changes have occurred that warrant plan adaptation. If a plan adaptation is flagged, the patient can be referred for a re-planning CT scan; however, dose calculations directly on the CBCT scans have also been investigated [6], [7], [8].

The image quality of CBCT scans is typically much lower than for CT scans which can reduce the accuracy of CBCT-based dose calculation [9], [10]. The artefacts in CBCT scans and resulting unreliable CT numbers in CBCTs are especially detrimental in proton therapy due to the steep fall-off of the proton depth-dose curve which requires a high accuracy of the CT number to stopping power ratio conversion to estimate the proton range [11]. Several techniques have been suggested for artefact reduction or correction in CBCT for both photon and proton therapy. These include look-up table-based approaches [12], deformable imaging registration (DIR) of the treatment planning CT (pCT) to the CBCT [13], [14], scatter correction methods using a prior based on the pCT [8], [15], methods which model the scatter contribution of the measured projections [16], or Monte Carlo-based approaches [17]. Among these methods there are approaches working at both image [12], [13], [14] or projection level [8], [15], [16], [17]. More recently, novel approaches for CBCT corrections based on neural networks and deep learning have been proposed [11], [18], [19]. Different network architectures have been applied; among the most successful approaches are cycle-consistent generative adversarial networks (cycle-GANs) [20], [21], [22]. The aim of CBCT correction based on cycleGANs is to create a so-called synthetic CT (sCT) from the CBCT, where the sCT should have an appearance, CT number accuracy, and image consistency of the same quality as the pCT. This is the research avenue of the study by Maspero et al. recently published in this journal [23]. Maspero et al. applied a cycleGAN to correct CBCTs for breast, lung and head-and-neck cancer patients to enable photon dose calculations on the corrected CBCTs, the sCTs [23].

An advantage of deep learning approaches compared to the physics based approaches utilizing, e.g., Monte Carlo simulations, is that deep learning is fast, and it can typically correct a whole CBCT image stack within seconds [23]. The training of well-functioning deep learning models, however, requires a lot of time, training data and extensive computational power [24]. The computer optimizes the neural network parameters, learning the similarities and differences between the two image sets – pCTs and CBCTs in this case – and how to convert one into the other. For cycleGANs, or generative adversarial networks (GANs) in general, this training process can be seen as a competition between two conflicting networks, where one tries to create a sCT from a CBCT with an image quality high enough that it resembles a regular CT, while the other aims at being able to distinguish between CT (real) and sCT (fake) images [25]. To improve the quality of the mapping for cycleGANs, the inverse transformation, in this case from a CT to a synthetic CBCT (sCBCT), is also created. The consistency over the cycle is then constrained such that the conversion from a CBCT to a CT and back to a CBCT should result in a CBCT, which is virtually the same as the original CBCT, and the same for the conversion from CT to CBCT and to CT again. This cycle consistency enables training based on unpaired data. This means that in each step of the training process, the algorithm is presented with CT and CBCT image slices but not necessarily from the same patient or the same anatomical location [23], as the algorithm is based on feature extraction rather than on creating a one-to-one correspondence. The correspondence is instead ensured by the constraint of regaining the same image after a full cycle of the process [25]. This simplifies the collection of training data since images can be included for patients where either the CT or CBCT is missing, or where large anatomical changes are seen between the CT and CBCT, which can create problems for other CBCT correction methods based on e.g. DIR.

Deep learning has been used in many fields already, and image processing tasks may be especially suitable to be tackled by artificial intelligence. A clear advantage is the versatility, as approaches already extensively used in one discipline can be easily transferred and reused in another field of application. Maspero et al. successfully apply a network to correct CBCTs which was originally developed for image processing tasks very remote from the medical physics world [23]. The cycleGAN network was originally designed to create naturally looking photographs from Van Gogh paintings, or aerial photos from Google map images [25]. Even though the two set of tasks seem to be very unrelated – correcting CBCT images by utilizing high quality pCTs and converting paintings to photographs – the common trait is that both tasks involve image-to-image conversions, i.e., mapping from one image domain (either CBCTs or paintings) to another (CTs or photographs). Deep learning is agnostic to the underlying characteristics; it tackles the two tasks in exactly the same manner: It learns the features of the two domains based on a large data set of training examples, and how to connect the features from the two domains. When the network after the end of the training stage is presented with a new, unseen example, it knows how to relate this image to an image of the other domain [25].

The novelty in the paper by Maspero et al. is the successful training of a single deep learning network to correct CBCT images of different anatomical regions: breast, lung and head-and-neck [23]. The single network was compared to three networks trained individually for each anatomical site in terms of image accuracy for the generated sCTs. The image similarity, measured as the mean absolute error and the mean error, between the sCT and the rescan CT (rCT), was comparable between the results obtained applying the networks trained only on a single anatomical site and the network trained for the three sites together [23]. This result has very favourable consequences for clinical implementation, as the very cumbersome and time-consuming task of training the network can be done only once and then used for all patients and indications. And maybe more importantly, only one model should be applied, which eliminates the risk of using the wrong model in a busy clinical situation.

The fact that a single network can be used for such diverse anatomical sites as the breast, lung and head-and-neck might also highlight the fact that deep learning models are often extracting features from the data with no (or little) physical meaning rather than building human interpretable models. There are both risks and benefits in using deep learning [26], and this debate is of high importance. But likely CBCT correction is a field where we can safely, though never uncritically, welcome artificial intelligence. Here the deep learning model is not used to diagnose or to predict the survival of the patient [27], but instead to improve the CBCT image contrast and consistency [23].

After the training stage, the cycleGAN model by Maspero et al. can generate a sCT from a CBCT within 10 s on a GPU and 40 s on a CPU [23], whereby image conversion is feasible while the patient is on the treatment table. The speed is one of the common and very important advantages of deep learning. Another advantage of their model is that it can conserve the anatomy observed in the CBCT images [23]. Unlike methods that rely on DIR to transform the CBCT to the pCT, cycleGAN uses only the CBCT image as the input after training. For one lung test patient that presented atelectasis on the CBCT, but not on the pCT, the atelectasis was correctly represented also in the sCT [23]; whereas methods relying on DIR to the pCT can lead to anatomical misrepresentation [11]. The improved image quality of the sCT, compared to the original CBCT, also allows for accurate dose re-computations with clinical acceptable voxel-vise dose differences and gamma-pass rates for photon-based radiotherapy plans [23]. It is known that proton dose calculations are more susceptible to subtle difference than photon radiotherapy and larger dose difference can be seen [11], [18]. It would therefore be of interest to see how well the method of Maspero et al. would perform in proton therapy.

To conclude, with the improvement in CT number accuracy for CBCT images that can be gained by deep learning approaches, such as the ones presented by Maspero et al. [23], we get closer to the goal of online ART, where the treatment plan can be recalculated while the patient is lying on the treatment couch. The next step towards this goal may be automated segmentation [28], [29] and fast dose calculations [30], [31], and deep learning might again be crucial to achieve the final goal of fully automated, adaptive re-planning.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.