Research on pseudo-CT imaging technique based on an ultrasound deformation field with binary mask in radiotherapy.

This study aimed to investigate the reliability of pseudo-computed tomography (pseudo-CT) imaging based on ultrasound (US) deformation fields under different binary masks in radiotherapy.We used 3-dimensional (3D) CT and US images, including those acquired during CT simulation positioning, and cone-beam CT (CBCT) and US images acquired 1 week after treating 3 patients with cervical cancer. Image data of 3 different layers were selected from the US images, and 3D CT images of each patient were selected. For US image registration, the following were created and applied: binary masks of the region of interest overlapping (ROIO) between the US image based on simulation positioning and US image for positioning verification, region of interest (ROI), whole overlapping (wholeO), and whole imaging region (whole). Accordingly, the deformation field was obtained and applied to CT images (CTsim), and different pseudo-CT images were acquired. Similarities between the pseudo-CT and CBCT images were compared, and registration accuracies between pseudo-CT images under different binary masks and CTsim were compared and discussed.A pair t test was conducted to normalized mutual information values of the registration accuracy between the pseudo-CT image based on ROIO binary mask and CTsim with other methods (P < .05), and the difference was statistically significant. A pair t test of normalized gray mean-squared errors was also performed (P < .05), and the difference was statistically significant. The similarity function means between pseudo-CT, that is, based on ROIO, ROI, wholeO, whole, and no binary mask, and CBCT were 0.9084, 0.8365, 0.7800, 0.6830, and 0.5518, respectively.Pseudo-CT based on ROIO binary mask best matched with CTsim and achieved the highest similarity with CBCT.

Introduction

The clinical application of image-guided radiation therapy (IGRT)[ improves the nonuniform or inadequate dose in tumor target area caused by organ movement and positioning error. Image registration, which is the key technology of IGRT, refers to strict matching of region of interest (ROI) in 2 images from different or same imaging device using appropriate spatial transformation. The basic framework of image registration is composed of characteristic space, search space, search strategy, and similarity measurement.[

Bones are more observable in computed tomography (CT) imaging than in other imaging methods. However, using projection data from a specific range of angles for CT image reconstruction is often necessary, so a CT image exhibits less detail in soft tissue.[ Ultrasound (US) imaging enables a high-resolution view of soft tissue and is often used to diagnose cardiovascular diseases.[ Thus, US-guided radiation therapy[ can increase soft-tissue imaging quality, decrease positioning errors, and protect patients from additional radiation. This technique is widely used in radiofrequency ablation,[ liver transplantation,[ and internal exposure therapy.[ Nevertheless, US image-guided systems[ (e.g., Elekta clarity) are used only to assist, rather than replace cone-beam CT (CBCT) in positioning verification.

Pseudoimaging technology, originally proposed by Pennec et al,[ was initially used for brain surgery. Yang et al[ converted zero-echo-time magnetic resonance imaging (MRI) to pseudo-CT and performed sinus/edge correction using a binary-mask method. The method can provide accurate positron emission tomography (PET) quantification in brain PET/MRI. Pseudo-CT under US-guided radiation therapy refers to a virtual CT image obtained using a deformation field acquired through registration between US images for simulation stage and US images for treatment stage. Van d et al[ recently performed pseudo-CT imaging through US deformation field and confirmed that a pseudo-CT image can reveal anatomical structural information in radiotherapy better than a CT image for positioning. Camps et al[ produced pseudo-CT images through 8 different methods and proved that pseudo-CT images from stepwise registration that combines rigid and nonrigid bodies are more applicable to adaptive radiotherapy. Few studies on pseudo-CT for radiation therapy exist at home and abroad. However, US images enable good viewing of soft tissues, so US image-guided external radiation therapy is frequently used to image breast and prostate tumors, among other.[ Moreover, pseudo-CT assisted positioning guidance can enable observation of the disease condition of patients everyday and provide real-time guarantee on relative deformation between amended tissues. This method also prevents the administration of an uneven dose to the target region or additional radiation to surrounding normal tissues and organs, thereby decreasing toxic and side effects. Thus, pseudo-CT imaging characteristics were discussed thoroughly in this paper, and different binary masks were added before the US image registration to improve pseudo-CT imaging quality. The feasibility for pseudo-CT to replace CBCT for the positioning verification of specific tumor therapy was also proven.

First, pixel information of images was adjusted through image preprocessing, and different binary masks were prepared to prevent deformation of bones in image registration and reduce errors. Second, deformation fields among US images of patients with cervical cancer were acquired by registration with ElastiX software (version 4.8, Image Sciences Institute, University Medical Center, Utrecht, the Netherlands)[ and were applied onto CTsim, obtaining pseudo-CT images. Finally, anatomy structural differences between pseudo-CT images based on different 2D binary masks and CBCT were analyzed and their feasibility in positioning verification of clinical radiotherapy for cervical cancer was verified.

Materials and methods

Image acquisition

The Clarity system[ consists of two 3D US-image workstations (Fig. 1A) in simulation and treatment rooms. Before positioning or positioning verification of patients, indoor laser lamp coordinates and the US probe must be calibrated by a calibration phantom (Fig. 1B) to ensure that the coordinates of every 3D US voxel collected by the system are known relative to the indoor laser lamp coordinates. CT-US 3D images of 3 postoperative cervical cancer patients were selected in this study, including CT and US acquired during CT simulation positioning as well as CBCT and US images for positioning verification, which were acquired 1 week after the treatment. However, due to patient motion and tissue deformation, the US images were prone to motion artifacts. Thus, we used a sequential 2D interslice registration method to correct motion artifacts.[ Each slice was considered as a template image Tk, where k ∊{0 … m} denotes the current position in the 3D image volume of m + 1 slices. The reference for slice k is defined as follows:

Figure 1

(A) 3D ultrasound image workstations. (B) Calibration phantom.

where factor α guides a convex combination of the 2 adjacent slices, and donates a registered image. By adjusting α, the difference between the reference and template images can be eliminated, so motion artifacts on the reference image can also be corrected.

When performing CT, CBCT, and US scans, patients were required to control the amount of water and the time of drinking to ensure that the bladder exhibited the same degree of filling at each scan. The direction of the US probe was also perpendicular to the patient, and the patients were scanned from head to toe to ensure repeatability of the probe scanning position. The US probe was mounted on a robotic arm that was attached to the couch and connected to the US imaging system to acquire images in real time. The device adjusted the height and angle of the arm in time to make the position of the US probe consistent with the patient for each scan. Images of 3 different layers were selected for each patient. These data included the largest section of the uterus, the largest cross-section of the cervix, and the cervical incision. At each fraction in a set of radiotherapy data, the deformations of organs and tissues in selected layers were more pronounced (Table 1).

Table 1

Image data of 3 different layers of every patient.

Image preprocessing

In clinical radiotherapy, images that are acquired under different image-guided devices frequently have different pixel information. US scans were acquired using the Elekta Clarity system [voxels: (0.577 × 0.577 mm2) × 0.577 mm slice thickness]. CT scans were acquired using SOMATOM Definition AS [voxels: (0.8 × 0.8 mm2) × 1 mm slice thickness]. The voxel information of images must be unified to increase registration accuracy and ensure accurate use of the deformation field. We initially unified the slice thickness, spatial resolution, bit depth, and matrix size of 2 modal images by using the multiscale adaptive mask rigid registration method.[ Whole US and CT images were first registered through 3D global mutual information registration. We then divided the registered US images into uniform blocks and analyzed their similarities to CT images. The local region with useful information can be extracted using the concept of region saliency. The selected local region was joined and used as the mask for global registration. The entire registration process alternated between global and local registrations until the regions converged. The resampled images were adjusted to a (1 × 1 mm2) × 1 mm slice thickness [400, 400, 100] image size, and 12-bit depth. Figure 2 shows that the central-point CT and US images have similar pixel information and were in the same space coordinate system after resampling. Images are deformed easily if bones are in the deformation field and further bring registration errors, because CT images contain rigid structures such as bone. Besides, different pressing strength and positions at abdomen during US image collection cause different deformation degree of the urinary bladder. Thus, the global registration of US images can easily cause wrong registration. The CT reconstruction technique almost completely eliminates the superposition of anatomic structures, leading to reduced “structural” noise. The CTsim image is smoothed using the high-order Gaussian curvature flow, which has been proposed by Lu et al.[

Figure 2

(A) CTsim and USsim images under the same coordinate system. (B) CBCT and USpv images under the same coordinate system.

Considering that the imaging range of each layer scan of the US image is approximately identical, the scan-range template is prepared. To obtain a mask based on the entire imaging area of the US, the pixel values outside the scan range are defined to be 0, and the pixel value in the scan range is 1. When preparing the mask based on the ROI region, the image is filtered horizontally and vertically using the Sobel edge operator to obtain the modulus and render the gradient amplitude image. The gradient image is then thresholded to eliminate the oversegmentation caused by the slight change in the grayscale. Finally, the ROI mask is obtained by performing watershed segmentation on the gradient-processed image. Binary masks based on the whole imaging region of USsim and USpv are shown in Fig. 3A and B. The surrounding background voxels are removed to expedite calculations. Binary masks based on ROI of USsim and USpv are shown in Fig. 3C and D. The deformation field exists only in the ROI to achieve local deformation. Binary mask based on wholeO of USsim and USpv is shown in Fig. 3E. Binary mask based on region of interest overlapping (ROIO) of USsim and USpv is shown in Fig. 3F. The overlapping area is the common area of 2 US scans.

Figure 3

(A) Binary masks based on the whole imaging region of USsim. (B) Binary masks based on the whole imaging region of USpv. (C) Binary masks based on ROI of USsim. (D) Binary mask based on ROI of USpv. (E) Binary mask based on wholeO of USsim and USpv. (F) Binary mask based on ROIO of USsim and USpv.

US image registration and application of deformation field

The US image registration and application of deformation field are shown in Fig. 4. First, resampled US images were selected. The multiresolution B spline deformation registration was conducted by ElastiX software using USpv as the fixed image and USsim as the moving image. We chose the ASGD (adaptive stochastic gradient descent) as the optimizer, and the advance mattes mutual information was used as metric method. The ASGD algorithm is defined as follows:

Figure 4

(A) Pseudo-CT image based on ROIO. (B) Pseudo-CT images without binary mask. (C) CBCT.

where [x]+ means max (x, 0), f denotes a sigmoid function, and μ0, t0, and t1 are user-defined initial conditions.[ The γ function is evaluated at time tk. The time is adapted depending on the inner product of the gradient and previous gradient . If the gradients in 2 consecutive steps point to the same direction, the inner product is positive so time is reduced. This phenomenon leads to increased step size γ(tk + 1) because γ is a monotonously decreasing function. In this way, the ASGD method implements an adaptive step-size mechanism.

The acquired deformation field was stored as a .txt file. Subsequently, the deformation field was used in CT image positioning by using Transformix software[ to obtain the pseudo-CT image. Finally, the imaging differences between pseudo-CT and CBCT were compared, and rigid registration between positioning CT and pseudo-CT and CBCT was performed to compare differences in their mutual information measurements. Different binary masks, including binary masks based on whole, wholeO, ROI, and ROIO, were added to the US image registration. The above operations were repeated, and then differences between pseudo-CT with and without binary mask and CBCT were compared. Pseudo-CT images based on different binary masks were further assessed.

Assessment

In this study, registration accuracies of pseudo-CT based on different binary masks and positioning CT were assessed by normalized gray mean-squared error (NMSE)[ and normalized mutual information (NMI)[ by using ElastiX software.

The gray mean-squared error reflects the average gray difference between registration images at the corresponding position. A smaller gray mean-squared error means a better registration effect. When the gray mean-squared error is 0, 2 images overlap completely. For the convenience of data comparison, this paper suggested to assess registration result by NMSE. The functional expression of NMSE is:

where F(r,c) and M(r,c) are the gray values of the fixed and moving images at (r,c) in the space coordinates system. and are the gray means of the reference and floating images. N is the number of total pixels in the image.

NMI is another quantitative index whose functional expression is as follows:

where I (A|B) is the mutual information value between images, H (A) and H (B) are information entropies, A is the fixed image, and B is the floating image. The registration effect increases as NMI approaches 1.

The pseudo-CT images based on different binary masks were compared with the CBCT images by using SSIM, which was proposed by Wang et al.[ The functional expression of SSIM is

where f and m stand for fixed image and moving image, μ and μ are the means of image gray values, σ and σ are the variances of gray value, σfm is covariance, c1 and c2 are the constants for maintaining the stability, L is the dynamic range of image gray value, k1 = 0.01, and k2 = 0.03. The value range of SSIM is [−1,1], and a higher value represents higher similarity between 2 images; when 2 images are the same, SSIM = 1.

Results

The images in this study were collected from voxel data of three patients with vertical cancer. Three groups of sectional images at different layers that meet experimental requirements were selected from each patient. CTsim was matched with pseudo-CT based on different binary masks, such as pseudo-CT based on no binary mask and CBCT. Registration accuracy was assessed using NMSE and NMI. Experimental results are listed in Table 2. The registration accuracy between pseudo-CT based on ROIO and CTsim was higher than that among pseudo-CT based on other binary masks. The NMI measures of the pseudo-CT and CTsim registrations obtained by the 3 patients based on the ROIO mask all exceeded 0.9, and the mean NMSE measures were below 0.2. The pair t test values of NMI between pseudo-CT based on ROIO and pseudo-CT based on other binary masks were 6.081, 14.025, 21.916, 31.997, and 5.857 (P < .0.5), indicating statistically significant differences. The pair t test values of NMSE were 9.072, 15.450, 20.524, 26.791, and 12.162 (P < .0.5), indicating statistically significant differences.

Table 2

Registration accuracy assessment by NMSE and NMI.

Pseudo-CT images based on ROIO and no binary mask are shown in Fig. 4A, B. The CBCT image on the same layer of CTsim is shown in Fig. 4C. Compared with Fig. 4B, the pseudo-CT image in Fig. 4A showed no large deformation of the cervix and its surrounding tissues and no erroneous deformation of bones; moreover, pseudo-CT shown in Fig. 4A was more similar to CBCT. Compared with Fig. 4C, the imaging of Fig. 4A showed high imaging quality in the soft-tissue region, especially at the junction among the bladder, cervix, and rectum. No artifact caused by the beam hardening effect was found in the figure. The structural difference between pseudo-CT and CBCT was further evaluated by SSIM, and experimental results are shown in Fig. 5. SSIM values between pseudo-CT based on ROIO, ROI, wholeO, whole and no binary mask, and CBCT were 0.9084, 0.8365, 0.7800, 0.6830, and 0.5518, respectively. pxy (x = 1, 2, 3; y = 1, 2, 3) represented the y layer of the x patients. The pseudo-CT images acquired by the 3 patients based on the ROI masks were found to be >90% similar to the corresponding CBCT, which was higher than in those obtained by other methods.

Figure 5

Structural difference assessment between pseudo-CT and CBCT by SSIM.

Discussion

In this study, different masks were used to obtain pseudo-CT images. We determined the registration region by adding masks to the US images to be registered. The obtained deformation field information was determined by the size and position of the registration region. As a voxel-matched displacement vector field, the deformation field can indicate the path of voxels movement between 2 registration images. Given that the CTsim and US images were resampled and registered in the preprocessing stage, the voxels in the 2 modal images were matched. When the US deformation field was applied to the CTsim image, the voxels in the CTsim were moved according to the path to which the deformation field was allocated. Finally, the pseudo-CT was obtained. The registration of US images based on the whole mask rendered all imaging areas in the deformation field and ensured the continuity and completeness of the deformation field. However, on the one hand, CTsim contained a structure as rigid as bone. If the US deformation field was applied to CTsim, the skeletal region would be deformed. On the other hand, in each acquisition of US images, the pressing position and pressure influenced the pressing effect, resulting in different degrees of bladder deformation. The pseudo-CT accuracy resulting from the inclusion of the bladder area in the deformable area was reduced. Registration based on the ROI mask can avoid the above problems. Herein, we defined ROI as an area of US imaging other than bladder and bone. However, a difference existed in the scanning direction and position of the US image acquired by the positioning stage and that acquired 1 week after treatment. In other words, differences existed among the imaging ranges of the US images obtained at different times, so the corresponding ROI masks also differed. The registration based on ROIO mask ensured that the overlapping area of the 2 scanned images was within the deformation range and further improved the registration accuracy.

Pseudo-CT based on different binary masks showed higher imaging quality than CBCT and achieved higher registration accuracy with CTsim than pseudo-CT without binary mask. The experiment proved that pseudo-CT can replace CBCT in radiotherapy for cervical cancer. However, pseudo-CT based on US deformation field cannot meet the imaging requirements of CBCT. Accordingly, binary masks were added before US image registration in this study. The binary mask mainly eliminated the image background beyond the imaging region, which can shorten registration time and increase registration efficiency, prevented deformation caused by the coverage of rigid structure in the deformation region, and shielded interference pixels outside the ROI and created deformation registration of local region.

The experiment demonstrated that pseudo-CT based on ROIO achieved the highest registration accuracy with CTsim. First, a rigid structure such as bones shall be separated from images to prevent registration error. Moreover, registration results that focused only on a single organ such as uterus and bladder and neglected the deformation of surrounding soft tissues cannot represent the actual tissue structural distribution in a patient. Due to different pressing strengths and positions at the abdomen of patients upon every collection of a US image, the deformation degree of bladder varied. Consequently, bladder was eliminated from ROI lest it caused registration errors. Hence, this study separated US images by performing watershed segmentation and eliminated the rigid structure (e.g., bone) from the imaging region.

The pseudo-CT based on ROIO possessed the anatomical structure information of CBCT and had a higher imaging quality than CBCT, thereby providing real-time CT image information for postoperative cervical cancer patients subjected to adaptive radiotherapy. In future works, we will focus on the dosimetry difference in radiation therapy between pseudo-CT and CBCT based on US deformation in different 3-dimensional binary masks. Subsequently, we will perform online adaptive radiotherapy based on pseudo-CT images. Deep learning methods based on convolutional neural networks (CNNs) have also been applied for cancer detection and identification of end-systolic and end-systolic frames of cardiac sequences.[ We will also use the CNN method to extract the image features (corner features, texture, and pixel information) of 2 modalities and thus generate a training set. The pseudo-image will be obtained on the basis of the feature matching training set.

Conclusion

The feasibility of using pseudo-CT imaging based on different binary masks to replace CBCT in radiotherapy for cervical cancer was proven by experiments. Pseudo-CT imaging based on ROIO masks achieved the best registration effect with CTsim.

Author contributions

The manuscript was written through contributions of all authors. S.H.-f. and N.X.-y. conceived the experiments, S.H.-f, N.X.-y., and X.K. performed the experimental characterizations. G.L.-g., S.J.-f., and L.T. analyzed the results. S.H.-f, N.X.-y., and X.K. revised the manuscript. All authors discussed the results and have given approval to the final version of the manuscript.

Data curation: Hongfei Sun, Liugang Gao.

Funding acquisition: Xinye Ni.

Methodology: Hongfei Sun, Kai Xie, Jianfeng Sui.

Software: Kai Xie, Tao Lin.

Writing – original draft: Hongfei Sun.

Writing – review & editing: Xinye Ni.