| Literature DB >> 36077662 |
Rie Tachibana1,2, Janne J Näppi1, Toru Hironaka1, Hiroyuki Yoshida1.
Abstract
Existing electronic cleansing (EC) methods for computed tomographic colonography (CTC) are generally based on image segmentation, which limits their accuracy to that of the underlying voxels. Because of the limitations of the available CTC datasets for training, traditional deep learning is of limited use in EC. The purpose of this study was to evaluate the technical feasibility of using a novel self-supervised adversarial learning scheme to perform EC with a limited training dataset with subvoxel accuracy. A three-dimensional (3D) generative adversarial network (3D GAN) was pre-trained to perform EC on CTC datasets of an anthropomorphic phantom. The 3D GAN was then fine-tuned to each input case by use of the self-supervised scheme. The architecture of the 3D GAN was optimized by use of a phantom study. The visually perceived quality of the virtual cleansing by the resulting 3D GAN compared favorably to that of commercial EC software on the virtual 3D fly-through examinations of 18 clinical CTC cases. Thus, the proposed self-supervised 3D GAN, which can be trained to perform EC on a small dataset without image annotations with subvoxel accuracy, is a potentially effective approach for addressing the remaining technical problems of EC in CTC.Entities:
Keywords: artificial intelligence; electronic cleansing; generative adversarial network; self-supervised learning; virtual colonoscopy
Year: 2022 PMID: 36077662 PMCID: PMC9454562 DOI: 10.3390/cancers14174125
Source DB: PubMed Journal: Cancers (Basel) ISSN: 2072-6694 Impact factor: 6.575
Figure 1(a) Overview of the training process in the 3D GAN. (b) Architecture of the generator in the 3D GAN. (c) Architecture of the discriminator in the 3D GAN.
Detailed architecture of the generator network of the 3D GAN in Figure 1b.
| Layers | Kernel | Stride | Padding | Output Shape | Activation | Batch Norm. | Dropout |
|---|---|---|---|---|---|---|---|
| Input: Image | 128 × 128 × 128 × 1 | ||||||
| Conv. Layer 1 | 4 | 2 | 1 | 64 × 64 × 64 × 64 | LeakyReLU | ||
| Conv. Layer 2 | 4 | 2 | 1 | 32 × 32 × 32 × 128 | LeakyReLU | True | |
| Conv. Layer 3 | 4 | 2 | 1 | 16 × 16 × 16 × 256 | LeakyReLU | True | |
| Conv. Layer 4 | 4 | 2 | 1 | 8 × 8 × 8 × 512 | LeakyReLU | True | |
| Conv. Layer 5 | 4 | 2 | 1 | 4 × 4 × 4 × 512 | LeakyReLU | True | |
| Conv. Layer 6 | 4 | 2 | 1 | 2 × 2 × 2 × 512 | LeakyReLU | True | |
| Conv. Layer 7 | 4 | 2 | 1 | 1 × 1 × 1 × 512 | ReLU | ||
| Deconv. Layer 8 | 4 | 2 | 1 | 2 × 2 × 2 × 512 | True | ||
| Concatenate (Layer 8, Layer 6) | |||||||
| Deconv. Layer 9 | 4 | 2 | 1 | 4 × 4 × 4 × 512 | ReLU | True | True |
| Concatenate (Layer 9, Layer 5) | |||||||
| Deconv. Layer 10 | 4 | 2 | 1 | 8 × 8 × 8 × 512 | ReLU | True | True |
| Concatenate (Layer 10, Layer 4) | |||||||
| Deconv. Layer 11 | 4 | 2 | 1 | 16 × 16 × 16 × 256 | ReLU | True | |
| Concatenate (Layer 11, Layer 3) | |||||||
| Deconv. Layer 12 | 4 | 2 | 1 | 32 × 32 × 32 × 128 | ReLU | True | |
| Concatenate (Layer 12, Layer 2) | |||||||
| Deconv. Layer 13 | 4 | 2 | 1 | 64 × 64 × 64 × 64 | ReLU | True | |
| Concatenate (Layer 13, Layer 1) | |||||||
| Deconv. Layer 14 | 4 | 2 | 1 | 128 × 128 × 128 × 1 | Tanh | ||
Detailed architecture of the discriminator network of the 3D GAN in Figure 1c.
| Layers | Kernel | Stride | Padding | Output Shape | Activation | Batch Norm. |
|---|---|---|---|---|---|---|
| Input 1: Real Image | 128 × 128 × 128 × 1 | |||||
| Input 2: Fake Image | 128 × 128 × 128 × 1 | |||||
| Concatenate (Input 1, Input 2) | ||||||
| Conv. Layer 1 | 4 | 2 | 1 | 64 × 64 × 64 × 64 | LeakyReLU | |
| Conv. Layer 2 | 4 | 2 | 1 | 32 × 32 × 32 × 128 | LeakyReLU | True |
| Conv. Layer 3 | 4 | 2 | 1 | 16 × 16 × 16 × 256 | LeakyReLU | True |
| Conv. Layer 4 | 4 | 2 | 1 | 8 × 8 × 8 × 512 | LeakyReLU | True |
| Conv. Layer 5 | 4 | 2 | 1 | 4 × 4 × 4 × 1 | Sigmoid | |
Figure 2Overview of our proposed self-supervised learning for the training of the 3D-GAN EC scheme.
Figure 3The mean value of the PSNR over 100 VOIs that were extracted from the fecal-tagging anthropomorphic colon phantom that was cleansed virtually by use of the proposed self-supervised 3D-GAN EC scheme. The values are shown for the initial supervised learning step and for the subsequent self-supervised training iterations for different numbers of convolutional and deconvolutional layers (N) of G. A high PSNR value indicates a higher cleansing quality than a low value.
Paired t-test of the differences of the PSNRs for different numbers of convolutional layers, N, in comparison to the five convolutional layers of the optimized self-supervised 3D-GAN EC scheme.
|
|
|
|
|
| ||||||
|
|
|
|
|
|
|
|
|
|
| |
| N = 4 | 1.929 | 0.057 | 0.778 | 0.439 | 1.250 | 0.214 | 3.477 | 0.001 | 3.414 | 0.001 |
| N = 6 | −4.621 | 0.000 | −6.610 | 0.000 | −3.187 | 0.002 | −1.294 | 0.199 | 0.279 | 0.781 |
| N = 7 | −4.121 | 0.000 | −3.331 | 0.001 | 0.244 | 0.808 | 4.253 | <0.0001 | 5.984 | <0.0001 |
|
|
|
|
| |||||||
|
|
|
|
|
|
|
|
| |||
| N = 4 | 5.254 | <0.0001 | 6.648 | <0.0001 | 6.253 | <0.0001 | 6.092 | <0.0001 | ||
| N = 6 | 1.808 | 0.074 | 4.255 | <0.0001 | 5.107 | <0.0001 | 4.987 | <0.0001 | ||
| N = 7 | 8.340 | <0.0001 | 10.010 | <0.0001 | 11.167 | <0.0001 | 12.740 | <0.0001 | ||
Figure 4Comparison of the mean numbers of EC artifacts (indicated by blue circles) observed on the virtual 3D fly-through views of the virtually cleansed clinical CTC cases generated by the commercial EC software and over the successive iterations of the self-supervised training of our proposed 3D-GAN EC scheme (N = 5).
Figure 5Visual comparison of the virtual cleansing by the self-supervised 3D-GAN EC and commercial EC. In the first column, the green arrows show the direction of the virtual camera in the virtual 3D fly-through views to the right. In the second column, the white arrows on the top and middle rows show locations of polyps partially submerged in residual feces, and on the bottom row, the cyan arrows indicate the location of a thin layer of fluid to be cleansed. In the third column, the orange arrows indicate locations of observed EC image artifacts by the commercial EC, which are not present on the EC images of the self-supervised 3D-GAN EC in the fourth column.