| Literature DB >> 36010201 |
Zhao Wang1,2, Yuxin Xu3,4, Linbo Tian1,2, Qingjin Chi5, Fengrong Zhao5, Rongqi Xu1,2, Guilei Jin5, Yansong Liu6, Junhui Zhen3,4, Sasa Zhang1,2,5.
Abstract
Targeted therapy is an effective treatment for non-small cell lung cancer. Before treatment, pathologists need to confirm tumor morphology and type, which is time-consuming and highly repetitive. In this study, we propose a multi-task deep learning model based on a convolutional neural network for joint cancer lesion region segmentation and histological subtype classification, using magnified pathological tissue images. Firstly, we constructed a shared feature extraction channel to extract abstract information of visual space for joint segmentation and classification learning. Then, the weighted losses of segmentation and classification tasks were tuned to balance the computing bias of the multi-task model. We evaluated our model on a private in-house dataset of pathological tissue images collected from Qilu Hospital of Shandong University. The proposed approach achieved Dice similarity coefficients of 93.5% and 89.0% for segmenting squamous cell carcinoma (SCC) and adenocarcinoma (AD) specimens, respectively. In addition, the proposed method achieved an accuracy of 97.8% in classifying SCC vs. normal tissue and an accuracy of 100% in classifying AD vs. normal tissue. The experimental results demonstrated that our method outperforms other state-of-the-art methods and shows promising performance for both lesion region segmentation and subtype classification.Entities:
Keywords: classification; convolutional neural network; deep learning; histopathological images; lung cancer; medical images; multiple tasks; segmentation
Year: 2022 PMID: 36010201 PMCID: PMC9406737 DOI: 10.3390/diagnostics12081849
Source DB: PubMed Journal: Diagnostics (Basel) ISSN: 2075-4418
Figure 1Structure of our research. A data pre-processing module was developed to annotate, crop and standardize original images into image patches and corresponding ground truth. With the training module and testing module, the multi-task model was trained and tested.
Demographic and clinical information (mean ± standard deviation) of the studied lung cancer subjects.
| Diagnosis | Age | Gender (M/F) 1 | Cancer Staging | Tumor Volume |
|---|---|---|---|---|
| SCC | 64.4 ± 7.8 | 14/1 | 10/4/1 | 10.74 ± 9.5 |
| AD | 53.8 ± 12.6 | 5/6 | 9/2/0 | 2.91 ± 2.1 |
| NC | 61.1 ± 9.1 | 8/2 | - | - |
1 M/F: male or female. 2 Cancer Staging: clinical Tumor Node Metastasis stage.
Figure 2The first row shows examples of the original images scanned from histopathologic slices. The second row shows the annotated images where pathologists highlighted the lesion regions. The third row shows binary masks for the segmentation task.
Figure 3Structure of the MCN. “Double Conv” denotes the two operations of convolution, in which the kernel size was 3 × 3, the stride was 1, and the padding was 1, batch-normalization and ReLU. “Down Conv” denotes max-pooling and the operation of “Double conv”. “Up Conv” denotes up-sampling, in which the kernel size was 2 × 2, and the operation of “Double conv”. “Out Conv” denotes an operation of convolution in which the kernel size was 1 × 1, the stride was 1, and the padding was 0. The “flatten” operation denotes the adaptive average pooling and was followed by two fully connected layers. The “⊕” operation concatenates feature maps in left side and the right side as input to next “Up Conv” module.
Performance of classification and segmentation after assigning different weights to their respective losses.
| Performance in | Classification | Segmentation | |||||||
|---|---|---|---|---|---|---|---|---|---|
| ACC | SEN | SPE | DSC | SEN | PRE | IoU | |||
| Γ1:Γ2 = 0:1 | SCC vs. NC | 97.8 | 95.2 | 100 | Mean | — | — | — | — |
| SCC | — | — | — | — | |||||
| AD vs. NC | 90.2 | 86.1 | 100 | AD | — | — | — | — | |
| NC | — | — | — | — | |||||
| Γ1:Γ2 = 0.5:1 | SCC vs. NC | 92.7 | 90.5 | 95 | Mean | 79.4 | 84.5 | 78.7 | 77.9 |
| SCC | 94.0 | 91.6 | 96.8 | 88.9 | |||||
| AD vs. NC | 98.1 | 100 | 100 | AD | 68.7 | 63.3 | 75.4 | 52.4 | |
| NC | 93.8 | 96.6 | 91.1 | 88.2 | |||||
| Γ1:Γ2 = 1:1 | SCC vs. NC | 95.1 | 95 | 95.2 | Mean | 92.3 | 92.2 | 91.9 | 85.2 |
| SCC | 93.5 | 90.1 | 97.3 | 87.9 | |||||
| AD vs. NC | 100 | 100 | 100 | AD | 89.0 | 89.4 | 88.6 | 80.2 | |
| NC | 94.5 | 97.2 | 89.7 | 87.5 | |||||
| Γ1:Γ2 = 1:0.5 | SCC vs. NC | 92.7 | 94.7 | 90.9 | Mean | 84.4 | 80.5 | 88.0 | 73.0 |
| SCC | 89.3 | 82.7 | 97.2 | 80.8 | |||||
| AD vs. NC | 95.7 | 100 | 90.9 | AD | 73.6 | 61.4 | 83.5 | 56.6 | |
| NC | 90.3 | 97.3 | 83.3 | 81.5 | |||||
| Γ1:Γ2 = 1:0 | SCC vs. NC | — | — | — | Mean | 89.4 | 88.6 | 90.5 | 82.1 |
| SCC | 95.6 | 94.2 | 97.8 | 92.3 | |||||
| AD vs. NC | — | — | — | AD | 76.8 | 73.9 | 80.1 | 62.4 | |
| NC | 95.9 | 97.6 | 93.7 | 91.7 | |||||
Mean: average of the evaluation indexes of SCC, AD and NC.
Comparison of our method with other multi-task methods for segmentation and classification.
| Method | Tumor | Segmentation Performance (%) | Tumor | Classification | |||||
|---|---|---|---|---|---|---|---|---|---|
| DSC | SEN | Pre | IoU | ACC | SEN | SPE | |||
| MGMLN | Mean | 84.1 | 79.6 | 91.5 | 74.6 | SCC vs. NC | 64.1 | 100.0 | 30.0 |
| SCC | 92.0 | 88.1 | 96.4 | 85.3 | |||||
| AD | 65.1 | 52.2 | 86.9 | 48.4 | AD vs. NC | 96.6 | 100.0 | 85.7 | |
| NC | 95.1 | 98.5 | 91.3 | 90.0 | |||||
| MDCN | Mean | 81.4 | 81.7 | 85.3 | 71.1 | SCC vs. NC | 92.7 | 90.0 | 95.2 |
| SCC | 85.2 | 74.6 | 99.6 | 74.4 | |||||
| AD | 69.4 | 80.9 | 60.2 | 52.7 | AD vs. NC | 100.0 | 100.0 | 100.0 | |
| NC | 89.7 | 89.5 | 96.0 | 86.3 | |||||
| MCN | Mean | 92.3 | 92.2 | 91.9 | 85.2 | SCC vs. NC | 95.1 | 95.0 | 95.2 |
| SCC | 93.5 | 90.1 | 97.3 | 87.9 | |||||
| AD | 89.0 | 89.4 | 88.6 | 80.2 | AD vs. NC | 100.0 | 100.0 | 100.0 | |
| NC | 94.5 | 97.2 | 89.7 | 87.5 | |||||
Figure 4Comparison of the segmentation results for SCC and AD obtained with our method (MCN) and other two methods (MDCN and MGMLN).
Figure 5Confusion matrixes of the classification results of MCN, MDCN, MGMLN. In each matrix, rows denote the histological subtype labels, and columns denote the predicted histological subtypes.