A contrast-enhanced-CT-based classification tree model for classifying malignancy of solid lung tumors in a Chinese clinical population.

BACKGROUND: To develop and validate a contrast-enhanced CT based classification tree model for classifying solid lung tumors in clinical patients into malignant or benign.
METHODS: Between January 2015 and October 2017, 827 pathologically confirmed solid lung tumors (487 malignant, 340 benign; median size, 27.0 mm, IQR 18.0-39.0 mm) from 827 patients from a dedicated Chinese cancer hospital were identified. Nodules were divided randomly into two groups, a training group (575 cases) and a testing group (252 cases). CT characteristics were collected by two radiologists, and analyzed using a classification and regression tree (CART) model. For validation, we used the decision analysis threshold to evaluate the classification performance of the CART model and radiologist's diagnosis (benign; malignant) in the testing group.
RESULTS: Three out of 19 characteristics [margin (smooth; slightly lobulated/lobulated/spiculated), and shape (round/oval; irregular), subjective enhancement (no/uniform enhancement; heterogeneous enhancement)] were automatically generated by the CART model for classifying solid lung tumors. The sensitivity, specificity, PPV, NPV, and diagnostic accuracy of the CART model is 98.5%, 58.1%, 80.6%, 98.6%, 79.8%, and 90.4%, 54.7%, 82.4% 98.5%, 74.2% for the radiologist's diagnosis by using three-threshold decision analysis.
CONCLUSIONS: Tumor margin and shape, and subjective tumor enhancement were the most important CT characteristics in the CART model for classifying solid lung tumors as malignant. The CART model had higher discriminatory power than radiologist's diagnosis. The CART model could help radiologists making recommendations regarding follow-up or surgery in clinical patients with a solid lung tumor. 2021 Journal of Thoracic Disease. All rights reserved.

Introduction

The diagnosis of lung nodules is a common and expensive challenge in medicine. Chest computed tomography (CT) is widely used for lung disease diagnosis (1,2). Lung nodules are divided into solid and sub-solid based on their density on CT, and the consensus of multiple guidelines is that lesions of different densities should be managed differently (3-6). Sub-solid nodules are often considered to be early stage adenocarcinoma (6,7), however, the histologic types of solid nodules or tumors vary widely. The 2015 World Health Organization Classification of Lung Tumors reports there are four major histological types of solid pulmonary malignancies, including adenocarcinoma, squamous cell carcinoma, large cell carcinoma, and small cell carcinoma; and four major types of benign lesions, including pulmonary hamartoma, inflammatory myofibroblastic tumor, sclerosing pneumocytoma and granuloma (8). This brings a challenge to the stratification of solid tumors into malignant or benign based on radiological imaging.

In recent years, research has focused on the work-up of smaller solid lung nodules. The Fleischner Society 2017 guidelines for the management of incidentally detected pulmonary nodules (4), advises that a solid nodule larger than 8 mm with suspicious morphology or upper lobe location should be considered for CT at 3 months, positron emission tomography/computed tomography (PET/CT), or tissue sampling. However, the challenge that radiologists often face is to give a definitive diagnosis of a certain nodule to be malignant or benign, especially in cancer centers where patients usually present with larger nodules or masses. Tissue biopsy is limited by difficulty to access locations and size of the tumor, and has potential complications such as pulmonary hemorrhage, pneumothorax, and risks associated with general anesthesia or sedation (9,10). PET/CT and contrast-enhanced CT scan are both effective noninvasive techniques widely used in clinical practice. However, PET/CT has high requirements for equipment and increases the financial burden for patients. Therefore, a contrast-enhanced CT scan is one of the main examination techniques performed before surgery, especially in underdeveloped countries and regions (11). Prior studies have demonstrated that the level of CT enhancement can differentiate the nature of the lung tumor (12,13). Absence of significant tumor enhancement (≤15 HU) at contrast-enhanced CT is strongly predictive of a benign nature (11). Therefore, the use of difference in CT density between non-contrast and contrast CT may help to establish a classification model to effectively improve the diagnosis of pulmonary nodules, especially for patients with a large solid tumor considered for surgery.

Our purpose was to develop and to validate a classification tree model based on contrast-enhanced CT characteristics of solid lung tumors to differentiate between the malignant or benign nature in clinical preoperative patients.

We present the following article in accordance with the TRIPOD reporting checklist (available at https://dx.doi.org/10.21037/jtd-21-588).

Methods

The institutional ethics committee board of Tianjin Medical University Cancer Institute and Hospital (No. bc2018039) approved this study. All participants signed an informed consent form before participating in this study, and this study conformed to the provisions of the Declaration of Helsinki (as revised in 2013).

Patients

A total of 1,789 consecutive patients from Tianjin Medical University Cancer Institute and Hospital who underwent surgical resection of a lung tumor with postoperative histopathological confirmation from January 2011 to October 2017 were considered. The inclusion criteria were: (I) solitary solid lung tumor on CT (diameter ≥8 mm); (II) preoperative thin-section non-contrast and contrast-enhanced CT images <1 month before surgery in the Picture Archiving and Communication System (PACS); (III) postoperative pathology report; (IV) high-quality images. We excluded 411 tumor cases with sub-solid appearance on CT, all 31 small cell carcinomas based on lymph node biopsy with poorly-defined border or unmeasurable lesions, 258 cases with multiple nodules on CT, 262 patients without available preoperative CT images (mainly because of preoperative imaging at a different hospital).

Computed tomography examination

CT examinations, consisting of an acquisition without and with iodine contrast, were performed on Somatom Sensation 64 (Siemens Medical Solutions, Forchheim, Germany) CT scanner, Lightspeed 16 (GE Medical Systems, Milwaukee, WI), or Discovery CT 750 HD (GE Medical Systems). The scan tube voltage was 120 kVp with automatic tube current modulation. The iodine contrast agent Visipaque (Iodixanol, 270 mg/mL) was administered intravenously through the upper extremity (1.5 mL/kg, injection rate: 2.5 mL/s). The scan range included the pulmonary apex level to below the diaphragm, and scanning was performed 70 seconds after injection of the contrast agent. For the Siemens CT system pitch was 0.95, acquired and reconstructed slice thickness of 1.5 mm, B70f and B30 reconstruction kernels were used. Pitch for the GE CT systems was 0.984, acquired and reconstructed slice thickness of 1.25 mm, and Stnd and Lung reconstruction kernels were used.

Evaluation of contrast-enhanced CT characteristics

All contrast-enhanced CT scans were reviewed by two experienced radiologists (6-year and 9 year reading experience in chest CT) blindly and independently by using the RadiAnt DICOM Viewer (version 2020.2). The two radiologists determined the final radiological characteristics after a mutual consultation. The radiologists evaluated the pulmonary solid tumor on pulmonary window setting (width, 1,450 HU; level, –500 HU), and mediastinal window setting (width, 350 HU; level, 40 HU). Included radiological characteristics were (see detailed description and example images in Appendix 1): (I) location (left/right; upper lobe/middle lobe/lower lobe); (II) centrality (peripheral/central); (III) shape (round/oval/irregular); (IV) margin (smooth/lobulated/spiculated); (V) calcification (yes/no); (VI) fat (yes/no); (VII) necrosis (yes/no); (VIII) cavitation (yes/no); (IX) air bronchogram (yes/no); (X) pleural indentation (yes/no); (XI) vascular invasion (yes/no); (XII) postobstructive pneumonia (yes/no); (XIII) satellite nodules (yes/no); (XIV) pleural effusion (yes/no); (XV) lymph nodes (yes/no); (XVI) size (in mm); (XVII) subjective enhancement [uniform/heterogeneous/no (<15 HU)]; (XVIII) plain CT value (in HU); (XIX) enhanced CT value (in HU); (XX) enhanced difference (in HU).

Decision analysis for the radiologist’s diagnosis

We used the clinical CT report to assess the diagnostic performance of radiologist evaluation. All CT diagnostic reports were double read by two radiologists at the time of CT acquisition and assigned to one of five categories: (I) benign; (II) probably benign; (III) undetermined; (IV) probably malignant; (V) high suspicion of malignancy. These categories represent the radiologist classification of malignancy probability of the evaluated nodule in real clinical setting. In total, 27 radiologists in the radiology department were involved in CT reporting. In this study, we classified into three management groups defined the radiologist diagnosis risk thresholds based on the malignancy probability of the five categories (observe =1, indeterminate =2 or 3 or 4, surgery =5). The “Observe” group was recommended for CT follow-up, the “indeterminate” group was recommended for further work-up (short interval CT follow-up, PET-CT or biopsy), and the “surgery” group was recommended for surgery.

Statistical analysis

We randomly divided the tumors (n=827) into a training group (n=575) and testing group (n=252). Student’s t-test, chi-square test, and Kolmogorov-Smirnov test were used to assess each indicator in the training and testing group. Finally, a classification and regression tree (CART) method was constructed to assess variables that might discriminate between benign and malignant tumors (P value <0.05 in univariate analysis) (14-16). CART model builds a tree through recursive partitioning and splits data into increasingly homogeneous subgroups. At each parent node the CART algorithm picks the distinguishing variable and gives the best discriminatory cutoff value between two child nodes. The depth of the classification tree was set to automatic. The parent node stops splitting when the size is less than 15 nodes, and the child node stops splitting when the size is less than 5 nodes. To show the probability of the classification tree model, ≥80% probability for each node was assumed as the cut-off value. To simplify the classification tree diagram, we made concise tables. Three threshold grades (ST1, ST2, and ST3) were made to describe the malignancy probability in each child node (ST1 = IF Malignant% ≤20%; ST2 = IF 20% < Malignant% <80%; ST3 = IF Malignant% ≥80%). ST1 was classified as ‘observe’ group, ST2 suggested for further work-up (see above), and ST3 advised for surgery, similar to the radiologist’s decision (Appendix 2). Finally, we used the testing group to validate the discriminatory performance of the CART model. P<0.05 indicated statistically significant results. Data were analysed using SPSS software version 20.0 (IBM Corp., NY, USA).

Results

Lung tumor and population characteristics

In total, 827 tumors with a median size of 27.0 mm (IQR 18.0–39.0 mm) in 827 patients were included. Of the patients, 56.3% were men (466/827). Mean age was 57.5 years (SD, 9.4 years). In total, 487 (58.9%) lesions turned out to be malignant. The most common malignant histological type was adenocarcinoma (n=319, 65.5%), then squamous cell carcinoma (n=118, 24.2%) and large cell carcinoma (n=71, 14.6%); 340 (41.1%) turned out to be benign, the most common benign pathological types were hamartoma (n=156, 45.9%), inflammatory pseudotumor (n=80, 23.5%), sclerosing pneumocytoma (n=54, 15.9%) and tuberculosis (n=50, 14.7%) (). We found no statistically significant differences between the training and testing group in patient and nodule characteristics.

Table 1

Characteristics of patients in the study

Characteristics	Total (n=827)	Training (n=575)	Testing (n=252)	P
Age, mean (SD) years	57.5±9.4	57.7±9.3	57.0±9.5	0.271a
Sex, n (%)
Man	466 (56.3)	328 (57.0)	138 (54.8)	0.543b
Woman	361 (43.7)	247 (43.0)	114 (45.2)
Histologic type, n (%)				0.338b
Malignant	487 (58.9)	352 (61.2)	135 (53.6)
Adenocarcinoma	319 (65.5)	226 (64.2)	93 (68.9)
Squamous cell carcinoma	118 (24.2)	87 (24.7)	31 (23.0)
Large cell	50 (10.3)	39 (11.1)	11 (8.1)
Benign	340 (41.1)	223 (38.8)	117 (46.4)
Inflammatory pseudotumor	80 (23.5)	55 (24.7)	25 (21.4)
Hamartoma	156 (45.9)	101 (45.3)	55 (47.0)
Sclerosing pneumocytoma	54 (15.9)	37 (16.6)	17 (14.5)
Tuberculosis	50 (14.7)	30 (13.5)	20 (17.1)

a, Student’s t-test (Normally distributed) and Kolmogorov-Smirnov test (non-normally distributed); b, Pearson’s chi-square test and Fisher’s exact test. HU, Hounsfield units.

Development of the CART model for predicting malignant and benign in training group

In the training group, nineteen variables (age, gender, nodule type, shape, margin, calcification, fat, cavitation, air bronchogram, pleural indentation, vascular invasion, postobstructive pneumonia, satellite nodules, lymph nodes, subjective enhancement, contrast-enhanced CT attenuation, plain CT attenuation, enhanced difference attenuation, size) were found to be different between the malignant and benign groups in univariate analysis ().These nineteen variables were entered into the CART model. Three categorical variables [subjective enhancement (no/uniform, heterogeneous); margin (smooth, lobulated/spiculated); shape (round/oval, irregular)] were automatically generated by the CART (). The optimized classification diagram was shown in . The sensitivity, specificity, PPV (positive predictive value), NPV (negative predictive value), and diagnostic accuracy of the CART model in the training group were 94.9%, 57.0%, 82.9%, 93.4%, and 80.2%, respectively.

Table 2

Characteristics of benign vs. malignant tumors in the training group

Characteristics	Subgroup	Malignant (n=352)	Benign (n=223)	P
Patient age, mean± SD, years		60.2±8.7	53.9±8.9	0.000a
Sex, n (%)	Man	227 (64.5)	101 (45.3)	0.000b
	Woman	125 (35.5)	122 (54.7)
Morphological
Location, n (%)	Left upper lobe	103 (29.3)	59 (26.5)	0.627b
	Left lower lobe	67 (19.0)	44 (19.7)
	Right upper lobe	91 (25.9)	52 (23.3)
	Right middle lobe	27 (7.7)	16 (7.2)
	Right lower lobe	64 (18.2)	52 (23.3)
Nodule type, n (%)	Peripheral	268 (76.1)	198 (88.8)	0.000b
	Central	84 (23.9)	25 (11.2)
Shape, n (%)	Round/oval	180 (51.1)	154 (69.1)	0.000b
	Irregular	172 (48.9)	69 (30.9)
Margin, n (%)	Smooth	21 (6.0)	95 (42.6)	0.000b
	Lobulated	182 (51.7)	71 (3.18)
	Spiculated	149 (42.3)	57 (25.6)
Calcification, n (%)	No	311 (88.4)	176 (78.9)	0.002b
	Yes	41 (11.6)	47 (21.1)
Fat, n (%)	No	351 (99.7)	189 (84.8)	0.000b
	Yes	1 (0.3)	34 (15.2)
Necrosis, n (%)	No	288 (81.8)	195 (87.4)	0.073b
	Yes	64 (18.2)	28 (12.6)
Cavitation, n (%)	No	278 (79.0)	201 (90.1)	0.000b
	Yes	74 (21.0)	22 (9.8)
Air Bronchograms, n (%)	No	247 (70.2)	214 (96.0)	0.000b
	Yes	105 (29.8)	9 (4.0)
Pleural indentation, n (%)	No	80 (22.7)	123 (55.2)	0.000b
	Yes	272 (77.3)	100 (44.8)
Vascular invasion, n (%)	No	247 (70.2)	222 (99.6)	0.000b
	Yes	105 (29.8)	1 (0.4)
Postobstructive pneumonia, n (%)	No	215 (61.1)	165 (74.0)	0.001b
	Yes	137 (38.9)	58 (26.0)
Satellite nodules, n (%)	No	347 (98.6)	206 (92.4)	0.000b
	Yes	5 (1.4)	17 (7.6)
Pleural effusion, n (%)	No	340 (96.6)	220 (98.7)	0.130b
	Yes	12 (3.4)	3 (1.3)
Lymph nodes, n (%)	No	248 (70.5)	218 (97.8)	0.000b
	Yes	104 (29.5)	5 (2.2)
Subjective enhancement, n (%)	No	6 (1.7)	89 (39.9)	0.000b
	Uniform	3 (0.9)	38 (17.0)
	Heterogeneous	343 (97.4)	96 (43.0)
CT attenuation (HU), mean± SD	Plain CT	28.6±11.4	23.0±27.0	0.001a
	Enhanced CT	63.1±19.9	50.8±40.2	0.000a
	Difference in HU	34.5±17.7	27.9±27.4	0.001a
Size, median (IQR), mm	Diameter	32.0 (23.0–42.8)	17.6 (13.5–28.4)	0.000a
Radiologist’s diagnosis, n (%)	Observe	12 (3.4)	117 (52.5)
	Indeterminate	24 (6.8)	52 (23.3)
	Surgery	316 (89.8)	54 (24.2)

a, Student’s t-test (Normally distributed) and Kolmogorov-Smirnov test (non-normally distributed); b, Pearson’s chi-square test and Fisher’s exact test. HU, Hounsfield units.

Figure 1

CART model for the classifying the malignancy of pulmonary solid tumor in the training group. CART, classification and regression tree.

Figure 2

The optimized classification diagram of CART model (ST1 = IF Malignant% ≤ 20%; ST2 = IF 20% < Malignant% < 80%; ST3 = IF Malignant% ≥ 80). CART, classification and regression tree.

Validation of the CART models and radiologist’s diagnosis in the testing group

Both the radiologists and the CART model used three thresholds to determine the solid tumor malignancy risk. The classification performance of the CART models and radiologist’s diagnosis were evaluated using the testing group. The sensitivity specificity, PPV, NPV, and diagnostic accuracy to differentiate malignant from benign tumors were 98.5%, 58.1%, 80.6%, 98.6, and 79.8% for CART model and 90.4%, 54.7%, 82.4% 98.5% and 74.2% for radiologist’s diagnosis, respectively ().

Table 3

Decision analysis using the three thresholds in the testing group

Method	Prediction	Risk threshold for malignancy	Pathological results (No. %)				Predictive value (NPV/PPV)
			Benign (n=117)		Malignant (n=135)
Radiologist’s diagnosis	Observe	1	64 (54.7)	T.N	1 (0.7)	F.N	98.5%
	Indeterminate	2–4	27 (23.1)		12 (8.9)		–
	Surgery	5	26 (22.2)	F.P	122 (90.4)	T.P	82.4%
CART model	ST1	≤20%	68 (58.1)	T.N	1 (0.7)	F.N	98.6%
	ST2	20–80%	17 (14.5)		1 (0.7)		–
	ST3	≥80%	32 (27.4)	F.P	133 (98.5)	T.P	80.6%

P, true positive; T.N, true negative; F.P, false-positive; F.N, false-negative; NPV, negative predictive value; PPV, positive predictive value; 1, benign; 2, probably benign; 3, undetermined; 4, probably malignant; 5, high suspicion of malignancy. ST1= IF Malignant% ≤20%; ST2= IF 20%< Malignant% <80%; ST3= IF Malignant% ≥80%.

Discussion

Three radiological CT characteristics of solid lung lesions were shown to have discriminatory ability for malignant and benign lung tumors. These characteristics were: subjective enhancement, margin and shape. The CART model for solid tumors in our clinical population resulted in a higher diagnostic accuracy compared to radiologist’s diagnosis. Our results suggest that our contrast-enhanced CT based CART model can support radiologists in making a diagnosis decision of solid lung tumors (median diameter 27.0 mm, IQR 18.0–39.0 mm) in case only CT is available as a diagnostic tool.

Radiological characteristics (nodule type, size, shape, margin and location) and clinical information (age, gender, cancer history, smoking history) have been used to develop classification models (such as the models developed by Mayo Clinic, Brock University and Veterans Affairs) to predict the probability of malignancy of pulmonary tumors (17-19). However, several previous studies showed that the performance of these models, especially in large tumors, is moderate (20-23). Our previous study (24), comparing classification models (Veterans Affairs, Mayo, and Brock model) to the diagnosis of the radiologist for differentiation of lung nodules (median size: 17.0 mm, IQR 13.0–21.0 mm) into benign and malignant in a Chinese population, showed a lower discriminatory power for the three models compared with the radiologist’s diagnosis. In that study, less than 15.5% out of 207 malignancies were classified at and above the surgical threshold using the three classification models, while the remaining malignant nodules were considered indeterminate (VA model 44.9%, Mayo model 85.5%; Brock model 93.7%). The radiologist’s diagnosis showed higher performance contrast to the three classification models, with 8 (3.9%) under the observe threshold, 42 (20.3%) as indeterminate, and 157 (75.8%) above the surgical threshold. Since the prior study only included non-contrast CT features, we decided to develop a classification model for larger tumors, based on more radiological features (contrast-enhanced CT) and the characteristics of a local population referred to a tertiary cancer center in China. In the current study, based on large lung tumors cohort, we added contrast-enhanced CT features to the traditional radiological characteristics and used a classification model (CART) instead of logistic models. Compared with the logistics regression equation, the CART model can display each step of the diagnosis more intuitively. Finally, a three thresholds ST 1–3 classification system, was used to describe the probability of malignancy of solid tumors which is more consistent with the radiologist’s diagnosis process. Our results showed only 18 (7.1%) tumors predicted as ST2 (indeterminate) in the CART model, compared with 39 (15.5%) based on radiologist’s diagnosis. The sensitivity and specificity were 98.5% and 58.1% for the CART model compared to 90.4% and 54.7% for the radiologist’s diagnosis. Our CART model showed higher discriminatory power than radiologist’s diagnosis. This suggests that contrast-enhanced CT characteristics (subjective enhancement) are useful factors for distinguishing malignant and benign pulmonary lesions.

Several previous studies showed that the lack of lung tumor CT contrast enhancement (≤15 HU) is strongly predictive for benign pathology. This was found to be helpful in the discrimination of malignant and benign pulmonary lesions at contrast-enhanced CT (12,13,25). The sensitivity reached 98%, but the specificity for malignancy was only 50–60% by using 15HU as enhancement cut-off (13).In a study by Yi et al. (26) the sensitivity for malignant nodules was 99% with specificity of 57% by using 30 HU as the enhancement cut-off value. The authors concluded that contrast-enhanced CT is highly sensitive to detect malignant tumors but has low specificity. The latter might be explained by the fact that some benign nodules, such as sclerosing hemangioma (27), can enhance as well. These findings are similar to our study’s training group, 93.7% (89/95) of the lesions with no enhancement (<15 HU), 92.7% (38/41) with uniform enhancement, and 21.9% (96/439) with heterogeneous enhancement being benign. The extent of enhancement may reflect underlying tumor angiogenesis. Therefore, radiologists cannot simply rely on an enhancement cut-off value to discriminate the benign and malignant tumors in clinical practice. In our CART model, we used subjective enhancement [(uniform, heterogeneous, no (<15 HU)] instead of a HU cut-off value. In total, 96.4% (54/56) of the lesions with uniform enhancement patterns were benign. Combined with tumor shape and margin features, our CART model achieved an overall diagnostic accuracy of 79.8%, which was higher than 74.2% by radiologist’s diagnosis. This difference is mainly determined by the larger indeterminate malignant group in the radiologist.

There are limitations in our study. First, selection bias is unavoidable in our study. We only included surgically resected solid tumors with known pathology. This may lead to more malignancy features to be present compare to non-resected benign solid tumors. Furthermore, this is a retrospective single-center study from a dedicated Chinese cancer hospital. The clinical application of the results may require more central data to verify. Finally, the CART model should include more clinical information (smoking history, cancer history, and family history of cancer) to improve the diagnosis performance.

To conclude, a CART model based on contrast-enhanced CT characteristics was more sensitive and accurate in the classification of malignancy of solid lung tumors than radiologist’s diagnosis. This classification model could assist radiologists to make recommendations regarding follow-up or surgery in clinical patients with a solid lung tumor.

The article’s supplementary files as