Developing of risk models for small solid and subsolid pulmonary nodules based on clinical and quantitative radiomics features.

BACKGROUND: Accurate evaluation of pulmonary nodule malignancy is important for lung cancer management. This current study aimed to develop risk models for small solid and subsolid pulmonary nodules based on clinical and quantitative radiomics features.
METHODS: This study enrolled 5-20 mm pulmonary nodules detected on thoracic high-resolution computed tomography (HRCT), which were all confirmed pathologically. There were 548 solid nodules (242 malignant vs. 306 benign) and 623 subsolid nodules (SSNs 519 malignant vs. 104 benign). Relevant clinical characteristics were recorded. The CT image prior to the initial treatment was chosen for manual segmentation of the targeted nodule using the ITK-SNAP software. Subsequently, the marked image was processed to quantitatively extract 1218 radiomics features using PyRadiomics. We performed five-fold cross-validation to select potential predictors from clinical and radiomics features using the LASSO method and to evaluate the performance of the established models. In total, four types of models were tried: random forest, XGBOOST, SVM, and logistic models. The established models were compared with the Mayo model.
RESULTS: Lung cancer risk models were developed among four nodule groups: all nodules (410 benign vs. 761 malignant; 1:1.86), nodules ≤10 mm (185 benign vs. 224 malignant; 1:1.21), solid nodules (306 benign vs. 242 malignant; 1.26:1), and SSNs (104 benign vs. 104 malignant; 1:1 matched). Significant clinical and radiomics predictors were selected for each group. The accuracy, area under the ROC curve, sensitivity, and specificity of the best model on validation dataset was 0.86, 0.91, 0.93, 0.73 for all nodules (XGBOOST), 0.82, 0.90, 0.86, 0.76 for nodules ≤10 mm (XGBOOST), 0.80, 0.89, 0.78, 0.82 for solid nodules (XGBOOST) and 0.70, 0.73, 0.73, 0.67 for SSNs (Random Forest). Except for the SSN models, the established clinical-radiomics models were superior to the Mayo model.
CONCLUSIONS: Predictive models based on both clinical and radiomics features can be used to assess the malignancy of small solid and subsolid pulmonary nodules, even for nodules that are 10 mm or smaller. 2021 Journal of Thoracic Disease. All rights reserved.

Introduction

The International Agency for Research on Cancer reported that lung cancer was the most commonly diagnosed cancer (11.6% of the total cases) and the leading cause of cancer death (18.4% of the total cancer deaths) for both sexes in 2018 (1). Hence, efficient strategies are necessary to manage the disease. Early stage lung cancer usually presents as a solitary pulmonary nodule on thoracic computed tomography (CT), namely, a rounded lesion measuring less than 30 mm in diameter, which is completely surrounded by pulmonary parenchyma without other pulmonary abnormalities (2). According to its texture, nodules can be classified into subsolid nodules (SSNs) or solid nodules (3). Accurate characterization of pulmonary nodules, especially regarding their likelihood of malignancy, can be very important in lung cancer management.

Traditionally, thoracic radiologists and clinicians rely largely on qualitative morphological features such as texture, spiculation, lobulation, calcification, pattern of enhancement, presence of blood vessels, impact on adjacent structures, and so on, to evaluate the nature of pulmonary nodules (4). For example, a smoothly marginated solid nodule with internal fat and calcification can be a hamartoma, and a solid triangular subpleural nodule with a linear extension to the pleural surface is typical of an intrapulmonary lymph node, where no further CT follow-up is recommended (3). In addition, nodule size, nodule type, spiculation, upper lobe location, 18F-fluorodeoxyglucose uptake, and volume doubling time (VDT) could be independent predictors of malignancy (5-8). However, morphological features can sometimes overlap between benign and malignant nodules, so the morphological features are not sufficient to assess the likelihood of malignancy (9). In addition, inter- and intraobserver agreement in evaluating nodule features was found to be highly variable (10). Moreover, nodules less than 10 mm in size, which account for approximately 80% of all non-calcified nodules, are less amenable than larger nodules when it comes to characterization (11).

At present, a rapidly evolving field called radiomics, which enables digital decoding of images into quantitative features, including descriptors of size, shape, and textural features, has shown promise in characterizing lung cancers (12). For example, previous radiomics studies were successfully performed to identify epidermal growth factor receptor mutations, classify tumor histologic subtype, and predict tumor invasiveness as well as lymph node and distant metastasis (13). Therefore, the current study aimed to develop risk models for small solid and subsolid pulmonary nodules (5–20 mm) based on clinical and quantitative radiomics features.

We present the following article in accordance with the TRIPOD reporting checklist (available at https://dx.doi.org/10.21037/jtd-21-80).

Methods

Patient enrollment

Data were retrospectively collected from the West China Hospital of Sichuan University between January 2010 and July 2017. Patients were enrolled if there was an untreated, pathologically confirmed, 5–20 mm, dominant, noncalcified SSN, or solid nodule on thoracic high-resolution CT (HRCT). The study included incidental pulmonary nodules and annually screen detected nodules. Patients were excluded if (I) multiple pulmonary nodules were observed, where nodules distributed diffusely or multiple nodules were confirmed to be tumors or if there were several problematic nodules; (II) pleural effusion, atelectasis, or lymph node enlargement was observed; and (III) the pathological diagnosis was not clear or it was a metastatic tumor. Pathological diagnoses of all nodules were based on sputum cytology or pathologic examination of lung tissues obtained from bronchoscopy, CT-guided percutaneous lung biopsy, or thoracic surgery. In total, 1,855 patients with 5–20 mm pathologically confirmed nodules were found, and 1,171 patients were enrolled for further analysis, as the segmentation of two patients failed and 682 patients did not have HRCT.

Collection of clinical variables

Clinical characteristics were recorded, including age, sex, smoking status, history of malignancy, and family history of lung cancer (demographic); diameter, location, shape, spiculation, lobulation (radiologic), red blood cells, white blood cells, blood platelets, neutrophil to lymphocyte ratio (NLR), prothrombin time, activated partial thromboplastin time (APTT), carcinoembryonic antigen (CEA), cytokeratin 19 fragment (CYFRA21-1) and neuron specific enolase (NSE, laboratory). This study was approved by the institutional review board of the West China Hospital of Sichuan University, and the requirement to obtain informed consent was waived as the privacy and identity information of the subjects were guaranteed. There were some missing data regarding CEA, CYFRA21-1, and NSE tests, which were populated by the median value. The data were not available as some of the patients did not undergo testing.

CT image acquisition

The CT image chosen was the one prior to the initial treatment. All images were acquired using several models of multi-row spiral CT scans from GE, Siemens or Philips scanners. Most CT scans were obtained between 100 and 120 kV. The pixel size was 0.7 to 0.9 mm, and the axial slice thickness was less than 1.25 mm, with 1 mm being the most common. Reconstruction was performed using a standard convolution kernel. Some patients underwent non-contrast CT examination while others underwent contrast-enhanced CT. All images were analyzed in both lung (width, 1,500 HU; level, −700 HU) and mediastinal (width =350 HU; level =40 HU) settings. During patient selection and radiologic feature evaluation, two specialists who were blinded to the pathological results of lesions evaluated all CT scans and resolved discrepancies by consensus.

Nodule segmentation and feature extraction

The target nodules were manually segmented in 3D using the ITK-SNAP software by one clinician in pulmonary and critical care medicine, with four years of experience in pulmonary nodule evaluation (14). The author was also blinded to the pathological results of all the lesions. Radiomics feature extraction was performed using the open-source platform Pyradiomics (version 2.0.0), which enables quantitative features to be extracted from HRCT images (12). Extracted from Pyradiomics were 14 shape features, 18 first-order features, 22 gray-level co-occurrence matrix (GLCM) features, 16 gray-level size zone matrix (GLSZM) features, 16 gray-level run length matrix (GLRLM) features, and 14 gray-level dependence matrix (GLDM) features. Except for shape features, other features were also calculated from the filtered images (LoG with five sigma levels and wavelet with eight derived images). In total, 1,218 radiomics features [14 shape features + 86 other features × (1 original image + 13 filtered images)] were generated for each nodule. A detailed list of the extracted features is provided in Table S1.

Model establishment

All risk models were established in two steps: feature selection and model development. The least absolute shrinkage and selection operator (LASSO) method was applied to select the most significant predictors from clinical and radiomics features simultaneously. Then, four types of models were tried for each nodule group based on selected features, including random forest, XGBOOST, SVM, and logistic model. We normalized the data using the Z-score to allow all the coefficients to be based on the same scale and performed five-fold cross-validation during both steps.

Performance of models

The model performance was evaluated using metrics such as the area under the ROC curve (AUC), accuracy, F1 score, recall, precision, sensitivity, and specificity. The performance of the models was compared with the predictive ability of the Mayo model (5). All cases in each group were utilized to validate the Mayo model, and the corresponding AUC, accuracy, sensitivity, and specificity were calculated.

Statistical analysis

The numerical data were presented as mean ± standard deviation and compared using Student’s t-test, while categorical data were described as percentages and compared using the chi-square test. All statistical tests were two-sided, and differences were considered statistically significant at P<0.05. All statistical analyses were performed with R version 3.6, and all models were established using Python 3.7.

Ethical statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the institutional review board of the West China Hospital of Sichuan University (No.59). Informed consent was waived as this was a retrospective study and the privacy and identity information of the subjects were guaranteed.

Results

Patient enrollment and pathology of nodules

presents the flowchart of the study process. A total of 1,171 patients who met the inclusion criteria were enrolled, including 548 patients with solid nodules (malignant, 242; benign, 306) and 623 patients with SSNs (malignant, 519; benign, 104). All nodules were confirmed pathologically. The benign group included inflammatory nodules (62.20%), benign tumors (20.00%), and other lesions (17.80%). Malignant nodules were mainly adenocarcinomas (97.37%), while 1.97% were squamous carcinomas and 0.66% were other types (Table S2).

Figure 1

The workflow of the current study. PN, pulmonary nodule.

Patient clinical characteristics

The clinical characteristics of the patients are summarized in . In patients with solid nodules, the benign and malignant groups exhibited some differences in age, history of malignancy, diameter, shape, spiculation, lobulation, white blood cell count, NLR, APTT, and CEA (P<0.05). As for SSNs, there was no difference in the proportion of history of malignancy, white blood cell count, NLR, and CEA between the benign and malignant groups. However, significant differences in age, diameter, location, shape, spiculation, lobulation, APTT, CYFRA21-1, and NSE were observed between the malignant and benign groups (P<0.05). Details of missing data for CEA, CYFRA21-1, and NSE are described in Table S3.

Table 1

Clinical characteristics of patients in the solid and subsolid group

Characteristics	Solid group				Subsolid group
	Benign (n=306)	Malignant (n=242)	P value		Benign (n=104)	Malignant (n=519)	P value
Age, years	52±12	59±10	0.000		51±10	57±11	0.000
Sex, female	47.7	53.3	0.193		67.3	71.1	0.439
Smoking	35.0	38.8	0.350		21.2	16.2	0.218
History of malignancy	5.2	10.7	0.016		5.8	8.7	0.325
Family history of LC	4.9	7.9	0.155		8.7	10.6	0.551
Diameter, mm	13±4	14±4	0.000		9±3	12±4	0.000
Location, upper lobe	45.4	50.0	0.287		54.8	66.5	0.023
Shape, irregular	55.6	82.6	0.000		53.8	68.0	0.005
Spiculation	32.7	58.7	0.000		12.5	33.7	0.000
Lobulation	38.6	52.1	0.002		16.3	35.5	0.000
Red blood cell, 1012/L	4.62±0.55	4.54±0.49	0.059		4.53±0.47	4.54±0.50	0.838
White blood cell, 109/L	5.75±1.60	6.12±2.01	0.015		5.87±1.65	5.79±1.75	0.702
Blood platelet, 109/L	176.68±56.73	177.11±56.96	0.930		172.81±53.63	177.12±57.96	0.484
NLR	2.06±1.24	2.34±1.70	0.023		2.27±1.49	2.17±1.43	0.518
PT, s	11.16±0.83	11.25±1.12	0.306		11.07±0.82	11.24±1.15	0.167
APTT, s	28.07±3.70	27.18±3.61	0.005		28.07±3.89	27.07±3.96	0.018
CEA, ng/mL	1.91±0.98	4.07±12.02	0.006		1.77±0.88	2.44±6.98	0.329
CYFRA21-1, ng/mL	1.96±0.73	2.04±0.87	0.242		1.85±0.60	2.09±1.05	0.001
NSE, ng/mL	12.78±4.46	13.26±4.24	0.200		12.71±4.06	14.57±5.14	0.001

Numeric variables were presented as mean ± standard deviation, category variables were presented in proportion. LC, lung cancer; NLR, neutrophil to lymphocyte ratio, PT, prothrombin time; APTT, activated partial thromboplastin time; CEA, carcinoembryonic antigen; CYFRA21-1, cytokeratin 19 fragment; NSE, neuron specific enolase.

Model establishment and selected radiomics features

We developed lung cancer risk models among four nodule groups: all nodules (n=1,171), nodules ≤10 mm (n=409), solid nodules (n=548), and SSNs (n=208). To avoid overfitting of models in the SSN group, only 104 malignant SSNs were included, which were 1:1 matched with benign SSNs.

Table S4 summarizes the selected features from LASSO analysis in detail for all nodules (α=0.00955, n=74 features), nodules ≤10 mm (α=0.0152, n=67 features), solid nodules (α=0.0152, n=51 features), and SSNs (α=0.12328, n=3 features). describes the top ten features sorted by the absolute value of coefficients and the rad-score distribution based on selected features for all nodules () and nodules ≤10 mm (). In addition, the top ten features for solid nodules, all three features for SSNs, and the corresponding rad-score distribution for solid nodules () and SSNs () are shown in . There was a higher proportion of radiomics features selected for nodules ≤10 mm (83.6% of 67 features and 80% of the top ten features).

Figure 2

Feature selection based on LASSO analysis for all nodules (A,B,C) and the nodules less than 10 mm (D,E,F). A,D, top ten predictors sorted by absolute value of coefficients; B,C,E,F, rad-score boxplots of all selected features. LRLGLE, long run low gray level emphasis; SZNN, size zone nonuniformity normalized.

Figure 3

Feature selection based on LASSO analysis for solid nodules (A,B,C) and subsolid nodules (D,E,F). A, top ten predictors for solid nodules sorted by absolute value of coefficients; D, selected three predictors for subsolid nodules; B,C,E,F, rad-score boxplots of all selected features. SRLGLE, short run low gray level emphasis.

Performance of risk models

We performed five-fold cross-validation to evaluate the predictive performance of the lung cancer risk models. According to the accuracy of the training dataset, the top three models for each nodule group were recorded. For all nodules (410 benign vs. 761 malignant; 1:1.86), nodules ≤10 mm (185 benign vs. 224 malignant; 1:1.21), and solid nodules (306 benign vs. 242 malignant; 1.26:1), the top three models were XGBOOST, random forest, and SVM, respectively. However, it was the XGBOOST, random forest, and logistic model for SSNs (104 benign vs. 104 malignant; 1:1). The performance of the established models on the validation dataset is summarized in , while that of the training dataset is shown in Table S5.

Table 2

Performance of established models on validation dataset

Group	Methods	Accuracy	F1 Score	Recall	Precision	Sensitivity	Specificity	AUC
All nodules	XGBOOST	0.86	0.89	0.93	0.87	0.93	0.73	0.91
	Random Forest	0.83	0.88	0.96	0.81	0.96	0.59	0.89
	SVM	0.77	0.83	0.88	0.79	0.88	0.55	0.81
Nodules≤10 mm	XGBOOST	0.82	0.84	0.86	0.82	0.86	0.76	0.90
	Random Forest	0.81	0.84	0.88	0.81	0.88	0.74	0.90
	SVM	0.74	0.78	0.83	0.74	0.83	0.64	0.80
Solid nodules	XGBOOST	0.80	0.78	0.78	0.77	0.78	0.82	0.89
	Random Forest	0.79	0.76	0.73	0.79	0.73	0.85	0.88
	SVM	0.71	0.66	0.65	0.68	0.65	0.75	0.77
Subsolid nodules	XGBOOST	0.62	0.63	0.64	0.62	0.64	0.60	0.65
	Random Forest	0.70	0.71	0.73	0.70	0.73	0.67	0.73
	Logistic	0.73	0.71	0.69	0.77	0.69	0.76	0.76

AUC, area under the ROC curve.

In addition, the ROC curves of the top three models in the training and validation datasets, as well as the performance of the best model in each fold on the validation dataset are shown in (all nodules, ; nodules ≤10 mm, ) and (solid nodules, ; SSNs, ). The performance of risk models for SSNs was quite weak (AUC in validation dataset, 0.65–0.76) and the XGBOOST model was even unstable (AUC, 1.00 in training vs. 0.65 in validation); therefore, the ROC curves of the random forest model are shown in .

Figure 4

Receiver operating characteristic curves of established models for all nodules (A,B,C) and the nodules less than 10 mm (D,E,F). A,B,D,E, performance of top three models; C,F, performance of the XGBOOST model in each fold.

Figure 5

Receiver operating characteristic curves of established models for solid nodules (A,B,C) and subsolid nodules (D,E,F). A,B,D,E, performance of top three models; C, performance of the XGBOOST model in each fold for solid nodules; F, performance of the Random Forest model in each fold for subsolid nodules. 

Comparison of established clinical-radiomics models with Mayo model

The Mayo model was validated in all cases in each nodule group and was compared with the best corresponding clinical-radiomics model. summarizes the detailed predictive results. We found that the Mayo model, which was based on only six clinical characteristics, showed comparable discriminating ability with our clinical-radiomics model for SSNs (accuracy, 0.73 vs. 0.70; AUC, 0.75 vs. 0.73). However, it was inferior to the clinical-radiomics models for the other three nodule groups (accuracy, 0.55–0.67 vs. 0.80–0.86; AUC, 0.59–0.70 vs. 0.89–0.91).

Table 3

The comparison between established clinical-radiomics model and Mayo model

Group	Model	Accuracy	Sensitivity	Specificity	AUC
All nodules (n=1,171)	XGBOOST	0.86	0.93	0.73	0.91
	Mayo	0.62	0.65	0.58	0.64
Nodules ≤10 mm (n=409)	XGBOOST	0.82	0.86	0.76	0.90
	Mayo	0.55	0.47	0.68	0.59
Solid nodules (n=548)	XGBOOST	0.80	0.78	0.82	0.89
	Mayo	0.67	0.60	0.74	0.70
Subsolid nodules (n=208)	Random Forest	0.70	0.73	0.67	0.73
	Mayo	0.73	0.72	0.76	0.75

The performance of established clinical-radiomics models were of validation dataset. AUC, area under the ROC curve.

Discussion

Accurate evaluation of the malignancy of pulmonary nodules plays an important role in lung cancer management. Moreover, solid nodules and SSNs exhibit different clinical courses, and guidelines for the management of pulmonary nodules have provided separate recommendations for the two types of nodules (9). Therefore, the current study based on clinical and radiomics features has established four models for all nodules (5–20 mm), nodules ≤10 mm, solid nodules, and SSNs.

We previously established lung cancer risk models solely based on the clinical features of solid nodules and SSNs of different sizes (15). The clinical models showed an AUC of 0.70 and 0.71 for SSNs and solid nodules <15 mm, 0.72 and 0.81 for SSNs and solid nodules between 15 and 30 mm, respectively (15). It was obvious that the current models combining clinical and radiomics features exhibited greater discrimination ability, especially for nodules ≤10 mm (highest AUC, 0.90) and solid nodules (highest AUC, 0.89), while there was only a slight improvement for SSNs (highest AUC, 0.76). Similarly, when compared to the Mayo model (six predictors: age, smoking, history of cancer, diameter, spiculation, and upper lobe location), the established models demonstrated absolute superiority for all nodules, nodules ≤10 mm, and solid nodules, whereas the diagnostic advantages for SSNs were not obvious. The possible reason was that there were only a small number of benign SSNs available, and for data balance, the same number of malignant cases were matched. Therefore, due to the small sample size (208), trained SSN models can be unstable, of which only three risk predictors were selected.

An increasing number of SSNs have been encountered in routine clinical practice. Most early stage lung adenocarcinomas can manifest as SSNs with different degrees of invasion (16). Consequently, previous radiomics studies for SSNs focused more on identifying the invasiveness of lung adenocarcinomas (16-18). However, Gong et al. studied 182 histopathology-confirmed SSNs using radiomics analysis for nodule diagnosis [59 benign nodules, 50 adenocarcinoma in situ (AIS), 32 minimally invasive adenocarcinoma (MIA), and 41 invasive adenocarcinoma] (19). Their results showed an average AUC of 0.75 in distinguishing benign and malignant SSNs, which was consistent with the current study (AUC, 0.65–0.76). Moreover, their models also demonstrated poor performance in benign and AIS nodules (AUC, 0.55), and benign and MIA nodules (0.77), respectively (19). Nevertheless, a high AUC of 0.93, was observed for benign nodules and invasive adenocarcinomas (19). Hence, radiomics features have shown potential in predicting the malignancy of SSNs, but sufficient data is a priority to train a good model.

We noticed that the clinical characteristics could be as important as radiomics features in lung cancer risk prediction for solid nodules. Sixty percent of the top ten selected features were clinical variables, which have been identified in previous studies (7,20-22). The clinical variables were age, spiculation, sex, shape, smoking, and history of malignancy. When predicting malignancy of solid nodules, the clinical-based models exhibited an AUC of 0.81 to 0.89 (7,20-22), and one study pointed out that a VDT of 25–400 days was highly suggestive of malignancy (7). On the other hand, quantitative radiomics models have also demonstrated potential for diagnosing solid nodules, especially radiomics models created from gross tumor volume instead of peritumoral volumes (23). Therefore, the combination of clinical and quantitative radiomics features improved the performance of the clinical models developed in our previous study (15).

The Brock model, which is based on two screening cohorts (PanCan and BCCA; rates of cancer, 5.5% and 3.7%, respectively) demonstrated good discrimination and calibration for nodules ≤10 mm, with an AUC of 0.89 to 0.94 (8). Predictors of cancer in the model were all clinical variables, including older age, female sex, family history of lung cancer, emphysema, larger nodule size, upper lobe location, part-solid nodule type, lower nodule count, and spiculation (8). Regarding incidental pulmonary nodules ≤10 mm, Xu et al. predicted the malignancy of 127 sub-centimeter nodules using radiomics features and achieved an AUC of 0.84 (24). However, in the current study of incidental pulmonary nodules, 67 clinical and radiomics features were selected for nodules ≤10 mm, and the risk models showed the highest AUC of 0.90. It is worth noting that radiomics features played a more important role in lung cancer risk prediction for incidental nodules ≤10 mm, with a high proportion of 80% among the top ten features, which were completely different from the other three nodule groups. Hence, radiomics features are meaningful in malignancy prediction for nodules ≤10 mm in routine clinical practice.

This study had some general and study-specific limitations. First, the study was carried out in a single medical center, in which bias could exist. In addition, this was a retrospective case-control study, which resulted in different scanner machines and non-uniform imaging protocols across patients. The difference should have a relevant impact on radiomics feature stability and thus prevented the establishment of robust risk models. Second, the growth rate of pulmonary nodules is one of the key characteristics associated with lung cancer probability, but we failed to apply the relevant parameters to build risk models in the current study. Usually, at least two thoracic CT scans prior to treatment are needed to calculate the growth rate of a nodule, but most of the patients we studied did not meet the criteria. Third, only a small number of SSNs were available for modeling; therefore, the performance of the established SSN models was unstable. Increasing the sample size, especially for benign SSNs, is warranted in future research. Last but not the least, we performed five-fold cross validation to evaluate the performance of established risk models, whereas the external validation is necessary.

Conclusions

In conclusion, based on both clinical and radiomics features, the current study established risk models to predict the malignancy of 5–20 mm pulmonary nodules. The models were developed for four nodule groups, including all nodules, nodules ≤10 mm, solid nodules, and SSNs. All models demonstrated excellent discrimination ability except for those of SSNs. Further studies are warranted to develop robust SSN models.

The article’s supplementary files as