The Influence of Different Ultrasonic Machines on Radiomics Models in Prediction Lymph Node Metastasis for Patients with Cervical Cancer.

Objective To investigate the effects of different ultrasonic machines on the performance of radiomics models using ultrasound (US) images in the prediction of lymph node metastasis (LNM) for patients with cervical cancer (CC) preoperatively. Methods A total of 536 CC patients with confirmed histological characteristics and lymph node status after radical hysterectomy and pelvic lymphadenectomy were enrolled. Radiomics features were extracted and selected with US images acquired with ATL HDI5000, Voluson E8, MyLab classC, ACUSON S2000, and HI VISION Preirus to build radiomics models for LNM prediction using support vector machine (SVM) and logistic regression, respectively. Results There were 148 patients (training vs validation: 102:46) scanned in machine HDI5000, 75 patients (53:22) in machine Voluson E8, 100 patients (69:31) in machine MyLab classC, 110 patients (76:34) in machine ACUSON S2000, and 103 patients (73:30) in machine HI VISION Preirus, respectively. Few radiomics features were reproducible among different machines. The area under the curves (AUCs) ranged from 0.75 to 0.86, 0.73 to 0.86 in the training cohorts, and from 0.71 to 0.82, 0.70 to 0.80 in the validation cohorts for SVM and logistic regression models, respectively. The highest difference in AUCs for different machines reaches 17.8% and 15.5% in the training and validation cohorts, respectively. Conclusions The performance of radiomics model is dependent on the type of scanner. The problem of scanner dependency on radiomics features should be considered, and their effects should be minimized in future studies for US images.

Introduction

Over the past decade, with the development of imaging technologies and efficient learning strategies, radiomics has emerged and is widely accepted in precision medicine by extracting mineable high-dimensional features from multimodal medical images, for example, computed tomography (CT), positron emission tomography (PET), MRI, or ultrasound (US).[1,2] Studies demonstrated that radiomics features could characterize the internal structure of tissue—such as tumor heterogeneity to enhance the diagnostic, prognostic, and therapeutic outcomes for a broad range of diseases.[3,4] Studies also indicated that with advanced analysis of images using artificial intelligence, radiomics could correlate imaging phenotypes with genomic and proteomic signatures,[5,6] and subsequently complement the information of tissue sampling and circulating biomarkers to reform clinical decision-making.[7,8]

However, one major shortcoming of radiomics is that features could be influenced by many factors, such as the type of scanner, imaging settings, reconstruction parameters, delineation of the tumor, etc,[9-11] which results in a low reproducibility of radiomics features and makes it difficult to compare and interpret radiomics studies for clinical application.[9,10] Standardization in imaging acquisition and analysis had been proposed to improve feature stability, but these standardization approaches are not always feasible as most radiomics studies are retrospective in nature. Awareness of the shortcomings of the radiomics features and studies is of critical significance for the feasibility and applicability of these radiomics studies.

Thanks to its advantages of nonionizing radiation, portability, accessibility, and cost-effectiveness, US has been one of the most used imaging modalities for screening and diagnosis in obstetrics and gynecology. Radiomics studies indicated that US radiomics features were highly associated with breast biologic characteristics, gestational age, neonatal respiratory morbidity, etc. Studies also demonstrated that US-based radiomics was able to predict lymph node metastasis (LNM) for patients with papillary thyroid carcinoma and cervical cancer (CC).[16,17] However, US images also suffer from some limitations, such as low imaging quality caused by noise and artifacts, high inter- and intra-observe variability across different scanners and institutes, and highly operator or diagnostician experience dependent, which will hinder the clinical application of US-based radiomics models. The purpose of this study is to investigate the influence of different US scanners on the reproducibility of radiomics features and the accuracy of LNM prediction for patients with CC.

Materials and Methods

Patients

By searching electronic medical records, a total of 1723 CC patients underwent radical hysterectomy and pelvic lymphadenectomy in our hospital between January 2014 and November 2018 were retrospectively reviewed. Inclusion criteria: (a) patients should have standard ultrasonography within 2 weeks before hysterectomy and (b) patients had confirmed histological characteristics and lymph node status after the operation. Exclusion criteria: (a) patients with incomplete or incorrect clinical data, (b) patients who had other malignant tumors, (c) patients who were treated by chemotherapy or radiotherapy before the operation, and (d) patients with unclear or missing images. A total of 536 patients were enrolled, including 148 with HDI5000, 75 cases of Voluson E8, 100 cases of MyLab classC, 110 cases of ACUSON S2000, and 103 cases of HI VISION Preirus. This retrospective study was conducted following the Declaration of Helsinki and approved by the Ethics Committee in Clinical Research (ECCR) of Wenzhou Medical University First Affiliated Hospital (ECCR No. 2019059). The written informed consent was waived by the ECCR due to the retrospective nature of this study with confirmation of following the Declaration of Helsinki with patient data confidentiality. Besides, all of the patient details in this study have been anonymized. The reporting of this study conforms to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline.

US Imaging and Machines

Transvaginal US images were acquired when patients lie in a lithotomy position with an empty bladder. In this study, the following 5-color Doppler ultrasonic machines were used to acquire the US images: ATL HDI 5000（Philips）using the transducer C8-4v at 4-8 MHz; Voluson-E8 (GE Healthcare) using the transducer RIC5-9-D at 5-9 MHz; Mylab classC (Esaote) using the transducer EC1123 at 3-9 MHz; ACUSON S2000 (Siemens) using the transducer MC9-4 at 1.5-6.0 MHz and HI VISION Preirus (Hitachi Ltd) using the transducer EUP-U531 at 4-8 MHz. All images were stored in PNG format and archived in the hospital's DICOM system.

Segmentation and Feature Extraction

Usually, each patient will have 10 to 20 standard US images collected. Regions of interest were delineated manually by a radiologist with 10 years of experience in gynecological US diagnosis using the LIFEx package (http://www.lifexsoft.org), and confirmed by another senior radiologist with 15 years of experience in gynecological US diagnosis. A typical transvaginal US image and contoured target volume are shown in Figure 1. Image preprocessing was performed before feature extraction. A total of 449 radiomics features were extracted from the contoured volumes after intensity normalization using Pyradiomics package (PyRadiomics, https://www.python.org), which includes shape features, first-order histogram statistics, gray-level co-occurrence matrix (GLCM), neighborhood gray-level different matrix, gray-level run length matrix (GLRLM), gray level size zone matrix (GLSZM), and wavelet filtered first-order, GLCM, gray-level different matrix (GLDM), GLRLM, and GLSZM, respectively.[21,22]

Figure 1.

A typical transvaginal ultrasound image with contoured target volume.

Feature Selection and Model Building

Optimal features correlated with LNM were further selected with a 2-step method in the training cohort. First, the Mann-Whitney U test was used to select the radiomics features which had a potential correlation with LNM with a significant p-value <.05. Second, the least absolute shrinkage and selection operator was applied to select the optimal features with 5-fold cross-validation to adjust the elastic network parameters to select the best combination of feature parameters to avoid overfitting. The dependent variables in the study are dichotomous and we use the parameters family="binomial”, α=1, nlambda=100, type.measure=“auc.”