Deep multi-modality collaborative learning for distant metastases predication in PET-CT soft-tissue sarcoma studies.

Soft-tissue Sarcomas (STS) are a heterogeneous group of malignant neoplasms with a relatively high mortality rate from distant metastases. Early prediction or quantitative evaluation of distant metastases risk for patients with STS is an important step which can provide better-personalized treatments and thereby improve survival rates. Positron emission tomography-computed tomography (PET-CT) image is regarded as the imaging modality of choice for the evaluation, staging and assessment of STS. Radiomics, which refers to the extraction and analysis of the quantitative of high-dimensional mineable data from medical images, is foreseen as an important prognostic tool for cancer risk assessment. However, conventional radiomics methods that depend heavily on hand-crafted features (e.g. shape and texture) and prior knowledge (e.g. tuning of many parameters) therefore cannot fully represent the semantic information of the image. In addition, convolutional neural networks (CNN) based radiomics methods present capabilities to improve, but currently, they are mainly designed for single modality e.g., CT or a particular body region e.g., lung structure. In this work, we propose a deep multi-modality collaborative learning to iteratively derive optimal ensembled deep and conventional features from PET-CT images. In addition, we introduce an end-to-end volumetric deep learning architecture to learn complementary PET-CT features optimised for image radiomics. Our experimental results using public PET-CT dataset of STS patients demonstrate that our method has better performance when compared with the state-of-the-art methods.