Artificial intelligence in musculoskeletal oncological radiology.

BACKGROUND: Due to the rarity of primary bone tumors, precise radiologic diagnosis often requires an experienced musculoskeletal radiologist. In order to make the diagnosis more precise and to prevent the overlooking of potentially dangerous conditions, artificial intelligence has been continuously incorporated into medical practice in recent decades. This paper reviews some of the most promising systems developed, including those for diagnosis of primary and secondary bone tumors, breast, lung and colon neoplasms.
CONCLUSIONS: Although there is still a shortage of long-term studies confirming its benefits, there is probably a considerable potential for further development of computer-based expert systems aiming at a more efficient diagnosis of bone and soft tissue tumors.

Introduction

Primary bone and soft tissue neoplasms present a minority among neoplastic lesions. Due to their rarity, precise radiologic diagnosis often requires an experienced radiologist with special interests in musculoskeletal oncology. To surmount the challenge of making precise diagnoses, and more importantly, to prevent overlooking potentially fatal conditions, attempts to incorporate artificial intelligence and its related techniques into medical practice have occurred in the last decades (Figure 1).

Figure 1

Schematic illustration of the hierarchy of artificial intelligence and its machine learning and deep learning subfields.

Being first introduced by McCarthy in the 1950s, artificial intelligence (AI) is a general term that describes computer machines that imitate human intelligence.1 Machine learning, a subset of AI, uses computational algorithms, which learn with experience and therefore improve the performance of tasks.2 The rapid progress of computational power and big data availability allowed the emergence of an even more specialized subfield of machine learning, called deep learning. It is a promising method capable of processing raw data to perform detection or classification tasks.3 Deep learning algorithms, implemented as artificial neural networks, mimic biological nervous systems.4 The network is organized in layers composed of interconnected nodes imitating architecture in a biologic brain.2,5 Nodes are weighted individually for the purpose of increasing data extraction. In order to classify data, weights are automatically and dynamically optimized during the training phase.5,6 Regarding the layers, three different kinds are present in each neural network. It begins with an input layer, which receives input data, followed by numerous hidden layers extracting the pattern within the data. It terminates with the output layer, which produces results or output data (Figure 2).5

Figure 2

Schematic presentation of a neural network. Regions of interests (ROI) are defined, either by user or by an automated computer process. These present the input cells (in pink) a neural network. For each ROI the neural network extracts and compute features within the hidden layers (in grey) by using pre-trained data sets. Finally, the output cell offers the final results in different possible forms (yes/no, final diagnosis, probability of malignancy etc.).

Among the different types of artificial neural networks, convolutional neural networks, in particular, gained attention in radiology due to their high performance in recognizing images.7 By calculating the intensity of each pixel or voxel, together with evaluating complex patterns in each image, they provide reliable quantitative image interpretations, eventually surpassing human performance.8

However, to build an intelligent machine a training phase is required, which requires sufficient computational power and large datasets. The latter is obtained from radiological images, which is in the domain of radiomics. Radiomics is a process of quantitative extraction of a high number of semantic and agnostic features from diagnostic images.9 Its approaches, like feature extraction and feature engineering techniques, are essential in the formation of AI applications.10

Artificial intelligence in cancer imaging (oncologic radiology)

Until recently, radiologists’ decisions were based predominantly on his or her experience of recognizing patterns and appreciating various features of each tumor including size, location, intensity, and surface characteristics, all combined with patients’ demographic data. However, after many consecutive image interpretations, they were consciously or subconsciously faced with fatigue11, which could lead to errors and potentially jeopardize patients’ health and own credibility.12

AI’s potentials are developing exponentially. Instead of qualitative and subjective image interpretations, it allows quantifiable and objective data extraction with the ability to reproduce the same results.13 Furthermore, by quantifying information otherwise not detectable to humans, AI may complement clinical decision-making.5,13

Computer-aided detection (CADe) and computer-aided diagnosis (CADx) algorithms have been used for the last two decades14 predominantly in mammography15, detection of lung16, and colon17 malignancies. In contrast to CAD algorithms, which only highlight the features they have been exactly trained for, actual AI systems continue to learn and improve in time. By focusing on the specific diagnosis, systems learn to discover new typical patterns that have not been linked with the disease before.18 For example, Beck et al. developed a machine learning-based computer program named “Computational Pathologist (C-Path)” to automatically analyze breast cancer and predict its prognosis.19 Importantly, regardless of all well-known histological characteristics that were implemented into the program, C-Path recognized surrounding stroma as another important prognostic factor.19 Convolutional neural networks have also given optimistic results in automated detection.6 It has been used for automated detection of liver tumors on CT scans with high detection accuracy and precision of 93% and 67%, respectively.20 Similarly, a deep learning-based CADe for detection of brain metastasis on magnetic resonance imaging (MRI) has been developed and achieved a sensitivity of 96% and reasonable specificity.21

In general, characterization is composed of segmentation, diagnosis, and staging of tumors.13 Image segmentation is the process used in cancer imaging to outline pathological area and distinguish it from non-pathological adjacent tissue. It can range from planar measurements to advanced 3-dimensional assessment of tumor volume.13 Tumors have traditionally been manually labelled by radiologists, which is indeed time-consuming, as well as a subject of interobserver variability.13,22 Thus, implementation of AI into automated image segmentation could potentially take over, increase the quality and reproducibility of measurements, and also save time.5,13 For example, machine learning has been used for breast density segmentation on mammography, which turns out to be as accurate as manual ones.23 Ye et al. successfully proposed and verified a fully automatic nasopharyngeal carcinoma segmentation method based on dual-sequence MRI and convolutional neural network.24 The mean dice similarity coefficient (DSC) of the models with only T1 sequence, only T2 sequence, and dual sequence were 0.620 ± 0.0642, 0.642 ± 0.118, and 0.721 ± 0.036, respectively. The combination of different features acquired from T1 and T2 sequences significantly improved the segmentation accuracy.24

Ability to quantitatively extract tumor features has great potential in the process of making diagnosis. With machine learning, Liu et al. quantitatively represented radiological traits characteristics of lung nodules and showed improved accuracy of cancer diagnosis in pulmonary nodules.25 Convolutional neural network has also shown to be an effective and objective method that provides an accurate diagnosis of pancreatic cancer.26

Another important aspect of tumor characterization includes staging. Mainly, TNM classification is used to assess the extent of primary tumor, lymph nodes, and metastases and therefore classify the lesion in a specific stage. However, attempts to extend the TNM cancer staging system have been made. For instance, CAD has shown to be a promising method of evaluation of tumor extent and multifocality in invasive breast cancer patients and therefore expanding the staging algorithms.27

In tumor response monitoring, many AI approaches have shown some potential. For example, AI and machine learning have been successfully implemented into the pre-procedural prediction of trans-arterial chemoembolization treatment outcomes in patients with hepatocellular carcinoma using clinical and imaging features.28 Ha et al. demonstrated promising results in the utilization of convolutional neural network to predict neoadjuvant chemotherapy response prior to the first cycle of therapy in breast carcinoma using baseline MRI tumor datasets.29 Positive response to chemotherapy led to decreased tumor metabolism, opening a potential opportunity to detect lower accumulation rates of a radiotracer.30 Differentiating between responders and non-responders based on low-dose 18F-FDG PET/MRI scans might, therefore, be another opportunity of the implementation of convolutional neural networks.6

Complementary to radiologic diagnosis, additional advanced methods have been proposed, promising even further advances in cancer management. Liquid biopsy based on circular tumor DNA (ctDNA) analysis may importantly improve early tumor detection, diagnosis, monitoring therapy, and progression in time.31 Contrary to standard tissue biopsies, liquid biopsies taken from blood may provide us with detailed biochemical characteristics of the neoplastic lesion and detect potential metastasis.32 What is more, a combination of liquid biopsies and radiomics, supervised by deep learning may significantly improve cancer management in the future.

Artificial intelligence in skeletal tumors

The first attempts to introduce computational power into diagnostic procedures of primary bone tumors date back to the 1960s.33 Based on Bayes’ formula, a computer program accurately predicted a bone tumor diagnosis in 77.9% of cases. Later in 1980, the same author set a milestone by publishing an article about computed-based radiographic grading of bone tumor destruction.34 This was a cornerstone for further research and implementation of neural networks into the diagnosis of focal bone lesions.35,36

A scarce number of articles regarding AI and primary bone tumors have been published so far, while considerably more has been done on the detection and segmentation of bone metastasis. For example, Burns et al. successfully identified and segmented sclerotic lesions in the thoracolumbar spine using CADe techniques. The sensitivity for lesions detection was 79%.37 What is more, Wang et al. developed a Siamese convolutional neural network to research the potential of automated spinal metastasis detection in MRI. The proposed approach accurately detected all spinal metastatic lesions with a false-positive rate of 0.4 per case.38 Another research proposed a machine learning-based whole-body automatic disease classification tool to distinguish benign processes and malignant bone lesions in 18F-NaF PET/CT images.39

Healthy and tumorous bone differs in numerous characteristics. Unlike healthy osseous tissue, which consists of cortical and trabecular part, primary bone malignancies may penetrate cortex and spread into adjacent soft tissue, as well as cause swelling around the bone or even weaken the bone architecture and lead to pathological fracture.40 Radiologically, they differ in absorption rate, which can be quantitatively evaluated. For example, CADx has been used to detect and classify primary bone tumors into benign and malignant lesions using x-ray images. In their study, Ping et al. an overall greater intensity of pixels for malignant bone tumors compared to benign bone tumors.41 Another study by Bandyopadhyay et al. proposed a CADx method to automatically analyze bone x-ray images. By integrating several classifiers, the method achieved accurate decisions regarding a bone-destruction pattern, stage, and grade of cancer in 85% of cases.42

When describing sarcomas, diagnosed on MRI, features like tumor size, shape, and enhancement pattern are estimated and taken into consideration along with patient’s demographic data.2 Machine learning and artificial neural network excel in quantifying and extracting supplementary features, which can correlate with clinical characteristics, diagnosis, and outcomes. Most of these are out of human visual perception and include inter-voxel relationships, image intensity analysis, and filtered images analysis.43 Deep learning-based algorithm has also been developed to predict survival rates in patients with synovial sarcoma.44 Its prediction was more precise compared to the Cox proportional hazard model, which is a commonly used regression model in medical research.

In primary bone tumors, bone tumor matrix, its density, and zone of transition represent suitable characteristics than may be classified through deep learning techniques. In fact, recurrent convolutional neural network outshined experienced musculoskeletal radiologists in bone tumor matrix classification with 86% vs. 72%, respectively.45 Li et al. proposed a super label guided convolutional neural network to classify CT images of bone tumors.46 In comparison, results exceeded the classic convolutional neural network. However, the classification included only nine types of the most common skeletal tumors.

Limitations and future directions

There are indeed some limitations of AI. First, it could potentially still be a subject of interobserver variability, due to different algorithms used in a neural networks of different AI systems or unequal learning stages in which the systems process a specific task. Standardization is mandatory to establish a large database. Data also needs to be accessible in order to integrate them into large sets. Prior to that quality check, labelling, classification, and segmentation need to be done manually by experts, making the process expensive and time-consuming.13 However, introducing deep learning-based techniques to the extensive quality ground-truth training datasets is essential for the development of accurate algorithms.47,48 Also, ethical dilemmas should be taken into consideration. When dealing with systems that operate with enormous amounts of data, patients’ privacy as well as human dignity may be jeopardized, unless meticulous safety mechanisms are implemented. There are also no long-term follow-up studies available thus far. On the contrary, the appreciating results of the already published studies and the presentation of a commercially available application is only a matter of time.

Undoubtedly, all kinds of artificial intelligence are persistently being integrated into the complex management of musculoskeletal tumors and tumors of other sites. Deep learning-based techniques are expected to minimalize false positive rates as well as assure accurate decisions and diagnoses.49 Further automatization of radiological tasks is expected to take place in the future. Among physicians, radiologists in particular are required to perform many time-consuming tasks, like image segmentation, delineation of regions of interest, and image annotation. AI techniques have an enormous potential to transform their workflow, which will allow them to focus on more meaningful tasks.11

On the other hand, “imaging is not an isolated measure of disease.”13 Neoplastic lesions are complex conditions, following DNA mutations that cause abnormal cellular proliferation.50 Despite many mutations being discovered and related to specific malignancies, intertumoural and intratumoural heterogeneity exist.51 Undoubtedly, molecular approaches, like genetic biomarkers and molecular imaging have already significantly contributed to a better understanding of cancer management. Finally, combining radiomics with other aspects of a broad family of “-omics”, including genomics, proteomics, and metabolomics, and therefore drastically expanding datasets available for advanced AI modalities, might move us closer to the precision medicine.52