Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine.

Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.

artificial intelligence

machine learning

deep learning

deep neural network

convolutional neural network

intra‐epithelial neoplasia

dual‐stained

area under the curve

circulating tumor DNA

cell‐free DNA

hematoxylin‐eosin

positron emission tomography‐CT

whole slide imaging

computed tomography

magnetic resonance imaging

three dimensional

nasopharyngeal carcinoma

gastrointestinal AI diagnostic system

microsatellite instability

tumor mutational burden

non‐small cell lung cancer

programmed death‐ligand 1

complete pathologic response

organs at risk

gross tumor volume

clinical target volume

randomized controlled trial

F‐box/WD repeat‐containing protein 7

discoidin domain receptor 1

distributed deep learning

Food and Drug Administration

neoadjuvant chemotherapy

Dice similarity coefficient

BACKGROUND

At a workshop in Dartmouth in the summer of 1956, McCarthy et al. [1] coined the term “artificial intelligence (AI)”, also known as “machine intelligence”. To put it simply, AI is defined as a programmed machine that can learn and recognize patterns and relationships between inputs and outputs and use this knowledge effectively for decision‐making on brand‐new input data [1, 2]. Machine learning (ML) and deep learning (DL) are the predominant methods used to actualize AI and are sometimes used synonymously. In the field of computer science, ML is a subfield of AI, and DL is a specific subset of ML that focuses on deep artificial neural networks (Figure 1). Over the past decade, following advances in big data, algorithms, computer power, and internet technology, DL has achieved unprecedented success in various tasks in various fields, including facial recognition, image classification, voice recognition, automatic translation, and healthcare [3]. Given the great number of patients diagnosed with cancers each year worldwide [4], there is an acute interest in the application of AI in oncology, and such interests include making accurate diagnosis of cancers using pathological slides and radiological images, predicting patient outcomes, and optimizing treatment decisions. AI therefore has the potential to solve the problem of unbalanced distribution of medical resources and improve cancer care.

FIGURE 1

The relationship between artificial intelligence, machine learning, and deep learning and commonly used algorithms as examples. CNN, convolutional neural network

Inspired by brain neural architecture, DL uses deep neural networks (DNNs) to develop sophisticated models with multiple hidden layers to analyze various types of data and develop prediction outputs (Figure 1) [5]. Unlike conventional ML techniques, which require careful engineering to design a feature extractor that transforms raw data (such as the pixel values of an image) into relevant discriminatory features before data input, DL algorithms feed the machine with raw data with which it can automatically learn the optimal deep features that best fit the task through a training process [6, 7]. This ability likely explains the fact that DL algorithms have been consistently improved in many common AI tasks, such as image recognition, pattern recognition, speech recognition, and natural language processing. Consequently, a majority of AI research within the oncology field involves the utilization of DL.

Among DNN models, convolutional neural networks (CNNs) are the most popular DL architectures. They have been used for cancer lesion detection, recognition, segmentation and the classification of medical images [8, 9, 10]. The architecture of a typical CNN (Figure 1) is structured by stacking three main layers: convolutional layers, pooling layers, and fully‐connected layers. In doing this, CNNs transform the original images layer by layer from pixel values to the final prediction scores. The convolutional layers involve combining input data (feature map) with convolutional kernels (filters) to form a transformed feature map. The filters in the convolutional layers are automatically adjusted based on learned parameters to extract the most useful features for a specific task. Yet, there is a drawback; it is difficult to tell what features are learned by the CNNs, which is known as the “black box”.

Over the past five years, large amounts of researches have applied DL to cancer diagnosis, precision medicine, radiotherapy, and cancer research (Figure 2). Moreover, the American Food and Drug Administration (FDA) have approved a number of AI algorithms related to oncology (Table 1) and published a fast‐track approval plan for AI medical algorithms in 2018. Here, we provided an overview of the recent and enormous progresses in the application of AI in oncology in this review (Figure 3). We also highlight the limitations, challenges, and future implications of AI‐powered cancer care.

FIGURE 2

Publication statistics of deep learning by cancer area over the past five years, searched on PubMed. A. Publication statistics of deep learning by cancer diagnosis, precision medicine, radiotherapy, and cancer research. B. Publication statistics of deep learning for different cancer sites

TABLE 1

Summary of FDA‐approved artificial intelligence devices in the field of oncology

AI algorithm	Company	FDA approval date	Indication
ClearRead CT	Riverain Technologies	09/09/2016	Detection of pulmonary nodules
QuantX	Quantitative Insights	07/19/2017	Diagnosing breast cancer
Arterys Oncology DL	Arterys	01/25/2018	Liver and lung cancer diagnosis
cmTriage	CureMetrix	03/08/2019	Detection of suspicious breast lesions
Koios DS Breast	Koios Medical	07/03/2019	Breast lesion malignancy evaluation
ProFound AI Software V2.1	iCAD	10/04/2019	Breast lesion malignancy evaluation
Transpara	ScreenPoint Medical BV	03/05/2020	Breast lesion malignancy evaluation
syngo.CT Lung CAD	Siemens Healthcare GmbH	03/09/2020	Detection of pulmonary nodules
MammoScreen	Therapixel	03/25/2020	Breast lesion malignancy evaluation
Rapid ASPECTS	iSchema View	06/26/2020	Detection of suspicious brain lesions
InferRead Lung CT.AI	InferRead Lung CT.AI	07/02/2020	Detection of pulmonary nodules
HealthMammo	Zebra Medical Vision	07/16/2020	Detection of suspicious breast lesions

Abbreviations: FDA, Food and Drug Administration; AI, artificial intelligence; CT, computed tomography; DL, deep learning; CAD, computer‐aided diagnosis.

FIGURE 3

Applications of AI in cancer diagnosis, treatment and research. OARs, organs at risk

CANCER SCREENING, DIAGNOSIS, CLASSIFICATION, AND GRADING

Cancer screening for early detection, accurate cancer diagnosis, classification and grading are the key determinants of treatment decisions and patient outcomes. Over the past few years, there is increasing interest in the applications of AI in these critical areas (Table 2), sometimes with performance equivalent to human experts and advantages in scalability and time‐saving. More importantly, AI has shown its potential in solving challenging problems that humans simply cannot do.

TABLE 2

Summary of key papers applying deep learning to cancer diagnosis and treatment

Application	Reference	Task	Performance
Screening
Pathology	[14]	Automation of dual stain cytology in cervical cancer screening	Sensitivity, 87%
Endoscopy	[15]	Automation of polyp detection	False positive rate, 7.5%
Radiology	[16]	Predicting invasiveness of pulmonary adenocarcinomas	AUC, 0.788
Radiology	[17]	Lung nodule classification: benign/malignant	Sensitivity, 98.45%
Radiology	[18]	Lung nodule classification: benign/malignant	Accuracy, 79.5%
Radiology	[19]	Lung nodule classification: benign/malignant	AUC, 0.944
Radiology	[20]	Breast lesion classification: benign/malignant	AUC, 0.909
Radiology	[21]	Breast lesion classification: benign/malignant	AUC, 0.860
Radiology	[22]	Breast lesion classification: benign/malignant	AUC, 0.870
Radiology	[23]	Breast lesion classification: benign/malignant	AUC, 0.860
Radiology	[24]	Breast lesion classification: benign/malignant	AUC, 0.890
Radiology	[25]	Breast cancer prediction	AUC, 0.8107
Diagnosis
Pathology	[30]	Invasive breast cancer detection	DSC, 75.86%
Pathology	[31]	Breast cancer nodal metastasis detection	AUC, 0.994
Pathology	[32]	Breast lesion classification: benign/malignant	Accuracy, 98.7%
Pathology	[33]	Detection of lymph node metastases in breast cancer	AUC, 0.994
Pathology	[35]	Diagnosis of gastric cancer	AUC, 0.990‐0.996
Pathology	[36]	Predicting origins for cancers of unknown primary	Accuracy, 80%
Pathology	[51]	Lung tumor classification: normal/ adenocarcinoma/squamous cell carcinoma	AUC, 0.97
Pathology	[52]	Automated Gleason grading of prostate adenocarcinoma	Cohen's quadratic kappa statistic, 0.75
Radiology	[37]	Brain tumor classification: normal/glioblastoma/sarcoma/metastatic bronchogenic carcinoma	AUC, 0.984
Radiology	[38]	Liver cancer detection	Accuracy, 99.38%
Radiology	[39]	Prostate lesion classification: benign/malignant	AUC, 0.84
Radiology	[40]	Detection of synchronous peritoneal carcinomatosis in colorectal cancer	Accuracy, 94.11%
Radiology	[41]	Detection of NPC using MRI	Accuracy, 97.77%
Radiology	[53]	Predicting grade of liver cancer	AUC, 0.83
Endoscopy	[42]	Gastric lesion classification: normal/malignant	Accuracy, 96.49%
Endoscopy	[43]	Upper gastrointestinal cancer detection	Accuracy, 99.7%
Endoscopy	[44]	Polyps identification	Accuracy, 96%
Endoscopy	[50]	Polyps identification	AUC, 0.984
Endoscopy	[45]	Invasive colorectal cancer diagnosis	Accuracy, 94.1%
Endoscopy	[46]	Diminutive colorectal polyps classification: hyperplastic/neoplastic	Accuracy, 90.1%
Endoscopy	[47]	cT1b colorectal cancer diagnosis	AUC, 0.871
Endoscopy	[49]	Nasopharyngeal lesion classification: benign/malignant	Accuracy, 88%
Prediction of mutation
Pathology	[51]	Predicting genetic mutations of lung cancer: STK11, EGFR, FAT1, SETBP1, KRAS, and TP53	AUC, 0.733‐0.856
Pathology	[56]	Predicting genetic mutations of lung cancer: CTNNB1, FMN2, TP53, and ZFX4	AUC>0.71
Pathology	[59]	Predicting MSI status in colorectal cancer	AUC, 0.93
Pathology	[60]	Predicting MSI status in colorectal cancer	AUC, 0.85
Pathology	[61]	Predicting TMB status in gastric cancer	AUC, 0.75
Pathology	[61]	Predicting TMB status in colon cancer	AUC, 0.82
Radiology	[62]	Predicting EGFR status in NSCLC	AUC, 0.81
Radiology	[63]	Predicting EGFR status in NSCLC	AUC, 0.81
Radiology	[70]	Predicting TMB status in NSCLC	AUC, 0.81
Predicting of prognosis
Pathology	[66]	Predicting outcome of colorectal cancer	AUC, 0.69
Pathology	[67]	Predicting outcome of mesothelioma	Concordance index, 0.643
Pathology	[68]	Predicting outcome of NSCLC	AUC, 0.85
Immunotherapy
Radiology	[70]	Predicting response to immunotherapy in advanced NSCLC using TMB	AUC, 0.81
Radiology	[74]	Predicting response to immunotherapy in NSCLC using MSI	AUC, 0.79
Pathology	[72]	Predicting response to immunotherapy in advanced melanoma	AUC, 0.80
Pathology	[73]	Predicting response to immunotherapy in gastrointestinal cancer using MSI	AUC > 0.99
Chemotherapy
Radiology	[75]	Predicting response to NAC in breast cancer	AUC, 0.851
Radiology	[76]	Predicting response to NAC in breast cancer	Accuracy, 88%
Radiology	[77]	Prediction response to NAC in rectal cancer	AUC, 0.83
Radiology	[78]	Prediction response to NAC in NPC	Concordance index, 0.719‐0.757
Radiology	[79]	Prediction response to NAC in NPC	Concordance index, 0.722
Radiotherapy
Radiotherapy	[84]	Segmentation of OAR in head and neck	DSC, 37.4%‐89.5%
Radiotherapy	[85]	Segmentation of OAR in NPC	DSC, 86.1%
Radiotherapy	[86]	Segmentation of OAR in head and neck	DSC, 74%
Radiotherapy	[87]	Segmentation of OAR in head and neck	DSC, 60‐83%
Radiotherapy	[88]	Segmentation of OAR in head and neck	DSC, 53‐90%
Radiotherapy	[91]	3D liver segmentation	DSC, 97.25%
Radiotherapy	[92]	Segmentation of CTV and OAR in rectal cancer	CTV: DSC, 87.7%
			OAR: DSC, 61.8‐93.4%
Radiotherapy	[93]	Segmentation of OAR in esophageal cancer	DSC, 84‐97%
Radiotherapy	[94]	Contouring of GTV in NPC	DSC, 79%
Radiotherapy	[95]	Segmentation of CTV and OAR in cervical cancer	CTV: DSC, 86%
			OAR: DSC, 82‐91%
Radiotherapy	[96]	Contouring of GTV in colorectal carcinoma	DSC, 75.5%
Radiotherapy	[97]	Contouring of CTV in NSCLC	DSC, 75%
Radiotherapy	[98]	Contouring of CTV in breast cancer	DSC, 91%
Radiotherapy	[99]	IMRT planning in NPC	Conformity index, 1.18‐1.42
Radiotherapy	[102]	Prediction of dose distribution of IMRT in NPC	Dose difference, 4.7%
Radiotherapy	[103]	Prediction of three‐dimensional dose distribution of helical tomotherapy	Dose difference, 2‐4.2%
Radiotherapy	[104]	Prediction of dose distribution of IMRT in prostate cancer	Dose difference, 1.26‐5.07%
Radiotherapy	[105]	Prediction of three‐dimensional dose distribution	Dose difference < 0.5%

Abbreviations: AUC, area under curve; NPC, nasopharyngeal carcinoma; MRI, magnetic resonance images; MSI, microsatellite instability; TMB, tumor mutation burden; NSCLC, non‐small cell lung cancer; NAC, neoadjuvant chemotherapy; DSC, Dice similarity coefficient; OAR, organs at risk; GTV, gross tumor volume; CTV, clinical target volume; IMRT, intensity‐modulated radiation therapy.

Cancer screening and early detection

Cancer screening has contributed to decreasing the mortality of some common cancers [11, 12]. The most successful examples are the identification of precancerous lesions (e.g., cervical intra‐epithelial neoplasia [CIN] for cervical cancer screening, and adenomatous polyps for colorectal cancer screening) where the treatment leads to a decrease in the incidence of invasive cancer [13]. Given the requirement for high throughput technology and a fast turnaround, automation is being used to improve the efficiency of cancer screening.

For cervical cancer screening, Wentzensen et al. [14] developed a DL classifier for p16/Ki‐67 dual‐stained (DS) cytology slides trained on biopsy‐based gold standards. In independent testing, AI‐based DS had equal sensitivity and substantially higher specificity compared with a Pap smear and manual interpretation of DS. Most importantly, AI‐based DS reduced unnecessary colposcopies by one‐third compared with Pap smears (41.9% vs. 60.1%, P < 0.001), while it had a similar performance in identifying high‐grade CIN, which indicates immediate treatment. For colorectal cancer screening, a prospective randomized controlled trial including 1,058 patients showed that AI‐assisted colonoscopy significantly increased adenoma detection rates and the mean number of adenomas found per patient compared with conventional colonoscopy (29.1% vs. 20.3%), which was attributed to a higher number of diminutive adenomas found [15]. This is particularly important because a 1% increase in the adenoma detection rate is associated with a 3% decrease in colorectal cancer incidence [13].

Automated nodule detection and classification on low‐dose computed tomography (CT) and mammography for lung and breast cancer screening have attracted significant attention. Several successful CNN‐based models have achieved classification accuracies of 80% to 95% [16, 17, 18], which shows their transformative potential in lung cancer screening. Ardila et al. [19] proposed a DL algorithm that uses patients current and prior low‐dose CT scans to predict the risk of lung cancer with outstanding results (area under the curve [AUC] of receiver operating characteristic = 0.944). Improvement in breast cancer screening with AI mammography has also been verified in preclinical studies [20, 21, 22, 23, 24], as well as in clinical settings [25]. McKinney et al. [25] established an AI system for breast cancer screening using an ensemble of three CNN‐based models. A reduction in the numbers of false positives and false negatives was observed compared with the original decisions made in the course of clinical practice. In an independent study by six radiologists, the AUC for the AI system was 11.5% higher than the average AUC achieved by the 6 radiologists. Notably, this AI system has the ability to generalize from the training data to multicenter data.

An emerging area for the early detection of cancers is liquid biopsies for circulating tumor DNA (ctDNA) or cell‐free DNA (cfDNA) obtained via a simple blood test. These are particularly important for cancer types that currently have no effective screening method. In a promising work, Cohen et al. [26] developed CancerSEEK for the early detection and prediction of eight cancer types using ctDNA. With CancerSEEK, samples are first classified as cancer‐positive using a logistic regression model applied to 16 gene mutations and the expression levels of 8 plasma proteins. The cancer type is then predicted using a random forest classifier, with accuracies ranging from 39% to 84%. Although liquid biopsies are promising for early cancer detection, so far, they have been limited to traditional ML algorithms [27, 28]. As data acquisition from liquid biopsies increases, we anticipate that DL models will eliminate the need for manual selection and curation of discriminatory features, as well as allowing for the combination of multiple data types to enhance early cancer detection.

Cancer diagnosis, classification, and grading

CNN‐based DL models that can accurately diagnose cancers, classify cancer subtypes, and identify cancer grades using histopathology (e.g., whole slide imaging [WSI]) [29], radiology (e.g., CT and magnetic resonance imaging [MRI]), and endoscopy images (e.g., esophagogastroduodenoscopy and colonoscopy) have been extensively reported, and most of them exhibit accuracies at least equivalent to that of professionals.

For cancer diagnosis, CNN‐based DL models have exhibited exceptional accuracy in identifying malignant tumors using histopathology slides [30, 31, 32, 33, 34, 35]. In an international competition (CAMELYON16) for diagnosing breast cancer metastasis in lymph nodes using WSI with hematoxylin‐eosin (HE) staining, the best CNN algorithm (a GoogLeNet architecture‐based model) yielded an AUC of 0.994, outperforming the best pathologist with an AUC of 0.884 and in a more time‐efficient manner [33]. DL algorithms have also been adopted to predict the origin of unknown primary cancers, which is extremely challenging in cancer diagnosis [36].

The success of DL has also been consistently reported in the diagnosis of malignant diseases using CT, MRI, positron emission tomography‐CT (PET‐CT) scans [37, 38, 39, 40, 41], and endoscopy [42, 43, 44, 45, 46, 47, 48, 49, 50]. Most recently, Yuan et al. [40] used CT scans to develop a classifier using a three‐dimensional (3D) ResNet algorithm to predict occult peritoneal metastasis in colorectal cancer with an AUC of 0.922, which was substantially higher than that achieved via routine contrast‐enhanced CT diagnosis (AUC = 0.791). In another work, Ke et al. [41] used MRI images from 4,100 patients with nasopharyngeal carcinoma (NPC) to train and test a self‐constrained 3D DenseNet that could distinguish NPC from benign nasopharyngeal hyperplasia with a reported AUC of 0.95‐0.97. As for endoscopy, in a multicenter study, Luo et al. [43] developed a gastrointestinal AI diagnostic system (GRAIDS) for the diagnosis of upper gastrointestinal cancers using a CNN‐based model and tested it in a prospective study involving six different tiered hospitals. While the diagnostic accuracies varied from 0.915 to 0.977 among the six hospitals, they were similar to those of expert endoscopists and superior to those of non‐experts, thus indicating the potential benefit in improving the diagnostic effectiveness of community‐based hospitals. All in all, such models, if their performance is confirmed in multicenter prospective studies, may play an important role in making cancer diagnosis more accurate, especially in local hospitals that lack experts.

Aside from dichotomous diagnosis, DL models are used for more challenging cancer classifications and grading tasks. Coudray et al. [51] developed DeepPATH, an Inception‐v3 architecture‐based model, to classify WSI for lung tissues into three classes (normal, lung adenocarcinoma, and lung squamous cell carcinoma) with a reported AUC of 0.97. The CNN was also successfully trained to perform automated Gleason grading of prostate adenocarcinoma, with a 75% agreement between the algorithm and pathologists [52]. Cancer grading can also be done using radiology images. Zhou et al. [53] developed a DL approach (based on SENet and DenseNet) to predict liver cancer grades (low versus high) using MRI images with a reported AUC of 0.83. Overall, these studies show the promising application of AI in cancer classification and grading, with performances equal to trained experts.

From a technical and practical aspect, these DL‐based diagnostic tools integrate features for fine‐tuning and enhancing performance, which simplifies the pipelines of conventional computer‐aided diagnosis and reduces false positive rates [54]. Although preclinical assessments of AI tools have paved the way for clinical trials to improve the accuracy and efficiency of cancer diagnoses, the robustness and generalizability of DL models need to be improved [55].

Predicting gene mutations in cancer

DL algorithms have also been used to characterize the underlying genetic and epigenetic heterogeneity using histopathology images. Using HE‐stained WSI of lung cancer, a CNN was trained to predict six different genetic mutations with an AUC from 0.733 to 0.856 as measured on a held‐out testing cohort [51]. Using WSI, the CNN model (Inception‐V3) also identified common mutations in liver cancer with AUCs >0.71 [56]. Using WSI, DL tools have also been developed for the prediction of whole‐genome duplications, chromosome arm gains and losses, focal amplifications and deletions, and gene variations for pan‐cancer [57, 58]. Expanded from predicting mutations in individual genes, DL models have been used to predict mutational footprints, such as microsatellite instability (MSI) status and tumor mutational burden (TMB) status, which are the most important biomarkers for responses to checkpoint immunotherapy. Most recently, Yamashita et al. [59] trained and tested MSINet, a transfer learning model based on MobileNetV2 architecture, to classify MSI status in HE‐stained WSI in a colorectal cancer cohort of 100 primary tumors and reported an AUC of 0.93. Using multiple instances of learning‐based DL, Cao et al. [60] also tried to classify MSI status using WSI in a colorectal cancer cohort and achieved an AUC of 0.85. In a work to classify TMB status using WSI, Wang et al. [61] compared eight different DL models and reported GoogLeNet as the best model for gastric tumors (AUC = 0.75) and VGG‐19 as the best for colon cancer (AUC = 0.82). The results indicate that features from histopathology images can be used to predict genetic mutations in cases in which obtaining tumor specimens for mutation analysis are not possible. Notably, it may be more cost‐effective than direct sequencing.

In addition to histopathology images, identifying cancer mutations using noninvasive radiology images such as CT or MRI scans has been explored. For example, the prediction of EGFR mutation status in non‐small cell lung cancer (NSCLC) can be achieved using CT and PET/CT scans using DL models both with AUCs >0.81 [62, 63]. In another work, Shboul et al. [64] introduced a ML approach to predict O6‐methylguanine‐DNA methyltransferase methylation, isocitrate dehydrogenase mutation, 1p/19q co‐deletion, alpha‐thalassemia/mental retardation syndrome X‐linked mutation, and telomerase reverse transcriptase mutation of low‐grade gliomas with radiomics, and achieved AUCs from 0.70 to 0.84. CT scans have also been used to predict TMB status in NSCLC (AUC = 0.81). The results were promising, but understanding what features are being learned by the CNN models to determine mutation status remains under researched.

PATIENT PROGNOSIS, RESPONSE TO THERAPY, AND PRECISION MEDICINE

Precision medicine refers to the tailoring of treatment to individual patients [65]. It aims to classify individuals into subgroups with differences in their disease prognosis or in their response to a specific treatment and thus make therapeutic interventions for those who will benefit and sparing expense and side effects for those who will not. DL algorithms are used to automatically extract features from medical data to build models that can accurately predict risk of tumor relapse and patients’ responses to treatments [66, 67, 68]. Based on the prediction results, physicians can provide more precise and suitable treatments.

Immunotherapy drugs have been approved for the treatment of metastatic melanoma, lung cancer, and other malignancies. However, more than 50%‐80% of cancer patients fail to respond to checkpoint inhibitor therapy. Currently, response prediction for immunotherapies is based on biomarkers of the immunogenic tumor microenvironment, such as programmed death‐ligand 1 (PD‐L1) expression, TMB, MSI, and somatic copy number alterations. However, these biomarker data were acquired via a biopsy, which is invasive, difficult to perform longitudinally, and limited to a single tumor region. Furthermore, the predictive value of biomarkers may be limited. In the KEYNOTE‐189 clinical trial, immunotherapy with pembrolizumab combined with standard chemotherapy provided survival benefits for all patients regardless of their PD‐L1 expression [69]. To achieve the goal of precision medicine, many researchers have established DL models to predict patient biomarkers related to immunotherapy using radiomics and pathomics data [70, 71, 72, 73]. Johannet el al. developed a pipeline that integrates DL on histology specimens with clinical data to predict immunotherapy response in advanced melanoma [72]. The results showed that the classifier accurately stratified patients into responders and non‐responders with an AUC of 0.80. Most excitingly, Arbour et al. [74] developed a DL model that directly predicts the best overall response and progression‐free survival using radiology text reports for patients with NSCLC treated with a programmed cell death protein‐1 blockade. These studies underscore the potential ability of AI to identify individuals who may benefit from immunotherapy without the aforementioned negatives of biopsies.

In addition to immunotherapy, other therapies (e.g., targeted therapy and neoadjuvant chemotherapy [NAC]) have achieved prominent clinical success in specific populations, driving the need for accurate predictive assays to inform patient selection. This requirement can be met by a combination of big data and AI. AI predictive models can identify imaging phenotypes that are associated with a targeted mutation. This AI‐based approach has the advantage of identifying the mutation status repeatedly and noninvasively. This approach was supported by a PET/CT‐based DL model for patients with NSCLC, which uses radiomic features to discriminate EGFR‐mutant types from wild‐type with an AUC of 0.81 [62]. Moreover, with a large amount of radiomics data, DL algorithms have shown power in estimating responses to NAC for patients with breast cancer [75, 76], rectal cancer [77], and NPC [78, 79]. After NAC, about 35% of patients with locally advanced breast cancer achieved a pathologic complete response (pCR), which was associated with improved survival [80]. Whereas, a poor response to NAC was associated with an adverse prognosis [81]. Therefore, the accurate prediction of treatment response is warranted, which can avoid unnecessary toxicity and delays to surgery. Using pretreatment MRI from patients with locally advanced breast cancer, Ha et al. [76] trained a CNN to predict pCR, and no response/progression after NAC, reaching an overall accuracy of 88%. In addition to predicting patient responses to therapies, AI now offers additional avenues to adjust drug dosage for single or combinational therapies for individual patients in a dynamic manner using patient‐specific data collected over time [82].

DEEP LEANING IN RADIOTHERAPY

Radiotherapy constitutes an integral modality in the treatment of cancers with half of patients receiving it. The image‐, data‐driven and quality assurance frameworks of radiotherapy provide an excellent foundation for the development of AI algorithms and their integration into radiotherapy workflows. There has been an acute interest in exploring AI to facilitate radiotherapy for target volume and organs at risk (OAR) delineation and automated treatment planning [83].

Target volume and OAR delineation is a labor‐intensive process, and its accuracy depends heavily on the experience of the radiation oncologists. CNN‐based semantic segmentation has been consistently established as a state‐of‐the‐art tool in the automated delineation of OAR in head and neck [84, 85, 86, 87, 88], thorax [89], abdomen [90, 91] and pelvic regions [92]. OAR is usually delineated on CT images, and the runtime for each patient lasts only several seconds. From these published studies, the segmentation accuracies of organs with large volumes, rigid and regular shapes were rather high, such as those of the mandible (Dice similarity coefficient [DSC] = 0.94), parotid (DSC = 0.84), kidney (DSC = 0.96), and liver (DSC = 0.97), while for organs with small volumes, movable and irregular shapes, the segmentation accuracies decreased, such as those of the optic nerve (DSC = 0.69), chiasm (DSC = 0.37), intestine (DSC = 0.65), and esophagus (DSC = 0.83). Of note, preliminary studies have shown that differences in dosimetry parameters between automatic and manual delineations were small, and automatic segmentations performed sufficiently well for treatment planning purposes [87, 93].

Given the variety of shapes, locations, and internal morphologies of tumors, automated contouring of tumor targets by DL is still a great challenge. Nonetheless, automatic contouring speeds up the process and improves consistency among radiation oncologists. Automated delineation of the gross tumor volume (GTV) and clinical target volume (CTV) have been investigated in many cancers, such as nasopharyngeal [94], cervical [95], colorectal [92, 96], lung [97] and breast cancers [98]. Lin et al. [94] first constructed an automated contouring tool for NPC by applying a 3D CNN model to MRI. In this independent test, they found acceptable concordance between the AI tool and human experts, with an overall accuracy of 79%. Moreover, in a multicenter test involving eight radiation oncologists from seven hospitals, the AI tool outperformed half of the physicians and was equal to the other four. With AI's assistance, substantial improvement in the contouring accuracy among five of the eight physicians as well as significant reductions in the interobserver variation (by 54.5%) and contouring time (by 39.4%) were observed.

Another important application of AI in radiotherapy is automated treatment planning. Radiotherapy planning is a complex process that involves “trial‐and‐error” based on physicists’ subjective priorities to achieve specific dosimetry objectives. As a result, treatment planning quality depends heavily on the experience of the clinical physicists. While automated planning using knowledge‐based techniques, such as RapidPlan in Eclipse, have improved the consistency of planning quality [99, 100], these methods are suboptimal since they cannot provide estimations of patient‐specific achievable dose distributions. Recently, DL‐based methods have become a promising approach for individualized 3D dose prediction and optimization [101, 102, 103, 104]. Fan et al. [105] first developed an automated treatment planning strategy based on ResNet to achieve an accurate 3D dose prediction and voxel‐by‐voxel dose optimization for head and neck cancers. The results showed no significant difference between the predicted and real clinical plans for most clinically relevant dosimetry indices. More importantly, with this strategy, patients with different prescription doses can be learned and predicted in a single framework.

Other applications of AI in radiotherapy include the prediction of radiation‐induced toxicities [106, 107, 108], image reconstruction [109, 110, 111], synthetic CT generation [112, 113, 114], image registration [115, 116, 117], and intra‐ and inter‐fraction motion monitoring [118, 119, 120]. In summary, AI has the potential to improve the accuracy, efficiency and quality of radiotherapy. Furthermore, MRI‐only radiotherapy [121] and real‐time adaptive radiotherapy [109] could be achieved with the implementation of effective and efficient automated segmentation, image processing, and automated treatment planning tools based on DL, which are significantly faster than standard approaches.

DL IN CANCER RESEARCH

DL approaches have been applied in various aspects of cancer research, including investigating biological underpinnings, developing anti‐cancer therapeutics, and implementing randomized controlled trials (RCTs). To uncover the biological mechanisms of cancer, studies have used DL to analyze the relationship between genotypes and phenotypes with a large number of achievements already reported. In a recent study leveraging DL algorithms, the role of F‐box/WD repeat‐containing protein 7 (Fbw7) in cancer cell oxidative metabolism was discovered via gene expression signatures from The Cancer Gene Atlas dataset [122]. Watson for Genomics also recognized genomic alterations with potential clinical effects that were not identified by the conventional molecular tumor boards across a spectrum of cancer types [123]. Identification of these genetic variants not only pinpoints relevant biological pathways but also suggests targets for drug discovery. ML methods have also been employed to accelerate the early discovery of potential anti‐cancer agents [124, 125, 126, 127, 128, 129]. Valeria et al. [129] reported the first perturbation model combined with ML to enable the design and prediction of dual inhibitors cyclin‐dependent kinases 4 and human epidermal growth factor receptor 2 with sensitivity and specificity higher than 80%. Another key aspect of drug discovery is the determination of compounds with good on‐target effects and minimal off‐target effects. Zhavoronkov et al. [130] developed a DL model and discovered powerful inhibitors of the discoidin domain receptor 1 (DDR1, a kinase target implicated in multiple cancers) in just 21 days versus conventional timelines of approximately one year. All in all, DL AI is accelerating drug discovery and is already successfully predicting drug behavior.

The adoption of novel cancer treatments is dependent on successful RCTs. However, successful recruitment of appropriate patients into these trials is regarded as one of the most challenging aspects. Matching complicated eligibility criteria to potential subjects is a tedious, labor‐intensive, and difficult task [131]. To automate this, Hassanzadeh et al. [132] used natural language processing and a Multi‐Layer Perceptron model to extract meaningful information from patient records to help collate evidence for better decision making on the eligibility of patients according to certain inclusion and exclusion criteria,. It achieved an overall micro‐F1 score of 84%. Selecting top‐enrolling investigators is also essential for the efficient execution of RCTs. To facilitate the automation of selection, Gligorijevic et al. [133] proposed a DL approach to learn from both investigator‐ and trial‐related heterogeneous data sources and rank investigators based on their expected enrollment performance in new RCTs. Here, DL shows the potential to optimize clinical cancer trials.

CHALLENGES AND FUTURE IMPLICATIONS

While AI is widely investigated in oncology, studies need to be performed to translate DL models into real‐world applications. Barriers to improving doctors’ acceptance and performance of clinically applied DL include the generalizability of its applications, the interpretability of algorithms, data access, and medical ethics.

Generalizability and real‐world application

Because of the great heterogeneity in medical data across institutions, the performance of DL models tends to decrease when applied at different hospitals, therefore, external validation sets may be required to confirm their performance [55]. Additionally, the extremely large number of parameters in DL results in a high likelihood of overfitting and limiting of the generalizability across different populations [134]. More importantly, in clinical settings, to make a precise decision, oncologists need to consider a variety of data, including clinical manifestations, laboratory examinations, imaging data, and epidemiological histories. However, most recent studies have only adopted one type of data (such as imaging) as the input model. To mimic real clinical settings, a multimodal DL model incorporating the aforementioned information plus imaging data needs to be constructed in future studies.

Interpretability: the black‐box problem

DL has been criticized for being a “black box” that does not explain how the model generates outputs from given inputs. The large number of parameters involved makes it difficult for oncologists to understand how DL models analyze data and make decisions. However, some efforts have been made to make this black box more transparent [135, 136]. For example, the heat map‐like class activation algorithm, visualizes which image regions are taken into account with DL models when making decisions and to what degree. These innovative studies render DL tools more interpretable and applicable in clinical oncology settings.

Data access and medical ethics

DL studies not only face technological challenges but also resource and ethical challenges. The power and believability of DL relies on a large amount of training data. Limited data may cause overfitting, yielding an inferior performance in an external test cohort [134]. Given the concerns of protecting patient information, medical data are often the property of individual institutions, and there is a lack of data‐sharing systems to link institutions. Fortunately, this obstacle is beginning to be overcome, with privacy‐preserving distributed DL (DDL) and multicenter data‐sharing agreements [137, 138, 139]. DDL provides a privacy‐preserving solution to enable multiple parties to jointly learn via a deep model without explicitly sharing local datasets. The Cancer Imaging Archive, which collects clinical images from different institutes and hospitals, also provides a good example of data sharing and may promote radiomic studies [140]. In the future, an authoritative framework should be developed by governments and enterprises to realize secure data sharing. In addition, several ethical issues need to be addressed prior to the clinical implementation of DL tools. First, the degree of supervision required from physicians should be determined. Second, the responsible party for incorrect decisions made by DL tools should also be determined.

CONCLUSIONS

DL is a newly developed AI method in oncology which is rapidly progressing. With the growth of high‐quality medical data and the development of algorithms, DL methods have great potential in improving the precision and efficiency of cancer diagnosis and treatment. Moreover, the positive attitude of the FDA towards AI medical devices further increases the prospect of DL's practical application in oncology. For the realization of clinical implementation, future researches should focus on the reproducibility and interpretability to make DL methods more applicable.

DECLARATIONS

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not applicable.

CONSENT FOR PUBLICATION

Not applicable.

AVAILABILITY OF DATA AND MATERIALS

Not applicable.

COMPETING INTERESTS

The authors declare that they have no competing interests.

AUTHOR CONTRIBUTIONS

ZHC, LL, YS, and RHX conceived this study. ZHC, LL, and CFW drafted the manuscript. YS, RHX, and CFL revised the manuscript. All authors read and approved the final manuscript.