Artificial intelligence and convolution neural networks assessing mammographic images: a narrative literature review.

Studies have shown that the use of artificial intelligence can reduce errors in medical image assessment. The diagnosis of breast cancer is an essential task; however, diagnosis can include 'detection' and 'interpretation' errors. Studies to reduce these errors have shown the feasibility of using convolution neural networks (CNNs). This narrative review presents recent studies in diagnosing mammographic malignancy investigating the accuracy and reliability of these CNNs. Databases including ScienceDirect, PubMed, MEDLINE, British Medical Journal and Medscape were searched using the terms 'convolutional neural network or artificial intelligence', 'breast neoplasms [MeSH] or breast cancer or breast carcinoma' and 'mammography [MeSH Terms]'. Articles collected were screened under the inclusion and exclusion criteria, accounting for the publication date and exclusive use of mammography images, and included only literature in English. After extracting data, results were compared and discussed. This review included 33 studies and identified four recurring categories of studies: the differentiation of benign and malignant masses, the localisation of masses, cancer-containing and cancer-free breast tissue differentiation and breast classification based on breast density. CNN's application in detecting malignancy in mammography appears promising but requires further standardised investigations before potentially becoming an integral part of the diagnostic routine in mammography.

Introduction

Screening mammography is the recommended tool for the early detection of breast cancer in women experiencing no symptoms and has been shown to decrease breast cancer mortality rate by 40–63%.1 However, current diagnostic frameworks for assessing mammograms such as the breast imaging reporting and data system (BI‐RADS), developed by the American College of Radiology (ACR) used to report findings into a number of well‐defined categories, can be limited by ‘detection errors’ (pathology missed) and ‘interpretation errors’ (pathology misinterpreted).2 One possible solution to reduce the errors is utilising artificial intelligence (AI) tools, which automatically find abnormalities (detection) and categorise them either as normal or as abnormal (interpretation).

In general, AI can be defined as a sophisticated computer program able to carry out tasks usually requiring human intellect in areas such as visual perception and decision‐making. Over the past few years, deep learning has dramatically transformed the AI industry. Deep learning is a subset of machine learning methods based on artificial neural networks. A convolutional neural network (CNN) is a class of deep neural networks, commonly applied to analyse image data. Similar to conventional machine learning methods, CNN allows the machine to perform specific tasks by relying on the inference of decision boundaries rather than explicit instructions by a user.

Furthermore, CNNs learn relevant features from the data, with limited input from a human expert, and make predictions via heuristics when new data are entered.3 CNNs can be used to assist with the detection of breast cancer on mammographic images with a high degree of accuracy as measured by sensitivity and specificity.4 It can also reduce the time required to assess a large volume of mammograms.5 These potential benefits have emphasised the importance of exploring CNN's ability to accurately diagnose abnormalities on mammograms promptly.4, 5 CNN is faster with similar accuracy when compared to radiologists without other limiting factors such as a lack of qualified personnel.6, 7

Considering recent advances in AI tools in mammography, it is essential for the imaging and the wider health community to better understand current trends and developments. However, there is a lack of comparisons in the literature between similar studies that this literature review aims to address. The purpose of this study therefore was to compare the recent developments of CNN in the diagnosis of mammographic images to enable medical imaging professionals to gain greater insight on how this technology can assist our patients in future.

Methodology

The focus of this review was to identify the current developments and uses of CNN in mammography diagnostics. Due to the review’s narrative nature, results were described by description and exploration with qualitative elements rather than systematically.

Using the keywords listed in Table 1, ScienceDirect, PubMed, MEDLINE, British Medical Journal and Medscape databases were searched for original English research studies published between 2011 and 2019. Studies which used imaging modalities other than mammography were excluded. A total of 33 relevant peer‐reviewed articles were found.8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 To compare the results among studies, relevant features were extracted: (1) author and publication year; (2) study aim; (3) utilised database; (4) number of images used; and (5) performance measures including sensitivity, specificity and area under the curve (AUC).

Table 1

List of keywords used to search for relevant topics related to the review.

Topic	Keywords
Artificial Intelligence	Artificial Intelligence; Deep learning; Artificial Neural Networks; Convolutional Neural Network
Anatomy; Pathology of Interest	Breast; Breast Cancer; Breast Lesion; Breast Carcinoma; Breast Neoplasm
Imaging Modality	Mammogram; Mammography
Others	Image Reading; Diagnostics; BI‐RADS; Breast Screening

Results

A total of 33 out of 65 articles were selected under the inclusion criteria; 32 studies were rejected due to lack of direct relevance. In the accepted literature, there were various recurring themes, so articles were further categorised into four different salient categories:

Category 1. Differentiation between benign and malignant tissues;

Category 2. Localisation of mass in breast tissue;

Category 3. ‘Cancer‐containing’ and ‘cancer‐free’ breast tissue differentiation;

Category 4. Breast classification based on breast density.

A total of 12 studies focused exclusively on Category 1, primarily testing CNN’s ability to distinguish between benign and malignant masses. Category 2 had seven studies that focused on mass localisation. Category 3 had seven studies which focused on screening images as ‘cancer‐free’ and ‘cancer‐containing’; Category 4 had seven studies that used CNN to categorise mammograms by breast density.

Twenty‐two of the 33 articles used the ACR’s BI‐RADS system to report mammograms assigning breast imaging studies to one of six assessment categories (0 – incomplete study, 1 – negative, 2 – benign, 3 – probably benign, 4 – suspicious for malignancy, 5 – highly suggestive of malignancy and 6 – known biopsy‐proven malignancy).41

In all 33 articles, four primary image databases were used:

Digital Database for Screening Mammography (DDSM) (Digitised Images): This database from the University of South Florida contains 2500 studies, with each study containing two images of each breast and patient information such as age, when the image biopsy proven, and ACR breast density classification. All cases are female.42

MIAS Mini Mammographic Database (Digital Images): The mini‐MIAS provided by the Mammographic Image Analysis Society contains 161 pairs of films and has commonly encountered pathologies and normal cases.43

INbreast database (Digital images): This database has 410 images from 115 cases: 90 cases have images of both breasts and 25 cases from mastectomy patients. All patients are females. Different pathologies include masses, calcification, asymmetries and distortions.44

Breast Cancer Digital Repository (BCDR) (Digital and Digitised images): BCDR is a public repository composed of breast cancer patient’s case studies hosted by the IMED Project in March 2009.8 Data classification uses BI‐RADS and is subdivided into two repositories: the Film Mammography Repository and Full Field Digital Mammography Repository. The BCDR has been used to train CAD schemes and is still in development.45 Digitised images contain 1010 patients (998 female and 12 male) ranging from ages 20 to 90, 1125 studies and a total of 3703 images. The digital database is in development with 724 patients (723 female and 1 male) ranging from ages 27 to 92, 1042 studies and a total of 3612 images.46

Discussion

Category 1: Differentiation between a benign and malignant mass

The differentiation between benign and malignant masses in CNN breast screening was the most prominent category of study. Akselrod‐Ballin’s CNN accurately diagnosed mammograms classified as BI‐RADS 1; however, it found difficulty diagnosing higher graded images.19 Importantly, they noted that CNN systems have issues differentiating benign and malignant masses in BI‐RADS 3 images.19 When BI‐RADS 3 images were excluded, their CNN system improved detection AUC from 0.60 to 0.72. Aside from this study, other studies did not mention BI‐RADS 3 exclusion.

Furthermore, Carneiro suggested that their CNN’s inability to diagnose BI‐RADS images >1 was due to training images being annotated for either microcalcifications or masses, but not for both in a single image, potentially affecting heuristics.15 Additionally, automated segmentation made by CNN generally resulted in lower accuracy, since detection of false‐positive masses impacts the system’s performance in mass classification.15

Category 2: Localisation of masses in breast tissue

Although many studies focused on benign and malignant differentiation, tumour localisation was another common topic. Five studies showed detecting a mass in dense breast regions and pectoral muscles to be difficult due to high intensities.21, 22, 23, 24, 36

Al‐Masni et al.47 employed a You Only Look Once (YOLO) system that uses a singular network for diagnostics by dividing the input image into multiple subregions, compared to traditional neural networks requiring multiple networks for each extracted region. As a result, their system operated quicker compared to traditional neural networks and successfully identified masses in pectoral muscles and dense tissues; they suggested that its ability to localise masses in radiopaque areas was due to augmented training by digitally manipulating images to produce more training images.47

Hwang et al.21 experimented with CNN’s ability to self‐learn via a weakly supervised CNN system, meaning that images used for training were minimally annotated with a weak description. Compared to other studies, the results were poor for AUC, sensitivity, specificity and accuracy with no results exceeding 0.77.21 This finding suggests that current CNN’s need highly annotated images for training for accurate results.

Overall, the results from the studies in Category 2 appear promising in highly supervised training methods, but weak in lowly supervised systems.

Category 3: ‘Cancer‐containing’ and ‘cancer‐free’ breast tissue differentiation

In 2018, a study using a CNN system was designed to differentiate between normal and abnormal images via biomarkers such as breast density, presence of mass and microcalcification.28 Their results indicated that abnormal images were more accurately identified when microcalcifications and benign cases were excluded. The authors suggested that due to the training data having an uneven number of cases of normal and cancer cases, this caused overfitting and database dependency. This creates difficulty distinguishing between benign and malignant cases due to overreliance on heuristics and an inability to learn and adapt from new cases.28

Al‐Antari modified Al‐Masni et al.'s YOLO system to differentiate between cancer‐containing and cancer‐free images, their study demonstrated an accuracy of 95.64% and noted that a robust CNN system heavily relies on the deep learning model in segmenting specific region of masses to reduce potential false positives from surrounding tissues.30

Category 4: Breast classification based on breast density

Breast tissue density may increase the likelihood of developing breast cancer, as increased breast density and glandular tissue may make it difficult to detect early signs of breast cancer such as microcalcifications.47 The importance of classifying breast density for breast cancer risk must be stressed as the relative risk of developing breast cancer in women with heterogeneously dense breasts is 1.2 times more compared to the average women, and for women with extremely dense breasts, the relative risk is 2.1 times higher. This is likely due to the masking effect, excess glandular tissue and breast density as an independent risk factor.48

Between 2018 and 2019, there were studies that tested CNN’s capabilities in categorising mammograms by breast density.37, 38, 40, 49 Ha’s preliminary system demonstrated an accuracy of 72% in predicting ‘high’‐risk and ‘low’‐risk mammograms by generating heat maps on the mammograms where ‘red’ areas had overlapping mammographic features suggesting high density, high and low risk defined as dense and non‐dense, respectively.37 They noted that while the red areas on the heatmaps suggest areas of high density, the pixel map generated does not necessarily indicate specific areas where breast cancer may develop.37 Notably, Mohammed’s study demonstrated a high accuracy on differentiating between BI‐RADS category ‘scattered density’ and ‘heterogeneously dense’ images, however, noted that their study was a retrospective study and the images used had little variation in mammographic vendors and imaging protocol, suggesting that their system may not guarantee the same results from different manufacturers and imaging standards.38

Fonseca et al conducted a preliminary study to diagnose breast density using the ACR’s breast composition categories (a – entirely fatty; b – scattered areas of fibro‐glandular density; c – heterogeneously dense tissue, potentially obscuring small masses; and d – breasts extremely dense, lowering mammography sensitivity)41 and achieved a mean accuracy of 0.7305, similar to a radiologist when categorising breast density.35 The study’s weakness was that the images used were sourced locally and not from a mainstream database. Additionally, a study found that inter‐reader agreement regarding breast density classification was only at 49% and agreement was usually on either fatty breasts or extremely dense breasts.50 Therefore, there is a great potential in improving breast density classification in assessing breast cancer risk using CNN’s capabilities.

Diniz in 201836 classified breast tissue by density and regions with or without a mass; by combining them, they localised and detected masses in dense (Cooper’s ligaments, mammillary glands and ducts) and non‐dense tissue (adipose tissue). Sensitivity was slightly higher in non‐dense tissue (0.9156 vs. 0.9036), yet dense tissue had significantly higher specificity (0.9635 vs. 0.9073), meaning that their CNN was better at recognising negative non‐dense images but identified abnormalities more accurately in dense tissues.36

Sensitivity and specificity, total accuracy and AUC of CNNs

Basic statistical analyses are employed for simple comparisons between a study’s AUC, sensitivity, specificity and total accuracy gathered from Table 2. The overall mean of CNN’s AUC, specificity, sensitivity and total accuracy is listed in Table 3. It is important to note that data sets used in the radiologist studies are not standardised and images are from patients with vastly different clinical history.

Table 2

33 studies listed by category, author, year, database used, number of images, AUC, specificity, sensitivity and accuracy.8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40

Author	Year	Database	#Cases (Images)	AUC	Specificity	Sensitivity	Accuracy
Category 1 studies
Ramos‐Pollán et al.8	2011	BCDR	286	0.996	0.77	0.95	0.97
Rouhi et al.9	2014	DDSM	170 (170)	0.951	0.96	0.97	0.96
Arevalo et al.10	2015	BCDR	344 (736)	0.86	–	–	–
Arevalo et al.11	2016	BCDR	344 (736)	0.7	–	–	–
Dhungel et al.12	2016	INbreast	115 (410)	0.87	–	–	0.91
Huynh et al.13	2016	Custom	– (–)	0.81	–	–	–
Jiao et al.14	2016	DDSM	– (–)	–	–	–	0.97
Carneiroet al.15	2017	INbreast	115 (410)	0.72	0.66	0.69	–
Carneiro et al.15	2017	INbreast	115 (410)	0.87	0.92	0.69	–
Carneiro et al.15	2017	DDSM	172 (680)	0.91	0.97	0.94	–
Kooi et al.27	2017	Custom	956 (1804)	0.8	–	–	–
Teare et al.17	2017	DDSM	0	0.92	0.91	0.8	–
Chougrad et al.18	2018	DDSM	1329 (5316)	0.98	–	–	0.97
Chougrad et al.18	2018	INbreast	50 (200)	0.97	–	–	0.96
Chougrad et al.18	2018	BCDR	300 (600)	0.96	–	–	0.97
Chougrad et al.18	2018	MD***	1529 (6116)	0.99	–	–	0.99
Chougrad et al.18	2018	MIAS	113 (113)	0.99	–	–	0.98
Akselrod‐Ballin et al.*, 19	2016	DDSM, INbreast	850 (850)	–	–	–	0.78
Akselrod‐Ballin et al.**, 19	2016	DDSM	850 (850)	–	–	–	0.77
	2016	INbreast (without BI–RADS 3)	– (–)
Akselrod‐Ballin et al.*, 19	2016	DDSM, INbreast	−850	0.6	–	–	–
Akselrod‐Ballin et al.**, 19	2016	DDSM, INbreast (without BI‐RADS 3)	−850	0.72	–	–	–
Ribli et al.20	2018	INbreast	115 (115)	0.85	0.9	–	–
Category 2 studies
Hwang et al.21	2016	DDSM and MIAS	10,363 (322)	0.68	0.76	0.58	0.7
Hwang et al.21	2016	DDSM and MIAS	10,363 (322)	0.54	0.66	0.44	0.66
Al‐masni et al.47	2017	DDSM	−600	–	–	–	0.9633
Sampaio23	2011	DDSM	566 (566)	0.87	0.86	0.8	0.85
Sun et al.24	2016	Custom	−1874	0.8818	0.72	0.81	0.8243
Shen et al.25	2019	DDSM	2223	0.922	–	0.9643	–
Savelli et al.26	2019	INbreast	115 (410)	–	–	0.763	–
Kooi et al.16	2017	Custom	44,090	0.941	–	–	–
Category 3 studies
Kim et al.28	2018	Custom	29,107	0.91	0.89	0.76	–
Jadoon et al.29	2017	DDSM and MIAS	– (–)	0.85	0.82	0.88	0.82
Al‐antari et al.30	2018	INbreast	115 (410)	0.9478	0.9241	0.9714	0.9564
Kaur et al.31	2019	MIAS	20	–	0.88	0.9	0.9
Anitha et al.32	2017	DDSM	300	–	–	0.925	–
Anitha et al.32	2017	MIAS	170	–	–	0.935	–
Wichakam et al.33	2016	INbreast	216	–	–	–	0.9727
Suzuki et al.34	2016	DDSM	198	–	–	0.899	–
Category 4 studies
Fonseca et al.53	2015	Custom	−1157	–	–	–	0.73
Bandeira Diniz et al.36	2018	DDSM	Non‐dense; − (1004)	–	0.91	0.92	0.91
Bandeira Diniz et al.36	2018	DDSM	Dense; – (1482)	–	0.96	0.9	0.95
Ha et al.37	2019	Custom	1474	–	–	–	0.72
Mohamed et al.49	2018	Custom	15,415	0.95	–	–	–
Ionescu et al.39	2019	Custom	67,520	–	–	–	–
Mohamed et al.38	2018	Custom	22,000	0.926	–	–	–
Trivizakis et al.40	2019	DDSM	2500 (10,239)	0.548	–	–	–
Trivizakis et al.40	2019	MIAS	161 (322)	0.798	–	–	–

Exp1.

Exp2.

MD = DDSM+ INbreast + BCDR.

Table 3

Comparison between mean, median, minimum and maximum values of AUC, specificity, sensitivity and total accuracy from the 33 CNN studies8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 and four radiologists studies.54, 55, 56, 57

	Mean		Median		Minimum		Maximum
	CNN	Radiologist	CNN	Radiologist	CNN	Radiologist	CNN	Radiologist
AUC	0.851	–	0.876	–	0.540	–	0.996	–
Specificity	0.851	0.905	0.890	0.808	0.660	0.889	0.970	0.988
Sensitivity	0.833	0.795	0.899	0.916	0.440	0.600	0.971	0.899
Total accuracy	0.883	–	0.930	–	0.660	–	0.990	–

For sensitivity, minimum and maximum value disparity is greater for CNN, but the lowest and highest sensitivity range is more significant. The lowest sensitivity and specificity are from a study by Hwang and Kim; their results are lowest across all parameters, reinforcing that a highly supervised learning requirement is required. They argued that the training database (DDSM) contained low‐quality images with many artefacts. However, this issue did not occur for other studies.

Of note, some studies omitted sensitivity and specificity results, so performance comparisons between the CNN and the radiologist are rudimentary. Furthermore, every study used a different number of images to train and test their systems, making exact comparisons difficult.

From these four parameters, the current development of CNN appears promising in mammographic diagnostics but requires highly supervised learning (images that are highly annotated with regions of interests to learn and be segmented), sufficient images for training and adjustment of learning algorithms to ensure overfitting and database dependency do not occur to achieve good diagnostic results.

In 2018, a retrospective study compared the performance of a CNN system to 101 radiologists and found that the CNN system slightly outperformed the radiologists with an AUC of 0.840 and 0.814, respectively.51 However, due to the study’s retrospective nature, results may suffer from the ‘laboratory effect’ and may not be applicable in a clinical setting.51

Limitations of this review and current literature

Although some studies have used the same databases for training, many have used a different combination of databases and various numbers of images for training. Hence, valuable comparisons between studies are difficult to make as each database uses mammograms from different hospitals, image parameters and manufacturers, resulting in a lack of reference standards between data sets. Since the study outcomes depend significantly on the database used for training, the variability between these databases can affect each AI’s performance algorithm. This variability is called ‘inter‐database variability’, causing optimisation of algorithm and standardised comparison between studies extremely challenging.52

Additionally, there are several ‘cases’ per database and studies may not use the same cases. For example, Rouhi et al.’s9 model aimed to differentiate malignant or benign lesions, while Diniz et al.'s36 design aimed to localise mass and categorise images based on breast tissue density. Although both used the same database, the cases used were different, meaning that direct comparisons are difficult. Furthermore, current data sets are archaic and lack realism; many images still use digitised rather than digital images; the data sets lack clinical history such as risk factors, ethnicity and age make thorough analysis difficult. Also, in real‐world screening approximately 0.5–0.8% cases contain cancer, meaning that data sets used had a high number of malignancies.

Likewise, the number of images used to pre‐train and test the CNN varies between studies and this potentially affects its calibration. A CNN extensively trained with small training schemes may lead to overfitting and database dependency which causes the CNN to be over‐reliant on heuristics taught in training, resulting in weak future predictions, analysis and adaptation to new images.3 To improve comparisons between CNN and radiologists, their performance must be examined on the same data set and investigations to identify overlapping errors between AI and humans are recommended.

To solve these issues, mainstream databases should be selected and standardised with the same cases and images used so meaningful statistical analysis can be performed between studies such as ROC curve analysis and homogeneity.

Conclusion

CNN diagnosis in mammography has been demonstrated as an expanding field in artificial intelligence diagnostics. The most significant limitation of the current studies is the lack of standardisation between studies. Despite this limitation, the mean accuracy across the studies is promising, showing the potential reliability of CNN in mammography diagnostics.

Current innovation in CNN diagnostics for mammography includes the differentiation of benign and malignant masses, the localisation of masses, cancer‐containing and cancer‐free breast tissues differentiation and breast classification based on density. Literature indicates that the differentiation of benign and malignant masses has the most developments, and the consensus suggests that CNN is comparable in performance to a radiologist. However, there are still some significant weaknesses within the 33 studies due to the inter‐database variability, inadequate original data reports and other aspects of AI training such as unsupervised learning and training schemes. Therefore, as promising as the introduction of a CNN application in detecting malignancy in breasts seems, it does not currently appear ready for the real clinical environment and requires further standardised investigations before it can be a part of the diagnostic routine for mammography imaging. With the current developments, instead of solely investigating CNN performance versus radiologists, the real opportunity is to examine CNN’s potential to support radiologist’s performance both for training purposes and as a computer aid to help them to further improve medical image diagnosis.

Conflicts of Interest

The authors declare no conflict of interest.