Can high school students help to improve breast radiologists in detecting missed breast cancer lesions on full-field digital mammography?

Aim: To investigate whether full-field digital mammography (FFDM) and contrast-enhanced mammography (CEM), evaluated by non-experienced high school students, improves detection of missed breast cancer lesions on FFDM, in the same cohort of patients.
Methods: Non-experienced first- and second year high school students examined fourteen cases of patients diagnosed with breast cancer. These cases consisted of missed breast cancer lesions on FFDM by a breast radiologist. Sensitivity of assessment of the students on FFDM and CEM was analysed and compared with the initial results of the breast radiologists.
Results: A total of 134 high school students participated in this study. Mean age was 12.8 years (range 10-14). Based on FFDM, mean overall sensitivity of the students was 29.2% (18.9 - 39.6%). When recombined CEM images were used, mean overall sensitivity of students improved to 82.6% (74.0 - 91.2%) (p=0.001). Mean overall sensitivity of FFDM exams evaluated by radiologists was 75.7% (64.2 - 87.3%), which was lower when compared to student's evaluations on recombined CEM exams, yet not statistically significant (p=0.098). Conclusions: Contrast-enhanced mammography evaluated by non-experienced high school students might improve detection rate of breast cancer when compared to evaluations of only full-field digital mammography by radiologists.

Introduction

Over the last decades, breast cancer incidence increased from 541,000 in 1975 towards more than 1,6 million in 2012 1; 2. Breast cancer death rates have been stable or have even slightly decreased since 1990 in Northern America and higher-resource European countries, because of the improved treatment strategies and early detection by (screening) mammography 2; 3. In breast cancer imaging mammography continues to play a pivotal role in detecting patients in an early phase of disease, in order to obtain the most beneficial and prognostic outcome.

The mean sensitivity of full-field digital mammography (FFDM) is 80% 4. Interpretation of FFDM can be especially challenging in more dense breast tissue or if lesions are smaller 5-7. Sensitivity can even drop to 61% when women have extremely dense breasts 8. As a result, a sensitivity of 80% is considered acceptable as reading performance among radiologists, but higher accuracy is strongly preferred to further optimize breast cancer detection 9.

Consequently, new techniques were and are developed in order to improve diagnostic performance of FFDM, like the introduction of contrast-enhanced mammography (CEM) 10. In CEM, an iodine-based contrast agent is intravenously administered prior to image acquisition. A typical CEM exam consists of a low- and high-energy image of both breasts in two different projection views 11. The low-energy image is comparable to FFDM, whereas the high-energy images are used to generate a recombined image, in which areas of iodine accumulation can be visualized 12; 13. A meta-analysis of Tagliafico et al. demonstrated that the sensitivity of CEM is superior compared to FFDM, with a pooled sensitivity of 98% compared to previously reported sensitivity of 80% on FFDM 4; 14.

Also, the learning curve of CEM seems to be shorter than FFDM. Prior studies have shown that CEM improves the diagnostic performance of experienced and less experienced readers (i.e. breast radiologists without previous CEM experience and radiology residents) when compared to FFDM 15; 16. If evaluation of CEM by non-experienced readers (even without any dedicated radiological training) could outperform FFDM assessed by experienced breast radiologists, the importance of CEM as breast imaging tool would be further underlined, especially in potential future settings where automated image-based feature analysis (i.e. radiomics) are involved to improve breast cancer detection rate 17.

Therefore, the current aim of this study was to investigate the diagnostic performance of FFDM and CEM in truly non-experienced readers (i.e. high school students) in a cohort consisting of patients with missed breast cancer lesions according to prior FFDM evaluations by breast radiologists.

Material and methods

Patient cohort

The local medical ethical committee of Maastricht UMC+ waived the requirement for obtaining informed consent from included patients and participants. In this study, we used cases from a previously published study by Lalji et al. 16. In their study, seven dedicated breast radiologists and three radiology residents independently evaluated a cohort of 199 patients, referred from the breast cancer screening programme, who underwent both FFDM and CEM exams.

Case selection and image analysis

For the present study, a total of fourteen cases with missed breast cancer lesions were collected from the study of Lalji et al. 16. All these cases consisted of unifocal breast cancer, but were missed on FFDM by at least one of the (expert) readers from the previous study. The ipsilateral images of both FFDM and CEM with best visualisation of the breast cancer lesion (either cranio-caudal view (CC) or mediolateral oblique view (MLO)) were collected and prepared as online test on commercially available online software system Qualtrics® (Qualtrics, Washington, District of Columbia, USA). A single view image of FFDM was followed by the same view (CC or MLO) on the recombined image of the CEM exam. The online test demonstrated the FFDM images first, followed by the recombined CEM images. A region of interest of the primary tumour on each FFDM and CEM image was drawn in consensus by a radiology resident (T.v.N.) and an expert breast radiologist with eight years of dedicated breast imaging expertise (M.L.) (Figure 1).

Figure 1

Case number 2 represented a patient with a 13 mm large invasive lobular carcinoma in her right breast. The red arrow demonstrates the breast cancer lesion, which is presented on the original FFDM and FFDM including heat maps (representation of student evaluations demonstrated in colours) (Figure 1a). Figure 1b demonstrates the corresponding recombined CEM image. Abbreviations: FFDM full-field digital mammography, CEM contrast-enhanced spectral mammography.

Participants

The participants in this study consisted of students that were involved in a six-year pre-university education, which is comparable to obtaining a General Certificate of Education A-level in the United Kingdom 18. All students were in their first or second year of education.

Reading instructions

First, a short introduction on breast cancer and CEM was presented. This introduction consisted of a brief overview of pathology, prognosis and treatment options of breast cancer (surgery, systemic therapy and radiotherapy), followed by a description of FFDM and CEM. For the online test, students were instructed to click on the most suspicious breast lesion on FFDM, according to their own interpretation, and subsequently click on the most suspicious (i.e. whitest coloured) breast lesion on the recombined CEM image within the breast.

Students were unaware of any clinical patient information. The online test was constructed to force one click per image before continuing to the next page. Students were unable to review previous pages of the test. Furthermore, students received no feedback on their performance while scoring. All fourteen cases were finished within one session. Total time of the test per student was automatically calculated by the online test.

Statistics

Data were analysed using Statistical Package for the Social Sciences software (version 24.0, Armonk, New York, USA). Sensitivity was calculated as the number of correctly identified breast cancer lesions (true positives) per case divided by the total number of students per case (true positives + false negatives). Variance of the sensitivity for all students was adjusted with the variance inflation factor (VIF). Adjusted 95% confidence intervals (CI) were calculated using an excel spreadsheet provided by Genders et al. 19. Difference in mean number of true positives between FFDM and recombined CEM exams evaluated by students was calculated by using paired samples T-test. Difference in mean number of true positives between recombined CEM exams evaluated by students and FFDM exams evaluated by radiologists was calculated by using independent samples T-test. Statistical significance was considered P-value < 0.05.

Results

General patient characteristics

Mean age of the fourteen included patients was 58 years (range 49 - 68 years). Most cases concerned invasive carcinoma NST (no special type, 71%), ER/PR + (estrogen/progesterone positive) and HER2- (Human Epidermal growth factor Receptor 2 negative) tumours (79%) and grade 1-2 tumours (86%). Mean pathologic tumour size was 13 mm (range 5 - 40 mm) (Table 1).

Table 1

Overview of general patient characteristics

Mean age (years) (range)	57.6 (49-68)
Histopathology (%)
Invasive carcinoma NST	10 (71.4)
ILC	3 (21.4)
DCIS	1 (7.2)
Receptor status (%)
ER/PR+, HER2-	11 (78.6)
Triple negative	2 (14.3)
Missing	1
Tumour grade (%)
1-2	12 (85.8)
3	1 (7.1)
Missing	1
Mean pathologic tumour size (mm) (range)	13 (5 - 40)

Abbreviations: Invasive carcinoma NST invasive carcinoma of no special type, ILC invasive lobular carcinoma, DCIS ductal carcinoma in situ, ER estrogen receptor, PR progesterone receptor, HER2 Human Epidermal growth factor Receptor

General student characteristics

A total of 134 high school students, with a mean age of 12.8 years (range 10-14 years), participated in this study. Male to female ratio was nearly 2:1 (64.9% versus 35.1%). Most students were in their first year of education (65.7%) (Table 2).

Table 2

Overview of high school students (n=134)

Age (years) (%)
10	1 (0.7)
11	5 (3.7)
12	44 (32.8)
13	54 (40.3)
14	30 (22.4)
Male (%)	87 (64.9)
High school class (%)
First year students	88 (65.7)
Second year students	46 (34.3)

Reading performance

Mean overall reading time of all students was 5.25 minutes (range 2.57 minutes - 14.60 minutes). Based on FFDM, overall mean sensitivity of students was 29.2% (18.9 - 39.6%). Using the recombined image of the CEM exam, mean sensitivity improved to 82.6% (74.0 - 91.2%) (p<0.001) (Table 3). Mean overall sensitivity of the radiologists on FFDM was 75.7% (64.2 - 87.3%), which was lower compared to student's evaluations on recombined CEM exams, yet not statistically significant (p=0.098).

Table 3

Overview of sensitivity for FFDM and CEM findings of high school students and radiologists

Case number	Sensitivity FFDM students (%) (95% CI) [n]	Sensitivity CEM students (%) (95% CI) [n]	Sensitivity FFDM radiologists (%) (95% CI) [n]
1	1.5 [0.2 - 5.8][2/134]	7.5 (3.8 - 13.6)[10/134]	60.0 (27.4 - 86.3)[6/10]
2	0 (0 - 3.5)[0/134]	95.5 (90.1 - 98.2)[128/134]	80.0 (44.2 - 96.5)[8/10]
3	3.0 (1.0 - 7.9)[4/134]	94.0 (88.2 - 97.2)[126/134]	80.0 (44.2 - 96.5)[8/10]
4	88.8 (81.9 - 93.4)[119/134]	36.6 (28.5 - 45.4)[49/134]	80.0 (44.2 - 96.5)[8/10]
5	3.7 (1.4 - 8.9)[5/134]	84.3 (76.8 - 89.8)[113/134]	40.0 (13.7 - 72.6)[4/10]
6	97.0 (92.1 - 99.0)[130/134]	100 (96.5 - 100)[134/134]	80.0 (44.2 - 96.5)[8/10]
7	3.0 (1.0 - 7.9)[4/134]	90.3 (83.7 - 94.5)[121/134]	50.0 (20.1 - 79.9)[5/10]
8	1.5 (0.3 - 5.8)[2/134]	94.0 (88.2 - 97.2)[126/134]	60.0 (27.4 - 86.3)[6/10]
9	8.2 (4.4 - 14.6)[11/134]	91.0 (84.6 - 95.1)[122/134]	90.0 (54.1 - 99.5)[9/10]
10	85.1 (77.6 - 90.4)[114/134]	97.8 (93.1 - 99.4)[131/134]	90.0 (54.1 - 99.5)[9/10]
11	15.7 (10.2 - 23.2)[21/134]	99.3 (95.3 - 100)[133/134]	90.0 (54.1 - 99.5)[9/10]
12	3.7 (1.4 - 8.9)[5/134]	98.5 (94.2 - 99.7)[132/134]	90.0 (54.1 - 99.5)[9/10]
13	3.7 (1.4 - 8.9)[5/134]	70.9 (62.3 - 78.3)[95/134]	90.0 (54.1 - 99.5)[9/10]
14	94.0 (88.2 - 97.2)[126/134]	97.0 (92.1 - 99.0)[130/134]	80.0 (44.2 - 96.5)[8/10]
Mean	29.2 (18.9 - 39.6)	82.6 (74.0 - 91.2)	75.7 (64.2 - 87.3)

Abbreviations: FFDM full-field digital mammography, CEM contrast-enhanced mammography, CI confidence intervals

To illustrate, evaluations of students regarding case number 2 are demonstrated in Figure 1. On FFDM, none of the students correctly identified the breast cancer lesion. When the recombined CEM image was shown, 95.5% of the students correctly identified the breast cancer lesion.

Figure 2 demonstrates an overview of the student evaluations concerning case number 4. On FFDM, mean sensitivity of the students was 88.8% (81.9 - 93.4%). According to the recombined CEM exams sensitivity decreased to 36.6% (28.5 - 45.4%), most false-negative results were caused by the calcification in the middle of the breast or the nipple.

Figure 2

Case number 4 represented a patient with a 5 mm large invasive lobular carcinoma in her right breast. The red arrow demonstrates the breast cancer lesion on the original FFDM and FFDM including heat maps (representation of student evaluations demonstrated in colours) (Figure 2a). Figure 2b demonstrates the corresponding recombined CEM image, including heat maps. Abbreviations: FFDM full-field digital mammography, CEM contrast-enhanced spectral mammography.

Discussion

This study investigated the diagnostic performance of non-experienced high school students in detecting breast cancer lesions with FFDM and CEM exams, in a cohort consisting of patients with missed breast cancer lesions according to FFDM evaluations previously performed by breast radiologists. According to our study, mean overall sensitivity of the students was 29% on FFDM and increased to 83% when CEM was used. In addition, sensitivity of CEM seemed slightly higher when compared to FFDM evaluations by breast radiologists, yet not statistically significant (p=0.098). Our results could underline a future potential for automated image-based feature analysis in CEM.

FFDM continues to play an important role in breast cancer imaging, not only for the detection of cancers, but also for evaluation of the extent of disease 20. However, appropriate interpretation of FFDM requires many years of experience and even after years of experience a large inter-observer variability among radiologists is observed with a sensitivity ranging from 55-82% 21. Hence, learning to accurately read FFDM exams is time consuming and requires dedicated training to develop expertise. Previous studies have shown that the learning curve of CEM as opposed to FFDM is almost non-existent, as radiologists both experienced and non-experienced in CEM readings achieved comparable diagnostic performances without any prior dedicated training in CEM reading, with an improved sensitivity of ≥90.3% on CEM compared to ≥59.7% on FFDM 15; 16. Simplification of reading mammograms, as is the case when CEM is performed, could help to train non-experienced physicians (such as surgeons, medical oncologists and radiotherapists). Our study emphasized the ease with which CEM exams can be interpreted, as even high school students without any knowledge of medical imaging are able to interpret these images.

Enhancement of breast cancer lesions on CEM is achieved by administration of a contrast agent in the circulation. The principle relies on the development of new microvessels, which is required to attain tumour growth. These microvessels are often rapidly formed and consequently may lead to 'leaky' basement membranes. This phenomenon enables a contrast agent, administered in the circulation, to leak through these vessels into the tumour itself, resulting in enhancement 22. Several imaging techniques, like CEM, use this phenomenon to improve visualisation of the tumour. In CEM, two minutes after administration of an iodine contrast agent a set of low and high-energy images are performed. Acquisition of low energy images are obtained at peak kilovoltage (kVp) values ranging from 26-31 kVp, which is below the k-edge of iodine at 33.2 keV. High enery images, at kVp values ranging from 45-49 (i.e. above the k-edge of iodine), are obtained to visualise areas with high concentration of the iodine contrast agent. Yet the high energy images are unable to use for radiologic evaluation. Therefore, recombined CEM exams, consisting of subtraction of low and high-energy mammography images, are performed to visualise areas of contrast enhancement while anatomic noise in the image is neutralized 11; 22.

Besides CEM, contrast-enhanced MRI is another imaging technique which uses the same principle of contrast enhancement to visualise breast cancer lesions 13; 23; 24. Yet, contrast-enhanced MRI is time consuming, more expensive and less favourable according to patient opinion as opposed to CEM 25; 26. Furthermore, a recent study of Li. et al demonstrated an equal performance of CEM compared to breast MRI for the detection of breast cancer, with a sensitivity of 100% in both techniques 27. Therefore, CEM might be able to replace MRI for several indications in near future 28.

The current study had several limitations. First, the cohort of patients consisted of breast cancer patients only. In contrast to the cohort of patients which were previously evaluated by radiologists in the study of Lalji et al., with a prevalence of approximately 30% of cases with breast cancer 16. However, the focus of this study was to investigate the ability to detect breast cancer lesions by non-experienced readers, in contrast to investigate a cohort of patients who underwent FFDM and CEM exams.

Second, students assessed FFDM and recombined CEM exams in one direction only (either CC or MLO view), in contrast to clinical daily practice which consists of two views per breast. In addition, students were focussed to detect enhancing lesions rather than detecting breast cancer on CEM, which could potentially be a confounding factor. Yet, the mean sensitivity of 83% already indicates the important added value of CEM in non-experienced readers, which could potentially even further increase when both views of FFDM and CEM exams were shown or when readers are used to have at least some kind of medical training, albeit not radiological.

Third, the cohort of cases in this study consisted of patients who had unifocal breast cancer. This is an important limitation, since radiologists do not focus only on the detection of breast cancer, but also the extent of breast cancer disease in order to recommend the most optimal surgical treatment plan 29; 30. Therefore, results of this study should be interpreted with this important limitation in mind. Future research should further explore reading performance regarding assessment of extent of breast cancer disease.

In our study, high school students, without any medical imaging experience, showed a substantial mean sensitivity (i.e. higher than 80%) for detecting breast cancer. Consequently, this could create possibilities for automated image-based feature analysis like radiomics in future to further improve breast cancer detection rate, for instance in cases of dense breasts on FFDM, which decreases breast cancer detection rate, or BIRADS 0 according to FFDM 17.

In conclusion, contrast-enhanced spectral mammography, evaluated by high school students, might improve the detection of breast cancer when compared to full-field digital mammography evaluations of radiologists. These observations emphasize the ease with which CEM can be interpreted when compared to FFDM, which potentially allows an important role for automated image-based feature analysis.