Role of Computer Aided Diagnosis (CAD) in the detection of pulmonary nodules on 64 row multi detector computed tomography.

AIMS AND
OBJECTIVES: To determine the overall performance of an existing CAD algorithm with thin-section computed tomography (CT) in the detection of pulmonary nodules and to evaluate detection sensitivity at a varying range of nodule density, size, and location.
MATERIALS AND METHODS: A cross-sectional prospective study was conducted on 20 patients with 322 suspected nodules who underwent diagnostic chest imaging using 64-row multi-detector CT. The examinations were evaluated on reconstructed images of 1.4 mm thickness and 0.7 mm interval. Detection of pulmonary nodules, initially by a radiologist of 2 years experience (RAD) and later by CAD lung nodule software was assessed. Then, CAD nodule candidates were accepted or rejected accordingly. Detected nodules were classified based on their size, density, and location. The performance of the RAD and CAD system was compared with the gold standard that is true nodules confirmed by consensus of senior RAD and CAD together. The overall sensitivity and false-positive (FP) rate of CAD software was calculated. OBSERVATIONS AND
RESULTS: Of the 322 suspected nodules, 221 were classified as true nodules on the consensus of senior RAD and CAD together. Of the true nodules, the RAD detected 206 (93.2%) and 202 (91.4%) by the CAD. CAD and RAD together picked up more number of nodules than either CAD or RAD alone. Overall sensitivity for nodule detection with the CAD program was 91.4%, and FP detection per patient was 5.5%. The CAD showed comparatively higher sensitivity for nodules of size 4-10 mm (93.4%) and nodules in hilar (100%) and central (96.5%) location when compared to RAD's performance.
CONCLUSION: CAD performance was high in detecting pulmonary nodules including the small size and low-density nodules. CAD even with relatively high FP rate, assists and improves RAD's performance as a second reader, especially for nodules located in the central and hilar region and for small nodules by saving RADs time.

INTRODUCTION

Computer-aided detection (CAD) system is one of the several approaches proposed to improve lung nodule detection. CAD systems are improving year by year and reached high-sensitivity levels at moderate false-positive (FP) rates. CAD algorithm depends on computing a “CAD score,” which corresponds to the probability that a given density is a lesion.[1] The sensitivity and specificity of CAD systems vary widely relative to the diversity of algorithms, computed tomography (CT) input, and varying populations of nodules. Improving CAD sensitivity proportionately increases the FP rates.[2]

CAD has been developed primarily to serve as a second reader, providing improved radiologist (RAD) sensitivity for nodule detection.[2] At present, a good indication for applying CAD is seen in the use of low-dose CT during lung cancer screening. In addition, CAD can also be applied to various oncological settings for identifying metastatic lung nodules as well as monitoring the therapy.[34567] An important feature of CAD is its ability to calculate size, volume, and density of nodules in one click, which is extremely important in screening lung cancer and also in assessing prognosis and response to therapy in metastatic workup cases. Thus, it saves lot of RAD's time, especially junior RADs of less experience; CAD increases RAD sensitivity.

The purpose of this study was to assess the performance of CAD system for nodule detection on thin section multi-detector row CT (MDCT) images of the thorax, by determining the overall detection rate and FPs and also to assess the nodule detection rate at various location, size, and density. Thereby, finding out the effect of CAD, as the second reader, on RAD performance.

MATERIALS AND METHODS

A cross-sectional prospective study was conducted over 2 years period after obtaining approval by the Institutional Ethics Committee. Written informed consent of the patients was obtained. The study subjects included 20 adult patients of either sex with pulmonary nodules of infective (4) or neoplastic (6) etiology and follow-up cases of metastasis (10). To achieve nodules with a wide variety of different morphologic features, CT studies having <40 nodules per patient were included. Patients with diffuse interstitial lung disease, extensive scarring, pneumonia, fibrosis, edema, and images with gross motion blur were excluded.

Computed tomography technique

Routine diagnostic chest CT studies were performed on a 64-slice MDCT scanner. Scanning was performed from the lung apices to the upper abdomen. Acquisition parameters were identical in all patients (high-speed mode; 1.5:1 pitch; table speed, 15 mm/s; 1.25 mm collimation, 120 kVp, 380 mAs, matrix 512 × 512). As we have taken referred cases, all patients underwent contrast study and reconstructed contrast series were loaded for CAD assessment.

Thin sections with a width of 1.4 mm were reconstructed with an increment of 0.7 mm by using a high-resolution reconstruction algorithm (window level −650 Hounsfield Unit [HU]; window width 1500 HU) and reformatted into coronal and sagittal planes for the evaluation by a RAD.

The acquired data were loaded onto a separate workstation with dedicated CT lung nodule assessment (LNA) software. Actual CAD requirement was minimum of 3 mm thickness with 50% overlap. As we wanted to study CAD performance with thinner section thickness, we used 1.4 mm thickness images reconstructed in a standard window. CAD software converted the images into maximum intensity projection (MIP) of 5 mm thickness with 0.7 mm increment. It also provided coronal and sagittal planes along with a three-dimensional image of selected nodule in 1 + 5 layout display.

Radiologist and computer-aided detection performance

Two radiologists of 2 yrs (RAD/radiologist) and 7 yrs of experince reviewed the thin section series separately, marked and evaluated nodules using nodule detection tools. Detection of pulmonary nodules was done initially by a radiologist and later by CAD lung nodule software was assessed. All the suspicious foci were confirmed either by viewing coronal or sagittal planes. Time taken for each case was documented. Once manual nodule detection and assessment were completed, the RAD then implemented the Lung Nodule CAD option on a different day to avoid any sort of bias. The CAD server accepts and analyzes the scans by segmenting the lung parenchyma and other bridging techniques. Results of the CAD were listed as candidates for sequential review and displayed with a marked region of interest on the CT image. This facilitated the review and then acceptance/rejection of the candidates by the RAD.

Nodules added by the RAD were circled in blue. CAD suspected nodule candidates were circled in pink. When a pink-circled “nodule” was selected, the appropriate axial, coronal slice and also the three-dimensional image of suspected nodule, colored violet will appear. The three-dimensional image can be rotated and otherwise modified for the RAD to decide if it was a true nodule or not. The size, volume, and density of the accepted and also FP nodules were displayed at the bottom of the monitor after selecting it [Figures 1 and 2].

Figure 1

Overall layout of the computer-aided detection application. Above nodules are identified by the computer-aided detection software displayed in axial, coronal, and sagittal planes. Below volume rendered display of the nodule and table showing the characteristics of selected nodule. Maximum intensity projection image showing all the nodules that are identified by computer-aided detection (yellow circle) and confirmed by radiologist (blue circle)

Figure 2

(a) Axial computed tomography section showing a solid true nodule in subpleural location of the right lower lobe; (b) volume rendered image and (c) lesion summary table of the same nodule

The lung nodule computer-aided diagnosis software

Used in this study was a part of LNA application in the Extended Brilliance Workspace postprocessing system and portal system. Automated nodule detection was executed through two major steps, i.e., initial nodule identification through multiple gray level thresholding and morphological revision of the segmented lung regions to include juxtapleural nodules.[89]

The consensus of senior RAD of 7 years’ experience and CAD results together was considered as gold standard for true nodules and sensitivity of CAD alone and CAD along with RAD was determined.

Nodule description

The suspected nodules detected by CAD were divided into four groups: True-positive (TP) referred to the true nodules detected by CAD or RAD and confirmed by the gold standard. False-negative (FN) referred to the true nodules missed by CAD/RAD but detected by the gold standard and FP referred to the structures detected by CAD as a “nodule” but rejected by the gold standard. A locus was considered as FPs after viewing coronal and three-dimensional image, only when could not be solved on axial MIP images. Time taken for overall CAD performance and time taken to accept or reject nodules was documented. Reason and location of FPs and negatives were assessed.

The location of the true nodule was classified as subpleural, peripheral, hilar, and central. A subpleural nodule was the one which had pleural contact. A peripheral nodule was the one within 2 cm of, but not touching the pleura. A hilar nodule was the one within 2 cm of the hilum. A central nodule was the one situated between the peripheral and hilar zones.[1011]

The nodules were separated into the following groups by diameter: Less than 4 mm, ≥4 mm but smaller than or equal to 10 mm, >10 mm but smaller than or equal to 30 mm and >30 mm.

We also classified them into “solid” pulmonary nodules and “nonsolid” pulmonary nodules using the peak HU of -100, which was automatically calculated by CAD on selecting the particular nodule.[1213]

Statistical analysis

All analysis was performed using SPSS (Statistical Package for the Social Sciences) software version 15. The performance of CAD system was evaluated in terms of additional nodules detected, number of FPs and FNs per CT study, reason for FPs and FNs. Sensitivity and positive predictive value of CAD were calculated. The influence of nodule size, location, and density on CAD sensitivity was also analyzed. The significance of these findings were analyzed by using Wilcoxon signed rank test, and the P values were calculated (P < 0.001 indicates significance).

OBSERVATIONS AND RESULTS

The twenty patients included in the study were in the age group of 38–72 years, with mean age 56.7 years and there were 18 males and 2 females.

Of the 322 “suspected nodules” detected by either CAD or RAD, 221 were finally scored as true nodules [Table 1] by consensus of CAD and senior RAD. 206 (93.2%) of 221 true nodules were detected by the RAD and 202 (91.4%) were detected by CAD. There were 15 (6.7%) true nodules, which would have been missed by the RAD without using CAD system. CAD missed 19 (8.5%) true nodules. A total number of density detected by CAD in 20 patients was 303, which included both true and false nodules. About 101 (33.3%) nodules detected by CAD were rejected by the RAD as FPs; the FP rate was 5 per patient (range: 2-11).

Table 1

Overall nodule detection performance by radiologist and computer aided diagnosis (n=322)

Per patient distribution and size-wise distribution of true nodules are shown in Figures 1 and 2. 149 (67.7%) of 221 true nodules were in the size range of 4-10 mm, 43 (19.5%) of 221true nodules were <4.0 mm.

Average time taken by RAD was 5 min 48 s per case and by CAD was 3 min 15 s after rejecting FPs and also with all the details of each nodule, which was statistically significant with P < 0.001.

Number of FNs were 19 for CAD, which were confirmed as true nodules by RAD before applying CAD protocol. Thus indicating the miss rate of CAD was 1 per patient which cannot be called as statistically significant. Of these 19 nodules, 17 were in the periphery of lung and 15 were <10 mm in size.

Overall sensitivity

CAD alone detected a total of 202 true nodules, giving an overall sensitivity and positive predictive value of 91.4% and 66.7%, respectively.

Assessment of nodule detection performance

Detection performance of CAD and RAD for the true nodules was assessed with respect to nodule's size, location, and density.

Nodule detection sensitivity with respect to size of nodules

The majority (67.7%) of true nodules were in the range of 4-10 mm, most of which were detected effectively by CAD with a sensitivity of 94.1% [Table 2]. Among the 141 nodules of size 4-10 mm detected by CAD, 12 nodules were missed during initial evaluation by RAD. CAD detected 36 of 43 nodules, which were <4.0 mm, giving a sensitivity of 82.5%. Among them, 3 were missed by RAD initially. CAD detected 22 nodules of 11-30 mm and missed 4 nodules, which were detected by RAD. Despite size criteria, 3 of the 4 nodules missed by CAD showed peak HU ≤−100 HU (nonsolid nodules). Applying the Wilcoxon signed rank test, these findings were found to be significant (P < 0.001).

Table 2

Comparison of nodule detection performance by computer-aided diagnosis and radiologist with respect to size of nodules (n=221)

Nodule detection sensitivity with respect to location of pulmonary nodules

We found, the detection performance of CAD and RAD varied according to the location of true nodules [Table 3]. The overall sensitivity of CAD alone for peripheral and subpleural nodule was 90.6% which was very high. However, CAD system showed better sensitivity for hilar and central nodules when compared to RAD's sensitivity. Applying the Wilcoxon signed rank test, these findings were found to be significant (P < 0.001). CAD missed 12 peripheral and 5 subpleural nodules, among these, 7 nodules were <4.0 mm. 3 nodules had lower density (peak HU <−100 HU). 5 nodules had contact with pleura or broncho vascular structure, which would have resulted in segmentation failure. There was no explanation for the other 2 nodules that were missed [Figure 3].

Table 3

Nodule detection performance by computer aided diagnosis and radiologist in different location (n=221)

Figure 3

(a) Axial and (b) Volume rendered image of lesion 6 in subpleural location. (c) coronal computed tomography sections showing computer-aided detection detected true peripheral nodule (lesion 6), true subpleural nodule (lesion 4) and false positive (lesion 5). Lesion 5 identified by computer-aided detection, is found to be focal fibrotic thickening on coronal images

Nodule detection sensitivity with respect to density of nodules

CAD system detected 196 of 212 solid nodules and the RAD identified 198 of 212 solid nodules. The detection sensitivity of CAD for solid pulmonary nodule was 90.5% [Table 4]. There were 14 solid nodules found only by CAD and missed by RAD. However, CAD missed 16 of 212 solid nodules. Applying the Wilcoxon signed rank test, these findings were found to be significant (P < 0.001). CAD detected 6 of the 9 nonsolid nodules while the RAD detected 8 of the 9 nonsolid nodules. The detection sensitivity of CAD for nonsolid nodules was 62.5% [Figure 4].

Table 4

Comparison of nodule density and detection performance by computer-aided diagnosis and radiologist (n=221)

Figure 4

Axial CT section showing two nodules (CAD 2 and CAD 4) detected by Computer aided diagnosis, of which nodule labelled CAD 2 is non solid with Mean HU of-601 and Peak HU of-27 as shown in summary table

False positive rate

101 (33.3%) of 303 suspicious nodules detected by CAD were rejected by the RAD as an FPs. The rate of FPs per patient was 5 (range: 2-11). Reasons for FPs included broncho vascular structure, 66 (65%); fibrotic scar, 17 (17%); pleural thickening, 1 (1%) and other causes such as subtle respiratory/cardiac motion artifacts, 17 (17%) [Figure 5]. These FPs-nonnodules were more predominantly seen in central (23.8%) and hilar (48.5%) location, as compared to peripheral (21.8%) and subpleural (5.9%) areas. Of 101, 73 were central and hilar location. 56 of these were due to broncho vascular markings picked up as nodule by CAD. Out of 101 FP, 74 were of >5 mm in size (50 were 5-10 mm and 24 were >10 mm). Because of large size and predominance in central or hilar location, it was easy for RAD to decide them as nonnodule and thus the time taken for rejecting these nonnodules was in the range of 60-70 s per patient.

Figure 5

False positive by computer-aided detection in two different cases, first case with (a) volume rendering and (b) axial computed tomography section showing nonnodule (false positive) broncho vascular structure near the right hilum is falsely identified as nodule by computer-aided detection. Another case (c) and (d) axial computed tomography sections showing the artifacts caused by respiratory/cardiac motion, being falsely identified as nodules by computer-aided detection

DISCUSSION

CT has been the main imaging modality for detection, characterization, and follow-up of pulmonary nodules, which are major radiologic findings of varied benign and neoplastic etiology.[567] CAD has been proposed as a solution for interpretation of the ever-expanding amount of radiologic information as early as in 1989. Since then, the majority of CAD for pulmonary nodules have been designed for and tested on conventional CT with a slice thickness of 5–10 mm.[141516]

As emphasized in recent past,[6151718] we evaluated the CAD scheme for automated detection of pulmonary nodules on thin section MDCT images. The major advantages were three-dimensional display of any of the nodule candidate which was very useful in resolving uncertainties between pulmonary nodules and vessels, and the use of thin slice allowed the direct utilization of the HU values; hence, partial volume effect was eliminated. A wide variation exists in the detection sensitivity of CAD for pulmonary nodules in previously published studies ranging from 38% to 100%.[6891017]

Because of the differences in CT technique, CAD algorithm, type, and number of nodules in the selected patients and the threshold of nodule size for CAD which can explain the wide range in sensitivity, a general comparison between our study and previous studies was not possible. The double reading, followed by CAD reading or multiple consensus will invariably pick up more nodules, thus reducing CAD sensitivity apparently.[19] Automated nodule detection by CAD includes two major steps, initially multiple gray level thresholding followed by morphological revision of the segmented lung regions. Then, analyzes three-dimensional features of nodule candidates through automated classifier. Because of this property sensitivity of CAD for subpleural nodule was high (90.9%). However, when compared to the RAD, the CAD sensitivity was less for the periphery of the lung. Three-dimensional shape information (sphericity feature, compactness, elongation factor, etc.) offered by our CAD system was essential since nodules were approximately spherical whereas bronchovascular structures were more tubular. It was an important feature in the detection of juxtavascular nodules and elimination of FPs (nonnodular structures).

The two principal advantages of MDCT data with thin section was first, the three-dimensional shape information of potential nodules and thus resolve ambiguities between pulmonary nodules and vessels. Second, the use of thin slices allows the direct utilization of the Hounsfield values due to the absence of the partial volume effect.

Comparison of nodule detection performance with respect to nodule size, location, and density

Earlier automated lung nodule detection systems were designed for use with thick section CT data and were tested primarily with nodules larger than 5 mm. Very few recently designed CAD systems described a shift from thick-to thin-section CT and from macronodule (>3.0 mm) to micronodule (≤3.0 mm) detection.

The Fleischner Society guidelines had stratified patients into low- and high-risk groups and had recommended that routine follow-up was not required for low-risk patients with very small nodules measuring 4 mm or less.[202122] However, in the case of known malignancy, nodules of size <5 mm might also be significant. Our CAD algorithm was designed to target nodules with diameter >4 mm and <30 mm. Further, lowering the size criteria increases the FPs and decreases the sensitivity, resulting in overall reduction of CAD performance. However, our CAD analyzed the nodular findings <4 mm size also with a sensitivity of 82.55% which is still significantly high. Similar trend was observed in previous studies.[101617] Thus, we inferred that CAD performance was influenced by nodule size, but because of its property of detecting small nodular findings, small size nodules are also picked up with relatively good sensitivity.

The difference in sensitivity was significant for hilar and subpleural nodules between CAD and RAD. The nodule detection performance can be influenced by nodule location and its relationship to surrounding anatomical structures. The RAD readily found nodules in peripheral and sub-pleural location even if they were small because there were no vessels of similar size and also good contrast between lung and nodules. In central lung regions, nodules were confused with blood vessels on axial sections. This is mainly because of more number of slices and busy schedule; RAD might overlook even bigger nodules. However, these nodules were detected in retrospective review after being detected on a CAD system. However, even with technical advancement CAD sensitivity was lower in peripheral and subpleural location because of nodules having pleural contact, the segmentation algorithm considers it as part of chest wall, rather than a nodule.[10112324] This may be one of the explanation for higher rate of FNs with CAD. In our study, majority of FN were located in peripheral or sub-pleural location. However, majority of FN were of <10 mm, this cannot be the explanation for higher FN rate as our CAD showed better sensitivity for picking up nodules of 4-10 mm size. The FN rate of CAD restricts its application as a standalone technique.[1011] Hence, further improvement in CAD segmentation algorithm might be the answer to reduce the CAD miss rate.

The detection sensitivity of CAD for solid pulmonary nodules was high even with older CAD algorithm.[10] Earlier CAD systems were specifically developed for solid nodules (peak HU: >−100 HU).[2526] However, the likelihood of malignancy for ground-glass or low density nodules was much higher than that for solid nodules especially in smoker and high-risk patients.[26] The ability of CAD to detect sub-solid nodules depends on setting of an attenuation range. With advancement in technology, present CAD systems can detect low density nodules;[1213] however, sensitivity was low in our study. Hence, one should be aware about this limitation of CAD and more attention needs to be given by the RAD in searching these low attenuation nodules. In this study, the percentage of nonsolid nodules was low (9 of 221 nodules) with detection sensitivity of 62.5%. However, since the number of sub-solid nodules was small, the CAD sensitivity cannot be matched or generalized further.

The considerable FP rate of 2-11 per CT for CAD in this study was similar to previous reports.[810171827] It is necessary for the RAD to inspect each suspected “nodule” before rejecting them. The majority of FPs were central and hilar in location, bigger size, and broncho vascular structures like end on vessel or branch points. With the availability of three-dimensional feature it was very easy to analyze them. Thus time taken for rejecting these FP were within a minute per case, which was not obviously significant. One more observation made in our study was artifacts also contributed for FP, especially in the basal segments of lung, however, easily eliminated as nonnodule on viewing single image. However, it is also true that, relatively high-rate of FP nodules, partially increases the time taken by CAD at least.

Many technological advances helped in bringing down the FP rate. However, considerable number of FPs are still known to occur. Hence, in spite of high FP rate, it was easier and less time taking for RAD to reject than search for a nodule. Overall time taken for each case by CAD assessment including processing time along with nodule details was significantly less than time taken by RAD.

CAD is already well established for second reading. However, we recommend applying CAD before initial reading by RAD. This can improve the RAD's performance and also saves time.

Limitations

There are few limitations in our study. First, the cases included were, only CT studies with nodules which might have biased the result of RADs. Second, RADs did not use thick MIP in detecting nodules which would apparently enhance CAD result.

CONCLUSION

The results of our study reinforces that CAD performance in pulmonary nodules, including the small size and low-density nodules are reaching high-sensitivity level and thus CAD assists RAD as a “second reader” by saving time. However, relatively significant FP rates indicate the need for further improvement in CAD software.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.