Computer aided detection (CAD): an overview.

Computer aided detection (CAD) is a technology designed to decrease observational oversights--and thus the false negative rates--of physicians interpreting medical images. Prospective clinical studies have demonstrated an increase in breast cancer detection with CAD assistance. This overview briefly describes the metrics that have been used to define CAD system performance. Copyright International Cancer Imaging Society.

Introduction

The use of computers to help radiologists in the acquisition (e.g. CT, MRI, US, computed radiography), management and storage (PACS), and reporting (RIS) of medical images is well established. More recently, computer programs have been developed and approved for use in clinical practice that aid radiologists in detecting potential abnormalities on diagnostic radiology exams. This application has been termed computer aided (or assisted) detection, commonly referred to as CAD.

As used in this overview, the term ‘computer-aided detection’ refers to pattern recognition software that identifies suspicious features on the image and brings them to the attention of the radiologist, in order to decrease false negative readings. As currently used, the radiologist first reviews the exam, then activates the CAD software and re-evaluates the CAD-marked areas of concern before issuing the final report. CAD is currently FDA and CE approved for use with both film and digital mammography, for both screening and diagnostic exams; for chest CT; and, for chest radiographs.

This is different from the concept of ‘computer aided diagnosis’ (which is also called CAD), which refers to software that analyses a radiographic finding to estimate the likelihood that the feature represents a specific disease process (e.g. benign versus malignant). To my knowledge, this technology has not yet been approved for clinical use.

Background

In the simplest of terms, the practice of radiology consists of (a) looking at an image (visual perception) and then (b) interpreting what is seen (cognition) [1]. Numerous studies have shown that radiographic abnormalities that are clearly present on an image are, at times, not reported. To address this, strategies such as double reading have been selectively used, such as in screening mammography, which yield an increase in the cancer detection rate. This, however, is labor intensive and is thus not widely used, except when mandated by medical or government agencies.

The primary goal of CAD is to increase the detection of disease by reducing the false negative rate due to observational oversights. The use of a computer rather than a second human observer has the advantage of not increasing the demands on the radiologist (or trained observer) pool. An important aspect of either approach is to increase disease detection without an undo impact on the recall and work up rates. Finally, in some applications CAD, with its associated automated software tools, has the potential to provide workflow efficiencies. This latter application is beyond the scope of this overview.

CAD algorithms are developed to search for the same features that a radiologist looks for during case review. Thus, for breast cancer on mammograms, the CAD algorithms search for microcalcifications and masses (both spiculated and non spiculated, architectural distortions and asymmetries). On chest radiographs and CT scans, current CAD applications search for pulmonary densities that possess certain physical characteristics, e.g. sphericity, that might represent lung nodules.

Not surprisingly, CAD algorithms will mark features that meet the algorithm requirements, but which do not represent findings that the radiologist considers to warrant further investigation, i.e. false CAD marks. Also, at times a true positive CAD mark, upon review by the radiologist, is dismissed as not warranting further investigation. In this instance, the false negative report would be the result of an interpretive—rather than a perception—error.

Clinical implementation of CAD

The CAD algorithms require a digital data set of the image for analysis. If the image is acquired on x-ray film, such as a film-screen mammogram, the analog image must first be digitized. However, the CAD algorithms can directly analyze images acquired in digital format, such as with digital mammography (FFDM) and CT.

In current practice (and as required by the FDA), the exam should first be reviewed and interpreted in the usual fashion. Only then are the CAD marks displayed, following which the radiologist re-reviews those areas that are prompted by the CAD system. Two important principles must be adhered to:

Current CAD systems do not mark all actionable findings. Therefore, the absence of a CAD mark on a finding the radiologist was concerned about on his/her pre CAD review must not deter further evaluation.

Current CAD systems generate many more false CAD marks than true CAD marks. Therefore, it is the responsibility of the radiologist to determine if a CAD mark warrants further evaluation.

How to evaluate a CAD system

A CAD system can be evaluated in several ways, which includes analysis of data generated in a laboratory or test setting, and by the impact of CAD on radiologist performance in an actual clinical practice setting.

‘Stand alone’ sensitivity and specificity

This information can be obtained by observing the performance of a CAD system on a set of ‘truth’ cases. Truth is generally established by histological verification of the presence (e.g. cancer) or absence (e.g. clinical follow-up) of disease. Sensitivity is determined by the percentage of positive cases in which the CAD system places a mark on the disease location. The number of false CAD marks per normal image or case is commonly used as a surrogate for specificity.

The results of this exercise are, of course, dependent on the case collection. Bias, intended or not, in collecting positive cases that have more conspicuous findings will result in apparent superior CAD performance compared to cases that are less conspicuous, even though the CAD algorithm is the same. Thus, the same CAD algorithm will demonstrate varying sensitivities and specificities (false marks per case) depending on the case composition.

A preferred method to compare CAD systems is to determine the sensitivity and false marker rates on the same set of ‘truth’ cases. These cases must be ‘unknown’ to the CAD system, that is, they should not have been used to train the CAD algorithms. Sufficient (and often large) numbers of cases will be needed in order to establish statistical significance of superiority or equivalence in performance when comparing CAD systems.

‘Laboratory’ studies of potential detection improvement

These studies recruit radiologists (or other ‘readers’) to evaluate a set of ‘truth’ cases to determine the sensitivity and call back rate of the unaided reader (pre-CAD) with that of the reader with CAD assistance. Such studies are useful to assess the potential benefit of CAD and provide estimates of expected changes in disease detection and workup/recall rates. However, the test setting often compromises the performance of the reader, in that the reader may either over or under call the reviewed cases in a test environment.

Actual clinical practice experience

In many ways, these results might be considered the best assessment of a CAD system, in that they evaluate the contribution (or lack thereof) of CAD in an actual clinical practice setting. In this situation, the impact of CAD on (a) the detection of disease and (b) the recall/work up rate is determined. These data, of course, are only reflective of that particular clinical practice; however, as with all such clinical research, the cumulative reports from different practices should demonstrate a trend on the value (or lack thereof) of the introduced intervention, i.e. CAD.

This can be done in a ‘sequential read’ clinical trial, in which the exam is first read prior to, and then following, CAD input [2-5]. The change in disease detection due to the CAD input, as well as the change in the recall/workup rates, will determine the contribution of CAD to patient management. Importantly, the percentage increase in disease detection should be concordant with, or less than, the percentage increase in the recall/workup rates.

Another approach is the ‘historical control’ trial, in which the percentage change in disease detection and recall/workup rates is determined by comparing data before and after the implementation of CAD into a clinical practice [6, 7]. These data are also useful, but changes in the patient demographics and practice patterns might account for changes in the pre-CAD and CAD periods that are independent of the introduction of CAD. When reporting results of such studies, care must be taken to consider those variables that might independently affect the disease detection rate [8].

Summary

Computer aided detection (CAD) is a clinically proven technology that increases the detection of breast cancer by assisting the radiologist in decreasing observational oversights (i.e. decreasing the false negative rate). The more recent clinical introduction of CAD to assist radiologists in the detection of actionable lung nodules will likely be followed by the development, clinical trials validation, regulatory approval and commercialization of a variety of CAD applications in diagnostic imaging.