Inter-observer agreement of the Coronary Artery Disease Reporting and Data System (CAD-RADSTM) in patients with stable chest pain.

PURPOSE: To assess inter-observer variability of the Coronary Artery Disease - Reporting and Data System (CAD-RADS) for classifying the degree of coronary artery stenosis in patients with stable chest pain.
MATERIAL AND METHODS: A prospective study was conducted upon 96 patients with coronary artery disease, who underwent coronary computed tomography angiography (CTA). The images were classified using the CAD-RAD system according to the degree of stenosis, the presence of a modifier: graft (G), stent (S), vulnerable plaque (V), or non-diagnostic (n) and the associated coronary anomalies, and non-coronary cardiac and extra-cardiac findings. Image analysis was performed by two reviewers. Inter-observer agreement was assessed.
RESULTS: There was excellent inter-observer agreement for CAD-RADS (k = 0.862), at 88.5%. There was excellent agreement for CAD-RADS 0 (k = 1.0), CAD-RADS 1 (k = 0.92), CAD-RADS 3 (k = 0.808), CAD-RADS 4 (k = 0.826), and CAD-RADS 5 (k = 0.833) and good agreement for CAD-RADS 2 (k = 0.76). There was excellent agreement for modifier G (k = 1.0) and modifier S (k = 1.0), good agreement for modifier N (k = 0.79), and moderate agreement for modifier V (k = 0.59). There was excellent agreement for associated coronary artery anomalies (k = 0.845), non-coronary cardiac findings (k = 0.857), and extra-cardiac findings (k = 0.81).
CONCLUSIONS: There is inter-observer agreement of CAD-RADS in categorising the degree of coronary arteries stenosis, and the modifier of the system and associated cardiac and extra-cardiac findings.

Introduction

Coronary artery disease is the leading cause of morbidity and mortality throughout the world. Clinical assessment is important in preliminary assessment of chest pain and in determining whether it is due to stenotic coronary artery disease. The degree of coronary artery stenosis is an important issue for treatment planning, whether by medical therapy or by revascularisation [1-3]. Different imaging modalities are used for the evaluation of the coronary arteries. Echocardiography is used for the examination of patients with chest pain, but it cannot delineate the course of the coronary arteries [4, 5]. The clinical application of coronary magnetic resonance angiography is reduced by the limited temporal and spatial resolution, partial volume artefacts, reliance on multiple breath-holds, and poor visualisation of the entire coronary artery [6, 7]. Invasive coronary angiography is considered as the gold standard technique for evaluation of the coronary arteries. However, it is an invasive and expensive procedure, with some periprocedural morbidity and mortality [1-5].

Coronary computed tomography angiography (CTA) is the most common method for assessment of coronary arteries. Coronary CTA allows for fast and safe triage of patients. This procedure is a relatively fast and simple procedure that does not require arterial access or hospital admission. The images can be post-processed and reconstructed during different cardiac phases to allow retrospective selection of the phases with the fewest motion artefacts [8-12].

Different reporting systems in medical imaging have introduced standardised reporting linked with information guiding subsequent steps in patient management. BI-RADS standardised reporting of screening mammograms allows clinicians to interpret the clinical relevance of imaging findings and to take action. Other reporting systems for evaluating thyroid, prostate, and parotid glands have also been developed [13-16]. The Coronary Artery Disease Reporting and Data System (CAD-RADSTM) is a classification system suggested by the Society of Cardiovascular Computed Tomography, which is used for grading the severity of coronary arteries stenosis. CAD-RADS is a standardised reporting system for coronary CTA results on a per-patient basis. It is applicable to coronary CTA in patients with suspected or known coronary artery disease. The CAD-RADS classification is meant to be complementary to the final impression of the report, particularly because the report will provide specific information regarding the location and extent of coronary plaque and stenosis. It includes suggestions regarding further patient management [17-20].

The aim of this work is to assess the inter-observer variability of CAD-RADS for classifying the degree of coronary artery stenosis in patients with stable chest pain.

Material and methods

Patients

The institutional review board committee approved the study protocol, and all patients provided written, informed consent. The prospective study was conducted upon 102 patients clinically suspected to have coronary artery disease, who were referred for coronary CTA. We excluded six patients from the study: due atrial fibrillation in four patients and impaired renal function in two patients. The final patients included in this study comprised 96 patients (82 male, 14 female, age range 45-74 years, mean age 63 years) who underwent coronary CTA.

Computed tomography angiography

Coronary CTA was performed using a 128-detector CT scanner [Ingenuity Core TM, Philips, Netherlands]. A β-blocker (100 mg propranolol) was administrated orally one hour before the examination in patients with heart rate greater than 65 beats per minute, and no adverse reactions to β-blocker were recorded. Non-contrast scan was done for calcium scoring to decide if the patient was fit for a contrast scan. A bolus of contrast medium (Ultravist, 370 mg of iodine per millilitre) was injected into an antecubital vein at the rate of 5.5 m/s, followed by a 50-ml saline flush with a dual-barrel power injector. The amount of contrast medium was determined according to the patient’s body weight and scanning time. The bolus tracking technique was used with the region of interest (ROI) positioned in the ascending aorta. After injection of the contrast bolus, monitoring scans at the level of the ascending aorta were obtained and attenuation values within the ROI are measured. Once the attenuation value reached 190 HU, the patient was automatically instructed to perform a breath hold during which the scan was started. Retrospective electrocardiographically gated CTA was performed with collimation of 128 × 0.6 mm, pitch 0.6 mm, tube current 800 mA, and tube voltage 100 kV.

Post processing

The images were archived as Digital Imaging and Communications in Medicine images (DICOM) and transferred to an Ingenuity CoreTM, Philips, Netherlands; 2010 CT apparatus (128 slices) workstation. Axial, cross-sectional, curved planar reformation, multiplanar reformation, and three-dimensional maximum intensity projection (MIP) images were reconstructed.

Image analysis

Analysis of the CTA images was performed by two radiologists (R.M. and A.G.) with five and three years of coronary imaging, respectively, who were blinded to the other clinical findings. They independently reviewed the coronary CTA images and analysed the degree of stenosis of the left main coronary artery, the left circumflex artery, the left anterior descending artery, and the right coronary artery. The CAD-RADS classified results into: CAD-RADS 0 (absence of atherosclerosis), CAD-RADS 1 (1-24% – minimal stenosis or plaque with no stenosis), CAD-RADS 2 (25-49% – mild stenosis), CAD-RADS 3 (50-69% stenosis), CAD-RADS 4 (4A – 70-99% stenosis or 4B – left main more than 50% or 3-vessel obstructive more than70%), and CAD-RADS 5 (the presence of at least one total occlusion). The modifiers of CAD-RADS were: CAD-RADS-N non-diagnostic, CAD-RADS-V for vulnerable plaques, CAD-RAD-G for graft patency, and CAD-RAD-S for stents. Each graft was assessed for course, patency of the ostia, body, and anastomotic segments, and each stent was assessed for its site, patency, and distal filling. The associated coronary anomalies, non-coronary cardiac findings, and extra-cardiac findings were detected. Finally, the standardised reporting in terms of the CAD-RADS reporting and data system was used.

Statistical analysis

The statistical analysis of data was done using SPSS software (Statistical Package for Social Science version 20). The kappa coefficient (K) including 95% confidence interval (CI) with percentage agreement was calculated to estimate the level of agreement between both reviewers for the degree of stenosis of coronary arteries, dominance of the coronary artery, each CAD-RADS category, CAD-RADS modifiers and associated congenital coronary anomalies, and non-coronary cardiac and extracardiac lesions. The k values were interpreted as follows: k values between 0.00 and 0.20 represented poor; k values between 0.21 and 0.40 were interpreted as fair; k values between 0.41 and 0.60 were interpreted as moderate; k values between 0.61 and 0.80 were interpreted as good; and k values between 0.81 and 1.00 were interpreted as excellent. A p value less than 0.05 indicated a statistically significant difference.

Results

Table 1 shows the inter-observer agreement of coronary arteries. Stenosis detected by both reviewers was seen in the left main coronary, the left anterior descending artery, the left circumflex artery, and the right coronary artery with excellent inter-observer agreement of CTA for detection of coronary artery and a percentage agreement 99.7% (k = 0.99, 95% CI = 0.98-1.00, p = 0.001). Table 2 shows the inter-observer agreement of the coronary dominance with a percentage agreement of 94.8% and excellent inter-observer agreement for coronary predominance (k = 0.852, 95% CI = 0.724-0.98, p = 0.001).

Table 1

Inter-observer agreement with 95% CI and percentage agreement of affected coronary artery

Affected artery	1st reviewer	2nd reviewer	k	95% CI	Percentage agreement	p value
LMT	20	20	1.000	1.00-1.00	100.0	0.001
LAD	66	64	0.990	0.98-1.00	98.5	0.001
LCX	46	46	1.000	1.00-1.00	100.0	0.001
RCA	52	52	1.000	1.00-1.00	100.0	0.001
Overall	184	182	0.990	0.98-1.00	99.7	0.001

LMT – left main coronary artery, LCX – left circumflex artery, LAD – left anterior descending artery, RCA – right coronary artery

Table 2

Inter-observer agreement with 95% CI and percentage agreement of coronary dominance

Dominance	1st reader	2nd reader	k	95% CI	Percentage agreement	p value
Right dominance	76	76	0.870	0.75-0.99	95.8	0.001
Left dominance	13	11	0.905	0.77-1.00	97.9	0.001
Co-dominance	7	9	0.728	0.47-0.99	95.8	0.001
Overall	96	96	0.852	0.72-0.98	94.8	0.001

Table 3 shows the inter-observer agreement of CAD-RADS. The CAD-RADS of both reviewers were: CAD-RADS 0 (Fig. 1), CAD-RADS 1 (Fig. 2), CAD-RADS 2 (Fig. 3), CAD RADS 3 (Fig. 4), CAD-RADS 4 (Fig. 5), and CAD-RADS 5 (Fig. 6) (n = 16 and 12). There was excellent inter-observer agreement for CAD-RADS 0 (k = 1.0, 95% CI = 1.0-1.0, p = 0.001), CAD-RADS 1 (k = 0.92, 95% CI = 0.80-1.00, p = 0.001), CAD-RADS 3 (k = 0.808, 95% CI = 0.64-0.97, p = 0.001), CAD-RADS 4 (k = 0.826, 95% CI = 0.68-0.97, p = 0.001), and CAD-RADS 5 (k = 0.833, 95% CI = 0.67-0.99, p = 0.001) and good inter-observer agreement for CAD-RADS 2 (k = 0.76, 95% CI = 0.58-0.95, p = 0.001). The percent agreement of both reviewers for CAD-RADS 0 was 100%, for CAD-RADS 1 it was 97.9%, for CAD-RADS 2 it was 93.8%, for CAD-RADS 3 it was 94.8%, for CAD-RADS 4 it was 94.8%, and for CAD-RADS 5 it was 95.8%.

Table 3

Inter-observer agreement with 95% CI and percentage agreement of CAD-RADS

Category	1st reviewer	2nd reviewer	k	95% CI	Percentage agreement	p value
CAD-RADS 0	20	20	1.000	1.00-1.00	100.0	0.001
CAD-RADS 1	15	13	0.920	0.80-1.00	97.9	0.001
CAD-RADS 2	14	16	0.760	0.58-0.95	93.8	0.001
CAD-RADS 3	15	16	0.808	0.64-0.97	94.8	0.001
CAD-RADS 4	16	19	0.826	0.68-0.97	94.8	0.001
CAD-RADS 5	16	12	0.833	0.67-0.99	95.8	0.001
Overall	96	96	0.862	0.78-0.94	88.5	0.001

Figure 1

CAD-RADS 0: Coronary computed tomography angiography shows normal left anterior descending artery without plaque or stenosis

Figure 2

CAD-RADS 1: Coronary computed tomography angiography shows minimal calcified plaque in the proximal left anterior descending artery with minimal luminal narrowing less than 25% diameter stenosis

Figure 3

CAD-RADS 2: Coronary computed tomography angiography shows predominantly calcified plaque in the mid left anterior descending artery with 25-49% diameter stenosis

Figure 4

CAD-RADS 3: Coronary computed tomography angiography shows non-calcified plaque in the proximal left anterior descending artery with 50-69% diameter stenosis

Figure 5

CAD-RADS 4: Coronary computed tomography angiography shows multiple mixed plaques in the mid left anterior descending artery with 70-99% diameter stenosis

Figure 6

CAD-RADS 5: Coronary computed tomography angiography shows focal, mixed occlusion of the proximal right coronary artery with distal filling

Table 4 shows the inter-observer agreement of modifiers of CAD-RADS. The modifiers of CAD-RADS of both reviewers were modifier N, modifier V (Figure 7), modifier G (Figure 8) and modifier S (Figure 9). There was good inter-observer agreement for the modifier N (k = 0.79, 95% CI = 0.4-1.0, p = 0.001), moderate inter-observer agreement for the modifier V (k = 0.59, 95% CI = 0.24-0.94, p = 0.001), and excellent inter-observer agreement for the modifier G (k = 1.0, 95% CI = 1.0-1.0, p = 0.001) and the modifier S (k = 1.0, 95% CI = 1.0-1.0, p = 0.001). The agreement between both reviewers for the modifier N was 98.9%, for the modifier V was 94.8%, for the modifier G was 100.0%, and for the modifier S was 100.0%.

Table 4

Inter-observer agreement with 95% CI and percentage agreement of modifier CAD-RADS

Category	1st reviewer	2nd reviewer	k	95% CI	Percentage agreement	p value
N – non-diagnostic	2	3	0.790	0.40-1.00	98.9	0.001
V – plaque	7	6	0.590	0.24-0.94	94.8	0.001
S – stent	5	5	1.000	1.00-1.00	100.0	0.001
G – graft	3	3	1.000	1.00-1.00	100.0	0.001
Overall	17	17	0.815	0.67-0.96	98.4	0.001

Figure 7

CAD-RADS modifier V. CAD-RADS 4B/V. Coronary computed tomography angiography shows focal non-calcified plaque in the proximal left anterior descending artery with 50-69% diameter stenosis. The plaque shows mixed plaque (non-calcified plaque with spotty calcification)

Figure 8

CAD-RADS modifier S. CAD-RADS 4/S. Coronary computed tomography angiography shows in-stent stenosis of the proximal left anterior descending artery with significant luminal narrowing (50-69% stenosis)

Figure 9

CAD-RADS modifier G. Coronary computed tomography angiography shows patent left internal mammary artery to the left anterior descending artery. No stenosis or luminal narrowing throughout the graft [0% stenosis]

Table 5 shows the inter-observer agreement of associated coronary anomalies, and non-coronary cardiac and extra-cardiac findings. The associated coronary artery anomalies detected by both reviewers revealed agreement of 94.8% and excellent inter-observer agreement for associated congenital anomalies (k = 0.845, 95% CI = 0.712-0.98, p = 0.001]. The associated non-coronary cardiac findings of both reviewers were myocardial infarction, thrombus (Figure 10), left ventricle aneurysm, and aortic valve calcification with agreement of 95.8% and excellent inter-observer agreement (k = 0.857, 95% CI = 0.72-0.99, p = 0.001). The associated extra-cardiac findings of both reviewers were lipoma, aortic coarctation, and pulmonary nodules with agreement of 91.7% and excellent inter-observer agreement (k = 0.81, 95% CI = 0.684-0.94, p = 0.001].

Table 5

Inter-observer agreement with 95% CI and percentage agreement of coronary anomalies, and associated non-coronary cardiac and extra-cardiac findings

Category	1st reviewer	2nd reviewer	k	95% CI	Percentage agreement	p value
Coronary anomaly	21	20	0.845	0.71-0.98	94.8	0.001
Non coronary cardiac	18	16	0.857	0.72-0.99	95.8	0.001
Extra cardiac	32	30	0.810	0.68 -0.94	91.7	0.001
Overall	57	53	0.836	0.75-0.91	94.3	0.001

Figure 10

CAD-RADS-associated findings. A) Computed tomography angiography shows myocardial infarction and thrombus at the apex of the left ventricle. B) Coronary computed tomography angiography shows infarction of the left ventricle and segment of myocardial bridging of the mid part of the left anterior descending artery

The suggested further investigations and management of patients with coronary artery disease were determined based on the CAD-RADS results. Patients with CAD-RADS 0 need no further investigation. Patients with CAD-RADS 1 and 2 do not undergo further investigation and are treated with preventive medical therapy. Patients with CAD-RADS 3 are referred for functional assessment with stress echo, and patients with CAD-RADS 4 and 5 are referred for conventional angiography.

Discussion

The present results show excellent inter-observer agreement between both reviewers for the degree of coronary artery stenosis with coronary CTA and the classification of stenosis according to CAD-RADS. There is an excellent inter-observer agreement for CAD-RADS as a standardized reporting system for coronary CTA. The CAD-RADS is reproducible between reviewers and can used by radiologists and clinicians in the future.

In this study, there is excellent inter-observer agreement between both reviewers for the degree of coronary stenosis. Previous studies reported that there is a good correlation between coronary CTA and conventional coronary angiography in the grading of stenoses, with a slight tendency to overestimate the degree of stenosis when using coronary CTA. Current coronary CTA systems are capable of accurately evaluating coronary vessels for the presence of stenosis for vessel diameters as small as about 1 mm [2, 21]. The main determinant of the false positive results for diagnosing more than 50% coronary luminal stenosis is small vessel size, and the diameter of stenotic segments tends to be underestimated by coronary CTA [3, 22].

In this study, there is excellent inter-observer agreement between both reviewers for the CAD-RADS categories as a standard method for reporting coronary CTA. The CAD-RADS categories range from CAD-RADS 0 for a complete absence of stenosis and plaque to CAD-RADS 5 for the presence of at least a totally occluded coronary artery. Categories should reflect the most clinically relevant finding per patient [18-20]. The k values for agreement between fellows are 0.48 (95% CI: 0.42-0.54] at one year. The k value of intra-observer agreement in the reading of conventional coronary angiography is 0.95 (95% CI: 0.92-0.98) for experts with more than 10 years’ experience [23].

The CAD-RADS categories are complemented with modifiers that indicate the presence of stents, grafts, and vulnerable plaque [19]. The presence of vulnerability markers, such as positive remodelling, napkin-ring sign, low CT attenuation, and spotty calcification, are associated with adverse clinical events [24, 25]. CT imaging of coronary stents depends on patient and stent characteristics; however, about 8% of stents are not accessible, mostly due to blooming or motion artefacts. Ideal candidates have stents with a diameter of 3 mm and more [26, 27]. Coronary CTA can detect the patency and course of by-pass grafts [28, 29]. In this study, there is excellent inter-observer agreement for the detection of modifiers of CAD-RADS, including stents, grafts, and vulnerable plaques.

In this study, there is excellent inter-observer agreement of associated coronary anomalies, and non-coronary cardiac and extra-cardiac findings. Coronary CTA has the added ability to depict extracardiac lesions of the chest and scanned upper abdomen. These incidental lesions can often present a challenge to physicians because of the potential benefits and risks of identifying such lesions [30-32].

The main goal of CAD-RADS is to standardise reporting of coronary CTA results and to facilitate the communication among physicians along with suggestions for further investigations and management [17, 18]. Thus, CAD-RADS provides a basis for the clinical use of CAD-RADS for the further investigation, diagnosis, management, and treatment planning of patients with stable chest pain [19, 20]. In this study, patients with CAD-RADS 1 and 2 were treated with preventive medical therapy, patients with CAD-RADS 3 underwent functional assessment, and patients with CAD-RADS 4 and 5 underwent invasive angiography.

The advantages of CAD-RADS are a standardised method of reporting coronary CTA: providing a single, per-patient severity score and providing recommendations for further investigation of the patients [8-10]. Thus, this reporting system provides a common language between radiologists and clinicians to increase the clarity in reporting and recommendation for the management of coronary artery stenosis. The achieved standardisation of reporting will be of benefit in education, research, peer review, and quality assurance and may ultimately result in improved quality of care [11, 12].

Limitations of the study

There are a few limitations of this study. First, this study was done in one centre that includes a small number of patients, and the degree of the stenosis was divided in increments of 25%, which resulted in relatively small groups; prospective multicentre studies of large numbers of patients will enhance the application of CAD-RADS among radiologists and cardiologists. Second, this study was done using 128-slice multi-detector CT scanners. Further studies using dual-energy CT scanners and higher multi-detector scanners such as 320-slice detectors are recommended [33, 34]. Third, there was no correlation between the results of coronary CTA and conventional angiography. Further studies to correlate the degree of stenosis with conventional angiography and MR angiography are recommended [35-39]. Fourth, the authors did not exclude any segment from the investigation and did not discuss the possible relationship between heart rate, calcium score, and inter-observer variability. Further studies are recommended with correlation of CAD-RADS with calcium scoring and its relation to heart rate.

Conclusions

We concluded that there is inter-observer agreement of CAD-RADS in categorising the degree of coronary arteries stenosis, system modifiers, and associated cardiac and extra-cardiac findings, and CAD-RADS can be a standard method for reporting coronary CTA in the future.

Conflict of interest

The authors declare no conflict of interest.