Diagnostic Performance on Low Dose Computed Tomography For Acute Appendicitis Among Attending and Resident Radiologists.

BACKGROUND: Low-dose computed tomography (LDCT) techniques can reduce exposure to radiation. Several previous studies have shown that radiation dose reduction in LDCT does not decrease the diagnostic performance for appendicitis among attending radiologists. But, the LDCT diagnostic performance for acute appendicitis in radiology residents with variable training levels has not been well discussed.
OBJECTIVES: To compare inter-observer and intra-observer differences of diagnostic performance on non-enhanced LDCT (NE-LDCT) and contrast-enhanced standard dose CT (CE-SDCT) for acute appendicitis among attending and resident radiologists. PATIENTS AND METHODS: This retrospective study included 101 patients with suspected acute appendicitis who underwent NE-LDCT and CE-SDCT. The CT examinations were interpreted and recorded on a five-point scale independently by three attending radiologists and three residents with 4, 1 and 1 years of training. Diagnostic performance for acute appendicitis of all readers on both examinations was represented by area under receiver operating characteristic (ROC) curves. Inter-observer and intra-observer AUC values were compared using Jackknife FROC software on both modalities. The diagnostic accuracy of each reader on NE-LDCT was compared with body mass index (BMI) subgroups and noise using independent T test.
RESULTS: Diagnostic performances for acute appendicitis were not statistically different for attending radiologists at both examinations. Better performance was noted on the CE-SDCT with a borderline significant difference (P = 0.05) for senior radiology resident. No statistical difference of AUC values was observed between attending radiologists and fourth year resident on both examinations. Statistically significant differences of AUC values were observed between attending radiologists and first year residents (P = 0.001 ~ 0.018) on NE-LDCT. Diagnostic accuracies of acute appendicitis on NE-LDCT for each reader were not significantly related to BMI or noise.
CONCLUSION: Attending radiologists could diagnose acute appendicitis accurately on NE-LDCT. Performance of senior residents on NE-LDCT is better than junior residents and comparable to attending radiologists.

1. Background

Acute appendicitis is one of the most common causes of right lower quadrant (RLQ) abdominal pain in the emergency room (ER) and the most common indication for emergency abdominal surgery (1). Due to its excellent diagnostic accuracy, abdominal computed tomography (CT) is widely accepted as the imaging modality of choice in diagnosing acute appendicitis. Using CT on patients with clinically suspected appendicitis has also been considered to be cost-effective, because it avoids negative appendectomies and delayed treatment (2). However, given the increasingly frequent use of CT today, concerns about the potential risk of carcinogenesis in patients undergoing CT scans have been raised (3, 4). Low-dose CT (LDCT) techniques have been shown to reduce exposure to radiation, but the trade-off is increased image noise and compromised image quality. However, several previous studies have shown that radiation dose reduction in LDCT does not significantly decrease the diagnostic performance for appendicitis among attending radiologists (5-8). In practice, however, attending radiologists are not always available during off-times in the ER. Thus, radiology residents become actively involved in emergent abdominal CT interpretation early on in their residency training program. The LDCT diagnostic performance for acute appendicitis in radiology residents with variable training levels has seldom been discussed.

2. Objectives

The aim of this study was to compare the diagnostic performance on non-enhanced LDCT (NE-LDCT) and contrast-enhanced standard-dose CT (CE-SDCT) for acute appendicitis between attending and resident radiologists with different training levels. In addition, we compared diagnostic performance between NE-LDCT and CE-SDCT within individuals. Finally, we analyzed the factors affecting the difference in diagnostic performance.

3. Patients and Methods

3.1. Patients

This retrospective study of prospectively collected CT data was approved by our institutional review board with patient informed consent obtained.

We included patients > 18 years old with RLQ pain and a clinical suspicion of acute appendicitis who were referred for CT from the ER. Patients who were pregnant, with an appendectomy history or impaired renal function were excluded. In total, 106 consecutive patients who underwent both NE-LDCT and CE-SDCT concurrently at ER from December 2008 to December 2009 were included in the analyses. Five patients were excluded from the study because they left the ER without surgical treatment or hospitalization and did not return to outpatient clinics for follow-up. The study population consisted of 44 males and 57 females (21 - 71 years, mean age 38.9 ± 12.6 years).

3.2. Technical Imaging Parameters

In our hospital, the standard CT protocol for RLQ pain includes both non-enhanced SDCT and CE-SDCT scans at the ER. The non-enhanced CT scan is usually preferred by the emergency physician due to its value in detecting trivial urinary tract stones. Therefore, in order to reduce the examination radiation dose, we replaced the non-enhanced SDCT by the NE-LDCT in our study. The NE-LDCT and CE-SDCT examinations were performed using a 64-slice multi-detector CT (Aquilion 64; Toshiba Medical Systems, Tokyo, Japan) from the lung base to the symphysis pubis. NE-LDCT was performed with the following parameters: 64 × 0.5 mm collimation, pitch 0.816, gantry rotation time 0.5 s, tube potential 120 kV, tube current time product per gantry rotation 49 mAs (80 mA × 0.5 s/0.816 = 49 mAs). CE-SDCT was performed immediately after completion of NE-LDCT and obtained with a tube potential of 120 kV and automatic tube current modulation. Iterative reconstruction technique was not used. All CE-SDCT examinations were obtained at the portal venous phase after intravenous administration of 100 mL, at 1 - 3 mL/s, of non-ionic iodinated contrast medium (Omnipaque 350; GE Healthcare, Princeton, NJ). No oral or rectal contrast medium was given for opacification of the bowels. NE-LDCT and CE-SDCT images were reconstructed into axial and coronal sections with 3-mm slice thickness and 3-mm intervals due to improved visualization of the appendix with use of thinner slice reconstruction (9). All images were anonymized before they were sent to a picture archiving communication system (PACS) for investigation. The volume computed tomography dose index (CTDI vol) and dose length product (DLP) of each patient were recorded in both examinations.

3.3. BMI and Noise

The body-mass index (BMI) of each patient was calculated and subgrouped as underweight (BMI < 18.5), normal weight (BMI between 18.5 and 24.9), overweight (BMI between 25.0 and 29.9), class I obesity (BMI between 30.0 and 34.9), and class II obesity (BMI ≥ 35) (6). Noise measurement was defined as the standard deviation of a CT number recorded by placing a rounded 200±10 mm2 region of interest (ROI) in the gluteal subcutaneous adipose tissue at the level of the right anterior superior iliac spine.

3.4. Image Analysis

At the end of the case collection, NE-LDCT and CE-SDCT images were interpreted independently by three board-certified attending emergency radiologists (readers 1, 2, and 3) with 20, 12, and 10 years of experience, respectively, and three resident radiologists (readers 4, 5, and 6) with 4, 1, and 1 year of training in radiology, respectively. All readers were blinded to the demographics, BMI, noise, clinical information, official CT report, and final diagnosis. NE-LDCT examinations were read prior to CE-SDCT examinations with a 1 month interval between the two interpretation sessions. During each interpretation session, readers were asked to record whether the appendix was visualized, and to record the presence or absence of each of the following signs (8, 10): dilatation of the appendix (> 7 mm diameter), appendicolith, periappendiceal fat stranding, periappendiceal abscess, extraluminal air, and extraluminal appendicolith (Figures 1, 2 and 3). When CE-SDCT examinations were interpreted, readers were further asked to assess whether wall enhancement of the appendix was stronger than that of the adjacent colonic walls, and whether the appendiceal wall was thicker than 2 mm.

Figure 1.

A, Axial NE-LDCT and B, CE-SDCT images of a 29-year-old woman with acute appendicitis show appendicolith impaction (black arrow) with enlarged appendix and wall thickening (white arrow). Both reader 1 and reader 6 (1-year resident) assigned a score of 5 (definitely appendicitis) for the diagnosis of appendicitis on NE-LDCT and CE-SDCT. The BMI was 18.4 kg/m2. The dose-length product was 291.2 and 630.8 mGy cm, respectively.

Figure 2.

A, Axial NE-LDCT and B, CE-SDCT images of a 51-year-old man with ruptured appendicitis show extraluminal appendicolith (black arrows) with extraluminal gas in the abscess formation (white arrows). Reader 1 and reader 6 assigned a score of 5 and 4 (probably appendicitis) for the diagnosis of appendicitis on NE-LDCT and CE-SDCT. The BMI was 19.8 kg/m2. The dose-length product was 267.7 and 627.8 mGy cm, respectively.

Figure 3.

A, Coronal NE-LDCT and B, Coronal CE-SDCT images of a 64-year-old woman with acute appendicitis show dilated appendix (white arrows), wall thickening, and surrounding fat stranding (black arrows). Reader 1 and reader 6 assigned a score of 5 and 1 (definitely not appendicitis) for the diagnosis of appendicitis on NE-LDCT and a score of 5 and 5 on CE-SDCT. The BMI was 23.6 kg/m2. The dose-length product was 239.6 and 647.9 mGy cm, respectively.

After recording each sign, readers were asked to assess the likelihood of appendicitis on a five-point Likert scale, based on their own evaluation of the signs above: (1) definitely not appendicitis, (2) probably not appendicitis, (3) equivocal appendicitis, (4) probably appendicitis, or (5) definitely appendicitis. In the absence of appendicitis, possible alternative diagnoses that may explain the RLQ pain were recorded.

3.5. Final Diagnosis

‘Official’ reports of the CT examination were issued immediately by the attending radiologist stationed at the ER to the referring clinician for patient disposition and treatment. Those immediate interpretations were not included in the analysis in the present study. For patients who underwent surgery, their final diagnoses were determined by surgical and pathological findings. For those who were hospitalized and treated medically, their final diagnoses were based on discharge diagnoses. For the patients who were discharged from the ER, the final diagnoses were determined by reviewing the clinical records of their follow-up outpatient clinic visits with symptom relief.

3.6. Statistical Analysis

We conducted a fully crossed, multi-reader, multi-case (MRMC) study with the 6 readers and two modalities. Receiver operating characteristic (ROC) curves and the corresponding area under the ROC curves (AUC) according to the five-point scale recorded by each reader were used to evaluate diagnostic performance of each reader for acute appendicitis on both examinations. Jackknife FROC software (Chakraborty, version 4.2) was used to compare the AUCs between each reader and within individuals on both modalities with MRMC-ROC data. The p values were calculated using the modified Dorfman-Berbaum-Metz method for ROC analysis summarized by Hillis, Berbaum, and Metz (11). For the following analysis of diagnostic accuracy, the diagnosis of acute appendicitis was classified as either absent (scores of 1, 2, or 3) or present (scores of 4 or 5). Using a score of 4 as a cut-off point was determined arbitrarily to decrease the negative appendectomy rate.

The subgroup analysis of the BMI and noise effect on accuracy of each reader for acute appendicitis on NE-LDCT was investigated with two independent samples t test. A P value of < 0.05 was considered to indicate statistical significance. The SPSS software (ver. 11.0 for Windows; SPSS, Chicago, IL) was used.

4. Results

4.1. Diagnoses and Treatments

In total, 56 (55.5%) of the 101 patients underwent surgery, and 48 (47.6%) of the 101 patients had a final diagnosis of acute appendicitis. An alternative diagnosis was reported in 44 (43.5%) patients. The remaining nine (8.9%) patients were discharged without a specific diagnosis after their symptoms subsided. The alternative diagnoses are summarized in Table 1.

Table 1.

Alternative Diagnoses and Their Respective Frequencies in 44 Patients who Were not Diagnosed as Acute Appendicitis

Alternative Diagnoses	44 Patients
Gastrointestinal disorder	26
Diverticulitis/diverticulosis	18
Colitis/ enteritis	5
Colon cancer	1
Small bowel perforation	1
Periappendicitis	1
Gynecological disorder	10
Tubal-ovarian abscess	3
Pelvic inflammatory disease	3
Hemorrhagic ovarian cyst	2
Endometrioma	2
Urinary tract stones	4
Peritonitis	2
Mesenteric adenitis	1
Acute pancreatitis	1

4.2. Comparison of Diagnostic Performance of NE-LDCT and CE-SDCT

Using the five-point Likert scale diagnostic performance for appendicitis of all readers on LDCT and CE-SDCT, the ROC curves were drawn. The AUC values were calculated and used to evaluate the overall diagnostic performance on both modalities of each reader. All AUC values are tabulated and compared within individuals in Table 2.

Table 2.

Intra-Observer Comparisons of AUCs Between NE-LDCT and CE-SDCT of Each Reader [a]

Reader	NE-LDCT		CE-SDCT		P Value
	AUC	95% CI	AUC	95% CI
R1	0.970	0.916 to 0.994	0.963	0.905 to 0.990	0.738
R2	0.952	0.891 to 0.985	0.944	0.880 to 0.980	0.768
R3	0.972	0.918 to 0.994	0.953	0.892 to 0.985	0.433
R4	0.924	0.854 to 0.967	0.973	0.920 to 0.995	0.050
R5	0.888	0.809 to 0.942	0.946	0.882 to 0.981	0.053
R6	0.824	0.736 to 0.893	0.882	0.803 to 0.938	0.252

a Abbreviations: AUC, area under the curve, CE-SDCT, contrast enhanced standard dose computed tomography; CI, confidence interval; NE-LDCT, non-enhanced low dose computed tomography; R, reader.

Intra-observer comparisons showed that differences in the diagnostic performances for acute appendicitis on NE-LDCT and CE-SDCT for attending radiologists (readers 1, 2, and 3) were not statistically significant. Larger AUC values were seen on CE-SDCT than NE-LDCT for all resident radiologists (readers 4, 5, and 6), although the differences were not statistically significant (P = 0.053 - 0.252) in the junior radiologists (reader 5, and 6). Borderline significance (P =0.05) was observed on the AUC values difference for senior resident radiologist (reader 4).

Inter-observer comparisons of AUC values among the six readers on NE-LDCT and CE-SDCT for the diagnosis of acute appendicitis are tabulated and compared in Figure 4. The AUC values on both NE-LDCT and CE-SDCT of the three attending radiologists (readers 1, 2, and 3) were comparable. A similar comparison between the three attending radiologists and the senior resident radiologist (reader 4) also showed no difference. A statistically significant difference in AUC values on NE-LDCT (P = 0.001 - 0.018) was observed when they were compared between attending radiologists and junior resident radiologists (readers 5 and 6). Similar comparisons of AUC values on CE-SDCT showed significant differences (P = 0.008 - 0.041) only between the attending radiologists (readers 1, 2 and 3) and one junior resident radiologist (reader 6).

Figure 4.

Bar charts of AUC values for the diagnosis of acute appendicitis on NE-LDCT and CE-SDCT of six readers. Statistically significant inter-observer comparisons (P < 0.05) of AUC values on both NE-LDCT (solid line) and CE-SDCT (dash line) are denoted by * (CE-SDCT, contrast enhanced standard dose computed tomography; NE-LDCT, non-enhanced low dose computed tomography; AUC, area under the curve).

Comparison of AUC values within the group of resident radiologists (readers 4, 5, and 6) demonstrated a significant difference between the senior resident (reader 4) and one junior resident (reader 6) on both NE-LDCT (P = 0.005) and CE-SDCT (P = 0.003). Comparison of AUC values within the group of junior resident radiologists (readers 5 and 6) demonstrated significant AUC value differences between the two readers on CE-SDCT (P = 0.035) only.

4.3. BMI, Radiation Dose, and Image Noise

Of the 101 patients included in our study, 8 (7.9%) were classified as underweight, 65 (64.4%) as normal weight, 23 (22.8%) as overweight, 4 (4%) as class I obesity, and 1 (0.9%) as class II obesity. The mean BMI was 23.3 ± 3.6. The mean dose-length products were 277.2 ± 20.7 mGy cm in the NE-LDCT group and 653.4 ± 209.8 mGy cm in the CE-SDCT group. Using a simplified approach with a conversion k-factor of 0.015 mSv mGy-1 cm-1 (12), the corresponding mean effective doses were 4.16 ± 0.31 mSv for NE-LDCT examinations and 9.8 ± 3.15 mSv for CE-SDCT examinations. The mean image noise values were 44.2 ± 9.11 in the NE-LDCT group and 18±2.06 in the CE-SDCT group.

4.4. Comparison of Diagnostic Accuracy with BMI and Image Noise

The sensitivity, specificity and, accuracy of each reader on both modalities are summarized in the Table 3. Diagnostic accuracies of acute appendicitis on NE-LDCT for each reader were not influenced significantly by BMI (P > 0.05). Moreover, diagnostic accuracies of acute appendicitis on NE-LDCT for each reader were also not influenced significantly by image noise (P > 0.05).

Table 3.

Diagnostic Performance of the Six Readers on NE-LDCT and CE-SDCT Examination for Acute Appendicitis [a,b]

	Sen	Spe	Acc
NE-LDCT
R1	91.7	92.5	92.1
R2	89.6	84.9	87.1
R3	85.4	96.2	91.1
R4	93.8	81.1	87.1
R5	85.4	81.1	83.2
R6	56.3	94.3	76.2
CE-SDCT
R1	93.8	94.3	94.1
R2	93.8	83.0	88.1
R3	97.9	86.8	92.1
R4	97.9	90.6	94.1
R5	91.7	92.5	92.1
R6	60.4	94.3	78.2

a Abbreviations: Acc, Accuracy; CE-SDCT, contrast enhanced standard dose computed tomography; NE-LDCT, non-enhanced low dose computed tomography; R, reader; Sen, Sensitivity; Spe, Specificity.

b Data are presented as percentage.

5. Discussion

Over the past two decades, abdominal CT for diagnosing acute appendicitis has been used more frequently (13, 14). However, such a high radiation exposure may have an effect on the incidence of carcinogenesis in these populations (3, 4). The issue of the radiation dose reduction is now taken very seriously. The often-cited standard radiation dose for abdominal CT is within the range of 7 10 mSv (15). In our cases, the mean NE-LDCT radiation dose was 4.16 mSv. Thus, it is about half the dose of SDCT and it is slightly higher than the annual effective dose from background radiation, about 3.1 mSv in the United States (15).

The diagnostic performance for acute appendicitis with NE-LDCT is comparable to that with CE-SDCT among each board-certified radiologist in our study. This finding is similar to a prior study (8). Better performance was noted on the CE-SDCT with a borderline significant difference (P = 0.05) for the senior resident radiologist. This result suggests that the senior resident radiologist still gains more information from the CE-SDCT for diagnosing acute appendicitis.

The senior resident radiologist had comparable diagnostic performance for acute appendicitis to that of the attending radiologists on both examinations. This finding implies that a resident with 4 years of training can achieve the diagnostic level of attending radiologists for acute appendicitis. The junior resident radiologists, however, showed inferior diagnostic performance on NE-LDCT compared with the attending radiologists. This phenomenon may indicate that diagnosis of acute appendicitis using NE-LDCT is not appropriate for junior resident radiologists. For CE-SDCT, the three attending radiologists had diagnostic performances superior to only one of the junior resident radiologists (reader 6). In addition, reader 5 showed significantly better diagnostic performance on CE-SDCT than reader 6. The finding suggests that heterogeneity exists among resident radiologists within the same training program. Reader 5 benefitted considerably from the CE-SDCT, but reader 6 did not. Overall, our results show that the diagnostic performance of NE-LDCT for acute appendicitis is acceptable for attending radiologists and senior resident radiologists, but still challenging for junior resident radiologists. There may be a need for extra training focusing on NE-LDCT and more experience to overcome the diagnostic threshold.

In our study, approximately 43.5% of the patients had a definite alternative diagnosis, similar to the ratio seen in previous studies (5, 8). However, because of the small number of cases in each category of alternative diagnoses and the large number of readers, diagnostic comparison between NE-LDCT and CE-SDCT was beyond the scope of this study. Based on CT interpretations by attending radiologists, 14 of 18 patients with diverticulitis and 3 of 4 patients with urinary stones were identified on both NE-LDCT and CE-SDCT. On the other hand, out of the 10 patients with gynecological disorders, 5 were identified on NE-LDCT and 9 were identified on CE-SDCT. These findings may indicate that NE-LDCT is adequate in diagnosing urinary stones and colonic diverticulitis as alternative diagnoses for patients with RLQ pain. These findings are comparable with prior studies (16, 17). On the other hand, the value of NE-LDCT is limited in diagnosing gynecological disorders, such as tubal-ovarian abscess and pelvic inflammatory disease. Additionally, the diagnosis of these diseases should also take clinical information, pelvic examinations, and laboratory data into consideration.

In our study, there was no significant diagnostic difference in the separate BMI subgroups when using NE-LDCT and CE-SDCT to diagnose acute appendicitis in each reader. This finding differs from one prior study. Platon and colleagues (8) stated that NE-LDCT was more limited than CE-SDCT for the detection of appendicitis in patients with BMI < 18.5. The sensitivity and specificity were 50% and 67%, respectively. However, only seven (8%) patients in the Platon study and eight (8%) patients in our study had a BMI < 18.5. Due to the limited number of patients, it is premature and hastily to judge the diagnostic performance of NE-LDCT in these slim patients with clinically suspected appendicitis.

Our study has several limitations. First, in the group of patients who did not undergo surgery, the final diagnosis was based on discharge diagnosis and clinical follow-up. Although imperfect, this is the preferred way of diagnosing patients who do not require surgery. Because this applied equally to NE-LDCT and CE-SDCT, there was no bias. Second, we compared the performance of NE-LDCT and CE-SDCT in the diagnosis of acute appendicitis. There are two factors (radiation dose and contrast medium enhancement) affecting the results. Only a study designed to compare the performances of NE-LDCT and non-enhanced SDCT or contrast-enhanced LDCT and CE-SDCT could truly evaluate the influence of radiation reduction on diagnostic performances. The issue was in part evaluated by Kim et al. (7), who reported that contrast-enhanced LDCT may have comparable diagnostic performance as CE-SDCT in the diagnosis of appendicitis in young adults. Third, there was a limited number of patients with BMI < 18.5 and > 30. We cannot draw definitive conclusions for these patient groups.

In conclusion, accurate diagnosis for acute appendicitis using LDCT is feasible for attending radiologists and senior resident radiologists. However, the diagnostic performance of junior resident radiologists was significantly inferior to the attending radiologists on NE-LDCT, and slightly inferior to the attending radiologists on CE-SDCT. There is a definite learning curve for the diagnosis of acute appendicitis with NE-LDCT. Radiology residents may require an extra training course on NE-LDCT images before interpreting acute appendicitis.