Can clinical scoring systems improve the diagnostic accuracy in patients with suspected adult appendicitis and equivocal preoperative computed tomography findings?

OBJECTIVE: Adult appendicitis (AA) with equivocal computed tomography (CT) findings remains a diagnostic challenge for physicians. Herein we evaluated the diagnostic performance of several clinical scoring systems in adult patients with suspected appendicitis and equivocal CT findings.
METHODS: We retrospectively evaluated 189 adult patients with equivocal CT findings. Alvarado, Eskelinen, appendicitis inflammatory response, Raja Isteri Pengiran Anak Saleha Appendicitis (RIPASA), and adult appendicitis score (AAS) scores were evaluated, receiver operating characteristic analysis was conducted, and the optimal, low, and high cut-off values were determined for patient classification into three groups: low, intermediate, or high.
RESULTS: In total, 61 patients were included in the appendicitis group and 128 in the non-appendicitis group. There were no significant differences between the area under the curve of the clinical scoring systems in the final diagnosis of AA for equivocal appendicitis on CT (Alvarado, 0.698; Eskelinen, 0.710; appendicitis inflammatory response, 0.668; RIPASA, 0.653; AAS, 0.726). A RIPASA score greater than 7.5 had a high positive predictive value (90.9) and an AAS score less than or equal to 5 had a high negative predictive value (91.7) in the diagnosis of AA.
CONCLUSION: The accuracy of clinical scoring systems in the diagnosis of AA with equivocal CT findings was moderate. Therefore, a high RIPASA score may assist in the diagnosis of AA in patients with equivocal CT findings, and a low AAS score may be used as a criterion for patient discharge. Most patients presented with intermediate scores. The patients with equivocal CT findings may be considered as a third diagnostic category of AA.

INTRODUCTION

Adult appendicitis (AA) is a common surgical condition that requires a prompt diagnosis for the minimization of morbidity, mortality, and unnecessary surgical interventions [1]. The diagnosis of AA using clinical presentation and physical examination is often limited because clinical presentation is atypical and often overlaps with other conditions [1]. Therefore, the diagnosis of AA needs to be supported by clinical scoring systems or imaging modalities such as ultrasonography and computed tomography (CT).

CT is widely used in cases of suspected AA because of its accuracy [2-5]. However, the diagnosis of AA is limited, particularly when CT findings are equivocal [6]. The incidence of equivocal CT findings of AA varies between 5.0% and 13.1% [7], and AA occurs in up to 30% of patients whose CT findings are considered equivocal [6]. Daly et al. [8] suggested that equivocal CT results be considered as a diagnostic category and that clinicians should keep in mind that symptomatic patients with equivocal CT scans will have an approximately 30% chance of having appendicitis. AA with equivocal CT findings remains a diagnostic challenge for clinicians.

In this context, clinical scoring systems may be effectively used to assist in the diagnosis of AA because they are simple and easy to use in clinical practice. To the best of our knowledge, no previous studies have evaluated the diagnostic performance of clinical scoring systems in patients with suspected AA and equivocal preoperative CT findings. Therefore, we aimed to evaluate the diagnostic performance of various clinical scoring systems in adult patients with suspected AA and equivocal preoperative CT findings. Alvarado score, Eskelinen score, appendicitis inflammatory response (AIR), adult appendicitis score (AAS), and Raja Isteri Pengiran Anak Saleha Appendicitis (RIPASA) scores were analyzed in this study [9-13].

METHODS

This study was approved by the institutional review board of Bundang Jesaeng General Hospital, Seongnam, Korea (protocol no. EMC 16-01), and informed consent was waived because the study was based on the retrospective analysis of data.

Study population and data collection

This single-center, retrospective, observational study evaluated patients that visited the emergency department between April 2011 and November 2013. We collected data on patients older than 15 years with suspected appendicitis and CT evaluation. The patients with equivocal appendicitis on their CT report were enrolled in this study. Patients with equivocal CT findings were divided into two groups according to the final diagnosis: appendicitis group and the non-appendicitis group. The final diagnosis was established via analysis of data from medical records, histological examination, and clinical follow-up collected for 2 weeks after the day of consultation. The final diagnosis of AA was based on the transmural infiltration of neutrophils in the appendix on histological examination. Negative AA was defined as the absence of the histologic findings above among the patients who underwent surgery, or absence of the need for surgical intervention within 2 weeks after the day of consultation in the patients that were not referred to immediate surgery.

We retrospectively collected patient demographics, history, and parameters of the scoring systems from the medical records and the laboratory report system (Table 1). All medical records were carefully reviewed by an emergency physician.

Table 1.

Parameters and scores in each clinical scoring system

Parameter	Proposed score
	Alvarado	Eskelinen	AIR	RIPASA	AAS
Demographics
Male	-	-	-	1.0	-
Female	-	-	-	0.5	-
Age (yr)
< 40	-	-	-	1.0	-
≥ 40	-	-	-	0.5	-
Foreign national registration ID card	-	-	-	1.0	-
Symptoms
Nausea/vomiting	1	-	-	1.0	-
Vomiting	-	-	1	-	-
Anorexia	1	-	-	1.0	-
RIF pain	-	3.51	1	0.5	2
Migratory pain to the RIF	1	-	-	0.5	2
Duration of pain (hr)
< 48	-	-	-	1.0	-
≥ 48	-	2.13	-	0.5	-
Signs
RIF tenderness	2	11.41	-	1.0	3/1[a)]
Rebound tenderness	1	4.25	-	1.0	-
Muscular defense/rebound		6.62		2.0
Mild	-	-	1	-	2
Intermediate	-	-	2	-	4
Strong	-	-	3	-	4
Temperature (°C)
≥ 37.3	1	-	-	1.0[b)]	-
≥ 38.5	-	-	1	-	-
Rovsing’s sign	-	-	-	2.0	-
Clinical results
White blood cell count (× 109/L)
≥ 10.0	2	5.88	1	1.0[b)]	-
≥ 7.2 and < 10.9	-	-	-	-	1
≥ 10.9 and < 14.0	-	-	-	-	2
≥ 14.0	-	-	-	-	3
≥ 15.0	-	-	2	-	-
Polymorphonuclear leukocytes (%)
> 75	1	-	-	-	-
≥ 70 and < 85	-	-	1	-	-
≥ 85	-	-	2	-	-
≥ 62 and < 75	-	-	-	-	2
≥ 75 and < 83	-	-	-	-	3
≥ 83	-	-	-	-	4
C-reactive protein concentration (mg/L)
≥ 10 and < 50	-	-	1	-	-
≥ 50	-	-	2	-	-
≥ 4 and < 11 (< 24-hour symptoms)	-	-		-	2
≥ 11 and < 25 (< 24-hour symptoms)	-	-		-	3
≥ 25 and < 83 (< 24-hour symptoms)	-	-		-	5
≥ 83 (< 24-hour symptoms)	-	-		-	1
≥ 12 and < 53 (≥ 24-hour symptoms)	-	-		-	2
≥ 53 and < 152 (≥ 24-hour symptoms)	-	-		-	2
≥ 152 (≥ 24-hour symptoms)	-	-		-	1
Negative urinalysis	-	-	-	1.0	-
Total or maximum score	10	67.60	12	16.0	25

The variables which were not included in each clinical scoring system were marked as ‘-’.

AIR, appendicitis inflammatory response; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis; AAS, adult appendicitis score; RIF, right iliac fossa.

Men and women older than 50 years/women aged 16 to 49 years.

The criteria for fever and elevated white blood cells have not been fully established in previous studies using the RIPASA score. Therefore, the score was evaluated using the criteria of our institution.

Imaging methods

All CT examinations were performed using a 16-slice multidetector CT scanner (Brilliance 16, Philips Healthcare, Eindhoven, Netherlands) with intravenous administration of a contrast agent, without the need for administration of oral contrast medium. The parameters used were as follows: anatomical range; scan from the diaphragm to the symphysis pubis with a collimation of 1.5 mm; rotation time, 0.75 second; and pitch, 1.188. Reconstructions consisted of axial and coronal sections of 5 or 3 mm. Tube voltage was 120 kVp, and tube current was 150 to 300 mA. Enhanced CT of the entire abdomen was performed with a 60-second delay after the administration of a dose of 2 mL/kg body weight of iohexol (Omnipaque 350, GE Healthcare, Princeton, NJ, USA) and iopamidol (Pamiray 370, Dongkook Pharmaceutical, Seoul, Korea), with an infusion rate of 4 mL/sec through the antecubital vein.

Image interpretation and definitions

All CT examinations were analyzed by one experienced abdominal radiologist (10 years’ experience). CT assessment of the appendix was based on a set of five criteria derived from previous studies: an appendix without intraluminal air with a distension of more than 6 mm; periappendiceal fat stranding and infiltration; appendiceal wall enhancement or thickening; cecal apical wall thickening compared with the normal thickness of the wall of the ascending colon; and presence of extraluminal fluid collection or gas bubbles around the appendix [14-16]. The diameter of the appendix was measured at the longest portion of the visible appendix on enhanced axial sections. In cases in which three or more of these findings were present, the patient was diagnosed with definitive appendicitis. In cases in which two findings were present, the patient was diagnosed with probable appendicitis. The presence of only one finding was considered as equivocal appendicitis. In cases in which the appendix was not observed, or the radiologist failed to trace the entire appendix, the radiologist concluded that the patient probably did not have appendicitis [17]. The appendices that were filled with air and measured less than 9 mm in diameter were considered normal [18].

Statistical analysis

All data were analyzed using IBM SPSS Statistics ver. 19 (IBM, Inc., Armonk, NY, USA) and MedCalc ver. 7.4 (MedCalc Software, Mariakerke, Belgium). Continuous data are expressed as means and standard deviation or medians (interquartile range) according to the results of normal distribution analysis. Categorical data and frequency distributions are shown as absolute values. A Mann-Whitney U-test and an independent t-test were conducted according to normal distribution analysis. Chi-square tests were conducted to compare data between the sexes.

The diagnostic performance of each of the scoring systems was compared by conducting a receiver operating characteristic (ROC) analysis using MedCalc ver. 7.4. The optimal cut-off value was calculated by maximizing the sum of sensitivity and specificity. The other two different cut-off values were obtained to score the probability of appendicitis as low, intermediate, or high. The low and high cut-off values included a point with high sensitivity (higher than 90%) and specificity of at least 30%, and a point with high specificity (higher than 85%) and sensitivity of at least 20% [13]. The cut-off value within these sensitivity and specificity limits was chosen where the diagnostic odds ratio (positive likelihood ratio/negative likelihood ratio) was the highest [13]. P-values smaller than 0.05 were considered statistically significant.

RESULTS

General characteristics of the study population

During the study period, 3,162 patients (older than 15 years) with suspected AA visited our medical institution and underwent intravenously enhanced abdominal CT to rule out AA. These patients received the following diagnoses: confirmed appendicitis (n=1,025), probable appendicitis (n=291), equivocal appendicitis (n=202), probably not appendicitis (n=248), and normal appendix (n=1,396). After excluding 13 patients with insufficient data on the clinical scoring systems from those with equivocal appendicitis (n=202, 6.4%), 189 adult patients with suspected AA and equivocal CT findings were enrolled in this study (Fig. 1).

Fig. 1.

Study design flowchart. CT, computed tomography; AA, adult appendicitis.

Of the 189 enrolled patients, AA was confirmed in 61 (32.3%) patients using the aforementioned diagnostic criteria. There was no significant difference in age or sex ratio between the groups (P=0.582, P=0.428). Negative appendectomy was defined as the absence of adult inflammation in the pathology reports. Seventy-seven operations were performed, and the rate of negative appendectomy was 20.8% (16/77). The alternative diagnoses in patients subjected to surgery but not diagnosed with appendicitis were as follows: adult gastroenteritis (n=6), pelvic inflammatory disease (n=1), diverticulitis (n=2), appendiceal mucocele (n=1), mesenteric lymphadenopathy (n=2), and non-specific findings (n=4). Fifty-four patients (70.1%) were referred to immediate surgery; of them, 41 (75.9%) patients had AA. Twenty-three patients (29.9%) were administered to antibiotic therapy before delayed surgery; of them, 20 (87.0%) patients had AA (Fig. 1).

The final diagnosis in 112 patients who did not undergo operation were as follows: adult gastroenteritis (n=42, 37.5%), adult pyelonephritis (n=3, 2.7%), pelvic inflammatory disease (n=18, 16.1%), diverticulitis (n=5, 4.5%), epiploic appendagitis (n=1, 0.9%), mesenteric lymphadenopathy (n=8, 7.1%), terminal ileitis (n=3, 2.7%), pseudomembranous colitis (n=1, 0.9%), ureteral stone (n=2, 1.8%), ovarian cyst rupture (n=3, 2.7%), and functional gastroenteritis (n=26, 23.2%).

Clinical scores, ROC analysis, and diagnostic performance of clinical scores

Table 2 shows the results of the median and interquartile range of each clinical score. The optimal cut-off values were higher than 5 using Alvarado score, 38.17 using Eskelinen score, 4 using AIR score, 5 using RIPASA score, and 8 using AAS score. Based on the chosen low and high cut-off values in the ROC analysis for each scoring system, the study participants were classified into three groups. The cut-off values, absolute and relative number of patients, and P-values are summarized in Table 2. Ten out of 11 (90.9%) patients in the high probability group were diagnosed with AA using RIPASA score, indicating a high positive predictive value (90.9). In addition, only 3 out of 36 (8.3%) patients in the low probability group using the AAS score were diagnosed with AA, indicating a high negative predictive value (91.7). There was no significant difference in the area under the curve for each clinical score (Alvarado, 0.698 [0.627 to 0.762]; Eskelinen, 0.710 [0.640 to 0.774]; AIR, 0.668 [0.596 to 0.734]; RIPASA, 0.653 [0.580 to 0.721]; AAS, 0.726 [0.656 to 0.788]) by pairwise comparison analysis between AAS and other scores. Fig. 2 and Table 3 summarize the results.

Table 2.

Clinical scores and classification of the patients into three groups according to probability values

	Appendicitis group	Non-appendicitis group	P-value
Sex, male	22 (36.1)	41 (32.0)	0.582
Age (yr)	31.0 (22.0–43.0)	35.0 (24.5–43.5)	0.428
Clinical score
Alvarado	6 (4–7)	4 (3–6)	< 0.001
Eskelinen	36.04 (34.41–40.29)	32.29 (30.16–36.04)	< 0.001
AIR	5 (3–5)	3 (2–5)	< 0.001
RIPASA	5.5 (5.0–7.0)	5.0 (4.0–6.0)	0.001
AAS	11 (9–14)	8 (5–11)	< 0.001
Patient classification according to probabilities
Alvarado			0.006
Low (≤ 2)	2	19
Intermediate (2 < score ≤ 7)	50	103
High (> 7)	9	6
Eskelinen			0.002
Low (≤ 28.03)	0	3
Intermediate (28.03 < score ≤ 40.29)	54	124
High (> 40.29)	7	1
AIR			0.001
Low (≤ 1)	3	24
Intermediate (1 < score ≤ 6)	51	102
High (> 6)	7	2
RIPASA			< 0.001
Low (≤ 3.5)	3	21
Intermediate (3.5 < score ≤ 7.5)	48	106
High (> 7.5)	10	1
AAS			< 0.001
Low (≤ 5)	3	33
Intermediate (5 < score ≤ 15)	50	90
High (> 15)	8	5

Values are presented as number (%), median (interquartile range, 25% to 75%), or number.

AIR, appendicitis inflammatory response; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis; AAS, adult appendicitis score.

Fig. 2.

Receiver operating characteristic (ROC) curve and area under the curve (AUC) of each clinical score for the diagnosis of adult appendicitis. (A) ROC of all scores, (B) ROC of Alvarado score, (C) ROC of Eskelinen score, (D) ROC of appendicitis inflammatory response (AIR), (E) ROC of Raja Isteri Pengiran Anak Saleha Appendicitis (RIPASA), and (F) ROC of adult appendicitis score (AAS).

There was no statistical difference between the scores by pairwise comparison of the ROC curves (P>0.05). P-values were calculated using MedCalc ver. 7.4 (MedCalc Software, Mariakerke, Belgium) to compare AAS with other scores. The square mark on each ROC curve indicates the best cutoff value for each clinical score. CI, confidence interval.

Table 3.

Sensitivity, specificity, positive predictive value, and negative predictive value of the clinical scores according to each cut-off value

Scores	Sensitivity (95% CI)	Specificity (95% CI)	PPV	NPV
Alvarado
>2	96.7 (88.6–99.5)	14.8 (9.2–22.2)	35.1	90.5
>5	55.7 (42.4–68.5)	74.2 (65.7–81.5)	50.7	77.9
>7	14.8 (7.0–26.2)	95.3 (90.1–98.2)	60.0	70.1
Eskelinen
> 28.03	100 (94.1–100.0)	2.3 (0.5–6.7)	32.8	100.0
> 38.17	45.9 (33.1–59.1)	89.1 (82.3–93.9)	66.7	77.6
> 40.29	11.5 (4.8–22.2)	99.2 (95.7–99.9)	87.5	70.2
AIR
>1	95.1 (86.3–98.9)	18.8 (12.4–26.6)	35.8	88.9
>4	52.5 (39.3–65.4)	72.7 (64.1–80.2)	47.8	76.2
>6	11.5 (4.8–22.2)	98.4 (94.5–99.8)	77.8	70.0
RIPASA
> 3.5	95.1 (86.3–98.9)	16.4 (10.5–24.0)	35.2	87.5
>5	62.3 (49.0–74.4)	58.6 (49.6–67.2)	41.8	76.5
> 7.5	16.4 (8.2–28.1)	99.2 (95.7–99.9)	90.9	71.3
AAS
>5	95.1 (86.3–98.9)	25.8 (18.5–34.3)	37.9	91.7
>8	82 (70.0–90.6)	53.9 (44.9–62.7)	45.9	86.3
> 15	13.1 (5.9–24.2)	96.1 (91.1–98.7)	61.5	70.0

CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value; AIR, appendicitis inflammatory response; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis; AAS, adult appendicitis score.

DISCUSSION

Patients with equivocal CT scans are a diagnostic challenge in the emergency department because the probability of appendicitis in this group is approximately 30% [8]. Ultrasound re-evaluation and reassessment of CT images improve the accuracy of diagnosis of AA [19]. In most institutions, ultrasound re-evaluation and reassessment of CT are not frequently used in the diagnosis of AA. Therefore, clinical scoring systems that are simple, non-invasive, and easy to use in routine clinical practice may provide useful information to clinicians. In this study, the diagnostic accuracy of clinical scoring systems was moderate (Alvarado, 0.698; Eskelinen, 0.710; AIR, 0.668; RIPASA, 0.653; AAS, 0.726). However, most patients presented with intermediate scores: 153 (81.0%) patients using the Alvarado score, 178 (94.2%) using the Eskelinen score, 153 (81.0%) using the AIR score, 154 (81.5%) using the RIPASA score, and 140 (74.1%) using the AAS score. However, the most significant results were that 1) using AAS score, the low probability group (≤5) showed a high negative predictive value (91.7) with potential to safely exclude the occurrence of AA; 2) using RIPASA score, the high probability group (>7.5) showed a high positive predictive value (90.9) with potential to help diagnose AA. In addition, AAS and RIPASA scores could help in 47 out of 189 (24.8%) patients with equivocal appendicitis to include or exclude the diagnosis of appendicitis.

The diagnostic accuracy of the clinical scoring systems was moderate probably because of the stage of appendicitis [20]. The patients with AA with equivocal CT findings were in the early stage of appendicitis, and their symptoms might not be characteristic of AA. The cut-off values obtained in previous studies differed from those of the present study. Accordingly, previous studies found cut-off values of 4 to 7 for Alvarado score, 55 to 57 for Eskelinen score, 4 to 8 for AIR score, 5 to 7.5 for RIPASA score, and 11 to 18 for AAS score [9-13]. The cut-off values in our study were lower than that found in previous studies probably because of the early stage of appendicitis in our patients.

The present study has several limitations. First, the retrospective design limits the diagnostic performance of the clinical scoring systems. In addition, it was impossible to use other scoring systems owing to the limited information in the medical records. Second, CT findings were interpreted by a single expert abdominal radiologist, and the inter-observer correlation was not determined. Therefore, selection bias for equivocal CT findings might have occurred. Moreover, standards or criteria of equivocal appendicitis are controversial. The criteria used in our study in cases of equivocal CT were established and used in a single institution; therefore, any generalized conclusions about these criteria could not be made, and additional large-scale studies are required to elucidate this issue. Third, 58 of 112 (51.8%) patients in the non-appendicitis group were treated with antibiotics. The success rate of antibiotic therapy alone in uncomplicated appendicitis is 72.7% [20]. Although the rates of equivocal CT findings and confirmed appendicitis in these patients were similar to those of previous studies [8], the true incidence of appendicitis in the patients with equivocal CT findings might have been underestimated in cases in which the disease was cured with antibiotics in any of the 58 patients. Therefore, the incidence of AA in equivocal CT findings and the rate of negative appendectomy in our study could not be generalized. Furthermore, the study population was relatively small. Therefore, other large-scale, randomized, controlled studies are required to validate our results.

In conclusion, the overall diagnostic accuracy of the clinical scoring systems was moderate. Forty-seven of 189 patients (24.8%) were accurately diagnosed with appendicitis or non-appendicitis using the AAS and RIPASA scores. Although it is difficult to make general conclusions because of study limitations (including the retrospective design and small sample size), the RIPASA score might be considered better to diagnose AA in the patients with equivocal CT findings and the AAS score might be considered a better criterion for the safe discharge of patients from the emergency department. Most patients presented with intermediate scores. The patients with equivocal CT findings might be considered as a third diagnostic group for AA.