Improving the value of ultrasound in children with suspected appendicitis: a prospective study integrating secondary sonographic signs.

PURPOSE: The purpose of this study was to determine whether the awareness and inclusion of secondary sonographic signs of appendicitis, in combination with a structured evaluation as part of engagement and training for sonographers, improved appendix visualization rates and reduced equivocal findings in children with suspected acute appendicitis.
METHODS: This was a prospective study of 230 children at a tertiary children's hospital in Australia referred for an ultrasound examination of suspected appendicitis. The ultrasound findings, radiology reports, histology, clinical results, and follow-up were collated. Secondary signs were used as an additional assessment of the likelihood of disease where possible, even in the absence of an identified appendix.
RESULTS: The implementation of a structured evaluation as part of sonographer engagement and training resulted in a 28% improvement in appendix visualization (68.7%) compared with a prior retrospective study in a similar population (40.7%). The diagnostic accuracy was 91.7%, with likelihood ratios suggesting a meaningful influence of the pre-test probability of appendicitis in children studied (positive likelihood ratio, 11.22; negative likelihood ratio, 0.09.). Only 7.8% of the findings were equivocal. A binary 6-mm diameter cut-off did not account for equivocal cases, particularly lymphoid hyperplasia.
CONCLUSION: Engagement of sonographers performing pediatric appendiceal ultrasound through training in the scanning technique and awareness of secondary signs significantly improved the visualization rate and provided more meaningful findings to referrers.

Introduction

Acute appendicitis is the most common cause of emergent surgery. Its documented incidence is roughly 100 admissions per 100,000 person-years in Eastern Europe and North America, and is higher in Western Europe and Oceania [1]. While there are regional and institutional variations, cases that present with a high clinical suspicion often proceed directly to treatment. Management typically involves surgical removal of the appendix. More recently, conservative treatment of cases of uncomplicated appendicitis with antibiotic therapy has been associated with a higher rate of major complications than primary appendectomy [2]. Medical imaging plays an important role in patients who present with an atypical clinical presentation or a relatively low clinical probability of appendicitis. International differences exist in diagnostic imaging pathways for suspected appendicitis, with computed tomography (CT), ultrasound, and more recently, magnetic resonance imaging shown to be of use [3]. It is widely accepted that, in children, ultrasound should be the first-line imaging modality, as it avoids the radiation burden of CT [4].

Appendiceal sonography has a wide range of reported accuracy, which is contingent upon the person performing the examination. In Australia, sonographers perform most appendiceal ultrasound examinations, usually in conjunction with a supervising radiologist, whilst in many European and Asian settings, scans are performed by radiologists themselves. Regardless of the person performing the scan, visualizing the appendix is problematic when investigating possible cases of appendicitis with ultrasound. Graded compression, first described by Puylaert in 1986 [5], is the most widely used sonographic technique for locating the appendix. In this method, a linear transducer is used to displace overlying bowel gas to reveal the appendix [5]. In an Australian study in children using a tightly-curved array transducer, 91.7% of appendices were seen [6]. Combining these principles, Lee et al. in South Korea [7] used different transducer shapes, varied the patient position, and extended the examination time up to 35 minutes, yielding a 99% appendix visualization rate. Studies demonstrating such high visualization rates show the usefulness of appendiceal ultrasound when time, training, and resources are available; however, in many clinical environments, visualization rates are commonly between 25% and 50% [8,9].

Making a definitive ultrasound diagnosis in suspected appendicitis relies on complete visualization of the appendix. When identified on ultrasound, an appendix is traditionally considered inflamed when it is greater than 6 mm in diameter, lacks compressibility, has wall hyperaemia, and has an appendicolith present [10,11]. A binary 6-mm cut-off criterion has been shown to result in a number of false-positive findings, leading to the proposal of a new diameter-based categorisation, with <6 mm being normal; 6-8 mm equivocal, and >8 mm positive for appendicitis [12]. When the appendix is not visualized on ultrasound, or when the sonographic criteria for acute appendicitis are not met, secondary or ancillary signs of appendicitis can be a useful adjunct. The most useful of these is the presence of peri-appendiceal echogenic mesenteric fat (Fig. 1) [13]. Other secondary signs may include free fluid, an appendicolith, phlegmon/abscess, prominent lymph nodes, and bowel dilatation/ileus (Fig. 2) [14].

Fig. 1.

A 13-year-old boy with acute appendicitis.

A. Grey-scale ultrasonography shows echogenic peri-appendiceal mesentery, which prompted a thorough examination of the right iliac fossa with a different transducer. B. Using the iliac vessels as an acoustic window deeper into the pelvis revealed an inflamed appendiceal tip (callipers).

Fig. 2.

A 5-year-old boy with acute appendicitis complicated by perforation.

Grey-scale ultrasonography shows a shadowing appendicolith (callipers) and the presence of phlegmonous changes around it.

The objectives of this study were to determine whether awareness of secondary sonographic signs of appendicitis, in combination with technical training for sonographers, improved appendix visualization rates and subsequently reduced equivocal findings in children with suspected acute appendicitis.

Materials and Methods

Study Design

This prospective study included patients up to 16 years of age, who were referred for ultrasound evaluation of suspected appendicitis or undifferentiated right iliac fossa pain, between December 2015 and April 2017 in an Australian tertiary children’s hospital. A sample size was of 138 patients was required to achieve a sensitivity of at least 90%, with 80% power (ß) and α<0.05 [15]. This was adjusted to 230 patients, as we estimated a 50% improvement in the visualization rate (to 60%) due to the training provided as part of the study, compared to the 40% visualization rate of our prior study in the same population [16]. Changes in the value of ultrasound examinations between the current and prior study were evaluated through a comparison of diagnostic accuracy calculations, primarily the change in the likelihood ratio, which would alter the post-test probability and therefore the value of the examination for a referring clinician. Examinations performed after-hours were not included in the study to limit costs through overtime. Children's Health Queensland Human Research and Ethics Committee approval was obtained (HREC/15/QRCH/125). Written informed consent was provided by the parent or guardian of all children enrolled. All examinations were performed using a Philips Epiq 7G ultrasound platform (Philips Healthcare, Bothwell, WA, USA).

Ultrasound Examination

A single customised pre-set was used across machines and transducers (L12-5, L18-5, C8-5, C9-2, and C5-1) with the colour Doppler scale set at 6.5 cm/sec for uniformity in the evaluation of hyperaemia. The L12-5 and C8-5 transducers were primarily used, although sonographers could also use higher- or lower-frequency transducers if indicated based on the patient’s body habitus. A sonographer worksheet was developed based on variables identified in a prior retrospective study [16]. This worksheet incorporated traditional appendiceal criteria, secondary signs of appendicitis, and clinical history. The sonographic signs evaluated as potential secondary signs were peri-appendiceal mesentery appearance (more echogenic than the contralateral side), free fluid (present or not), bowel dilatation (fluid-filled hypoperistaltic or aperistaltic small bowel), prominent lymph nodes (a single node greater than 10 mm in the short axis, or a cluster of greater than 5 nodes with short axes greater than 5 mm), bladder debris, and presence of an abscess/ phlegmon (complex intraperitoneal collection).

The sonographers performing the examinations were trained on the research protocol and the suggested scanning technique, including the various transducers (as listed above), patient positions (supine and left-lateral decubitus), graded compression and posterior-manual compression [7], and acoustic windows (urinary bladder and iliac vessels). The clinical experience of sonographers was diverse, from a student in their final year of training to some with decades of dedicated pediatric experience. Full-time-equivalency (FTE) also varied from full-time to 1 day per week. The decision to include all available sonographers was made to better reflect employment status and experience levels in most practices. The radiologists reporting the examinations included in the study were also made aware of the protocol. The study imposed no formal or structured changes to the radiology report, which was considered to be the ultimate finding of the ultrasound examination.

Data Collection

The completed worksheets were collected and electronically collated. Clinical information was retrieved from the patients’ electronic medical records, including the surgical report, histology findings, C-reactive protein (CRP) level, white blood cell (WBC) count, weight, height, age, duration of symptoms, and any differential diagnosis. The histology report, where available, was the reference standard for the final diagnosis. Patients who did not undergo appendectomy or re-present within 1 month were considered not to have suffered from acute appendicitis. Ultrasound findings were allocated into one of five diagnostic categories integrating the presence of secondary signs and the three diameter-based classifications (Table 1).

Table 1.

Categorization of ultrasound findings

Category	Appendix identified	Diameter	Secondary sign	Binary classification
Positive for appendicitis	Y	>6 mm	Y	Positive
Equivocal (probably positive)	N	N/A	Y	Excluded or considered positive
Equivocal	Y	6-8 mm	N	Equivocal
	N	N/A	Not adequately assessed	(excluded or considered negative)
Equivocal (probably negative)	N	N/A	N	Excluded or considered negative
Negative for appendicitis	Y	<6 mm	N	Negative

Y, yes; N, no; N/A, not available.

Statistical Analysis

Statistical analysis was performed using IBM SPSS Statistics ver. 22 (IBM Corp., Armonk, NY, USA). Categorical data were evaluated using descriptive statistics, sensitivity, specificity, accuracy, and likelihood ratios. The diagnostic accuracy analyses required condensing the five diagnostic categories into binary outcomes (Table 1). This was performed using three methods to better permit comparisons with studies that may have treated equivocal studies in different manners by excluding the equivocal categories altogether (method 1); collapsing the probable positive and negative results into the positive and negative findings, respectively, and excluding the remaining equivocal findings (method 2); and considering the remaining equivocal examinations as negative findings (method 3). A receiver operating characteristic (ROC) curve was used to analyse the area under the curve (AUC) for appendix diameter, WBC count, and CRP. As WBC count can vary with age, a binary variable for elevated WBC count was calculated based on the patient’s age and relevant upper reference limit [17]. The chi-square test was used to assess visualization rates and employment status (<0.4, 0.4-0.7, or 0.8-1.0 FTE) and years of experience (<5, 5-10, or >10 years), with post hoc cell-wise residual analysis used to identify significance between groups [18]. Logistic regression analysis was used to assess the associations of the age of the patient, length of symptoms, and body mass index for age percentile (BMI-FAP) with the visibility of their appendix. Logistic regression was also used to analyse the relationships among the secondary signs of appendicitis, traditional ultrasound criteria and the presence of acute appendicitis. The threshold for statistical significance was set at P<0.05.

Results

A total of 230 imaging studies were included during the 16-month period, with 109 in female patients (47.4%) and 121 in male patients (52.6%). The patients’ age ranged from 21 months to 16 years, with a mean of 9 years and 10 months (standard deviation [SD], 3 years and 5 months). The appendix was identified on ultrasound in 158 cases (68.7%). The incidence of appendicitis was 19.1%, with 63 instances of alternative diagnoses, the most common being mesenteric adenitis (n=36) (Table 2). There were 20 negative appendectomies performed (8.7%) (Table 3). These negative appendectomies included four cases of parasite infestation, six cases of lymphoid hyperplasia (including four false-positives), a case of ovarian torsion, a perforated Meckel diverticulum, and an omental infarct that was suspected pre-operatively on ultrasound.

Table 2.

List of alternative diagnoses

Alternative finding	No. of cases
Mesenteric adenitis	36
Ileitis/Colitis	9
Appendiceal lymphoid hyperplasia	6
Parasite/Giardia	5
Renal (obstruction/calculus)	4
Ovarian (cyst/torsion)	3
Retrograde menses	2
Juvenile polyposis	1
Henoch-Schönlein purpura	1
Perforated Meckel diverticulum	1
Cholelithiasis	1
Hepatitis	1
Omental infarction	1

Table 3.

Ultrasound results and patient outcomes

Ultrasound result	No. of cases (%)	Sex (M/F)	Mean age (yr)	Pain duration (hr)	WBC count (×109/L)	CRP (mg/L)	Surgical management		LH	No surgery
							Positive	Negative
Positive for appendicitis	48 (20.9)	30/18	10.16±3.48	34.48±23.21	14.75±5.72	61.47±79.05	40	5	3	3
Equivocal (probably positive)	7 (3.0)	4/3	11.11±2.82	374.50±436.03	7.17±1.94	41.33±101.25	-	1	1	5
Equivocal	18 (7.8)	9/9	10.31±3.85	54.00±44.28	10.61±4.42	46.56±71.53	3	3	1	12
Equivocal (probably negative)	59 (25.7)	28/31	9.82±3.47	92.67±125.97	9.64±4.19	13.23±24.71	1	5	1	53
Negative for appendicitis	98 (42.6)	50/48	9.53±3.38	134.76±254.89	9.81±5.07	17.44±38.48	-	5	-	93
Total	230 (100)	121/109	9.83±3.42	94.42±171.42	10.83±5.25	29.84±56.99	44	20	6	166

M, male; F, female; WBC count, white blood cell count; CRP, C-reactive protein; LH, lymphoid hyperplasia.

Diagnostic accuracy calculations were conducted using the three aforementioned methods of classifying findings, and also including only cases where the appendix was visualized on ultrasound (Table 4). A logistic regression model analysing the effects of pain duration, BMI-FAP, and the patient’s age on the likelihood of visualizing the appendix showed statistically non-significant results (χ2 (3)=7.393, P=0.060), explaining 6% (Nagelkerke R2 ) of the variance in appendix visualization and correctly classifying 68% of cases. Increasing age was associated with a decreased likelihood of appendix visualization. Overall, the visualization rate in this study was markedly improved from our prior study and compared favourably to other recent pediatric studies (Table 5).

Table 4.

Summary of diagnostic accuracy results by the method of classifying ultrasound findings and by appendix visualization, including the results of our prior study for comparison

Method	Ultrasound result[a)]		Equivocal	No. of studies	SN (%)	SP (%)	Accuracy (%)	PPV (%)	NPV (%)	LR+	LR-
	Definitive	Probable
1	Included	Excluded	Excluded	146	100	96.8	97.5	88.9	100	31.25	1.03
2	Included	Included	Excluded	171	97.6	91.9	92.4	72.7	99.4	11.09	0.03
3	Included	Included	Included	230	90.9	91.9	91.7	72.7	97.7	11.22	0.10
Excluded non-visualized	Included	Included	Included	158	95.1	89.7	91.1	76.5	98.1	9.23	0.06
Prior study [16]	Included	Included	Included	457	88.1	91.4	90.4	78.0	96.3	10.20	0.13
Excluded non-visualized	Included	Included	Included	186	96.1	71.1	84.9	80.5	93.7	1.63	0.055

SN, sensitivity; SP, specificity; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR-, negative likelihood ratio.

Ultrasound results were definitive (appendix seen), probable (appendix not seen and secondary signs present or absent), or equivocal (borderline cases where the appendix was seen or where the examination was inadequate).

Table 5.

Comparison of recently published appendix visualization rates in children

Study	Year	Country	Study type	SS included[a)]	Exams performed by	Sample size	Visualized (%)
Cundy et al. [6]	2016	Australia	Retrospective	NS	Sonographers	3,799	91.7
This study	2018	Australia	Prospective	Y	Sonographers	230	68.7
Lofvenberg et al. [9]	2016	Sweden	Retrospective	Y	Radiologists	438	47
Cohen et al. [19]	2015	USA	Retrospective	N	Sonographers and radiologists	1,383	43.8
Reddan et al. [16]	2016	Australia	Retrospective	NS	Sonographers	457	40.7
Partain et al. [20]	2016	USA	Retrospective	Y	Sonographers	825	36
Alter et al. [8]	2017	USA	Retrospective	N	Sonographers	441	21

Were the secondary sonographic signs of appendicitis incorporated in the examination findings: Y, yes; N, no; NS, not specifically.

The mean uncompressed appendiceal diameter for normal appendices was 5.0 mm (SD, 1.4 mm; 95% confidence interval [CI], 4.8 to 5.4), and 9.2 mm for inflamed appendices (SD, 2.9 mm; 95% CI, 8.1 to 10.4) (Fig. 3). When compression could be applied to appendices during diameter measurements, the mean was reduced: normal, 3.9 mm (SD, 1.5 mm; 95% CI, 3.7 to 4.3) and inflamed, 8.6 mm (SD, 2.9 mm; 95% CI, 7.4 to 9.7). When the appendix diameter measurements were analyzed with a ROC curve, there was little difference between non-compressed measurements (AUC, 0.964; 95% CI, 0.902 to 0.985; P<0.001) and those made with compression of the appendix (AUC, 0.950; 95% CI, 0.937 to 0.990; P<0.001), with optimal cut-offs of 6.8 mm and 5.8 mm respectively (Fig. 4). Using the DeLong test, the difference between the area under the ROC curves of the compressed and uncompressed appendix diameters of 0.006 was not significant (P=0.670) [21]. A logistic regression model analysing the effects of secondary sonographic signs (mesentery, nodes, bowel dilatation, free fluid, bladder debris, and abscess) and traditional ultrasound criteria (compressed diameter, compression, hyperaemia, or appendicolith) on the likelihood of a child having acute appendicitis yielded statistically significant results (χ2 (10)=98.179, P<0.001), explaining 85.2% (Nagelkerke R2 ) of the variance in acute appendicitis and correctly classifying 96.7% of cases. Lack of echogenic mesentery and appendiceal wall hyperaemia were associated with a decreased likelihood of appendicitis.

Fig. 3.

Box-and-whisker plot of appendix diameter when compressed and uncompressed for positive and negative findings.

Figure 4.

Receiver operating characteristic curve of appendix diameter (compressed and uncompressed), white blood cell count (WBC count), and C-reactive protein (CRP).

Analysis of common pathology results ordered for suspected appendicitis revealed that WBC count (AUC, 0.757; 95% CI, 0.680 to 0.834; P<0.001) and CRP levels (AUC, 0.701; 95% CI, 0.613 to 0.789; P<0.001) were potentially useful indicators of appendicitis (Fig. 4). An optimal WBC count cut-off of 10.55×109 /L was identified in the ROC analysis. The Fisher exact test revealed a significant relationship between having appendicitis and a WBC count above the upper reference limit for the patient’s age (P<0.001) [17].

Of the secondary signs evaluated, only echogenic mesentery appeared to be an independent predictor of the presence or absence of appendicitis: sensitivity, 95.5%; specificity, 90.3%; positive predictive value (PPV), 70%; and negative predictive value (NPV), 98.8%. While the PPV of the remaining secondary signs was poor, they did demonstrate some potential value for negative prediction, as follows: free fluid: PPV, 24.2% and NPV, 84.1%; prominent lymph nodes: PPV, 13.6% and NPV, 74.3%; and dilated bowel loops: PPV, 33.3% and NPV, 82.3%.

Fifteen sonographers with varying levels of experience were involved in the study examinations, including seven with more than 10 years of experience (47%); five with 5-10 years (33%); and three with fewer than 5 years, including a student (20%). Full-time employment equivalency also varied: five sonographers worked more than 8 days per fortnight (≥0.8 FTE, 33%); six between 4 and 7 days per fortnight (0.4-0.7 FTE, 40%); and the remainder worked 3 days per fortnight or less (≤0.3 FTE, 27%). The visualization rates based on the clinical experience of sonographers performing the examinations were 77% for those with 5-10 years of experience (n=104), 73% for those with fewer than 5 years (n=37), and 59% for those with over 10 years (n=89). The visualization rate according to the full-time employment status of sonographers was highest amongst those working ≥0.8 FTE (81%, n=124). Staff who worked 0.4-0.7 FTE had a rate of 64% (n=42), whilst those who worked ≤0.3 FTE had the lowest rate of 50% (n=64). Sonographer experience (χ2 (2)=7.232, P=0.027) and FTE (χ2 (2)=17.942, P<0.01) were both associated with appendix visualization. Post hoc cell-wise residual analysis revealed that those working greater than 0.8 FTE (P<0.001) were more likely to detect the appendix, while those working less than 0.3 FTE (P<0.001) or with more than 10 years of experience (P=0.007) were significantly less likely to successfully visualize it.

Discussion

Complete visualization of the appendix during an ultrasound examination is ideal, but this is often not achieved, leading to a high proportion of what are often considered non-diagnostic or equivocal studies [22]. This prospective study was designed to ascertain whether engagement of sonographers through technical training and improved awareness of the secondary signs of appendicitis would lead to improvements in appendix visualization and diagnostic accuracy. The results of this study clearly show that better awareness and recording of secondary sonographic signs by sonographers improved the post-test probability and reduced the proportion of equivocal findings. The presence of secondary signs was recorded regardless of whether the appendix was visualized, permitting probable radiological findings to be conveyed for what would otherwise be non-diagnostic studies in the event of not identifying the appendix.

There was a marked improvement in appendix visualization between this study and our previous retrospective investigation in a similar patient population (Table 5) [16]. Whilst this was likely due to training in appendiceal ultrasound techniques for the sonographers involved, we also noted a cultural shift in terms of an enhanced expectation that the appendix should be located during every examination, which was previously recognized by Cundy et al. [6] as an important factor in their study, which reported an excellent visualization rate. An acknowledged source of bias is that the sonographers were not blinded to the study objectives. Our improved visualization rate is in contrast with another recent study, which sought to implement a more structured ultrasound report including secondary signs, and reported a decrease in visualization from before (43.9%) to after the intervention (32.7%) [23]. Interestingly, that study did not detail any involvement of the sonographers performing the examinations in the quality improvement process. The engagement of staff performing these examinations, be they sonographers or doctors, appears crucial in achieving high visualization rates.

Of our false-positive findings, two had normal histology, one had a perforated Meckel diverticulum that the surgical team felt clinically warranted laparoscopy, and eight cases did not proceed to surgery. The remainder of the false-positive cases were histologically proven lymphoid hyperplasia with the presence of one or more secondary signs (appendicolith, echogenic mesentery, free fluid, or hyperaemia) and a wide range of increased appendiceal diameters noted on ultrasound. A previous study demonstrated that it was possible to differentiate lymphoid hyperplasia on ultrasound from acute appendicitis by identifying a thickened, hypoechoic lamina propria in a moderately enlarged appendix capable of resisting compression [24].

This prospective study was designed to overcome some of the limitations of a prior baseline retrospective study in the same patient population, particularly the lack of documentation of secondary signs and the availability of some clinical data, such as height and weight [16]. A limitation of this study was the lack of pathological evidence in children that did not undergo appendectomy, which may not have captured cases of spontaneously resolving appendicitis. Another limitation was the potential for children to be lost to follow-up if they did not re-present at our institution; however, this was considered unlikely, as our institution is the only tertiary pediatric hospital in the region.

Studies with excellent visualization rates tend to have higher patient volumes, providing more opportunity to develop and maintain the skills necessary to achieve such results [6,7,25,26]. Our findings mirrored this effect of patient volume through differences in visualization rates based on sonographer full-time employment equivalency. When we considered only definitive radiological diagnoses in this study (161 of 230 patients), sensitivity and specificity were very high (100% and 96.8%, respectively). Excluding only the 7.9% (n=18) of cases that were equivocal and integrating the presence of secondary signs into what would otherwise be probable radiological findings maintained a high degree of accuracy (sensitivity, 97.6%; specificity, 91.9%) with an appendix visualization rate of 68.7%. To make a meaningful comparison of the diagnostic accuracy of this study and our prior one, we compared the examinations where the appendix was visualized, revealing an increase in the positive likelihood ratio from 1.63 to 9.23, with little difference in the negative likelihood ratio, which changed from 0.055 to 0.056 [16].

As children with suspected appendicitis do not always present to a tertiary pediatric health facility, there is a need to engage with centres providing non-dedicated pediatric ultrasound services that have lower imaging study volumes. It is also sometimes not feasible to perform ultrasound. If ultrasound cannot be performed in a way that affects pre-test probability, there may be a temptation to perform CT for further clarification or to proceed to surgery, both of which have inherent risks. Whilst Partain et al. [20,23] saw the visualization rate diminish after implementing a structured radiology report with integrated secondary signs, they found that admission and CT use were nonetheless reduced, as secondary signs provided guidance to clinicians in cases that would otherwise be equivocal and of little value.

Different academic curricula and the nature of sonography training in some centres with low pediatric or emergent caseloads can lead to varied levels of knowledge or confidence in pediatric appendiceal ultrasound. The results of this study provide clear evidence that engagement of sonographers through training in an adaptive technique to locate the appendix can improve appendix visualization. Better awareness and documentation of secondary signs of appendicitis can reduce the number of truly equivocal findings, especially in practices with a lower visualization rate. This approach can better equip radiologists to convey the likelihood of disease to referrers even in the absence of a visualized appendix. Ultrasound plays an important role in children with suspected appendicitis, reducing the need for CT and associated ionizing radiation concerns, without increasing the risk of morbidity through missed appendicitis. The value of ultrasound findings to clinicians can be improved through sonographer engagement in the recognition of secondary sonographic signs, leading to a higher visualization rate and reduced equivocal findings.