A QI Partnership to Decrease CT Use for Pediatric Appendicitis in the Community Hospital Setting.

The primary aim of this quality improvement initiative was to decrease the use of computerized tomography (CT) in the evaluation of pediatric appendicitis in a community general emergency department (GED) system by 50% (from 32% to 16%) in 1 year.
METHODS: Colleagues within a State Emergency Medical Service for Children (EMSC) community of practice formed the quality improvement team, representing multiple stakeholders across 3 independent institutions. The team generated project aims by reviewing baseline practice trends and implemented changes using the Model for Improvement. Ultrasound (US) use and nondiagnostic US rates served as process measures. Transfer and "over-transfer" rates served as balancing measures. Interventions included a GED pediatric appendicitis clinical pathway, US report templates, and case audit and feedback. Statistical process control tracked the main outcomes. Additionally, frontline GED providers shared perceptions of knowledge gains, practice changes, and teamwork.
RESULTS: The 12-month baseline revealed a GED CT scan rate of 32%, a US rate of 63%, a nondiagnostic US rate of 77%, a transfer to a children's hospital rate of 23.5%, and an "over-transfer" rate of 0%. Project interventions achieved and sustained the primary aim by decreasing the CT scan rate to 4.5%. Frontline GED providers reported positive perceptions of knowledge gains and standardization of practice.
CONCLUSIONS: Engaging regional colleagues in a pediatric-specific quality improvement initiative significantly decreased CT scan use in children cared for in a community GED system. The emphasis on the community of practice facilitated by Emergency Medical Service for Children may guide future improvement work in the state and beyond.

INTRODUCTION

Problem Description

The diagnosis of appendicitis in children is facilitated by the use of computed tomography (CT).[1,2] However, emerging practice patterns have demonstrated comparable diagnostic accuracy in diverse care settings through the use of a pathway that integrates clinical decision rules, basic laboratory results, and ultrasound (US) to minimize childhood exposure to ionizing radiation.[3-8]

We implemented a pediatric transfer follow-up and feedback program within an ongoing statewide Emergency Medical Services for Children (EMSC) partnership to improve pediatric emergency care in the community hospital general emergency department (GED) setting.[9] Colleagues from this initiative identified several opportunities for collaborative quality improvement (QI) work. Pediatric appendicitis represented a significant cohort of commonly transferred children. The perception of CT scan overuse emerged and became the focus of this EMSC led QI partnership between a community hospital system and 2 regional pediatric centers.

Available Knowledge

In the United States, 90% of children seek emergency medical care outside specialized children’s hospitals.[10] However, variability in the quality of pediatric care exists between specialized pediatric centers and community GEDs.[11-15] Many GEDs have low pediatric volumes and little access to pediatric subspecialists and ancillary services. They have high transfer rates and low pediatric “readiness” concerning proper equipment, protocols, and personnel to manage a pediatric emergency.[11,12,16-18]

Given the low volume of pediatric patients seen in individual GEDs, QI work can offer a more significant impact when it focuses on common conditions.[19] Abdominal pain is a common chief complaint of children presenting for emergency care, with the incidence of pediatric appendicitis reported between 1% and 8%.[7] The CT scan offers an accurate and timely diagnosis of appendicitis; however, it exposes the child to ionizing radiation.[8] The long-term sequela of such exposure is a driver behind current clinical practice recommendations favoring the US as the initial imaging modality for pediatric appendicitis.[3,7]

The national rate of CT scan use for pediatric appendicitis diagnosis is unknown. However, a recent publication showed that 27% of subjects received a CT scan, and those presenting initially to a community GED experienced significantly higher odds of receiving a CT scan.[20] Another study uncovered that low pediatric visit volume was an independent risk factor for CT scan use.[21] These findings conflict with regional practice patterns at the children’s hospitals, reporting a CT rate below 8% in their appendicitis evaluations.[22]

Rationale

Several solutions have been suggested to reduce the use of CT scans for undifferentiated abdominal pain in pediatric patients.[4,6,7] Specifically, care pathways exist, each with well-performing risk stratification scores preceding the decision to obtain a CT scan; some of which have been validated in the GED setting.[23-25] Further, the Society of Pediatric Radiology and the American Pediatric Surgical Association promote the practice of obtaining diagnostic imaging at the institution best prepared to interpret the study and act on the results.[2,17]

Despite solid evidence and readily available tools to help address disproportionate CT use in the GED setting, additional factors influence why a GED may favor CT scan use in pediatric appendicitis evaluations. First, there may be a delay in knowledge translation and integration of pediatric best practice recommendations to frontline community GED providers charged with staying up to date on a vast breadth of evidence-based recommendations.[26] Second, US use for any application requires a frequency of practice and expertise at interpretation, both challenges for low pediatric volume community GEDs.[1-3,5,25] And third, logistical barriers may encourage providers and families to favor immediate and definitive diagnosis with a CT over transfer to another institution.[18,27]

To address the above, we formed a community of practice within a state EMSC partnership to decrease pediatric CT scan use in the community GED setting.[28] In applying the Model for Improvement framework, the team needed to collaborate to understand baseline performance and local culture to impact this specific pediatric practice.[29]

Specific Aims

The QI team aimed to decrease CT use in children ages 3–18 years presenting to a community GED system for an appendicitis evaluation by 50% (from 32% to 16%) in 1 year. Further, within this patient population, the team aimed to reduce the proportion of patients obtaining a CT scan without a prior US by 25% (from 85% to 60%). Secondary goals included increasing US use by 15% (from 63% to 78%), and decreasing the incidence of nondiagnostic US scans by 20% (from 77% to 57%), a figure more consistent with the literature.[2,5]

The team tracked transfer and “over-transfer” rates of 23.5% and 0%, respectively, as balancing measures. These balancing measures were chosen to monitor for increased and potentially unnecessary resource utilization for patients in this cohort due to this improvement work.

METHODS

Context

Middlesex Health has 3 GEDs that care for approximately 82,000 total annual visits, of which 10,000 are pediatric. These GEDs are staffed by the Middlesex Health department of emergency medicine, composed of approximately 45 providers, most of whom (65%) are physicians. Pediatric topics are reviewed annually through lectures or simulation, and pediatric cases are included in departmental quality assurance reviews as indicated. This pediatric-focused work is facilitated by the Middlesex Health Pediatric Care Champion (WL), who has extensive EMSC experience.[11] Before the appendicitis QI project, no formal pediatric care pathways existed in the Middlesex Health System.

The state of Connecticut has 2 dedicated children’s hospitals, both within a few hours’ drive from all quadrants of the state. Transfers between Middlesex and one of the children’s hospitals are based on geographic proximity, family preference, and provider preference. The Middlesex system has 2 campuses near Connecticut Children’s Medical Center (CCMC) and 1 campus closer to Yale-New Haven Children’s Hospital (Yale). Middlesex has no written transfer agreements or formal policies dictating the minimum patient age upon which a Middlesex general surgeon would operate.

Both CCMC and Yale have fully staffed pediatric radiology and pediatric surgery teams collaborating with pediatric emergency medicine (PEM) providers to reach appropriate patient dispositions for appendicitis evaluations. As such, stakeholders on the QI team represented emergency medicine, radiology, and surgery departments at all 3 institutions.

Interventions

The PDSA interventions are annotated alphabetically in parenthesis within the article text. These same annotations are also reflected in the key driver diagram (Fig. 1), listed chronologically in Table 1 and correspond to the labels on the main results statistical process control chart (SPCC) p-chart (Fig. 3).

Fig. 1.

Key Driver Diagram. *GED: general emergency department; US: ultrasound; CT: computed tomography.

Table 1.

PDSA Interventions over the Course of the QI Project

	Intervention	Description	Date
A	Baseline planning phase	Project team assembly, data analysis, and project aim development.	July–September 2019
B	QI team & frontline GED provider meeting	Meeting with frontline GED providers, baseline data share, best available evidence presentation, feedback solicited on QI project aims, and initial care pathway.	October 2019
C	Case audit and Feedback	Every case transferred between Middlesex and Yale was reviewed. Deviations from the pathway were respectfully brought to the attention of the frontline provider by the local project champion.	October 2019
D	Draft #1 of care pathway	Pathway rolled out to the 3 Middlesex GEDs.	November 2019
E	Radiology planning meetings	A series of meetings and proposals aimed at improving the use and reliability of US in the GED.	November 2019
F	Frontline provider scripting	Plain language explanations for pathway endpoints offered to GED providers to use with patients and families.	December 2019
G	Ultrasound internal case review	Quality assurance review by the chair of Middlesex Radiology to gain understanding of US practice and guide training of US technicians.	December 2019
H	Appendicitis US report templates	Standardized appendicitis reporting to offer frontline providers insight into the US findings, as opposed to simply a positive, negative, or nondiagnostic result.	January 2020
I	Progress review #1	Email reports of the primary, process, and balancing measures were shared with the frontline providers and project stakeholders.	January 2020
J	Revised care pathway	Frontline providers expressed interest in having the probability of appendicitis at different nodes in the care pathway to guide their decision-making and conversations with families.	February 2020
K	Progress review #2	Email reports of the primary, process, and balancing measures were shared with the frontline providers and project stakeholders.	June 2020

Fig. 3.

Annotated Primary Outcome SPCC p-chart – The Rate of CT use for Pediatric Appendicitis evaluations in the Middlesex Health System. *Annotations as noted in the article body, Table 1, and key driver diagram (Fig. 1).

In the summer of 2019, the QI team formed and analyzed the baseline practice patterns for pediatric appendicitis evaluations at Middlesex and generated project aims (annotation A, Fig. 1, Table 1, Fig. 3). These aims originated from the global mission of EMSC, and the planned interventions to reach these aims were organized into a key driver diagram (Fig. 1).

In October 2019, we initiated the project with a meeting between Middlesex frontline providers and the QI team (annotation B). This meeting had 2 primary purposes. First, to present the Middlesex frontline providers the best available evidence for pediatric appendicitis evaluation, and second, to share the results of Middlesex’s baseline practice patterns. Both offered context and rationale behind the project aims. During this meeting, time was allotted for feedback by reviewing the key driver diagram drafts (Fig. 1) and the original Middlesex Health Pediatric Appendicitis Care Pathway (Fig. 2A).

Fig. 2.

The Middlesex Health Pediatric Appendicitis Care Pathway. A, B, Initial and Revised Middlesex Health Pediatric Appendicitis Care Pathway.

The Middlesex Health Pediatric Appendicitis Care Pathway

The Middlesex Health Pediatric Appendicitis Care Pathway (Middlesex Pathway) uses the Pediatric Appendicitis Score to risk-stratify patients for diagnostic imaging, promotes US as the preferred initial imaging modality, and encourages consultation with regional pediatric surgeons before ordering a CT scan (Fig. 2A, annotation D).[4,7,25,30] The pathway underwent 2 updates. The first occurred immediately after the October 2019 project meeting. The second update reflected frontline providers’ later request for additional data on the pathway offering the likelihood of appendicitis based on the different endpoints (Fig. 2B, annotation J).[30,31] By including this information, providers were able to tailor discussions with families about the role of CT or to justify a transfer to a children’s hospital. Additionally, frontline providers were offered scripting examples to explain their clinical decision-making to patients and families (see appendix 1, annotation F, Supplemental Digital Content, which shows Provider Scripting Example, http://links.lww.com/PQ9/A322).

Advancing Ultrasound Value

Consistent with the literature, Middlesex frontline providers reported dissatisfaction with the US use in their practice setting, noting high rates of nondiagnostic appendicitis studies.[2,4,5] Meetings between the QI team, the department of pediatric radiology at Yale, and the radiology department at Middlesex resulted in several proposed interventions targeting US yield (annotation E). Administrative credentialing prevented cross-institutional, in-person training of US technicians. However, local internal quality assurance review began for all Middlesex pediatric appendicitis scans (annotation G). Additionally, Middlesex implemented an US report template and scoring system used at Yale, which assists frontline providers to interpret nondiagnostic studies (see appendix 2, annotation H, Supplemental Digital Content, which shows Middlesex Health Ultrasound Report Template for Appendicitis Scans, http://links.lww.com/PQ9/A323).[31]

Audit and Feedback

Throughout the project, the primary investigator (MPG) provided audit and feedback on individual appendicitis evaluation transfers to the Middlesex project champion (WL) (annotation C). This feedback was intentionally filtered through the local point person to ensure that deviations from the Middlesex Pathway were addressed between the frontline provider and a trusted colleague.[9,32,33]Additionally, quarterly progress reports were shared between the QI team and Middlesex frontline staff (annotations I and K).

Study of the Interventions

The QI team queried the Middlesex Health electronic medical record for all children between the ages of 3 and 18 who presented with the following chief complaints: fever, nausea, vomiting, abdominal pain, appendicitis, small bowel obstruction, and acute gastroenteritis. Patients were excluded if a clear alternative diagnosis was reached (eg, UTI, strep throat, blunt abdominal trauma, inflammatory bowel disease, or history of prior abdominal surgery). These patients were narrowed for inclusion in the analysis if they were transferred to a children’s hospital for further appendicitis evaluation and management or if their appendicitis evaluation at Middlesex included imaging with an US or a CT.[4] As interventions began in October 2019, the prior 12 months were used to define Middlesex Health’s baseline performance.

The QI team also solicited frontline Middlesex GED providers to complete a questionnaire targeting the themes of knowledge, systems-based practice, and inter-institutional teamwork in their evaluation of pediatric appendicitis (see appendix 3, Supplemental Digital Content 3, which shows Middlesex Provider Questionnaire, http://links.lww.com/PQ9/A324). The QI team iteratively developed the questionnaire and piloted it with colleagues outside the project for intention and clarity of construct.[34]

Measures

The QI team calculated the monthly mean rates of CT, US, and transfer to children’s hospitals among the total number of children who met analysis criteria. Monthly quotients of nondiagnostic US over the total US’s generated the rates of nondiagnostic US. The QI team calculated the number of CT scans ordered without a prior US by the number of such instances over the total number of CT scans ordered. The balancing measure of “over-transfers” was the number of children who did not receive a CT, US, or pediatric surgical consult at the children’s hospital over the total number of transfers. Seventy-two-hour recidivism to Middlesex GEDs was also tracked.

The QI team administered the questionnaire to Middlesex frontline providers at the project’s initiation and again after 9 months. Respondents were asked to rank their level of agreement to probes targeting the themes mentioned above on a 5-point Likert scale from strongly disagree to strongly agree. Additionally, the 9-month questionnaire solicited feedback on the project from respondents via free text.

Analysis Plan

The team used p-charts abiding by conventional statistical process control chart interpretation rules to identify special cause and guide centerline shifts of the critical outcome metrics.[35-37] Given the infrequent monthly occurrence of children receiving a CT scan without a prior US, “over-transfers,” and 72-hour recidivism throughout all phases of the project, the Chi-Square test compared these proportions between baseline and intervention periods. The team analyzed questionnaire data as follows: t-tests and Chi-Square tests compared questionnaire respondent characteristics, and t-tests compared mean Likert Scale responses between the 2 data sets. Finally, the team performed a thematic interpretation of free-text responses of the feedback.

Ethical Considerations

All institutional review boards deemed the project protocol exempt from formal review by local guidance around QI work.

RESULTS

One hundred eighty-four patients met analysis criteria, 81 patients formed the baseline sample (October 2018–September 2019), and 103 were included in the intervention period (October 2019–November 2020). Patients included in analysis were similar between baseline and intervention periods with respect to age (12.14 versus 12.47, P = 0.61), biologically male status (43% versus 39%, P = 0.61), and the rate of clinical appendicitis diagnosed at Middlesex (9% versus 16%, P = 0.16).

Figure 3 depicts the p-chart of the project’s primary outcome demonstrating the decrease in the rate of CT scan use in Middlesex GEDs from 32% to 4.5%, surpassing the project’s primary aim. Special cause variation was observed in September 2020 when the CT scan rate of 33% (3/9) surpassed the new upper control limit.[35-37] On review, all of these patients were above 16 years old, and the eldest had their appendectomy at the main Middlesex campus (as opposed to one of the children’s hospitals). After September 2020, 2 additional months of data fell back into form with the preceding eleven months.

Figure 4A shows the rate of US use in the evaluation of pediatric appendicitis in the Middlesex system. The team did not observe an appreciable change from the baseline practice of 63%. Figure 4B shows the nondiagnostic US rate in these evaluations, noting an unchanged baseline practice of 77%. Additionally, the proportion of CT scans obtained without a prior US was 85% (23/27) during the baseline period and 70% (7/10) during the intervention period (difference of 15%, P = 0.31).

Fig. 4.

Key Process and Balancing Measures. A–C, SPCC p-charts for the process measures of US rates and nondiagnostic US rates and the balancing measure of transfer rates to the children’s hospitals.

Figure 4C reports the balancing measure of transfer rate to a children’s hospital. Increased transfers are noted, with 7-consecutive points falling above the baseline centerline from October 2019 to April 2020. In May 2020, only 1 patient met the analysis criteria and was not transferred. After that, 6-consecutive points from June 2020 to November 2020 remained above the centerline, with special cause appreciated in October and November 2020. This signal meets the criteria for centerline shift as 13 of the 14 intervention period data points fall above the baseline.[35-37] Further, when comparing the proportion of patients transferred between the baseline and the intervention periods, an increase of 32.8% was noted (23.5% (19/81) versus 56.3% (58/103), P < 0.0001). Finally, there were zero “over-transfers” within our study population during the entire project and no change in pediatric abdominal pain recidivism to Middlesex (7% versus 3%, P = 0.19).

Respondents to the questionnaire were similar at both samplings: response rate [54% (25/46) versus 61% (26/43), P = 0.44]; age (in years) of respondents (40 versus 42, P = 0.59); role [MD/DO 84% (21/25) versus 77% (20/26), P = 0.39]; years working at Middlesex (9.6 versus 7.8, P = 0.44); and years working in emergency care (13.2 versus 12.1, P = 0.67).

As noted in Table 2, respondents perceived gains in pediatric appendicitis knowledge, standardization of practice, and improved teamwork between the 3 institutions. Finally, the respondents’ free-text feedback emphasized the desire for improvements in pediatric appendicitis US capability and an appreciation for how the Middlesex Pathway facilitated standardization of care to decrease CT use.

Table 2.

Average Frontline Providers’ Perceptions of Knowledge Gains, Practice Changes and Teamwork as Measured by 5-point Likert scales (1 = strongly disagree; 5 = strongly agree)

	Pre (n = 24)	Post (n = 24)	P
Knowledge gains
I am confident in my ability to evaluate pediatric abdominal pain/“rule out appendicitis” in my current practice setting.	3.58	4.38	P = 0.007
I am comfortable with my understanding of the evidence behind pediatric abdominal pain/“rule out appendicitis” workups.	3.58	4.50	P < 0.001
Practice changes
I feel there is a consistent practice pattern amongst my colleagues in the Middlesex system for pediatric abdominal pain/“rule out appendicitis” workups.	2.50	3.50	P < 0.001
Teamwork cultivated
A practice guideline agreed upon between transferring and receiving facilities for pediatric abdominal pain/“rule out appendicitis” workups can improve patient care.	4.21	4.70	P = 0.049

DISCUSSION

Previous works described the emergency care of children in the United States as “uneven,” attributed to discrepancies in provider experience, inadequate access to equipment, lack of pediatric-specific care protocols, and a lack of a pediatric care champion all impact a GED’s pediatric “readiness.”[11,12,15] Through a multi-institutional, multi-departmental collaborative approach, the QI team surpassed and sustained the primary aim of decreasing CT use in pediatric appendicitis evaluations in the GED setting. Through continuing case audit and feedback, quarterly data review, and novel PDSA cycles addressing US utility, we aim to sustain the near elimination of CT scan in pediatric appendicitis evaluations in the Middlesex system.

Consistent with prior publications, CT scan use in the Middlesex GED system was higher than reported rates at regional children’s centers.[20-22] To combat this practice, the Middlesex Pathway used a clinical prediction rule shown to assist providers to risk-stratify those needing advanced imaging.[4,7,30] Importantly, the QI team did not design the Middlesex Pathway to diagnose appendicitis but rather to facilitate continuity of care between a large GED system and 2 distinct regional children’s hospitals. In essence, the QI project aimed to diagnose appendicitis in the ideal location for the patient, all while preventing over-transfers. This practice is supported in the literature, and, in our experience, it cultivated a perception of teamwork amongst varied stakeholders at 3 large, independent institutions.[2,7,28] However, this focus was a necessary clarification for Middlesex frontline provider buy-in and has implications to consider before broad application and spread.

Unfortunately, the US use rate and the rate of nondiagnostic US studies were not impacted by the project’s interventions. The Middlesex Pathway emphasized US use as a primary imaging modality for children whose pediatric appendicitis score alone did not sufficiently eliminate the suspicion of appendicitis.[4,7] To this end, several interventions to improve US utility were initiated, but few came to fruition due to administrative barriers. As a result, the Middlesex US practice did not improve as nondiagnostic studies remained above the national average.[1,5]

Interpretation of these data is 2-fold. On the one hand, novel interventions may improve GED US practice, such as innovative training practices, image sharing networks, or point of care US.[38] On the other hand, regional partners may elect to acknowledge US operator dependence and the fact that a low volume of pediatric studies inhibits expertise. In doing so, one could consider a future pathway that foregoes the US step in the GED and focuses on the decision between immediate or delayed referral to a pediatric center for diagnostic imaging.[5,25] Future studies comparing the performance of such pathways to the Middlesex Pathway may add essential data points. Notably, in response to this QI project, Middlesex Radiology has launched an internal QI project. To date, they are promoting the practice of identifying the appendix by technicians regardless of an US’s indication.

The balancing measure of transfer rates to pediatric centers significantly increased while no “over transfers” occurred during the QI project.[35-37] Interpretation of these data starts with a closer review of Middlesex’s baseline transfer practice, noting a process that is both unstable and reflects a possible secular trend of increased transfers to pediatric centers.[35-37] We hypothesize this trend may represent Middlesex’s prior engagement in pediatric readiness QI work with EMSC.[9,11,39] The increased transfer rate does reflect the finding that all children who were transferred received the “evidence-based” appendicitis practice.[2] But while this practice may be “evidence-based,” a transfer that ends up in a subsequent children’s hospital ED discharge may not be the best way to meet the family’s expectations. After all, a pediatric transfer carries significant health care costs, may pose safety risks, and imposes challenging family logistics.[27] Focus groups of families whose children were transferred, regardless of their ultimate disposition, would add a valuable dataset to understand the QI project’s implications beyond the primary outcome achieved. Further, should this initiative spread, a reassessment of the children’s hospital stakeholders on the receiving end of additional transfers would also be important.

There are significant limitations to review. First, the interventions of this regional QI project suited a specific context, which may limit generalizability. Second, the Pediatric Appendicitis Score’s test characteristics have been improved upon with the Pediatric Appendicitis Risk Calculator, which has recently been validated in the GED setting.[24,30] As such future iterations of the Middlesex Pathway or any future projects with similar aims should incorporate the Pediatric Appendicitis Risk Calculator to improve validity and generalizability, which may also curb transfers.[24] Third, it should be acknowledged that Middlesex Health has a long-standing relationship with EMSC and an experienced pediatric care champion, both instrumental factors in this QI project’s success.[9,39] Therefore, these factors may limit the generalizability in regions where these relationships amongst colleagues at different institutions remain underdeveloped.[11] Fourth, the primary outcome of this work has crucial pediatric patient safety considerations. Yet, the project did not evaluate all possible balancing measures such as the final diagnosis of pathology-proven appendicitis, combined emergency department length of stay, or the patient/family experience, all of which may be valuable data to consider before spread. Finally, the validity analysis of our questionnaire is limited.[34]

In conclusion, through a multi-institutional and multi-departmental pediatric-specific QI project, the QI team surpassed and sustained the primary aim of decreasing the rate of CT scan use in pediatric appendicitis evaluations in the community GED setting. Future directions include careful consideration of US’s role in GED settings and a deeper understanding of stakeholders’ perspectives such as families and children’s hospital personnel that the spread of this work would impact. The next steps will include ongoing data tracking, new iterations to the Middlesex Pathway, and the stepwise spread of the project to similar GED systems. Finally, the demonstration of this successful collaboration between multiple institutions offers a blueprint for conducting novel QI work under the coordination and guidance of state EMSC organizations.

DISCLOSURE

The authors have no financial interest in relation to the content of this manuscript.

ACKNOWLEDGEMENTS

The authors acknowledge Doruk Ozgediz, MD, MSc and Adam Berkwitt, MD, for their contributions to the study.