Quality of care for acute abdominal pain in children.

OBJECTIVE: To assess quality of care for children presenting with acute abdominal pain using validated indicators.
DESIGN: Audit of care quality for acute abdominal pain according to 21 care quality indicators developed and validated in four stages. SETTING AND PARTICIPANTS: Medical records of children aged 1-15 years receiving care in 2012-2013 were sampled from 57 general practitioners, 34 emergency departments (ED) and 28 hospitals across three Australian states; 6689 medical records were screened for visits for acute abdominal pain and audited by trained paediatric nurses. OUTCOME MEASURES: Adherence to 21 care quality indicators and three bundles of indicators: bundle A-History; bundle B-Examination; bundle C-Imaging.
RESULTS: Five hundred and fourteen children had 696 visits for acute abdominal pain and adherence was assessed for 9785 individual indicators. The overall adherence was 69.9% (95% CI 64.8% to 74.6%). Adherence to individual indicators ranged from 21.6% for assessment of dehydration to 91.4% for appropriate ordering of imaging. Adherence was low for bundle A-History (29.4%) and bundle B-Examination (10.2%), and high for bundle C-Imaging (91.4%). Adherence to the 21 indicators overall was significantly lower in general practice (62.7%, 95% CI 57.0% to 68.1%) compared with ED (86.0%, 95% CI 83.4% to 88.4%; p<0.0001) and hospital inpatient settings (87.9%, 95% CI 83.1% to 91.8%; p<0.0001).
CONCLUSIONS: There was considerable variation in care quality for indicator bundles and care settings. Future work should explore how validated care quality indicator assessments can be embedded into clinical workflows to support continuous care quality improvement. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Evidence-based or consensus-based clinical practice guidelines (CPG) and recommendations are accepted as central in supporting care quality and safety.1 CPGs can only be effective if implemented into practice by front-line clinicians, and yet adherence is rarely measured.2 The CareTrack studies in adults1 and in children3 report that approximately 60% of care is delivered as recommended in CPGs for common conditions. Acute abdominal pain in children is a common reason for accessing medical care in all healthcare settings, including general practice (GP) and emergency departments (ED). It is associated with high hospital admission rates and significant morbidity.4 5 There are no reliable Australian prevalence estimates for childhood acute abdominal pain, but 10% of all visits with general practitioners (GPs) are for acute abdominal pain and approximately 13% of children aged <18 years who experience abdominal pain will seek medical attention.6 7

Acute abdominal pain is an important symptom of many clinical conditions, ranging from mild and self-limiting conditions (eg, gastroenteritis, constipation and urinary tract infections) to life-threatening conditions requiring urgent surgical intervention (eg, intestinal obstruction, incarcerated inguinal hernia, testicular torsion, intussusception, volvulus and appendicitis).4 5 8 In adolescent girls, ovarian torsion or cyst rupture and ectopic pregnancy should also be considered as differential diagnoses.4 9 Abdominal pain may also be a feature of systemic conditions like diabetic ketoacidosis.4 10 Appropriate and definitive management of acute abdominal pain therefore relies on an accurate and timely differential diagnosis of the aetiology. This, in turn, requires clinicians taking and documenting a detailed history, thorough physical examination and ordering appropriate investigations when indicated.

The CareTrack Kids (CTK) study developed and validated care quality indicators for 17 common childhood conditions including acute abdominal pain, according to available international CPGs.3 The overall findings of CTK have been previously reported.3 In this paper, we focus on the CTK care quality indicators for acute abdominal pain in Australian children presenting to the GP, ED and inpatient hospital settings. To our knowledge, the quality of care in children with acute abdominal pain has not been previously measured in a systematic manner and at scale.3 11–13

Methods

Study design

This study is part of the CTK programme of research that assessed the quality of documented care in Australian children aged 0–15 years during 2012 and 2013 for 17 childhood conditions.3 CTK is the second large-scale population-based study to assess the quality of care provided to children, and the first to include acute abdominal pain.3 The CTK methods have been described in detail elsewhere.3 14–16 Here, we describe those aspects of the methods that relate specifically to acute abdominal pain.

Development of indicators

We defined a clinical care quality indicator as a measurable component of a standard or guideline, with explicit criteria for inclusion, exclusion, time frame and practice setting. Thirty recommendations were extracted from eight available CPGs which covered recommendations for GP, ED and inpatient settings.15 Of the 30 recommendations three were excluded because they were imprecise, for example, used indefinite auxiliary verbs such as ‘may’ or ‘consider’.

Candidate recommendations underwent a three-round modified Delphi internal review by two paediatricians and one general practitioner involved in CTK. Four paediatricians and two GPs external to CTK further reviewed and modified the recommendations, again using a modified Delphi method.14 A modified RAND-UCLA method was used for both internal and external reviews.17–22 Additionally, reviewing clinicians recorded the level of clinical impact and whether it was feasible to extract information about the recommendations from patient records.14 The final seven recommendations were reformatted as 21 care quality indicator questions (see online supplementary appendix table 1). For example, a recommendation that children should not be inappropriately referred for imaging was used to create three separate indicators, ABDO13–ABDO15 (table 1).

Table 1

Description of care quality indicators and components of bundles

IndicatorID	Bundle*	Indicator description
ABDO01	A	Children who presented with acute abdominal pain had their pain history documented (eg, onset, location, severity, progression, character).
ABDO02	A	Children who presented with acute abdominal pain were screened for other associated features (eg, fever, cough, vomiting, pallor, lethargy, anorexia).
ABDO03	A	Children who presented with acute abdominal pain were assessed for possible urinary tract infection (eg, offensive urine, dysuria, frequency).
ABDO04†		Children who presented with acute abdominal pain had their gynaecological history documented.
ABDO05	A	Children who presented with acute abdominal pain had their history of bowel movements documented (eg, stool pattern, stool quality (size, hard/soft, odour), constipation, diarrhoea).
ABDO06	A	Children who presented with acute abdominal pain had their medical history documented (eg, surgical, medical, family and travel).
ABDO07	B	Children who presented with acute abdominal pain had their vital signs (including HR and temperature) documented.
ABDO08	B	Children who presented with acute abdominal pain had the severity of their dehydration (eg, absent, mild, moderate or severe dehydration) documented.
ABDO09	B	Children who presented with acute abdominal pain received an abdominal assessment for tenderness (eg, local or generalised tenderness).
ABDO10	B	Children who presented with acute abdominal pain received an abdominal assessment for signs of acute abdomen (eg, rebound, guarding or rigidity).
ABDO11	B	Children who presented with acute abdominal pain had other abdominal findings (eg, masses, distention, palpable faeces, bowel sounds) documented.
ABDO12†		Children who presented with acute abdominal pain received an assessment of their inguinoscrotal area (eg, swelling or tenderness).
ABDO13	C	Children who presented with non-traumatic acute abdominal pain who do not require exclusion of a differential diagnosis of acute obstruction or perforation did not receive an abdominal X-ray or CT scan.
ABDO14	C	Children who presented with non-traumatic acute abdominal pain, and NO bile (yellow or green) stained vomit, did not receive an abdominal X-ray or CT scan.
ABDO15	C	Children who presented with non-traumatic acute abdominal pain, and NO suspected ingestion of radiopaque foreign objects, did not receive an abdominal X-ray or CT scan.
ABDO16‡		Children who presented with acute severe abdominal pain were administered intravenous morphine or intranasal fentanyl.
ABDO17§		Children who presented with acute mild abdominal pain, who require analgesia, were administered paracetamol or ibuprofen.
ABDO18		Children who presented with acute abdominal pain who were moderately dehydrated had their blood sugar measured.
ABDO19		Children who presented with acute abdominal pain who were severely dehydrated OR shocked had their electrolytes measured.
ABDO20		Children who presented with acute abdominal pain who were severely dehydrated OR shocked had their blood sugar measured.
ABDO21		Children who presented with acute abdominal pain who were severely dehydrated OR shocked received fluid resuscitation (initial bolus 20 mL/kg normal saline).

*Indicators were bundled into three bundles: A, B and C; not all indicators belonged to a bundle.

†ABDO04 restricted to females over the age of 10 years, ABDO12 restricted to males.

‡Emergency department setting only.

§In the general practice (GP) setting this was interpreted as paracetamol or ibuprofen was ‘recommended’ as general practitioners generally do not administer analgesia.

HR, heart rate.

Sample size, sampling process and data collection

The general sampling methods have been published3; additional details specific to acute abdominal pain can be found in online supplementary appendix 2. CTK sampling targeted at least 400 medical records for acute abdominal pain and 6000 medical records for the 16 other conditions. Nine purpose-trained paediatric nurses screened medical records for visits for acute abdominal pain. They reviewed the selected records for adherence to quality indicators and collected relevant data.14 16 Participating sites (GP, EDs and hospitals) were selected from randomly chosen administrative units (‘health districts’) in Queensland (Hospital and Health Services), New South Wales (Local Health Districts) and South Australia (Local Health Networks). Of the invited sites, 92% of hospitals and 24% of GPs agreed to participate (see online supplementary appendix 2). Records of children aged 1–15 years receiving care in 2012 and 2013 were assessed in each site. Records of children aged less than 12 months were not assessed because of the difficulty in confirming whether the presenting problem was acute abdominal pain or not.

Analysis

Adherence was measured on three levels: overall adherence to the 21 indicators, adherence with each individual indicator and adherence with indicator ‘care bundles’. At the individual indicator level, adherence was measured as the percentage of each indicator scoring a ‘yes-adherent’ for all eligible visits. Individual indicators describing similar or related aspects of care were grouped into three bundles: bundle A-History, bundle B-Examination and bundle C-Imaging (table 1). Indicators within a bundle were given equal importance as the CPGs from which the indicators originate did not rank their importance. Adherence with a bundle required all indicators in that bundle be scored ‘yes-adherent’ for a particular visit. Sampling weights were constructed as specified in online supplementary appendix 2 to adjust for oversampling of states and healthcare settings and for sampling within health districts.3 Similar methods were used to calculate estimates for each healthcare setting across all indicators, and across all indicators in each of the three indicator bundles.

The weighted data were analysed in SAS V.9.4 using the SurveyFreq procedure. Variance was estimated by Taylor series linearisation and the primary sampling unit (health district) was specified as the clustering unit. Stratification and, where appropriate, domain analyses were used (see online supplementary appendix 2, online supplementary appendix table 2). Exact 95% CIs were generated using the modified Clopper-Pearson method, except when adherence was 100%, where the unmodified method was used. In both individual indicator and indicator bundle reports, results were suppressed if there were <25 eligible assessments. Statistical significance, where calculated, was based on the F-test approximation of the Rao-Scott χ2 test, which adjusts for the design effect.

Ethical considerations

We received primary ethics approvals from all hospital networks and the Royal Australian College of General Practitioners (HREC/14/SCHN/113; HREC/14/QRCH/91; HREC/14/WCHN/68; NREEC 14-008), in addition to 34 site-specific approvals allowing for data collection from medical records without individual patient consent as the study entailed minimal risk to healthcare professionals and patients.16 To protect participants, statutory immunity from litigation was obtained in recognition of CTK as a quality assurance activity, from the Federal Minister for Health under Part VC of the Health Insurance Act 1973 (Commonwealth of Australia).

Results

Characteristics of audited medical records

Five-hundred and fourteen children had 696 visits for abdominal pain across three states of Australia (figure 1). Visits for acute abdominal pain were retrospectively reviewed in 57 GP, 34 ED and 28 inpatient settings. A median of 14 indicators were assessed per record with a total of 9785 indicator assessments undertaken. Most children (484, 94.2%) had one or two visits. Only 4.3% were 1–4 years old, 14.6% were 3–4 years old, 59.1% were 5–11 years old and 22.8% were 12–15 years old.

Figure 1

Geographical and clinical settings of the study participants. Total number of consultations (n=696): 228 in New South Wales, 254 in Queensland, 214 in South Australia. 327 emergency department, 81 hospital admissions and 288 general practice (GP) consultations (adapted from https://mapchart.net/, CC BY-SA 4.0).

Overall adherence

The overall adherence for all indicators and settings was 69.9% (95% CI 64.8% to 74.6%). Overall adherence was significantly lower in the GP setting (62.7%; 95% CI 57.0% to 68.1%) compared with ED setting (86.0%; 95% CI 83.4% to 88.4%; p<0.0001) and inpatient hospital setting (87.9; 95% CI 83.1 to 91.8; p<0.0001).

Adherence to individual indicators

Table 2 summarises adherence to individual indicators. The highest adherence was recorded for indicator ABDO13, appropriate use of imaging, at 91.4% (95% CI 77.1% to 98.1%). The lowest adherence was recorded for indicator ABDO08, assessment of severity of dehydration, at 21.6% (95% CI 13.0% to 32.5%). Other indicators with less than 50% adherence included: documentation of vital signs (ABDO07; 43.9%; 95% CI 30.2% to 58.3%) and examination of the inguinoscrotal area in boys (ABDO12; 31.6%; 95% CI 16.0% to 51.0%) (table 2).

Table 2

Proportion of encounters where practice was assessed to be adherent to the care quality indicators

IndicatorID	Children, n	Visits, n	Proportion adherent% (95% CI)
ABDO01	513	695	74.1 (55.7 to 87.8)
ABDO02	513	695	86.2 (76.7 to 92.9)
ABDO03	511	690	53.5 (40.0 to 66.7)
ABDO04*	65	106	69.6 (43.8 to 88.6)
ABDO05	514	696	81.0 (69.7 to 89.5)
ABDO06	514	696	62.5 (47.4 to 76.0)
ABDO07	513	695	43.9 (30.2 to 58.3)
ABDO08	509	688	21.6 (13.0 to 32.5)
ABDO09	514	696	90.4 (83.5 to 95.1)
ABDO10	512	694	71.7 (60.2 to 81.5)
ABDO11	503	677	78.0 (73.7 to 81.9)
ABDO12*	234	307	31.6 (16.0 to 51.0)
ABDO13	469	614	91.4 (77.1 to 98.1)
ABDO14	493	651	89.9 (79.0 to 96.3)
ABDO15	501	663	89.8 (79.1 to 96.2)
ABDO16‡	58	83	76.2 (60.6 to 87.9)
ABDO17†	315	405	55.9 (43.5 to 67.7)
ABDO18	23	25	64.6 (26.8 to 92.2)
ABDO19	4	4	Insufficient data
ABDO20	3	3	Insufficient data
ABDO21	2	2	Insufficient data

For detailed descriptions of the quality indicators please see table 1.

*ABDO04 restricted to females over the age of 10 years, ABDO12 restricted to males.

†ABDO17 in the general practice (GP) setting was interpreted as paracetamol or ibuprofen was ‘recommended’ as general practitioners generally do not administer analgesia.

‡Emergency department setting only.

Adherence to indicator bundles

Across all settings, adherence to bundle A-History (29.4%) and bundle B-Examination (10.2%) was lower than to bundle C-Imaging (91.4%) (table 2). Adherence to bundle A-History was significantly lower in the GP setting at 12.6% (95% CI 4.8% to 25.1%) compared with 67.9% (95% CI 57.1% to 77.5%; p<0.0001) in the ED setting and 73.3% (95% CI 57.4% to 85.8%; p<0.0001) in the hospital inpatient setting (table 3). Adherence to bundle B-Examination was also significantly lower in the GP setting at 2.3% (95% CI 0.5% to 6.8%) compared with 27.9% (95% CI 20.3% to 36.5%; p<0.0001) in the ED setting and 35.7% (95% CI 17.5% to 57.6%; p<0.001) in the inpatient setting.

Table 3

Adherence to bundles of related indicators by healthcare setting

BundleID*	Bundle description	Included indicators	Healthcaresetting	Children, n	Visits, n	Indicators assessed, n	Proportion adherent% (95% CI)
A	History: Children who presented with acute abdominal pain had appropriate history and associated features documented.	01–03,05–06	GP	244	286	1430	12.6 (4.8 to 25.1)
			ED	256	325	1625	67.9 (57.1 to 77.5)
			Inpatient	71	79	395	73.3 (57.4 to 85.8)
			All settings	511	690	3450	29.4 (19.3 to 41.3)
B	Examination: Children who presented with acute abdominal pain had appropriate assessment.	07–11	GP	240	280	1400	2.3 (0.5 to 6.8)
			ED	247	314	1570	27.9 (20.3 to 36.5)
			Inpatient	69	75	375	35.7 (17.5 to 57.6)
			All settings	498	669	3345	10.2 (7.7 to 13.1)
C	Imaging: Children who presented with non-traumatic acute abdominal pain, or without documented prespecified justification†, did not receive an abdominal X-ray or CT scan.	13–15	GP	238	274	822	89.0 (68.6 to 98.2)
			ED	220	274	822	97.7 (95.2 to 99.1)
			Inpatient	51	56	168	99.4 (92.6 to 100)
			All settings	464	604	1812	91.4 (76.9 to 98.2)

*The indicators in each bundle were equally weighted. For the bundle to be scored ‘yes adherent’ all individual indicators in that bundle had to be adherent.

†The three prespecified justifications were for children who: (1) required exclusion of a differential diagnosis of acute obstruction or perforation; (2) had bile (yellow or green) stained vomit; or (3) had suspected ingestion of radiopaque foreign objects.

ED, emergency department;GP, general practice.

Discussion

For the first time we have measured and demonstrated gaps in care quality for paediatric abdominal pain according to validated indicators. The overall adherence to care quality indicators for acute abdominal pain was 69.9%, which is higher than the overall average adherence for all 17 CTK conditions combined (59.8%).3 In the current study, health professionals in all healthcare settings demonstrated a high level of adherence to indicators related to avoidance of unnecessary imaging when there was no history of abdominal trauma, and no signs or history of obstruction or perforation, or ingested foreign objects. This aligns with the recommendations in the current Evolve guidelines of Royal Australasian College of Physicians.23 24 The WHO recently highlighted the potential risks of ionising radiation for children due to unnecessary medical imaging.25 In our study, inappropriate imaging may have occurred in approximately 10% of encounters in the GP setting, representing low-value care that could be avoided.

Our results suggest that education about the importance of thorough physical examination and assessment of vital signs is needed to prevent potential misdiagnosis of serious conditions associated with paediatric acute abdominal pain. For example, regularly assessing and documenting vital signs helps with the early detection of deteriorating patients,26 yet adherence with this indicator occurred in less than half of the visits in our study. Similarly, a comprehensive physical examination is needed to avoid missing serious diagnoses such as incarcerated inguinal hernia or testicular torsion, but examination of the inguinoscrotal area was done in only ~30% of visits for abdominal pain in boys. Approximately a quarter of children presenting to ED with severe acute abdominal pain did not receive appropriate pain relief. Pain relief in children is thought to make healthcare safer, alleviating anxiety, allowing for a more thorough examination, and it does not interfere with differential diagnosis or treatment.10 27

Adherence to indicator bundle A-History and bundle B-Examination was low across all settings which may indicate that clinicians adhered to some quality indicators within a bundle, but rarely adhered to all. Adherence to bundle A-History and bundle B-Examination was significantly lower in the GP setting. There are several potential reasons for this. The assessment of adherence to quality care indicators relied on appropriate documentation in patients’ medical records. It is possible that GPs asked all recommended history-taking questions and conducted all examinations and assessments but failed to document this sufficiently. We know that medical records are often incomplete and sometimes inaccurate.28 29 An average visit with an Australian GP is approximately 15 min.30 Undertaking and documenting a thorough assessment and physical examination in a sick child in this short time frame is challenging for GPs.

Clinicians tend to look first to their own professional college or association for guidance, however, there were no specific guidelines for paediatric acute abdominal pain endorsed by the Royal Australian College of General Practitioners at the time of data collection for our study. Although the indicators used in our study were derived from several guidelines, including those intended for use in ED and hospital settings, all indicators were deemed appropriate by independent paediatrician and general practitioner reviewers for the Australian clinical context, including the GP setting.14 16 Adherence to guidelines in the ED and hospital settings is often supported by sophisticated clinical governance structures, more reliance on teamwork, use of decision aids such as flow charts and prompts embedded in electronic medical records, but this is seldom the case in the GP context. GPs see over 160 different conditions in their daily practice,31 and clinical recommendations are ever proliferating, but few are embedded into GP workflows and medical software.32 Initiatives such as the development and dissemination of clinical pathways have the potential to support GPs in their clinical decision-making and referral practices.33

Strengths and limitations

The CTK study has several strengths including population-based sampling, robust methods for the development of quality care indicators, its size and scope, with data drawn from 113 healthcare delivery settings including GP, ED and inpatients, from three Australian states, and assessment of 9785 individual visits.3 14 Only 24% of invited GPs participated in the CTK study which might represent a selection bias towards those GPs more likely to be adherent to care quality indicators.3 It is estimated that approximately 10% of non-adherence to the indicators may be due to lack of documentation,12 and this is a limitation as the study assessed what was documented and not what was done. Furthermore, the clinical outcomes for the audited visits were not collected, limiting the ability to assess the consequences of adherence or non-adherence for patients.

Auditing of patients’ medical records in person, by trained nurse auditors, is a barrier to repeating the study to monitor changes over time, due to cost and logistical constraints, multiple ethics permissions, variability of medical record platforms and their quality. The current lack of unified electronic medical record platforms and the inability to harvest data across jurisdictions, sectors and facilities remains a significant barrier.

Practical implications and next steps

Our study confirms previous findings showing that embedding recommendations into clinical practice to effect changes in the care delivered to patients remains a long-standing challenge for healthcare systems.34 35 However, developing and validating a set of care quality indicators for acute abdominal pain that are applicable across healthcare settings affords opportunities to apply these indicators to monitor care quality improvement over time. To do this effectively without overburdening clinicians, automated solutions that interface with clinical software could support routine data collection to increase the completeness of documentation submitted by clinicians. Real-time data analysis and feedback to individual clinicians, teams and organisations shows promise to support benchmarking and to reduce unwarranted variation in care.36

Adoption of clinical recommendations could be further improved by encouraging codevelopment and codesign of dissemination, implementation and evaluation plans alongside the development of content of CPGs.37 Embedding recommendations into electronic decision support tools,32 using clinician champions,35 and developing companion recommendations for consumers are increasingly used to support adoption.38 39 Evaluation of the relationship between adherence to care quality indicators and patient outcomes and experiences may be a strong incentive for clinicians to change practice.40 Research in this area should be actively encouraged. There may be opportunities to embed the measurement of care quality indicators in pay-for-performance programmes such as the soon-to-be-implemented Australian Practice Incentive Program—Quality Improvement, which aims to reward GPs who collect and share data about quality performance indicators.41 42

Conclusion

Our study has, for the first time, enabled measurement of gaps between recommended care and documented care for acute abdominal pain in children in all healthcare settings in Australia. The significantly lower adherence in GP may reflect setting-specific challenges routinely faced by GPs, such as limited time, seeing children infrequently and limited decision support and clinical governance structures. Future work should explore how the validated indicators can feasibly be applied by individual clinicians and organisations in a structured and automated manner using clinical software, where possible, to help them to continuously assess and improve care quality.