Risk score to stratify children with suspected serious bacterial infection: observational cohort study.

OBJECTIVES: To derive and validate a clinical score to risk stratify children presenting with acute infection. STUDY DESIGN AND PARTICIPANTS: Observational cohort study of children presenting with suspected infection to an emergency department in England. Detailed data were collected prospectively on presenting clinical features, laboratory investigations and outcome. Clinical predictors of serious bacterial infection (SBI) were explored in multivariate logistic regression models using part of the dataset, each model was then validated in an independent part of the dataset, and the best model was chosen for derivation of a clinical risk score for SBI. The ability of this score to risk stratify children with SBI was then assessed in the entire dataset. MAIN OUTCOME MEASURE: Final diagnosis of SBI according to criteria defined by the Royal College of Paediatrics and Child Health working group on Recognising Acute Illness in Children.
RESULTS: Data from 1951 children were analysed. 74 (3.8%) had SBI. The sensitivity of individual clinical signs was poor, although some were highly specific for SBI. A score was derived with reasonable ability to discriminate SBI (area under the receiver operator characteristics curve 0.77, 95% CI 0.71 to 0.83) and risk stratify children with suspected SBI.
CONCLUSIONS: This study demonstrates the potential utility of a clinical score in risk stratifying children with suspected SBI. Further work should aim to validate the score and its impact on clinical decision making in different settings, and ideally incorporate it into a broader management algorithm including additional investigations to further stratify a child's risk.

Introduction

Serious bacterial infections (SBIs) are a leading cause of morbidity and mortality among children worldwide. A recent review of child deaths in England, Wales and Northern Ireland confirmed that the largest single cause of death in children dying of an acute physical illness was infection, and found evidence that healthcare professionals in both hospital and primary care had difficulty identifying serious illness in children.1 In another study of over 400 children with meningococcal sepsis in the UK, only half were referred to hospital at their first primary care consultation, usually after a delay of several hours following the first symptoms and/or signs of sepsis.2 Early recognition of SBIs is associated with better treatment outcomes3 4 but is challenging as many of the early clinical features also occur in self-limiting illnesses. In developed countries, discriminating serious infection from self-limiting illness is further complicated by the low prevalence of serious infection in children presenting to hospital emergency departments as well as primary care settings.5 Better clinical tools are therefore required to improve the recognition of children with serious acute illness, particularly SBIs.1 6 7

Serious bacterial infections (SBI) are a leading cause of childhood illness and death in the UK and worldwide.

In clinical practice, discriminating children with SBI from those with other self-limiting infections is challenging.

A simple, validated clinical tool to risk stratify and guide further management of children with suspected SBI would greatly improve their care.

The high specificity for SBI of some clinical signs supports their use in clinical guidelines to identify seriously ill children.

The utility of a simple clinical score to stratify children according to SBI risk is clearly demonstrated.

Combining this clinical tool with laboratory tests in a broader management algorithm may further improve SBI diagnosis pathways.

Clinical prediction rules utilising key clinical features to generate a diagnostic or prognostic score are increasingly used to risk stratify patients in a wide variety of clinical situations, particularly those in which diagnostic decisions are complicated, and in some areas have had a major impact on clinical service delivery.8–15 We aimed to develop a simple clinical score with the ability to risk stratify children with possible SBIs presenting to a paediatric emergency department.

Methods

Patients and setting

As part of a large prospective study of patient care pathways, detailed clinical data were collected prospectively on all children presenting to the Queen's Medical Centre Emergency Department in Nottingham between September 2000 and March 2001, and September 2001 and March 2002, with the exception of neonates and children requiring immediate emergency resuscitation at presentation. A triage nurse recorded vital signs prior to assessment by emergency department clinical staff. All clinical data including investigations, management and final diagnoses were directly entered onto a standard proforma. Completed proformas were then scanned using Formic electronic data capture software to minimise data entry errors. Study clinicians checked the data for completeness, resolved data gaps and inconsistencies by re-review of the clinical notes, and recorded additional clinical data on children who were admitted. Children who re-attended hospital within 1 week of discharge from either the emergency department or the ward were identified from the electronic patient register, their notes reviewed, and final diagnoses and SBI classification amended in the light of their second presentation. A consultant paediatrician re-reviewed the patient records of all those admitted to check the accuracy of the data, particularly in relation to the final diagnosis recorded.

Children were eligible for inclusion in this analysis if their differential diagnosis at presentation included acute infection. We excluded from the analysis children for whom data were insufficient to confidently assign outcome, or who had missing dates of birth.

Clinical definitions

Automated measurements were used for temperature (tympanic thermometer), pulse rate, blood pressure and pulse oximetry. Tachypnoea, tachycardia and hypotension were defined according to UK Advanced Paediatric Life Support guidelines16; children for whom no blood pressure recordings were available were assumed not to be hypotensive for the purpose of the analysis. We adopted pragmatic definitions of mild hypoxia (SaO2<95% and/or documented oxygen therapy) and severe hypoxia (SaO2<90% and/or SaO2<92% despite documented oxygen therapy), and defined a ‘risk factor for infection’ as a documented history of any condition known to increase a child's risk of invasive bacterial infection. Developmental delay was included separately as a potential risk factor, since several studies have documented an increased rate and severity of acute illness including infection among this group.17 18 Other definitions reflected those commonly used in clinical practice (table 1); data collected routinely on all children included level of consciousness, capillary refill time, hydration status, and presence and type of rash.

Table 1

Univariable associations between clinical variables and risk of serious bacterial infection (SBI) in children presenting to the emergency department

Clinical variable		N*	SBI, n (%)	OR	95% CI	p Value
Age group	Infants (1–11 months)	701	26 (3.7)	1.35	0.31 to 5.81	0.784
	Preschool (12–59 months)	855	31 (3.6)	1.32	0.31 to 5.62
	School age (5–11 years)	323	15 (4.6)	1.70	0.38 to 7.65
	Adolescents (12–15 years)	72	2 (2.8)	1.00	–
Sex	Male	1081	39 (3.6)	1.00	–	0.562
	Female	850	35 (4.1)	1.15	0.72 to 1.83
Risk factor for infection†	No	1898	66 (3.5)	1.00	–	<0.0001
	Yes	53	8 (15.1)	4.93	2.23 to 10.9
Developmental delay	No	1923	70 (3.6)	1.00	–	0.003
	Yes	28	4 (14.3)	4.41	1.49 to 13.1
Consciousness level (AVPU score)	Alert	1913	70 (3.7)	1.00	–	0.027
	Responds to voice	18	0 (0.0)	0.00	–
	Responds to pain	5	2 (40.0)	17.6	2.86 to 107.7
	Unresponsive	1	0 (0.0)	0.00	–
State variation	Awake	1909	68 (3.6)	1.00	–	<0.0001
	Eyes close briefly	31	6 (19.4)	6.50	2.57 to 16.4
	Falls asleep	1	0 (0.0)	0.00	–
Temperature‡	35.0–36.4°C	426	9 (2.1)	0.82	0.37 to 1.82	<0.0001
	36.5–37.5°C	780	20 (2.6)	1.00	–
	37.5–38.4°C	205	17 (4.4)	1.86	1.01 to 3.43
	≥38.5°C	305	28 (9.2)	4.10	2.39 to 7.05
Tachycardia	Not tachycardic	974	22 (2.3)	1.00	–	0.0001
	Tachycardic	651	39 (6.0)	2.76	1.61 to 4.71
Capillary refill time	<2 s	1755	65 (3.7)	1.00	–	<0.0001
	≥2 s	40	7 (17.5)	5.52	2.34 to 13.0
Hydration status	Well hydrated	1834	58 (3.2)	1.00	–	<0.0001
	Dry mucous membranes	102	12 (11.8)	4.08	2.11 to 7.90
	Reduced skin turgor	9	4 (44.4)	24.5	6.30 to 95.3
Hypotension§	Not hypotensive	1947	74 (3.8)	1.00	–	0.691
	Hypotensive	4	0 (0.0)	0.00	–
Tachypnoea	Not tachypnoeic	656	19 (2.9)	1.00	–	0.028
	Tachypnoeic	932	48 (5.2)	1.82	1.06 to 3.13
Hypoxia	Not hypoxic	1620	49 (3.0)	1.00	–	<0.0001
	Mild hypoxia	265	16 (6.0)	2.06	1.15 to 3.68
	Severe hypoxia	66	9 (13.6)	5.06	2.36 to 10.9
Purpuric rash	None	1666	66 (4.0)	1.00	–	0.641
	Purpuric rash	16	1 (6.3)	1.62	0.21 to 12.4
Petechial rash	None	1666	66 (4.0)	1.00	–	0.793
	Petechial rash	42	2 (4.8)	1.21	0.29 to 5.13
Macular rash	None	1757	70 (4.0)	1.00	–	0.184
	Macular rash	194	4 (2.1)	0.51	0.18 to 1.41

N indicates the total number children in a particular category. Note that the sum of N across all categories of a particular variable is equal to the total number of children for which data were available for that variable. For most variables this is less than the total number of children (1032) included in the analysis due to missing data for that particular variable.

Identified risk factors for bacterial infection in this analysis included neutropaenia, sickle cell disease, primary ciliary dyskinesia, Down's syndrome, glycogen storage disease 1b, Hunter's syndrome, Hurler's syndrome, cryptogenic cirrhosis, biliary atresia, vesico-ureteric reflux, hydronephrosis, congenital pneumonia and a documented history of recurrent bacterial infections.

Temperature categories 35.0–36.4°C and 36.5–37.5°C were combined in the multivariate analysis to allow temperature to be modelled as a linear variable. No children had a temperature <35.0°C in this study.

Blood pressure recordings were only available for 94 children; the remaining children were classified as not hypotensive for the purposes of the analysis.

SBI was defined a priori as admission to hospital plus any of the following (in the absence of an alternative non-infective or non-bacterial diagnosis to explain the clinical and laboratory findings): positive bacterial cultures from blood or another normally sterile site in the appropriate clinical context, radiological signs of pneumonia, clinical meningitis plus a cerebrospinal fluid polymorphonuclear leukocytosis, acute febrile purpura, deep collection(s) requiring intravenous antibiotics±surgical drainage, a white blood cell count ≥20×109/l, a C reactive protein ≥120 mg/l, or a final diagnosis of septic arthritis, osteomyelitis, empyema or mastoiditis.

Statistical methods

Analyses were performed using Stata v 10 (StataCorp LP). The distribution of each variable was summarised with respect to SBI, and crude OR derived. The sensitivity, specificity, positive and negative predictive values (PPV, NPV) and positive and negative likelihood ratios (LR+, LR−) were reported for each variable.

We randomly assigned each child to either a ‘derivation’ or a ‘validation’ dataset, such that ∼85% of children were in the derivation set. Multivariate logistic regression models were then derived using the derivation set to develop a clinical prediction score for SBI. We included in the model only variables with at least a weak association with SBI in the univariate analysis (likelihood ratio test (LRT) p value ≤0.1), first confining this to categorical variables (which are more easily applied in clinical practice), and then repeating the analysis using the available continuous data to explore to what extent this improved the model.

Backwards stepwise logistic regression was performed first using ‘standard’ selection criteria, such that variables not significantly associated with SBI (Wald p value <0.5; t test p value ≤0.1 for continuous variables) are sequentially dropped from the model, and then using ‘sign OK’ selection,19 whereby variables are eliminated from the model only if the sign (+ or −) of the regression coefficient (β) is discordant with known or expected clinical associations (since standard selection criteria may not be optimal in the analysis of smaller datasets15 19–21).

Each model took the form: SBI=α+β1X1+β2X2+β3X3+…+βiXi, where α is a constant, βi are the regression coefficients for each variable (Xi) in the model, and Xi are coded as 0 or 1 (for binary variables), integers (for ordered categorical variables) or continuous numbers (for continuous variables). For each model we generated a score for each child, such that the score=β1X1+β2X2+β3X3+... +βiXi. The ability of each score to discriminate children with SBI was quantified in the derivation dataset, and validated in the validation dataset, by calculating the area under the receiver operator characteristics (ROC) curve (AUC) of score as a predictor of SBI.

Finally, the best model was selected to derive a simpler score for clinical use. Criteria for model selection included both discriminant ability (defined by the AUC) and model simplicity. We generated a simplified clinical prediction score using weightings defined by the β coefficients of the chosen model, multiplied if necessary by a constant, and rounded as far as possible to the nearest integer. The discriminant ability of this final ‘SBI score’ was tested by plotting an ROC curve of the score as a predictor of SBI in the entire dataset (derivation and validation sets combined). Risks of SBI were presented stratified by score.

Ethics approval

Ethics approval was granted by the Nottingham Research Ethics Committee and the ethics committee of the London School of Hygiene and Tropical Medicine.

Results

There were 2341 eligible children, of whom 17 were excluded because insufficient data were available to confidently determine outcome. A further 373 children were excluded because their date of birth and therefore their age could not be positively determined from the hospital records due to uncertainty in interpreting their handwritten unique patient identifier number on the proforma; the proportion of children with SBI did not differ significantly between those for whom age data were and were not available (p=0.47).

The final analysis therefore included 1951 children. Their median age was 19 months (range 1 month to 15 years) and 55.4% were boys. Five hundred and forty-six (28.0%) children were admitted at first presentation, of whom 72 had SBI. Thirty-six children re-attended hospital within a week of their initial presentation and within the same illness episode, 18 of whom were admitted and 2 of whom had SBI. The principal final diagnoses of the 74 (3.8%) children with SBI included pneumonia (52), sepsis without a clear focus (14), soft tissue infections (5), urinary sepsis (2) and tonsillitis plus sepsis (1).

Several clinical variables were strongly associated with SBI in the univariate analysis (table 1), including signs of a child's a neurologic, haemodynamic and respiratory state, and a known risk factor for infection. Although hypotension was not significantly associated with SBI, it is important to note that blood pressure data were only available for 103 (5.3%) children.

The predictive values of each variable are summarised in table 2. Apart from tachypnoea (sensitivity 71.6%), the sensitivity of most clinical signs was poor. However, some clinical signs, such as those of reduced level of consciousness/alertness and impaired perfusion were highly specific, and demonstrated more clinically useful positive likelihood ratios. The low prior probability of SBI in this population explains the apparently high negative predictive values of many variables, reflected in the poor utility of most negative likelihood ratios.

Table 2

Predictive value of clinical variables for serious bacterial infection

Clinical variable	Sensitivity, % (95% CI)	Specificity, % (95% CI)	PPV, % (95% CI)	NPV, % (95% CI)	LR+ (95% CI)	LR− (95% CI)
Risk factor for infection	10.8 (4.8 to 20.2)	97.6 (96.8 to 98.2)	15.1 (6.7 to 28.0)	96.5 (95.6 to 97.3)	4.5 (3.3 to 6.2)	0.91 (0.67 to 1.25)
Developmental delay	5.4 (1.5 to 13.3)	98.7 (98.1 to 99.2)	14.3 (4.0 to 32.7)	96.4 (95.4 to 97.2)	4.2 (4.2 to 4.3)	0.96 (0.95 to 0.97)
Consciousness level
Not alert*	2.8 (0.34 to 9.7)	98.8 (98.2 to 99.3)	8.3 (1.0 to 27.0)	97.5 (96.7 to 98.1)	2.4 (1.7 to 3.3)	0.98 (0.7 to 1.4)
No response to voice	2.8 (0.34 to 9.7)	99.8 (99.5 to 99.9)	33.3 (4.3 to 77.7)	96.4 (95.4 to 97.2)	13.0 (9.2 to 18.2)	0.97 (0.69 to 1.4)
Unresponsive	0.0 (0.0 to 5.0)†	99.9 (99.7 to 100.0)	0.0 (0.0 to 97.5)†	96.1 (95.2 to 97.0)	0.0	1.0
State variation
Eyes close briefly or falls asleep	8.1 (3.0 to 16.8)	98.6 (98.0 to 99.1)	18.8 (7.2 to 36.4)	96.4 (95.5 to 97.2)	5.8 (7.0 to 4.9)	0.93 (0.78 to 1.1)
Falls asleep	0.0 (0.0 to 4.9)†	99.9 (99.7 to 100.0)	0.0 (0.0 to 97.5)†	96.2 (95.2 to 97.0)	0.0	1.0
Temperature
≥37.5°C	60.8 (48.8 to 72.0)	64.5 (62.2 to 66.7)	6.5 (4.8 to 8.6)	2.4 (1.6 to 3.4)	1.7 (0.65 to 4.5)	0.61 (0.23 to 1.61)
≥38.5°C	37.8 (26.8 to 49.9)	84.8 (83.1 to 86.4)	9.2 (6.2 to 13.0)	97.1 (96.1 to 97.9)	2.5 (1.1 to 5.7)	0.73 (0.32 to 1.7)
Tachycardia	63.9 (50.6 to 75.8)	60.9 (58.4 to 63.3)	6.0 (4.3 to 8.1)	97.7 (96.6 to 98.6)	1.6 (0.67 to 4.0)	0.59 (0.24 to 1.5)
Capillary refill time ≥2 s	9.7 (4.0 to 19.0)	98.1 (97.3 to 98.7)	17.5 (7.3 to 32.8)	96.3 (95.3 to 97.1)	5.1 (4.0 to 6.5)	0.92 (0.72 to 1.2)
Hydration status
Dry mucous membranes	21.6 (12.9 to 32.7)	94.9 (93.8 to 95.9)	14.4 (8.5 to 22.4)	96.8 (95.9 to 97.6)	4.3 (2.3 to 7.8)	0.83 (0.45 to 1.5)
Reduced skin turgor	5.4 (1.5 to 13.3)	99.7 (99.4 to 99.9)	44.4 (13.7 to 78.8)	96.4 (95.4 to 97.2)	20.2 (20.1 to 20.4)	0.95 (0.94 to 0.96)
Hypotension	0.0 (0.0 to 4.9)†	99.8 (99.5 to 99.9)	0.0 (0.0 to 60.2)†	96.2 (95.3 to 97.0)	0.0	1.0
Tachypnoea	71.6 (59.3 to 82.0)	41.9 (39.4 to 44.4)	5.2 (3.8 to 6.8)	97.1 (95.5 to 98.2)	1.2 (0.47 to 3.2)	0.68 (0.26 to 1.8)
Hypoxia
Mild or severe	33.8 (23.2 to 45.7)	83.7 (81.9 to 85.3)	7.6 (4.9 to 10.9)	97.0 (96.0 to 97.8)	2.1 (0.95 to 4.5)	0.79 (0.36 to 1.7)
Severe	12.2 (5.7 to 21.8)	97.0 (96.1 to 97.7)	13.6 (6.4 to 24.3)	966 (95.6 to 97.3)	4.0 (2.8 to 5.8)	0.91 (0.63 to 1.3)
Rash
Purpuric rash	1.5 (0.04 to 8.0)	99.1 (98.5 to 99.5)	6.3 (0.2 to 30.2)	96.0 (95.0 to 96.9)	1.6 (0.8 to 3.2)	0.99 (0.50 to 2.0)
Petechial rash	2.9 (0.4 to 10.2)	97.6 (96.7 to 98.3)	4.8 (0.6 to 16.2)	96.0 (95.0 to 96.9)	1.1 (0.79 to 1.6)	1.0 (0.71 to 1.4)
Macular rash	5.4 (1.5 to 13.3)	89.9 (88.4 to 91.2)	2.1 (0.6 to 5.2)	96.0 (95.0 to 96.9)	0.53 (0.53 to 0.54)	1.1 (1.0 to 1.1)

Not alert indicates only responds to voice or pain, or unresponsive.

One sided, 97.5% CI.

LR+, likelihood ratio of a positive test; LR−, likelihood ratio of a negative test; NPV, predictive value of a negative test; PPV, predictive value of a positive test.

After random weighted allocation of children to the derivation and validation sets, the derivation set comprised 1600 (85.2%) children without SBI and 64 (86.5%) children with SBI. The models with the best discriminative ability were those derived using ‘sign OK‘ selection. For the ‘sign OK’ model confined to categorical variables, the AUCs in the derivation and validation sets were 0.76 (0.68 to 0.82) and 0.85 (0.71 to 0.99), respectively. Inclusion of continuous data in the model did not markedly improve performance (AUC 0.76 and 0.86 in the derivation and validation sets, respectively), so the simpler model confined to categorical variables was selected. Regression coefficients and adjusted ORs and 95% CIs for each variable included in this model are given in table 3. A modified SBI score was derived based on the β coefficients of this model, such that:

Table 3

Regression coefficients (β), and adjusted OR and 95% CI for variables included in the chosen model

Variables	β Coefficient	OR	95% CI	p Value
History of developmental delay	1.6807	5.4	1.6 to 18.6	0.008
Risk factor for infection	1.4408	4.2	0.9 to 20.9	0.077
State variation	0.2336	1.3	0.3 to 4.7	0.728
Temperature category	0.6643	1.9	1.4 to 2.7	<0.001
Capillary refill time	0.6595	1.9	0.6 to 5.8	0.243
Dehydration category	1.3651	39	2.0 to 7.6	<0.001
Tachypnoea	0.1760	1.2	0.6 to 2.2	0.585
Hypoxia category	0.4734	1.6	1.1 to 2.4	0.026

The AUC for SBI score as a predictor of SBI in the derivation and validation sets combined was 0.77 (95% CI 0.71 to 0.83; figure 1). Table 4 shows the SBI score in tabular form more conveniently for clinical use, and risks of SBI stratified by SBI score are shown in table 5.

Figure 1

Receiver operator characteristics (ROC) curve of serious bacterial infection score as a predictor of serious bacterial infection in the entire dataset.

Table 4

Serious bacterial infection (SBI) score for prediction of serious bacterial infection

Clinical variable		Score*
Developmental delay	No delay	0
	Delay	4
Risk factor for infection	No risk factor	0
	Risk factor present	2
State variation	Eyes open	0
	Eyes close briefly	1
	Falls asleep	2
Temperature	<37.5°C	0
	37.5–38.4°C	1
	≥38.5°C	2
Capillary refill time (s)	<2	0
	≥2	1
Hydration status	Well hydrated	0
	Dry mucous membranes	2
	Reduced skin turgor	4
Respiratory rate	Not tachypnoeic	0
	Tachypnoeic	1
Hypoxia	Not hypoxic	0
	Mild hypoxia	1
	Severe hypoxia	2

Score indicates the sum of scores for each clinical variable (maximum score 18). Table 5 shows risks of serious bacterial infection stratified by SBI score.

Table 5

Risks of significant bacterial infection (SBI) stratified by SBI score

SBI score	Children (N)	Children with SBI (n)	Risk of SBI, % (95% CI)
≤5	1085	21	1.9 (1.2 to 2.9)
6	214	17	7.9 (4.7 to 12.4)
7	68	9	13.2 (6.2 to 23.6)
8	51	9	17.7 (8.4 to 30.9)
>8	29	10	34.5 (17.9 to 54.3)
Total*	1447	66	4.6 (3.5 to 5.8)

Total represents the total number of children in the entire dataset (derivation and validation sets combined) for whom data on all clinical variables included in the final model were available.

Discussion

By exploring the feasibility of developing a clinical prediction score for risk stratification of suspected serious bacterial illness in children, this study addresses an important challenge in everyday clinical practice in the UK and other developed countries.1 2 5 7 Among children presenting to emergency care in whom infection was suspected (where the prior probability of SBI was 5%), our SBI score was able to rule out SBI with reasonable confidence and precision in three quarters of children in the dataset to whom the score was applied (SBI score ≤5 associated with <2% risk of SBI), to identify children at the highest risk of SBI (a third of children with an SBI score >8 had SBI) and to risk stratify the remaining children into groups with increasing risk.

We modelled continuous data as categorical and continuous variables in separate models. Although less suitable for the development of a clinical prediction score for clinicians, continuous variables maximise information15 22 and can be utilised in computer-assisted diagnostic pathways. However, their inclusion in the model did not improve discriminative ability sufficiently to justify the increased complexity.

Previous studies in developed countries have attempted to derive clinical scores as a tool to identify seriously ill children at presentation.23–35 Most were carried out in populations with a higher incidence of SBI prior to the introduction of routine vaccination against Haemophilus influenza type b (Hib) and Streptococcus pneumoniae; others were designed to predict the need for admission to intensive care.9 Moreover, promising initial reports of diagnostic performance have not always been consistently reproducible.27 36 37 This probably reflects a combination of the inherent over-optimism of data-driven model predictions based on small datasets,15 19 21 38–40 failure to validate models and/or scores in an independent dataset,15 19 and spectrum bias arising from the use of case–control designs.41–43 In a recent study based mainly in primary care, a clinical algorithm was derived with the ability to rule out SBI with reasonable confidence.44 However, these results have not been validated in an independent dataset, and negative predictive values are likely to be lower in emergency department settings where the prior probability of SBI is higher.

The strengths of our study include the large number of children studied, detailed review of all clinical and laboratory data to assign outcome, validation of the model in an independent part of the dataset, and inclusion of all children presenting with suspected SBI in a cohort study design, which suggests the predictive value of our SBI score should be generalisable to other similar settings. Furthermore, focusing on children with suspected SBI rather than all children with serious illness allowed inclusion of clinical risk factors specific to infection, and may have helped improve model specificity. It is important to stress that while the SBI score should be generalisable to emergency departments in the developed world, it is unlikely to be applicable to developing world settings where the prevalence of SBI among paediatric presentations is much higher and there may be other important risk factors for infection such as malnutrition and HIV.5

A potential limitation of the study concerns the definition of SBI. No single, reliable gold standard exists for the diagnosis of invasive bacterial infections in children. Although cultures from blood and other sterile sites are highly specific, sensitivity is limited.45–47 The definition of SBI used has been formulated to take account of all available clinical and laboratory information in defining each clinical syndrome, including a child's response to treatment, and is designed to capture all children with invasive or otherwise SBIs who require admission to hospital. A potential criticism is the risk of incorporation bias, since clinicians are not blind to the admission clinical variables studied, which are likely to influence admission decisions; inclusion of clinical and laboratory data from the entire admission in assigning outcome is likely to only partly mitigate this bias. Nevertheless, we believe the definition adopted represents a pragmatic working definition of SBI. Importantly, we were also able to identify those children discharged from hospital who re-attended within a week, and to amend their SBI classification as required. While we cannot exclude the possibility that other children may have re-presented elsewhere, the nearest emergency department, which does not have a dedicated paediatric section, is 18 miles away, so the chance of this is small. Similarly, since sudden childhood death is rare in this population and most cases are brought to hospital, the chance of a child in this study dying at home and not re-attending hospital is also small.

Missing data also limit the power of our analysis. Although the proportion of data missing for individual variables was generally small, this became more significant in the multivariate analysis. Blood pressure is not routinely measured in paediatric clinical practice except in the sickest children, so data were missing for most patients. Nevertheless, blood pressure may have an important role in predicting SBI that this analysis would not be able to identify. The number of children for whom age data were missing is unfortunate as it further limits study power; however, this is unlikely to have introduced any bias since the underlying problem (poor legibility of the proforma) is unlikely to be related to the clinical presentation and was not associated with SBI.

Finally, measurement error is also possible, since emergency department staff assessed each clinical sign using routine clinical tools. For some variables this is likely to be minimal since automated measurements were used. Arguably, even for other variables this may be seen as a strength of the study, since it increases its generalisability to the routine clinical setting.

The implications of this study for clinical practice are threefold. First, the specificity of some clinical signs for SBI (such as reduced activity or level of consciousness, prolonged capillary refill time and reduced skin turgor) support the National Institute for Health and Clinical Excellence clinical guidelines7 which use these clinical features as ‘red flags’ to identify seriously ill children. The presence of these features in a febrile child should prompt a careful clinical assessment. Second, it clearly demonstrates the potential utility of a clinical score to stratify children according to their risk of SBI. Such a score might be incorporated into electronic decision support and medical records. Finally, it underlines the need to use such a clinical score as part of a broader algorithm to guide management decisions (including additional investigations, plans for review or admission for observation) to further stratify a child's risk. Future research should explore the benefit of including in the model other risk factors such as immunisation status, routine blood pressure measurement, and other potential markers of sepsis such as limb pain or cool peripheries,2 investigate how laboratory tests such as C-reactive protein, procalcitonin and white blood cell count may best be incorporated into a clinical algorithm to refine a child's risk status, and further validate the performance and impact14 48 of the score in different acute paediatric settings, including primary care.