Determining immunisation status of children from history: a diagnostic accuracy study.

OBJECTIVES: Children presenting unplanned to healthcare services are routinely asked about previous immunisations as part of their assessment. We aimed to assess the accuracy of screening children for immunisation status by history.
DESIGN: Diagnostic accuracy study. We compared information from patient history by a retrospective review of notes and used a central database of child immunisation records as the reference standard.
SETTING: Paediatric emergency department in a tertiary hospital in Oxford, UK. PARTICIPANTS: Consecutive children aged 6 months to 6 years presenting over a 2-month period. OUTCOME MEASURES: Proportion of children with documented immunisation history; sensitivity and specificity of detecting overdue immunisations by history compared to central records.
RESULTS: 1166 notes were surveyed. 76.3% children were asked about immunisations. The proportion of children who were fully immunised on central records was 93.1%. History had a sensitivity of 41.3% (95% CI 27% to 56.8%) and a specificity of 98.7% (95% CI 97.5% to 99.4%) for detecting those who were overdue. Negative predictive value was 95.8% (95% CI 93.9% to 97.2%). Only around a third of children with overdue immunisations are detected by the current screening methods, and approximately 1 in 20 children stated as being up to date are in fact overdue.
CONCLUSIONS: History had poor sensitivity for identifying overdue immunisation. Strategies to improve detection of children overdue with immunisation should focus on alternative strategies for alerting clinicians, such as linkage of community and hospital electronic records.

Childhood immunisation is essential to reduce the incidence of vaccine-preventable disease, but there is incomplete uptake of immunisations in the UK.

Contacts with healthcare providers are a useful setting in which to address incomplete immunisation and to provide information about vaccines and immunisation.

In a hospital or out-of-hours setting, immunisation status is usually determined by medical history, but there have been no UK studies assessing the accuracy of this method.

An immunisation history is not obtained in significant numbers of admissions to the emergency department and this is more likely for certain categories of admission.

Immunisation history is an insensitive method of detecting those who have incomplete immunisation particularly in children ≥16 months of age.

Clinicians should be aware that children reported to be up to date may in fact be overdue, and there is scope for educational interventions to clinicians to improve accuracy of questioning.

This is the first study assessing diagnostic accuracy of immunisation history in relation to the UK child health record system.

Over 1000 notes were surveyed, but, owing to the low prevalence of being overdue, CI for sensitivity are wide.

It is possible that the child health record database may overestimate children who are overdue because of incomplete records; this would have the effect of increasing the estimate of sensitivity of the immunisation history.

Background

An effective childhood immunisation programme is essential to reduce the incidence of childhood disease,1 and this is emphasised by UK data demonstrating a reduction in Measles, Mumps and Rubella immunisation (MMR) coverage resulting in an increase in measles notifications.2

Children who are overdue with immunisations may present more frequently as unplanned visits to healthcare settings such as emergency departments.3 It is important to identify these children, both to identify children who may be at increased risk of vaccine-preventable diseases, and to provide the opportunity for education about the importance of immunisation or for the missing immunisation(s) to be given at the time of presentation. NICE guidance states that the immunisation status of children should be checked at every available opportunity.4

In the UK, patient health records are not generally available in an acute hospital or out-of-hours general practice (GP) setting. Immunisation status is therefore usually determined by medical history or by reviewing the parent-held child health record (the ‘Red Book’). While a child health record will usually be accurate, it may not always be available, and it is often quicker to review immunisation status by history alone. Our aim was to assess the accuracy of ascertainment of immunisation status by history in children presenting to a paediatric emergency department (PED).

Methods

We undertook a retrospective review of medical notes of consecutive presentations to PED of children aged 6 months up to 6 years of age over a 2-month period in early 2012. We recorded age, clinical presentation, presence or absence of a documented immunisation history and any details of immunisation recorded. For children who were identified as overdue, the discharge documentation was reviewed to identify whether further action had been taken.

In the UK, although childhood immunisation generally takes place in GP settings, records are maintained by a central child health organisation for each region, which sends out initial invitations to parents and then reminders if immunisations have been missed. GP practices provide information to the child health organisation on a weekly basis on immunisations administered. In our region of Oxfordshire, this service is run from within the local primary care trust (PCT).

For each child who had an immunisation history taken, we compared the information obtained from the documented history with the PCT records, using this as the reference standard for immunisation status. We accessed PCT records at least 8 weeks after the date of PED presentation to allow time for records to have been updated, but immunisation status was recorded as it would have been on the date of presentation. For all children found to be overdue in PCT records, the accuracy of this information was checked by PCT staff against the original data provided by GP practices.

A child was defined as overdue if, according to the PCT record, they had not received a complete course of primary infant immunisations by 6 months, first dose MMR and PCV/Hib-MenC boosters (ie, 12–13 month immunisations) by 16 months or second dose MMR and dTaP-IPV (ie, pre-school booster, given at 3 years 4 months (40 months)) by 44 months.

We performed subgroup anaysis of diagnostic accuracy of history taking in relation to age group.

Calculations were done in Stata ‘diagt’ command with ‘sf’ option enabled.

Results

Screening and intervention

There were 1278 presentations of children aged 6 months to 6 years to the PED during the study period, of whom 1166 had written notes available to survey. Of these, 890/1166 (76.3%) had an immunisation history documented. Children more likely not to be asked about immunisation were those with a minor injury (making up 45.6% of those not asked compared to 18.1% of a sample of those who were asked) or surgical presentation (19.2% and 9.2%, respectively). Asking about immunisation status also reduced in frequency with increasing age of the child: 80.2% of children under 36 months were asked, compared to 68.9% of children 36 months or older.

For 2 of the 890 children with a documented history, the parents were uncertain of their immunisation status and these were excluded from the analysis, leaving 888 children with an unambiguous documented history. From this group, 49 children (5.5%) were recorded as having incomplete immunisation. Of note, only two had reference made to their incomplete immunisation status in their medical discharge documentation.

Comparison with PCT records

Upon review of the PCT records for the 888 children with a definite documented history, 92 were from outside the region or could not be linked with a National Health Service number. A further 17 children had a history of incomplete immunisation but were not sufficiently overdue to fulfil our criteria, the majority of whom were of ages shortly after immunisation due dates. We dealt with this by excluding all children in these age groups (13–15 and 40–43 months, 80 and 46 children, respectively). This left 670 children included in the main analysis. Table 1 shows the relationship of history to PCT immunisation records. From PCT data, the overall prevalence of incomplete immunisation was 6.9% (95% CI 5.1% to 9.1%). Of the 46 who were overdue, 19 had been identified in the medical history, giving a sensitivity of 41.3% (95% CI 27% to 56.8%). The specificity was 98.7% (95% CI 97.5% to 99.4%). Negative predictive value was 95.8% (95% CI 93.9% to 97.2%). Full diagnostic accuracy data are available in the online supplementary files.

Table 1

Comparison of history and PCT assessment of immunisation status

History	PCT
	Incomplete (positive)	Complete (negative)	Total
Incomplete (positive)	19	8	27
Complete (negative)	27	616	643
Total	46	624	670

PCT, primary care trust.

The sensitivity of the medical history in identifying children with incomplete immunisation varied with age (table 2), with a much higher sensitivity in the youngest age group.

Table 2

Diagnostic accuracy of history taking in determining immunisation status

Age range	n	Number incomplete by history	Number incomplete by PCT* data	Sensitivity (%)	Specificity (%)
Overall (6–71 months†)	670	19	46	41.3 (27.0 to 56.8)	98.7 (97.5 to 99.4)
6–12 months	173	7	11	63.6 (30.8 to 89.1)	96.3 (92.1 to 98.6)
16–39 months	334	7	20	35.0 (15.4 to 59.2)	99.4 (97.7 to 99.9)
≥44 months	163	5	15	33.3 (11.8 to 61.6)	100.0 (97.5 to 100.0)

95% CIs are given in parentheses

*Including those detected on history.

†Excluding age subgroups 13–15 and 40–44 months reflecting those periods before our cut-off for being overdue with a set of immunisations.

Sensitivity analyses including those children excluded from the main analysis above due to age criteria (both including and excluding borderline overdue children) had minimal effect on sensitivity and specificity results (see online supplementary data).

Discussion

Efficacy of history taking as a screening method for detecting incomplete immunisation relies on two factors: first, that the question is asked, and second, that the information given by parents is accurate.

Our data show that approximately three-quarters of children were asked about their immunisation status. The question was less likely to be asked for non-medical presenting complaints and with increasing age of the child. In our ED, only medical presentations are seen by paediatricians and this may explain why an immunisation history is taken less frequently with other presentations. Older children may be asked less frequently because it is perceived as less relevant, or there may be the confounding factor that they are more likely to present with non-medical complaints.

When an immunisation history was taken, there was a low sensitivity for detecting incomplete immunisation. Fewer than half of those who were in fact overdue correctly identified themselves as such. Of those who reported that they were incomplete, over half were insufficiently overdue to meet our criteria, and PCT records showed that the majority did receive their outstanding immunisations shortly following their PED presentation. The negative predictive value of 95.8% means that approximately 1 in 20 patients reported as up to date were in fact overdue, which has important implications for ED or out-of-hours clinicians assessing unwell children by highlighting that the possibility of a vaccine-preventable disease should not be completely excluded from the differential diagnosis by an up-to-date immunisation history. In our sample population, immunisation coverage was 93.1% (95% CI 90.9% to 94.9%). This is slightly lower than the coverage for our PCT region of 95.7% (estimate from published immunisation coverage by 5 years)5 and may reflect a reduced uptake in children who attend PED.

The strengths of our study include the large number of notes surveyed, although, because of the low prevalence of being overdue, our CI for sensitivity is still relatively wide. Using a retrospective review of medical notes, it is possible that discussions occurred about immunisation status but were not documented. We have no record of the way in which the question about immunisations was asked, and it is possible that the method of asking led to incorrect responses rather than inaccurate knowledge on the part of the parent. Studies in Australia in relation to the Australian Childhood Immunisation Register found that for those identified as overdue on the register, immunisation history was incorrect in the notes for more than half of these children.6 This and other studies also found from surveys of parents that between 40% and 85% of children showing as overdue on the register were actually fully immunised.7 8 However, we believe that there are important differences between the Australian and UK health systems, particularly that in the UK all residents have an NHS number which enables transfer of health records with them when they move location, which should improve the accuracy of records and mean they are suitable for use as a reference standard. In our study, PCT staff also checked with the original data from GP practices for children identified as overdue. However, it is possible nonetheless that some of the children who we classified as overdue were in fact completely immunised, and if this were the case, the estimated sensitivity would be somewhat higher.

There are several reasons which may account for the poor sensitivity of history taking. The parent may simply have forgotten that the immunisations are incomplete or the question may be asked in insufficient detail to correctly ascertain if there are omissions from the schedule. For some children, parents may know that their child is overdue but not wish to open a discussion about this with health professionals. Sensitivity of screening through history was lower for older children, and this may be related to a reduced parental focus on child health as children get older.

The combination of incomplete coverage of questioning about immunisation and low sensitivity of screening by history resulted in identification of only about a third of children with incomplete immunisation. This is a missed opportunity for opportunistic immunisation or discussion about the importance of vaccination. Previous studies have demonstrated the importance of health professionals in determining parental attitudes towards immunisation9 and that opportunistic immunisation would be taken up if offered.3

While coverage of immunisation history taking could be improved, for example, by making it a required part of admission or discharge documentation, this alone is unlikely to significantly improve detection of those overdue because of the low sensitivity of the history. Our study shows that older children with minor injuries are less likely to be asked, and so education to clinicians or modifications to the clerking proforma, to target this group may be particularly beneficial. An ideal strategy for improving detection could instead focus on alternative tools to assess immunisation status of PED attendees, such as linking the PCT database to the PED system, so a child who is overdue can be flagged up on arrival and steps taken to rectify this while they are in hospital. However, the cost implication of this is likely to outweigh the benefit in the population studied, where overall coverage is quite high and the number of children who could be offered opportunistic immunisation relatively small. Future research could involve parent interviews to better assess the relationship between medical history, parental knowledge of immunisations, and the PCT database. In the interim, highlighting the cohort of patients who present as fully immunised, but may actually be incomplete for immunisations, may be helpful to clinicians.