Real-time registration of adverse events in Dutch hospitalized children in general pediatric units: first experiences.

UNLABELLED: The objectives of this study are to describe the number and nature of adverse events occurring in general pediatric practice, to describe factors contributing to the occurrence of these adverse events, and to report on the experience of pediatricians with reporting adverse events. It is a prospective study on 11 pediatric units in a 3-month period; adverse events were registered for all newly admitted patients. Ninety-four adverse events were registered in 88 of 5,669 patients, amounting to a 1.6 per 100 admissions rate and a 0.4 per 100 patient days rate. Ninety percent of the adverse events did not cause serious harm. Failed diagnostic procedures were most common.
CONCLUSION: Adverse event registration in general pediatric practice is a first step in assessing quality and safety of care. It yields a considerable number of adverse events. Compliance to adverse event registration in daily practice is difficult but also key to optimal monitoring of quality of care.

Introduction

Since the publication of the report “To Err is Human” [6], quality and safety of care have been improved to decrease harm and adverse events. Nevertheless, a recent report showed little progress in reducing harm for adult patients in ten acute care hospitals in North Carolina [8]. Annual adverse event rates of 1% to 3.4% for hospitalized children have been reported in the USA [10, 13, 16, 23]. Adverse event rates for Dutch hospitalized children cannot be determined from available databases.

The Dutch Ministry of Health decreed that by January 1, 2008, all Dutch hospitals were to have a working patient safety management system. The Dutch Health Care Inspectorate subsequently established adverse event registration as a quality indicator for Dutch health care. These developments triggered the Pediatric Association of the Netherlands in collaboration with the Dutch Order of Medical Specialists to develop an easy-to-use reporting system for adverse events. The goal is to establish the exact rate and nature of adverse events in pediatric practice. The pediatric registration system is modeled after the surgical adverse event registration system and is intended to be used by all pediatricians in the Netherlands, so it can serve as a benchmarking tool for patient safety management. A previous study by Van den Beuken et al. tested the design of the registration system [21]; the system was subsequently approved by the Paediatric Association of the Netherlands. In this study, we report the first experience with prospective voluntary registration of adverse events in general pediatric units in Dutch hospitals. We describe the numbers, nature, contributing factors, and consequences of the adverse events and report on the findings of the pediatricians using this registration system.

Methods

Study design

We conducted a prospective study in 11 general pediatric units in the Netherlands, geographically evenly distributed over the Netherlands and recruited by open invitation. During a 3-month period, pediatricians collected general data and adverse events for all newly admitted patients on general pediatric units.

Setting

The units employed an average of nine pediatricians (range, 6 to 15) and a changing number of residents in training or not in training for pediatrics. Ten units were located in regional hospitals; one was a general pediatric unit in a university hospital, providing care for children with pulmonary or airway problems requiring a tracheal canula and children with gastro-intestinal problems. Four of the ten regional hospitals are pediatric teaching hospitals. The mean number of beds in the regional hospitals was 556 (median, 570; range, 267–1,070). All the pediatricians in each unit agreed to participate in the study, and one to three in each unit volunteered to be the primary contacts for the study.

Registration

The general data collected were patient details (name, date of birth, sex, hospital ID, and dates of admission and discharge). An adverse event was defined as any harm inflicted on the patient by medical care, whether or not the result of an error, with consequences such as adjustment of medical management, (permanent) scarring, or lengthening of hospital stay. As a first step in the registration, the localization is recorded, i.e., the body site or organ involved, and second, the nature of the event, i.e., what actually happened. Furthermore, potentially contributing factors are registered, for instance placement of a nasogastric tube or administration of medication. Thus, pneumonia due to aspiration caused by vomiting during placement of a nasogastric tube would be registered as a respiratory adverse event (pneumonia) with placement of nasogastric tube as contributing factor. Finally, the consequence of the adverse event was registered. An adverse event was considered to have a serious consequence if it resulted in longer hospital stay (≥1 day), (temporary) disability, or death. If a consequence was adjustment of medical management or temporary discomfort (e.g., extra blood sampling), it was deemed minor. The adverse events could be registered in handheld computers, in computer databases, or on paper.

All 21 participating pediatricians (one to three per unit) were assembled twice and instructed on what constitutes an adverse event and were trained in the use of the registration system. Formal testing of inter-rater reliability on the assessments of adverse events was not performed. The participants were requested to register all adverse events in newly admitted patients in the stated period. They were recommended to ask nursing and medical staff during daily rounds and handovers about occurrences of adverse events. It was left to the participants how the adverse events would be brought to their attention, at what time they registered the event, and whether or not to invite others to register adverse events as well. At the end of the study period, the data were collected by the author and imported in a database. The adverse events were reviewed by the author with the pediatricians, and in cases where there was doubt whether an event was actually validly registered as an adverse event, consensus was reached after discussing the adverse event. The adverse events were then classified in the categories “diagnostic,” “therapeutic,” “medication,” “non-surgical procedure,” and “other.” Four weeks after the study, all participating units received evaluation forms by mail including both multiple choice and open questions on the use of the registration.

Results

In 3 months, 94 adverse events were registered for 88 of 5,669 newly admitted patients. Thus, the mean adverse event rate was 0.0165 per admission (range, 0.004–0.046; median, 0.015) and 0.4 adverse events per 100 patient days (range, 0.05–0.9; median, 0.3). The distribution over the participating units is detailed in Table 1; the rates vary widely, with the number of adverse events per unit ranging from 1 to 29.

Table 1

Number of adverse events, percent adverse events per discharge, and number of adverse events per 100 patient days per unit

	Unit A (n = 546)	Unit B (n = 629)	Unit C (n = 602)	Unit D (n = 528)	Unit E (n = 535)	Unit F (n = 543)
Adverse events (% of AEs per discharge)	5 (0.9%)	29 (4.6%)	3 (0.5%)	10 (1.9%)	5 (0.9%)	4 (0.7%)
Number of AEs/100 patient days	–a	0.9	–a	0.4	0.2	0.1
	Unit G (n = 274)	Unit H (n = 785)	Unit I (n = 493)	Unit J (n = 482)	Unit K (n = 252)	Total (n = 5,669)
Adverse events (% of AEs per discharge)	6 (2.2%)	12 (1.5%)	10 (2.0%)	9 (1.9%)	1 (0.4%)	94 (1.65%, median 1.5%)
Number of AEs/100 patient days	0.3	0.3	0.5	0.4	0.05	0.4 (median 0.3)

AEs adverse events

aMissing data

The localizations of the adverse events are listed in Table 2. The localization “other” (n = 54) related to the adverse event of “pain” in 27 cases and to “other adverse event” in 26 cases, for instance failures in diagnostic procedures. The categories of the adverse events are detailed in Table 3. Most were of a diagnostic nature. These include blood sampling failure (n = 13), lumbar puncture failure (n = 6), urine sampling failure (n = 6), and missed diagnosis (three cases of appendicitis and one case of urosepsis).

Table 2

Distribution of adverse events in localization, n (percent)

Localization	Number	%
Other	54	57.4
Biochemistry	12	12.8
Skin	9	9.6
Respiratory system	8	8.5
Gastro-enterologic system	5	5.3
Circulatory system	3	3.2
Hematological system	2	2.1
Urinary tract	1	1.1
Total	94	100

Table 3

Classification of adverse events

Category	n = 94 (100%)
Diagnostic	38 (40.4%)
Therapeutic	20 (21.3%)
Medication	19 (20.2%)
Non-surgical procedures	11 (11.7%)
Other	6 (6.4%)

Potentially contributing factors to the occurrence of the adverse event were registered in 48 of the 94 adverse events (51.8%) (Table 4), in 16 cases (17.2%), a failure in the medication process and in 6 cases, lumbar puncture (in five cases, a dry tap and in one case, the liquor sample was lost). A urinary catheter or catheterization was related to an adverse event in five cases: three catheterizations failed, one urine sample was lost, and one patient developed urinary tract infection while having a urinary catheter in situ.

Table 4

Specification of potentially contributing factors to adverse events

External factor	No.	% of AEs	Corresponding adverse events	No.
Medication failure	16	17.2	Hypoglycemia	3
			Hyperglycemia	2
			Hyponatremia	2
			Hypokalemia	1
			Bleeding	1
			Other	7
Lumbar puncture	6	6.4	Diagnostic failure	6
Urinary catheter	5	5.4	Diagnostic failure	4
			Cystitis	1
Peripheral venous access	5	5.4	Phlebitis	4
			Hypoglycemia	1
Suprapubic aspiration	4	4.3	Diagnostic failure	4
Endotracheal tube	3	3.2	Hypoxia	3
External heater	2	2.2	Burn	2
Central venous line	2	2.2	Bleeding	1
			Hypoglycemia	1
Mechanical ventilation	1	1.1	Hypoxia	1
Nasogastric tube	1	1.1	Aspiration	1
SpO2 sensor	1	1.1	Burn	1
pH probe	1	1.1	Diagnostic failure	1
Other (scissors)a	1	1.1	Other	1
Total	48	51.8%

aFather cut umbilical cord below the umbilical clamp

As to the consequences, medical management was adjusted in 51% of the adverse events. The consequences were serious in 18 cases (20%): 16 patients needed to stay longer in hospital, with a median lengthening of 1 day, 1 was readmitted, and 1 died due to a Gram-negative sepsis. The consequences classified as “other” (n = 28, 30%) ranged from scarring following a burn, intensified monitoring after a fall, to discomfort from extra blood sampling.

Eight of the 11 participating units returned the evaluation forms. Five of eight units had no adverse event registration system in place before the study. At the time of the study, seven units used a voluntary incident reporting system. The adverse events were mostly registered not only during daily rounds (in five of the eight units), but also upon discovery and during hand overs or weekly multidisciplinary rounds. In four units, a designated pediatrician registered the adverse events whereas in the other four units all physicians (pediatricians and residents) participated in registering. In two units, the adverse events were registered directly in the handheld computer; in five units, a paper registration was used, and the adverse events were later entered in the handheld computer. Two units had initiated improvements: a protocol was reviewed, and a different type of oxygen sensor was purchased because burns had occurred with the previously used type.

Discussion

In this study, adverse event registration as part of standard care in general pediatric practice yielded a considerable number of adverse events. Most adverse events registered were of a diagnostic nature. The adverse events resulted in longer hospital stay for one in six of the patients that suffered an adverse event.

This study is the first to register real-time data on adverse events in general pediatric practice in the Netherlands. Most of the earlier studies used administrative data to retrospectively evaluate harm to hospitalized children. Miller and Zahn studied the 2000 Healthcare Cost and Utilization Project Database that included 5.7 million pediatric discharges [12]. Using the AHRQ's Patient Safety Indicators [1], they uncovered 1.2 adverse events per 100 discharges [12], including events related to birth, procedures, and surgery in all children, whereas our study was limited to general pediatric units, and birth trauma and surgical or anesthetic adverse events were not recorded. Woods et al. found an adverse event rate of 1 per 100 discharges by retrospectively analyzing 3,719 pediatric hospitalizations in the Colorado and Utah Medical Practice Study [22]. Again, in that study, most adverse events occurred in newborns and were birth related; 16% of the adverse events were surgical. Dunn and colleagues found an adverse event rate of 0.26 per 100 discharges in a retrospective review of 1,612 selected records [3].

The rate of 0.0165 adverse events per admission in our study is higher than the rates of adverse events in those studies—even though obstetrical and surgical adverse events were not registered in our study. This may be due to the prospective nature of our study and the commitment of the participating pediatricians. Also, our study used real-time data, whereas previous studies used administrative data to identify adverse events, a method which is neither highly sensitive nor specific [7]. A striking finding is the wide range of numbers of registered adverse events by the different units (1 to 29). We speculate this to be a reflection of the zeal of the involved pediatricians, rather than of a difference in level of care, number of admissions, or severity of sickness of the patients. Part of the differences between units can be explained by the fact that some physicians felt all events resulting in harm for the patient were to be considered adverse events, where others would consider such an event an anticipated result of a procedure (for instance a dry tap during a lumbar puncture). It was argued that in some of these cases, the adverse event might have occurred because of poor supervision of inexperienced residents and thus should be registered. Another reason for the differences between the units could be that some participants felt that any adjustment of management after an event (for instance loss of a blood sample, necessitating more drawing of blood samples, thus causing pain to the patient) needed to be registered, where others considered this to be (an unfortunate) part of pediatric practice. Contrary to the expectations, the different case mix and level of care in the university hospital unit did not result in a higher number of registered adverse events (data not shown). It can be speculated that this was also caused by different interpretations by the participating pediatricians.

A limitation to this study is the lack of a gold standard for detecting adverse events; several methods have been proven to underestimate the actual occurrences of adverse events. It is likely that not all adverse events were detected with the real-time registration, and we did not compare the results of the registration with other promising new methods such as the trigger tool methodology [9].

Another limitation to this study is the lack of inter-rater reliability testing. Still, participants were taught that only events that led to harm and necessitated a change in medical management were to be registered.

However, a number of advantages to real-time registration can be identified. Real-time registration as applied in our study is thought more reliable in detecting actual harm than voluntary incident reporting. In incident reporting, it is encouraged to also report near-misses, which did not reach the patient and thus did not cause any harm or errors that did reach the patient but did not lead to harm. Also, voluntary reporting is well known for its underreporting, especially of more serious events such as nosocomial infections or diagnostic failures [2, 4, 5, 15]. Nevertheless, incident reporting has been introduced in Dutch pediatric practice under the name NEOSAFE Project and is widely used [17-19]. The NEOSAFE study by Snijders et al. reported significant harm in 70 of 4,846 incident reports (1.4%), a considerably lower rate than the serious consequences for the patients with adverse events in our study (n = 18 in 88 patients, 20%) [18].

Herein lies one of the arguments for direct registration of adverse events by the pediatricians themselves. By registering adverse events locally, underreporting might be less of an issue. Also, by gathering data locally, as opposed to centrally in a national database, the number and nature of the adverse events are very accessible and thus provide direct feedback to the pediatricians of the quality of the care delivered in their unit and hospital. Moreover, this direct registration provides information on the causes of the adverse event and on contributing factors that may be helpful in giving direction to improvement initiatives. Last but not least, it allows the focus of further investigations to be directed away from the role of the individual in the origin of the event towards the whole care delivery system as the source of the adverse event [14, 15, 20].

Another limitation of this study was to keep the physicians engaged in registering adverse events. Participants were very motivated, though, but quite often found that their colleagues were less likely to report adverse events. This can partially explain the large differences in numbers of adverse events per unit. Underreporting prevents good benchmarking of quality of care, so this issue should be addressed by each physician, unit, or hospital committed to improving quality of care. Prospectively registering adverse events could well be a means to engage physicians and convince them that studying the adverse events and their causes can guide improvement programs and help assess the effects of those programs [11]. Nevertheless, very few improvement initiatives were reported in the evaluation forms. A number of pediatricians told they were going to deploy initiatives after the study. The prospective registration of adverse events was likely an important step, and for some units a first step, in creating awareness of quality and safety issues.

Conclusion

We believe that prospective registration of adverse events is a first step towards good monitoring of quality of care in general pediatric practice. A nationwide used registration system allows benchmarking within pediatrics. The registration system described in this study could be a useful tool, provided it is integrated in the daily practice of everybody working in pediatric units: nurses, residents, and pediatricians. Prevalence, effects, and causes of adverse events need to be further researched so that the findings may serve in the development of programs for the prevention of adverse events in hospitalized children.