What are incident reports telling us? A comparative study at two Australian hospitals of medication errors identified at audit, detected by staff and reported to an incident system.

OBJECTIVES: To (i) compare medication errors identified at audit and observation with medication incident reports; (ii) identify differences between two hospitals in incident report frequency and medication error rates; (iii) identify prescribing error detection rates by staff.
DESIGN: Audit of 3291 patient records at two hospitals to identify prescribing errors and evidence of their detection by staff. Medication administration errors were identified from a direct observational study of 180 nurses administering 7451 medications. Severity of errors was classified. Those likely to lead to patient harm were categorized as 'clinically important'.
SETTING: Two major academic teaching hospitals in Sydney, Australia. MAIN OUTCOME MEASURES: Rates of medication errors identified from audit and from direct observation were compared with reported medication incident reports.
RESULTS: A total of 12 567 prescribing errors were identified at audit. Of these 1.2/1000 errors (95% CI: 0.6-1.8) had incident reports. Clinically important prescribing errors (n = 539) were detected by staff at a rate of 218.9/1000 (95% CI: 184.0-253.8), but only 13.0/1000 (95% CI: 3.4-22.5) were reported. 78.1% (n = 421) of clinically important prescribing errors were not detected. A total of 2043 drug administrations (27.4%; 95% CI: 26.4-28.4%) contained ≥ 1 errors; none had an incident report. Hospital A had a higher frequency of incident reports than Hospital B, but a lower rate of errors at audit.
CONCLUSIONS: Prescribing errors with the potential to cause harm frequently go undetected. Reported incidents do not reflect the profile of medication errors which occur in hospitals or the underlying rates. This demonstrates the inaccuracy of using incident frequency to compare patient risk or quality performance within or across hospitals. New approaches including data mining of electronic clinical information systems are required to support more effective medication error detection and mitigation.

Background

Incident reporting within hospitals is accepted as a key quality and safety mechanism [1-5], although under-reporting is a serious limitation [6-9]. Poor understanding of the strengths and limitations of incident data can lead decision-makers astray [10-13]. Interviews with administrators from 34 US hospitals revealed that many found incident report data difficult to interpret, and raised concerns that resources may be misdirected to solve problems that are easily identifiable and more likely to be reported to incident systems [14]. The risk of incorrect conclusions is exacerbated by small sample sizes for specific incident types, a lack of denominators and advice to monitor ‘trends’ in incident report frequency [15]. Despite attempts to guard against incident report frequency as an indicator of patient risk, many hospital administrators and safety committees interpret increases negatively [16]. Graphing of incident data may further reinforce perceptions of the significance of small changes in the frequency of incident reports. Studies using incident data as an outcome indicator to assess the effectiveness of safety interventions also continue to be published [17-19].

While studies have demonstrated significant under-reporting of adverse incidents overall [3, 14, 20, 21], few have focused on medication incidents and whether the profile of incidents reported reflect the types of medication errors not reported. Medication errors are one of the most frequently reported incident types [22], and are more likely to result in serious harm and death than other types of incidents [23, 24]. Levinson et al. [14] in their study of US Medicare patients found that of 111 medication events identified at audit, only 14 (13%) were reported. Importantly, 50% of all medication-related events were estimated to be preventable, clearly identifying the potential to intervene to reduce such errors if they were reported and better understood [25].

Reasons why clinicians do not report incidents include lack of feedback and time to complete forms, along with concerns about repercussions or disciplinary actions [2, 26–28]. A frequent underlying assumption is that clinicians are aware of errors but fail to report. The extent to which this is an accurate assumption has rarely been investigated.

In this study, we aimed to (i) identify the extent to which medication errors identified at record audit and observation are reported to hospital incident reporting systems; (ii) use these data to identify differences between two hospitals in their frequency of incident reports and their underlying medication error rates identified and (iii) test the hypothesis that low rates of prescribing error detection by staff may be a contributing factor to low medication incident reporting rates.

Methods

We enrolled two major Sydney teaching hospitals (400 and 326 beds, respectively). All hospitals across the state of New South Wales (NSW) are required to comply with the State incident management policy [29] which requires the electronic reporting of incidents with prescribed documentation of incident details which will allow investigation and development of remedial action if required. The 60-page policy outlines the steps to be followed by hospitals throughout the reporting, severity classification and follow-up processes for individual incidents. Very limited guidance is provided regarding interpretation of changes in incident data over time, or the strengths or limitations of aggregate incident data.

The hospitals in this study use different electronic incident reporting systems [Riskman [30], and the NSW Health Incident Information Management System (IIMS)] [29]. Hospital staff are required to report all incidents they observe, or errors they themselves make, as well as incidents which become apparent after the event, for example when reviewing a patient record. Incidents are defined as ‘Any unplanned event which causes, or has the potential to cause, harm to a patient’ [22]. Staff may report incidents anonymously. Both hospitals actively encourage reporting and previous research has shown staff within NSW have generally positive views about the reporting of incidents [31]. Serious incidents are reviewed by the local clinical governance (quality improvement) units. Medication incidents are reported to the hospitals’ medication review committees where they are expected to be used to initiate further investigations, identify safety risks and recommend action. Medical and nursing staff are represented on the medication review committees. At one site, this committee prepares a monthly email to hospital staff regarding their discussions as well as presenting a report to the hospital's drug and therapeutic committee as a further review mechanism. At the second site, the medication review committee also reports to a district medication safety committee. At that hospital, copies of reports summarizing incidents are provided to the heads of medical departments and a ‘Medication Safety’ newsletter is prepared by the hospital pharmacy and distributed to staff. These newsletters included details of de-identified incidents with a discussion of preventive actions. Actions taken as a result of incident investigations are communicated through a range of mechanisms including policy changes, presentations at grand rounds and initiation of specific quality and safety activities.

All incident data are provided to the Clinical Excellence Commission (CEC), an organization responsible for leading healthcare quality and safety improvement programs across NSW, a State of 7.2 million people. The CEC was established in 2004 to promote and support improved clinical care and safety in NSW. The organization aims to foster these developments within a framework of collaboration, openness, respect and empowerment. Since 2007 CEC has produced aggregated data regarding all incidents reported by hospitals across the State and these data inform quality and safety programs.

Prescribing errors identified at record audit and errors detected by staff

Data on prescribing errors occurring on six medical and surgical wards (respiratory, cardiology, renal/vascular, psychiatry and two acute aged care wards) over two periods of time at each hospital (Hospital A: May–August 2006 and May–August 2008; Hospital B: November 2007–March 2008 and March 2008–February 2010) were obtained from a dataset of clinical audits of 3291 patient records conducted by pharmacists independent from the study hospitals [32]. During chart review, any evidence that medication errors had been detected by staff (either before or after the error reached the patient), such as a correction or note made by a doctor, nurse or pharmacist in the medication chart, progress notes or the existence of a pharmacist's intervention report (e.g. indicating that the doctor had been contacted to review the order), was recorded. Prescribing errors identified were classified as procedural (e.g. unclear, illegal or incomplete orders) or clinical (e.g. wrong drug, dose, route). Definitions for each error type have been published previously [32].

Medication administration error data

Data on medication administration errors were obtained for six medical/surgical wards (neurology, orthopaedics, respiratory, renal/vascular and two acute aged care wards) at the same two hospitals, derived from a second dataset [33]. That dataset contained direct observations of 180 nurses as they prepared and administered 7451 medications to 1397 patients. We had highly trained observers (nurse researchers independent from the hospitals) who watched nurses prepare and administer medications to patients. Inter-rater reliability was conducted on multiple occasions and kappa scores for agreement calculated. Researchers arrived on the study wards during peak administration times (7:00 am–9:30 pm) and closely shadowed individual nurses. Errors were classified as procedural (e.g. failure to read medication label) or clinical (e.g. wrong drug, rate, route, timing) [33]. Observational data of drugs administered were compared with patients' medication charts to identify clinical administration errors. Unlike the prescribing error study, no information about whether staff had detected the medication administration error was collected. Data were collected in two periods (Hospital A: August 2006–March 2007; and October 2009–June 2010; Hospital B: November 2005–March 2006; and November 2007–March 2008). The data collected did not represent all medications administered to patients on these wards.

Procedures

The potential severity of all medication errors was rated on a 5-point scale (Table 1). All errors rated as serious/major/moderate were classed as ‘clinically important’ errors. Two pharmacists independently rated the actual or potential severity of errors; disagreement was settled by consensus with input from a clinical pharmacologist when required. Severity review committees involving physicians, hospital pharmacists and nurses from both hospitals were also given subsets of errors to classify during the study in order to verify the ratings provided by the clinical pharmacists.

Table 1

Potential severity scale [34]

Rating	Description	Categories used in analyses
Insignificant	Incident is likely to have little or no effect on the patient	Minor errors
Minor	Incident is likely to lead to an increase in level of care e.g. review, investigations or referral to another clinician
Moderate	Incident is likely to lead to permanent reduction in bodily functioning leading to e.g. increase length of stay; surgical intervention	Clinically important errors warranting an incident report
Major	Incident is likely to lead to a major permanent loss of function
Serious	Incident is likely to lead to death

During both medication error studies, prescribing and administration, researchers were required to notify hospital staff if they identified an error which could cause serious patient harm. This occurred 15 times during the medication administration error study and 17 times during the prescribing error study. One of these prescribing errors was subsequently reported by staff to the hospital's incident system.

Ethics approval

The study was approved by the human research ethics committees of both study hospitals.

Data linkage and analyses

Incident data were extracted relating to all prescribing and medication administration errors from the hospitals' incident reporting systems for the study wards over the same time periods as the prescribing error and administration error datasets. Reported medication incidents and observed errors were then linked and each possible match reviewed by a research pharmacist to confirm agreement.

Incident report data extraction occurred after both observational studies had been completed, and thus auditors and observers were unaware of reported incidents. If multiple errors were reported on the same incident form, for example two drugs failed to be charted, this was categorized as a match for both observed errors.

Frequency tables of prescribing errors are presented by hospital. Incident reporting rates were calculated using the number of errors reported over total errors observed at audit. Ninety-five percent CIs were calculated based on the assumption of a normal distribution. Incident reporting and detection rates by hospital were compared using z-tests and Fisher's exact tests with significance set at 0.05.

Results

Comparison of prescribing errors observed at audit with those reported to the hospitals' incident reporting systems

A total of 12 567 prescribing errors were identified at audit across the two hospitals. The majority 68.9% (n = 8664) were procedural errors (e.g. missing allergy identification). Clinical errors constituted 31.1% (n = 3903) and 539 (4.3%) were rated as clinically important errors (Table 2).

Table 2

Prescribing errors observed at audit, those detected by staff, and the numbers and rates reported to the hospitals' incident reporting systems

	Numbers of prescribing errors identified at audita in 3291 patient admissions			Number of prescribing errors detectedb by hospital staff (rate per 1000 prescribing errors: 95% CI)		Numbers of prescribing errors reportedc to the hospital incident reporting systems (rate per 1000 prescribing errorsd: 95% CI)
Hospital	Total prescribing errors observed	Clinical errors	Clinically important errorse	All prescribing errors detectedb by staff	Clinically important errors detected by staff	Total prescribing errors reported	Clinical errors reported	Clinically important errors reported
Hospital A	8621	2373	343	878 (101.8: 95.5–108.2)4	84 (244.9: 199.4–290.4)5	3 (0.3: <0.1–0.7)1	2 (0.8:<0.1–2.0)2	2 (5.8: <0.1–13.9)4
						12 (3.0: 1.3–4.8)1	11 (7.2: 3.0–11.4)2
Hospital B	3946	1530	196	404 (102.4: 92.9–11.8)4	34 (173.5: 120.5–226.5)5			5 (25.5: 3.4–47.6)3
Total	12 567	3903	539	1282 (102.0: 96.7–107.3)	118 (218.9: 184.0–253.8)	15 (1.2: 0.6–1.8)	13 (3.3: 1.5–5.1)	7 (13.0: 3.4–22.5)

aIdentified errors: Prescribing errors detected at record audit.

bDetected errors: Prescribing errors detected by staff as evidenced by documented information which indicated that at least one staff member was aware of the error.

cReported errors: Prescribing errors reported to the hospitals’ incident reporting systems.

dper 1000 prescribing errors identified at audit.

eClinically important errors = those classified as serious, major or moderate potential severity (see Table 1).

P-values for comparisons between hospitals: 1P< 0.0001; 2P = 0.001; 3P = 0.052; 4P = 0.9; 5P = 0.054.

During the period covered by the record audit, 15 prescribing errors were reported to the hospitals' incident systems. All incident reports identified prescribing errors found at audit. Thirteen (87%) of the 15 incidents related to clinical prescribing errors, and two were procedural errors.

There was a very low rate at which prescribing errors identified at audit had a matching incident report (Table 2), with 1.2 incident reports per 1000 identified errors. Incident reporting rates were higher for clinical errors, at 3.3 per 1000 identified clinical prescribing errors. The highest reporting rate was for clinically important prescribing errors at 13.0 reports per 1000.

Table 3 reports a sample of prescribing errors with incident reports. Table A2 shows a sample of clinically important prescribing errors identified at audit for which there was no incident report.

Table 3

Examples of prescribing errors reported to the hospitals' incident reporting systems

Severity rating	Error description	Error type	Clinical error category
Serious	None reported	–	–
Major	Warfarin PO 3 mg evening at 1600 h. Dose given despite INR being 5.5 at 0830 h that day	Clinical error	Medication not indicated
Moderate	Long-acting Risperidone IM 50 mg in the morning (should be given every 2 weeks)	Clinical error	Wrong frequency
Minor	Esomeprazole PO 40 mg daily (not transcribed onto new chart)	Clinical error	Drug required but not prescribed
Insignificant	Phenytoin IV 300 mg STAT (order should have been recharted)	Procedural error	Legal/procedural

PO, oral administration (per os); INR, international normalized ratio; IM, intramuscular; IV, intravenous; STAT, immediately (statim).

Table A2

Medication incidents detected by patients

Error description	Error category	Potential outcome severity
Glyceryl trinitrate 5 mg/24 h patch was prescribed for ‘on’ at 0800 h and ‘off’ at 2200 h. Patient reported using the patch overnight (on at 2200 h and off at 0800 h) at home.	Wrong timing	Minor
Atorvastatin 40 mg oral in the morning at 0800 h. Patient refused morning medication as usually takes the tablet at night.	Wrong timing	Insignificant
Hydrocortisone 20 mg oral twice daily at 0800 h and 2200 h. Patient reported taking the second dose at 1400 h as per endocrinologist.	Wrong timing	Minor
Montelukast 10 mg oral daily was ordered. A dose was given to the patient in the morning (but should have been administered at night). Patient took morning dose and then was given a second evening dose. Record indicated that patient identified that they had been given two doses (one in error).	Wrong timing	Minor
Carmellose (Refresh Tears) 0.5% was ordered to be given orally instead of in the patient's eyes. Patient refused administration orally as they reported using the eye drops in their eyes. The order was changed, but not recharted (as is legally required), to carmellose 0.5% eye 1 drop both sides TDS.	Legal/procedural	Insignificant

TDS, three times a day.

Prescribing errors detected by staff and patients

Of the 12 567 prescribing errors identified at audit, there was evidence that 1282 (10.2%) had been detected by hospital staff (n = 1277), or patients (n = 5) (Table 4). Of errors detected, pharmacists identified over two thirds (Table 4). Only 15 prescribing errors were reported in a total of 7 incident forms, as two patients had multiple errors documented in the same report. All incident reports were made by nurses. Table A1 summarizes the five errors detected by patients, which included no clinically important errors.

Table 4

Number and percentage of prescribing errors and clinically important prescribing errors detected by staff group

Hospital	Number of prescribing errors detected (%)					Number of clinically important prescribing errors detected (%)
	Nurse	Pharmacist	Patient	Doctors	Total	Nurse	Pharmacist	Patient	Doctors	Total
Hospital A	116 (13.2)	531 (60.5)	2 (0.2)	229 (26.1)	878 (100)	13 (15.5)	49 (58.3)	0 (0)	22 (26.2)	84 (100)
Hospital B	50 (12.4)	321 (79.5)	3 (0.7)	30 (7.4)	404 (100)	10 (29.4)	23 (67.6)	0 (0)	1 (2.9)	34 (100)
Total	166 (12.9)	852 (66.5)	5 (0.4)	259 (20.2)	1282 (100)	23 (19.5)	72 (61.0)	0 (0)	23 (19.5)	118 (100)

Prescribing error detection patterns by professional group were different between the two hospitals (Fisher's exact test, χ2 = 41.41 with df = 3, P< 0.0001). Prescribing errors overall at Hospital A were more likely to be detected by pharmacists and doctors, while errors were more likely to be detected by pharmacists and nurses at Hospital B. Doctors at Hospital A detected a greater proportion of errors at their hospital than their peers at Hospital B. Pharmacists at Hospital B detected more errors than their colleagues at Hospital A.

Similar differences were observed for the detection of clinically important errors (Fisher's exact test, χ2 = 9.47 with df = 2, P = 0.008), except that nurses at Hospital B detected a greater proportion of clinically important errors than their colleagues at Hospital A.

Table A1

Examples of clinically important prescribing errors identified by research pharmacists at audit for which there was no evidence the error had been detected by clinical staff and no incident report was completed

Error description	Error type	Error category
No designated allergy recorded in system yet patient is allergic to penicillin (anaphylaxis), cefotaxime (rash) and sulphur (anaphylaxis) as per previous charts.	Documentation	Legal/procedural
Potassium chloride (Slow K) 16 mmol PO BD after food (order still active despite potassium levels of 6.1 mmol/l)	Clinical	Drug not indicated
Warfarin 3 mg PO evening at 4 pm (given despite INR of 5.5)	Clinical	Drug not indicated
Prescriber withheld ‘warfarin target range’ (instead of ‘warfarin order’, dose given)	Clinical	Drug not indicated
Fentanyl patches 75 µg 3 daily (instead of every 3 days; patient had three patches applied afresh each day)	Documentation	Unclear order (which resulted in a wrong dose)
Fludrocortisone 0.1 mg PO TDS (previous dose was 0.1 mg BD but BD was charted unclearly and patient received TDS dosing for 6 days)	Clinical	Wrong rate/frequency
Vancomycin 1 g IV BD. Trough levels of up to 32.6 mg/l (TDM recommend trough levels 10–20 mg/l)	Clinical	Inadequate monitoring
Patient admitted from nursing home on meloxicam 7.5 mg PO nocte + perindopril 4 mg PO nocte + furosemide 40 mg PO mane and 20 mg PO midi (NSAID + ACEI + diuretic = triple whammy)	Clinical	Drug–drug interaction
Simvastatin 40 mg PO nocte (eGFR 9 ml/min, recommended dose as per renal handbook GFR <10 ml/min is 10 mg daily). The patient was admitted for acute renal failure. Taking statin during severe inter-current illness, increases the risk of myopathy, rhabdomyolysis and renal failure.	Clinical	Wrong drug
Meloxicam 15 mg PO mane prescribed to patient with history of CVA and seizures (black box warning in Micromedex that meloxicam may be fatal in patient with history of cardiovascular disorder; MI and stroke risk increased).	Clinical	Wrong drug
Trimethoprim 300 mg PO nocte (peak potassium levels of 7.0 mmol/l with ECG changes but patient remained asymptomatic)	Clinical	Wrong dose/volume
Potassium chloride (Slow K) 2 tablets PO BD (potassium level increased from 4.0 to 6.0 mmol/l over 5 days)	Clinical	Inadequate monitoring
Concurrent orders of naproxen SR 750 mg PO nocte + irbesartan 150 mg PO mane + furosemide 20 mg PO mane (NSAID + ARB + diuretic = triple whammy)	Clinical	Drug–drug interaction
No designated allergy recorded in system yet—previously documented to react to amiodarone (collapse requiring adrenaline), metoprolol (collapse) and morphine (itch)	Documentation	Legal/procedural
Ciprofloxacin 150 mg PO BD and tramadol 50 mg PO TDS ordered (increased risk of serotonin syndrome)	Clinical	Drug–drug interaction
Furosemide 120 mg IV TDS was recharted without ceasing old order	Clinical	Duplicate drug
Digoxin 625 µg PO mane (instead of 62.5 µg PO mane)	Clinical	Wrong dose/volume

PO, by mouth; BD, twice a day; TDS, three times a day; IV, intravenous; TDM, therapeutic drug monitoring; to measure peak or trough concentrations of a medication in the blood; S/C, subcutaneous; nocte, at night; mane, in the morning); midi, at mid-day; NSAID, non-steroidal anti-inflammatory drug; ACEI, angiotensin-converting enzyme inhibitor; triple whammy, a combination of diuretics; NSAIDs, ACEI/ARB that may impair renal function; ARB, angiotensin receptor blocker; eGFR, estimated glomerular filtration rate; CVA, cardio vascular accident; MI, myocardial infarction; ECG, electrocardiogram; SR, slow release.

Of the clinically important prescribing errors identified at audit, 21.9% (n = 118) had been detected, of which seven had an incident form. Table 2 presents a summary of the rates at which prescribing errors were identified at audit, detected by staff and reported to the incident systems, by hospital. Figure 1a shows the distribution of prescribing errors identified, detected and reported and Fig. 1b shows this information for clinically important prescribing errors.

Figure 1

Distribution of (a) prescribing errors observed by researchers, and detected and reported by clinical staff; (b) clinically important prescribing errors observed by researchers, and detected and reported by clinical staff.

Rates of clinically important prescribing errors identified at audit, detected by staff and reported for each hospital

For the 3291 patient admissions reviewed, 539 had clinically important prescribing errors, a rate of 16.4/100 patient admissions. Of these, 118 were detected by staff, a rate of 3.6/100 admissions and the rate at which these were reported via the incident reporting systems was 0.21/100 admissions (n = 7).

We examined each hospital's data. At Hospital A, 8621 prescribing errors were identified at audit for 1948 patient admissions, a rate of 4.43/100 admissions. At Hospital B, 1530 prescribing errors for 1343 admissions (1.14/100) were found at audit. However, the number of reported incidents for Hospital A was 0.34/1000 prescribing errors and for Hospital B 3.04/1000 prescribing errors (Table 2). Thus, Hospital A with the higher overall prescribing error rate had an incident reporting rate significantly lower than Hospital B (P < 0.0001).

Comparison of medication administration errors observed with those reported to the hospitals' incident systems

The direct observational study of 7451 drug administration yielded 10 955 medication administration errors. Of these, the vast majority (79%) were procedural errors (e.g. failing to check a patient's identification prior to drug administration). One or more clinical errors occurred in 27.4% (n = 2043; 95% CI: 26.4–28.4%) of drugs administered. 10.2% (n = 209; 95% CI: 8.9–11.5%) of all drug administrations involved errors which were rated as clinically important.

Of the 10 955 administration errors observed none had an incident report. Table A3 provides examples of clinically important medication administration errors observed. We had no information about administration errors detected by staff. During the period of the observational study, 173 medication administration errors, involving administrations not observed by the research staff, were reported to the hospitals' incident systems.

Table A3

Clinically important medication administration errors identified at observation but with no incident report

Drug name	Drug form	Error type/s	Comment
Heparin	Injection	Wrong drugWrong doseWrong strength	Heparin 5000 units/0.2 ml subcutaneously was ordered but enoxaparin was administered.
Metoprolol	Tablet	Wrong dose	Metoprolol 12.5 mg was ordered, nurse signed for metoprolol 25 mg but administered metoprolol 50 mg.
Tramadol	Capsule	Wrong doseWrong formulation	Tramadol 100 mg SR was ordered but tramadol 50 mg was administered.
Gentamicin	Injection	Wrong routeWrong IV rate	Gentamicin 80 mg/2 ml administered as an IV bolus injection. Gentamicin is not recommended to be given as a bolus injection.
Irbesartan/hydrochlorothiazide	Tablet	Wrong strengthWrong dose	Irbesartan/hydrochlorothiazide 125 mg/12.5 mg 2 tablets were ordered but 300 mg/12.5 mg 2 tablets were administered.
Clopidogrel	Tablet	Extra dose	Ordered to be given Mondays, Wednesdays and Fridays. Was in addition administered, but not signed for, on a Thursday.
Ampicillin	Injection	Wrong IV rate	Recommended administration rate according to MIMS is 3–5 min. It was administered over 0.31 min.

SR, slow release; IV, intravenous; MIMS, Monthly index of medical specialties, an independent medicine information source for health care professional.

Discussion

Only 1.3% of clinically important prescribing errors, with the potential to cause patient harm, were reported to the hospital incident systems. Of all clinically important prescribing errors identified at record audit, 21.9% (n = 118) had been detected by hospital staff, but only 6% of those detected were reported. The remaining 78.1% (n = 421) appear to have failed to be detected, although it is possible that some of these errors were detected but no information to this effect was recorded in patients' records.

Overall, one in every 1000 prescribing errors identified at audit were reported to the incident systems. For only 10% of these errors was there any evidence that staff had detected the error. Clinically important prescribing errors were twice as likely to be detected as other prescribing errors. Barriers to reporting, such as a lack of time, and an absence of evidence that the data are used for good purpose, appear to be powerful deterrents [2, 26, 27], despite survey evidence which indicates that many Australian nurses and doctors indicate they always report drug errors [26]. Removing barriers to reporting will improve incident data, but a major problem also lies in the very low rate at which prescribing errors are detected by staff.

We found no relationship between the number of reported medication incidents by hospital and the ‘actual’ rate of prescribing errors. The hospital with the higher number of incident reports had lower ‘actual’ prescribing errors and vice versa. Thus, in this instance, the higher number of medication incidents reflected a lower patient risk.

None of the 2043 clinical medication administration errors identified during direct observation were reported by staff to the incident systems. This included 209 clinically important errors observed. Flynn et al. [35] also found a very low rate at which observed medication administration errors were reported, with one report (0.4%) across 36 US hospitals. Husch et al. [36], investigating errors associated with the use of intravenous pumps in a US hospital, identified 55 rate deviation and wrong medication errors during 9 h of observation. They noted that in the previous 2 years only 48 of these error types had been reported to the hospital's incident system, suggesting significant under-reporting. Unlike prescribing errors, retrospective identification of medication administration errors is often not possible. For example an IV pump set at the wrong rate, or an incorrect dose administered, are largely undetectable once the event has passed, greatly hindering learning from such incidents.

Our results clearly indicate that the incident data hospitals generate have significant limitations in reflecting both the frequency and nature of medication errors occurring. Hospital committees and management boards charged with the responsibility for monitoring medication incident reports require clear guidance regarding the strengths and limitations of these data, including guidance on how to interpret changes in the frequency of incident data. This guidance should include that: an increase in incident reports should be encouraged and viewed as an indicator of an open and safe reporting culture; the value of incidents lie in their ability to identify hazards on which to focus quality improvement activities [28]; attention should focus on groups within the hospital who are not reporting incidents, but this must be accompanied by direct feedback regarding the actions taken when incidents are reported; presentation of incident data with histogram and line graphs should be discouraged (other than to monitor the ‘reporting culture’) without the availability of a meaningful denominator; and alternative approaches such as the use of control charts and specific measures better suited to rare event data should be applied.

Finally, health care organizations need to take advantage of new opportunities to obtain more reliable and comprehensive medication incident data. The increasing introduction of electronic clinical information systems presents an opportunity to revolutionize the quality and timeliness of medication-related data available for monitoring medication safety and the quality use of medicines. Electronic prescribing and medication administration record systems capture details of all drugs prescribed and administered. They are highly effective at reducing prescribing errors, particularly with well-designed decision support [32, 37], and when linked with bar-coding of medications they are also effective at reducing administration errors [38]. The potential of these technologies to deliver up-to-date data about the quality and safety of medication practices has been under-explored. Automatically mining data derived from these electronic systems, can provide information about both medication errors and near-miss situations. For example an automated report from an electronic prescribing system database could identify dangerous drug combinations prescribed and ceased within a defined time period. A clear advantage is the ability to search across all patients and thus generate a denominator to calculate incident rates. Novel approaches to examine the rates at which decision-support alerts are fired, when and by whom, as well as actions taken subsequently, can provide insights into approaches to effectively mitigate prescribing errors. Such analyses may allow identification of doctors at higher risk of making serious prescribing errors [39]. Data on missed doses can be automatically identified [40]. Data logs of smart pumps can be used to identify orders entered, abandoned or not allowed as they conflicted with the pumps' administration rules [41]. There is enormous potential for these new electronic data sources, and health care organizations must ensure that it is as easy to retrieve information from these systems as it is to enter data [42]. These systems will not provide the detailed contextual information available from well-documented incident reports, but they provide a valuable new source of data that should be exploited. Quality improvement staff in hospitals should be looking to these new data sources to better inform medication safety and improvement policies and strategies, and to reduce the reporting load on clinical staff. An important implication of our findings is the need for staff responsible for monitoring medication safety to use multiple data sources and more sophisticated approaches, which we suggest are now readily available with electronic medication management systems.

A strength of our study was the availability of robust audit and observational data from previous studies. A limitation was that we relied upon documented evidence of error detection and thus our detection rates may be an under-estimation. However, for clinically important prescribing errors, which had the potential to cause permanent harm to patients, some form of documentation would be expected if they had been detected. While poor documentation in medical records is widely recognized in terms of reflecting all care provided, it seems less likely that health professionals who detected potentially serious medication errors would not document any information about corrections made or steps taken to remediate errors or their effects. Studies have attempted to estimate the extent to which medical records under-report care provided and have estimated that record reviews may lead to under-estimations in the order of 10% [43]. Even if our results regarding error detection represent only 50% of those errors ‘truly detected’, our findings still demonstrate a significant problem in relation to the extent to which existing processes effectively support the detection of serious medication errors by clinical staff on hospital wards. Unfortunately, we had no information about the detection of medication administration errors by staff as these data were not collected in that study.

In conclusion, our findings demonstrate that for individual health care organizations medication incident report data provide an inaccurate reflection of the types or severity of medication errors occurring to patients. As such, incident data may provide only limited insights into patient risk. Steps should be taken to ensure that incident data are used appropriately to investigate specific types of hazards, but the frequency of reported incidents should not be used to monitor changes over time, compare hospitals or as a source of outcome measurement in relation to the effectiveness of specific interventions.

Many clinically important prescribing errors fail to be detected and this is a significant barrier to reporting. Given the recognized limitations of incident reporting systems, greater attention should be placed on more effective ways to monitor medication safety in hospitals. Electronic medication management technologies are not only highly effective error prevention interventions; they also present a powerful tool by which to generate up-to-date and comprehensive data regarding medication practices and potential and actual errors. Such data should be used to monitor medication safety, identify new strategies to reduce risk (e.g. introduction of specific electronic decision-support rules) and to provide feedback to clinicians regarding their medication practices.

Funding

Funding to pay the Open Access publication charges for this article was provided by a National Health and Medical Research Council (NHMRC) Program grant APP1054146.