Using risk management files to identify and address causative factors associated with adverse events in pediatrics.

We report a retrospective analysis of 84 consecutive pediatrics-related internal review files opened by a medical center's risk managers between 1996 and 2001. The aims were to identify common causative factors associated with adverse events/adverse outcomes (AEs) in a Pediatrics Department, then suggest ways to improve care. The main outcome was identification of any patterns of factors that contributed to AEs so that interventions could be designed to address them. Cases were noted to have at least one apparent contributing problem; the most common were with communication (44% of cases), diagnosis and treatment (37%), medication errors (20%), and IV/Central line issues (17%). 45% of files involved a child with an underlying diagnosis putting her/him at high risk for an adverse outcome. All Pediatrics Departments face multiple challenges in assuring consistent quality care. The extent to which the data generalize to other institutions is unknown. However, the data suggest that systematic analysis of aggregated claims files may help identify and drive opportunities for improvement in care.

Introduction

Providing quality care for children and reducing malpractice risks have long been goals of pediatricians, medical centers, and their insurers. Identifying the etiology(ies) of serious medical events is an important first step toward those goals. Various methods, including patient chart audits, satisfaction surveys, audits of unsolicited complaints, accreditation/certification evaluations, and morbidity/mortality reviews have been used in attempts to understand the drivers underlying malpractice claims (Luft and Hunt 1986; Hall and Dornan 1988; Berwick 1989; Brennan et al 1990; Hickson et al 2002).

Systematic reviews of risk management claim files have proved useful for identifying common causes of alleged adverse outcomes or threatened litigation that prompted risk management activity on behalf of pediatricians (Pichert et al 1997). The study described herein was designed to 1) conduct a more targeted analysis of a risk management database to identify common causes of pediatrics-related adverse events (AEs) at an academic medical center (AMC), and 2) identify any characteristics that may increase risk for serious medical events and litigation involving children. We conclude by identifying ways in which these data have been employed to drive organizational improvements.

Method

Study design

This study was a structured retrospective analysis of consecutive risk management records from a single institution.

Study population and setting

All patients were treated in an urban academic Department of Pediatrics. For perspective, annual clinical activity during the study period included approximately 820,000 pediatric encounters (inpatient and outpatient). Care was provided by attending physicians, residents, and medical students. All care is supervised by an attending physician.

Data

The source of the raw data for this project included 116 internal review files opened by the Office of Risk Management at the AMC. The target files met several criteria: (a) they were opened from 1996 through 2001; (b) cases were limited to those involving children less than 21 years of age at the time the risk management file was opened; (c) care was provided by the insured medical center’s physicians, hospitals, and/or clinics; and (d) more than three years had passed since the incident or outcome of concern without any indication that a malpractice claim might be pursued, the claim had been dropped, resolved or settled out of court, or the courts had dismissed or rendered judgment regarding the claim.

Routine risk management review process

Table 1 depicts the sequence by which these cases were identified and analyzed. The AMC’s routine risk management process is initiated upon receiving a patient/family member’s allegation of an AE, a formal (incident) report or informal report of an AE from a staff member, or an inquiry from an attorney. All reports are screened. Files are opened only in cases judged by risk managers to represent potential liability or potential need to defend threatened claims. Well trained and experienced risk management claims investigators conduct interviews with staff associated with the case; obtain complete medical records as needed, then review and copy relevant portions for the file; meet with expert consultants as needed; and prepare summaries for subsequent review by a committee of physician/nurse/administrative leaders. As a result, files contain a copy of all relevant medical records, summaries of any interviews with the personnel involved, expert medical and legal opinions, and any other material pertinent to the event. The risk managers participating in this project have an excellent history of fully investigating AEs and identifying events that could lead to lawsuits. Over the previous ten years, fewer than 2% of claims had not been anticipated. In other words, cases are well-vetted and documented prior to the reviews described below for purposes of this study.

Table 1

Sequence of events in this project

Routine Risk Management Processes

1. Occurrence of adverse event or potential adverse event. 2. Adverse event reported to Risk Management Office. 3. Risk Manager evaluates the event and opens a case review file if potential liability is identified or legal action threatened. 4. Risk Manager populates the file with case-related materials, including pertinent medical records, summaries of interviews with physicians and staff, expert reviews and other case-related documents relevant for decision-making.

Subsequent Project-Specific Processes

5. All cases associated with the Department of Pediatrics are distributed randomly to members of a multidisciplinary review group, each of whom is the primary analyst. 6. Primary reviewers present case and analysis to multidisciplinary review group. 7. Review group achieves consensus about likely cause(s) of each alleged adverse event or evaluates additional case information until consensus is achieved.

Project-specific measurement process

Case files were distributed to and initially evaluated by individual members of a team that included two nurses employed full time in risk management and trained as risk management claims investigators, three first year medical students, a Ph.D. researcher, and a pediatrician reviewer (PH). All but the physician served as “primary screener” for a number of cases, a process employed in similar studies (Pichert et al 1997; White et al 2004; White et al 2005). The person serving as primary screener for a case identified all actions, events, and environmental circumstances that risk management’s expert reviews suggested had potentially contributed to the event. The screener also looked at each case for other potential deviations from standard care. So, for example, when trainees were involved in a child’s care, one question was whether the file indicated that they had been adequately supervised. When essential medications were prescribed, the reviewers assessed whether they were delivered as ordered. Similar questions were asked with respect to the timeliness and accuracy of critical tests and films. In other words, screeners were not only to identify probable causes of AEs highlighted by the risk managers’ reviews, but they were also challenged to ask whether risk managers had considered other case-relevant factors that could have contributed to the AE. All participants signed a strict pledge of confidentiality.

Screeners created “cause-and-effect diagrams” (also called Ishikawa or “fishbone” diagrams) for each case to depict relationships between an AE and its apparent contributing factors (Ishikawa 1982). Figure 1 illustrates a fishbone diagram of a fictitious case composed of elements drawn from many cases. In brief, a 6 month old female weighing 15.5 pounds was admitted to the wards with a diagnosis of seizures. Having mistakenly read the recorded weight as kilograms, the house officer on call ordered 310 mg (20 mg/kg) of phenobarbital as a loading dose. The admitting team then went off-call and left the hospital. The nurse noted that the dose seemed high and called the covering intern. The nurse was told, “Loading doses are supposed to be high.” The dose was given and the patient had a respiratory arrest.

Figure 1

A hypothetical 6 month old female weighing 15 1/2 pounds was admitted with a diagnosis of seizures. The cause-and-effect diagram depicts aspects of her care and the adverse outcome following a medication error. Risk management code categories are bolded in the diagram. The elements of the case are as follows: RS, a 6 month-old healthy female, was admitted to the hospital with new onset seizures. An intern was instructed by an attending neurologist, Dr. Neurologist, to administer 20 mg/kg of phenobarbital. The intern noted the weight from the ED triage sheet (recorded as 15.5 kg) and ordered 310 mg of phenobarbital IV. An error message was generated by the computer order entry system, but was overridden by the intern (error messages are always given for loading doses - the system was programmed to check maintenance doses only). Because the on-call period was ending, the intern checked out to the new on-call team. Shortly after the new shift began, RS’s nurse received the loading dose from the pharmacy and was concerned by the amount (310 mgs). She saw the covering intern finishing a note and said “sure is a large dose of phenobarbital for such a small child.” The intern replied, “Loading doses are supposed to be large; the parents have already complained to me about the delay. Get the drug in — NOW. I am NOT answering to a d*** nurse.” The phenobarbital was given and RS stopped breathing. A general code was called. No ambu bag was found in the room, so another intern began mouth to mouth resuscitation. The parents witnessed the event. RS was intubated and transferred to the PICU, where her parents insisted on accompanying her. A staff member refused them entry, explaining that the PICU protocol did not allow parents to be admitted until the patient is stabilized.

The methods for coding such a case have been previously described (Pichert et al 1997). The primary screener assigned codes to the apparent contributing causes from a list of 120 descriptions (Harvard Risk Management Foundation, Malpractice claims description codes [unpublished]). For example, if risk management’s review suggested that a physician failed to educate a parent about signs of a child’s post-operative complication, and this led to delayed care, the code for “Inadequate discharge instruction” from the general category of communication-related items was assigned to the case. As many codes as needed were assigned to each case in order to capture all contributing factors associated with the alleged AE. Given the quality and detail of the risk managers’ case reviews and summaries, medical students and other nonmedical personnel have proved competent as screeners in previous studies. (Pichert et al 1997; Morris et al 2003; White et al 2004, 2005).

Primary case screeners presented their analysis to the rest of the review team. The team met to consider each file together, challenging the primary screener to support the code assignments and offering alternate explanations until the team reached a consensus. Disputes were resolved by applying precedents established in previous projects, including obtaining, analyzing, and coding additional information. Complete medical records were available for review, but copies of relevant portions were sufficient for gaining consensus in all but six files where the review team wanted confirmatory or contradictory information before deciding on a cause. In all six instances the complete record confirmed the reviewers’ judgments based on the file summaries. The physician reviewer (PH) did not assign codes, but clarified and arbitrated questions about technical and procedural issues.

The index of concordance between primary screeners’ initial code assignments and the final consensus-based codes was 87% (number of agreements divided by number of agreements plus disagreements), suggesting adequate initial coder training and general consistency among the judges. The codes that changed, however, reinforce the importance of multidisciplinary team reviews. Specifically, in our experience, judges’ varied backgrounds, professional training, perspectives and inevitable biases promote revelation and evaluation of a case’s “entire picture.” Individual reviewers may focus on selected aspects of a case, but discount, overlook or not fully appreciate the significance of others until reviewers with complementary expertise point them out. Traditional methods of determining “interrater reliability” do not apply when raters’ expertises vary so considerably as in multidisciplinary case review teams. The team approach is essential to achieve comprehensive evaluations.

Data analysis

After stripping the data of identifiers in order to preserve patient confidentiality, codes for each case were collected in a spreadsheet (MS Excel®, Microsoft, Redmond, WA) and analyzed with descriptive statistics (frequency counts and proportions). We also recorded the patient’s reason for presentation.

The Vanderbilt University Institutional Review Board reviewed and approved this project on an exempt basis.

Results

Out of some 820,000 pediatric hospital discharges and ambulatory visits, 116 risk management files were opened, a rate of less than 0.2 file openings per 1,000 visits. After analyzing the files, 84 (72%) were identified that had both an AE (Table 2) and at least one potential causative issue related to that AE. The other 32 were opened for reasons not relevant to the analysis. For example, in one case judged irrelevant to this project, a referring medical center was sued about its care prior to a child’s transfer to the target institution. In this and similar instances the risk managers opened a file simply in order to deal with communications associated with that suit. Other examples of file openings not relevant to the analysis included such events as parents being reported to the State Department of Children’s Services for concerns about potential child abuse, a court-order for blood products administered to a child whose parents objected on religious grounds, known complications of surgery, previously existing conditions, and children who died of sudden infant death syndrome (SIDS). In several cases parents threatened to sue because they became angry about perceived failures of communication between themselves and various care providers, but neither these communications nor any other problem appeared had any bearing on the care provided or the AE; files were opened simply to prepare a defense if the parents chose to pursue a claim. Overall, 32 cases were excluded from further analysis.

Table 2

Adverse outcomes associated with case files

Types of Adverse Outcomes	N of Cases
Death	20
Procedural/Surgical misadventure (eg, nicked aorta, patient left with a limp, instrument burns)	16
Prolongation of stay/course	14
Line Complication	11
Cardiac/Respiratory Arrest or Myocardial Infarction	6
Medication/Transfusion error (eg, erroneous administration of product resulting in adverse side effect or potential side effect requiring additional monitoring)	5
Brain injury (one each: actual injury or potential injury due to hypoxia)	2
Hearing Loss	2
Significant treatment delays that increased treatment challenges and risk of problems	2
Unplanned extubation (NG or Respiratory) requiring additional procedure	2
Burn (treatment-related)	2
Failure to perform test(s) for which patient was admitted, preventing diagnosis and complicating care	2
Total	84

The first goal was to learn the issues in the remaining cases that led to AEs and creation of a risk management file. The cases involved a range of outcomes that might have been avoided or ameliorated (Table 2). Of the 84 cases, several categories of potential causes of or contributors to the adverse event stood out (each file can have more than one code): Communication 44% (37 files), Diagnosis and Treatment 37% (31 files), Medication Errors 20% (17 files), and IV or Central Line Issues 17% (14 files).

The “Communication” category was further examined to evaluate specific types of communication problems identified during the case reviews (each file can have more than one type of communication issue). A common communication issue that occurred in cases with an AE (and a codable contributor to it) was a communication failure that angered a family during a child’s evaluation or treatment, but that had no apparent relevance to the adverse medical outcome (15 files). For example, these included two cases where a “patient was not initially informed of an AE [or its causes], but eventually found out.” Note that all cases in which this issue was identified also had at least one problem that caused or contributed to the AE. On the other hand, communication problems that did appear to contribute to the AE included “communication failure among caregivers inside the institution” (15 files), “communication failure between patient and caregiver” (4 files), “communication failure among care givers outside the institution” (4 files), and “jousting among medical professionals” (3 files).

Another question was whether causes of adverse outcomes clustered around certain underlying diagnoses. A significant portion of the risk management file openings involved “high risk” diagnoses. Specifically, 16% of the children were premature, 13% had congenital heart disease, 11% had other congenital anomalies, 3% had cerebral palsy, and 2% involved trauma. In total, 45% of risk management file openings in the review period carried a high risk diagnosis. The specific causes for adverse outcomes in each subset were evaluated. Premature infants (16% of cases) suffered 5 of the 10 (50%) IV infiltrates identified in this review. No pattern of causes was identified for the other groups, perhaps due to small numbers of cases.

Discussion

The purposes of this project were to 1) use a method of abstracting risk management data at an AMC to identify common factors associated with pediatric AE’s, and 2) describe case characteristics that may increase risk for serious medical events and claims involving children. A previous review covering pediatric risk management files opened at the AMC between 1987–1995 had revealed several opportunities for improvement (Pichert et al 1997), and the medical center had responded over the next several years with several initiatives designed to create awareness of problem patterns (eg, via grand rounds presentations) and improve safety, (eg, via improved care and safety protocols, and targeted training).

Therefore, in addition to identifying current issues, a question was whether the issues themselves had changed from the previous time period. Because risk management file openings still represent rare events for spotting trends, the AMC’s risk managers consider at least five years of data to be most reliable (and generally persuasive). The periods reflected in the previous and current reviews were otherwise arbitrarily chosen. Communication issues, medication errors, and IV/central line problems stood out in both reviews, so remained areas of concern. Payouts and expenditures for pediatrics claims cost the insurer roughly equal amounts during the two time periods. Expenditures during both periods occurred in fewer than half of all cases reviewed; payments ranged from less than $100 to hundreds of thousands of dollars. These expenses included the cost of documentation, legal fees, expert witnesses, and settlements and awards.

A commonly rated communication problem was one that made a family unhappy (eg, perceived rudeness or failure to answer questions), but had no bearing on the AE or its underlying cause(s). Note that in every one of these cases there had been an AE and a separate problem that appeared to contribute to the AE. While it is tempting to ignore or segregate this category because a particular communication problem had no direct effect on the adverse patient outcome, it is important to remember that unhappiness about communications in the face of an adverse outcome can contribute to decisions to sue, and certainly contributes to the costs of risk management. In a Florida survey of families’ reasons for filing suits, nearly one quarter cited a need for more information or the belief that there was a cover up (Hickson et al 1992). In that same study, nearly half of those filing suit felt that they were either misled by a doctor or that a doctor would not be open with them about the case. Other articles have suggested that at least as many medically nonvalid malpractice suits are filed as valid ones (Brennan et al 1991; Studdert et al 2000; Baker 2005;). Communication that leaves families unhappy contributes to these numbers (Vincent et al 1994). Due in part to the data obtained in this study and similar findings in other projects (Pichert et al 1997; Morris et al 2003; White et al 2004, White et al 2005), the AMC now offers case-based instruction on “the how and when of communicating adverse outcomes and errors” (Pichert et al 2007).

Another common apparent communication failure involved “caregivers inside the institution.” Most dyads were represented in the data, including attending–attending, attending–resident, resident–resident, and resident–nurse. Almost half (47%) of all cases involving potential diagnosis or treatment problems also included faulty communication among caregivers, and more than half (53%) involved some type of communication problem. While there is no substitute for thoughtful, timely care, the cases we reviewed suggested that some apparent diagnostic and treatment problems likely would have been avoided if communication difficulties had not also been present, such as during patient handoffs or when a team member failed to voice a concern while care was proceeding (Weinger et al 2003, Weinger et al 2004).

The aviation industry has studied ways to reduce or eliminate the kinds of faulty communication that can result in AEs, and those techniques are being applied to medicine as a way to improve healthcare systems (Wilf-Miron et al 2003). Health professionals’ willingness to speak up in a traditionally hierarchical system, and the ability of those higher in the chain-of-command to listen, are paramount to improving communication (Morey et al 2002). Due in part to the present data, Crew Resource Management, which is based on the aviation-industry team development model, has been deployed throughout the medical center as one strategy for improving professional-to-professional communications (France et al 2005).

Probable medication issues were associated with approximately one-fifth of pediatric risk management files. Again, the current data and other medical center quality improvement initiatives led to integration of a computer order entry system with a pharmacy dose checking system and the results have been very promising (Potts et al 2004).

The extent to which these findings generalize is unknown. Studies such as this must acknowledge that incident reporting varies among institutions, and risk managers apply different thresholds for opening a case file. Nevertheless, so long as risk managers receive sufficient reports to identify common underlying causes of AEs, the system will have value so long as it is used to drive quality improvements (Firth-Cozens et al 2004; Vincent 2006). The more important issue is how such descriptive data can serve physicians, risk managers and administrators whose groups or institutions might have similar challenges. Note, however, that in a litigious environment, any quality improvement-related uses for these data depend upon their protection from legal discovery. Justifying this protection in turn depends upon actually using such data for identifying and working to overcome common causes of adverse outcomes. Consider, then, individual and organizational changes that may be worthy of discussion wherever pediatric care is delivered.

Real or perceived communication failures anger families and thereby promote risk management activity. Identifying and rectifying miscommunications that, for example, lead family members to believe information is being withheld or that the family is being misled may reduce some patient/family dissatisfaction even in the face of an AE and, perhaps, reduce the number who pursue legal proceedings.

Poor communication among caregivers can lead to adverse outcomes and may frequently underlie what appear to be potential or alleged diagnosis and treatment errors. It behooves institutions to empower all caregivers, regardless of role on the health care team, to speak up when something seems not to be right.

Patients with high risk diagnoses account for a disproportionate share of risk management activity. While this is no particular surprise, these numbers reinforce the importance of promoting vigilance with respect to communication, technical aspects of care (eg, in this AMC, placing and monitoring IV lines for premature infants), and documentation in such cases. Caregivers who work with high risk patients may benefit from targeted education and feedback in these areas.

Risk management files provide a rich data source for analyzing adverse outcomes. “Near misses” may offer an even better source were they to be systematically reported and analyzed. User-friendly error reporting systems may help generate sufficient data to more quickly and clearly see patterns of system failures. To that end, the AMC has implemented an online reporting system. A secure web-based system accessible from any medical center computer terminal allows employees, without regard to position, to report concerns. The system’s intent is to capture data as early as possible, allow early error pattern recognition, permit corrections, and avoid poor outcomes that might otherwise have occurred.

The process of aggregating and coding risk management files can be used by other institutions to help understand underlying causes of adverse outcomes. We believe that this process has value because it engages personnel in quality improvement, and it de-emphasizes a culture of blame. The cause-effect analysis process can be taught in continuing medical education programs (Hain et al 2003) in order to help medical staffs understand how underlying systems problems—not just individual physician failures—can affect outcomes. Another valuable aspect of the process is that it provides both data and anecdotes for “closing the feedback loop” by reporting the findings back to the staff. Only after the personnel in a medical area are made aware of the problems can they actively engage in finding solutions.