Quality improvement practices in academic emergency medicine: perspectives from the chairs.

OBJECTIVE: To assess academic emergency medicine (EM) chairs' perceptions of quality improvement (QI) training programs.
METHODS: A voluntary anonymous 20 item survey was distributed to a sample of academic chairs of EM through the Association of Academic Chairs of Emergency Medicine. Data was collected to assess the percentage of academic emergency physicians who had received QI training, the type of training they received, their perception of the impact of this training on behavior, practice and outcomes, and any perceived barriers to implementing QI programs in the emergency department.
RESULTS: The response rate to the survey was 69% (N = 59). 59.3% of respondents report that their hospital has a formal QI program for physicians. Chairs received training in a variety of QI programs. The type of QI program used by respondents was perceived as having no impact on goals achieved by QI (χ(2) = 12.382; p = 0.260), but there was a statistically significant (χ(2) = 14.383; p = 0.006) relationship between whether or not goals were achieved and academic EM chairs' perceptions about return on investment for QI training. Only 22% of chairs responded that they have already made changes as a result of the QI training. 78.8% of EM chairs responded that quality programs could have a significant positive impact on their practice and the healthcare industry. Chairs perceived that QI programs had the most potential value in the areas of understanding and reducing medical errors and improving patient flow and throughput. Other areas of potential value of QI include improving specific clinical indicators and standardizing physician care.
CONCLUSION: Academic EM chairs perceived that QI programs were an effective way to drive needed improvements. The results suggest that there is a high level of interest in QI but a low level of adoption of training and implementation.

INTRODUCTION

Academic emergency medicine (EM) is a practice prone to medical errors due to the volume and complexity of activities that take place.1 Factors such as crowding, resource constraints, cognitive workload, diagnostic uncertainty, high activity levels, interruptions and distractions, shift changes, poor feedback and time pressures can often lead to medical errors.1 Emergency physicians (EP) have an especially critical role, since emergency departments (ED) are increasingly the gateway to acute care and hospitalizations. Between the years 1993 – 2003, there was a 26% increase in the number of annual visits to EDs, but a 12% decrease in the total number of departments.2 An increase in patient demand coupled with a decrease in EDs suggests potential for further crowding, as well as corresponding reductions in physician-patient interaction times. With these increased time pressures and workloads, we propose that it is critical for EPs to focus on continual learning to improve processes of care in EDs and ensure high quality efficient care.

In addition to adapting their behavior to reflect current clinical best practices, other larger organizational issues often affect their ability to practice medicine. EPs often encounter structural and process barriers that impact patient care. For example, structural issues in an ED that can affect quality of care include the design of the department, how it is equipped, availability of ancillary services, staffing mix, and departmental policies and procedures.3 Process issues can also impact the quality of care delivered by EPs. For example, bottlenecks in the ED can create delays in sending and receiving information pertinent to patient care, which can result in either delayed care or misdiagnosis.4 Physician training in quality improvement (QI) programs has the potential to prevent medical errors by training EPs to identify and correct process and systemic weaknesses. However, little is known about the impact of QI training programs on EP behavior and practice.

QI tools, such as Plan, Do, Check, Act (PDCA), total quality management (TQM), Six Sigma and LEAN, are management techniques that focus on improving processes. They are built on a foundation of constant learning by participants, adaptation to better practices and behaviors, and therefore reinforce change. As a result, most QI tools result in structural or process modifications within the organization. These tools are commonly used to identify, analyze, reduce, and monitor medical errors in healthcare organizations today.5, 6 Several tools, such as Six Sigma and TQM, are data driven and focus on the measurement of defects and errors, while others, such as Lean and PDCA, focus on removing waste and unnecessary steps from processes.7–10 Both TQM and Lean focus on cultural changes within the organization to improve quality.6,11

However, getting physicians to change their behavior is often difficult. Educational efforts such as continuing medical education (CME) are largely ineffective at changing physician behavior and performance.12 Additionally, CME rarely trains physicians how to solve issues involving other departments where there is an impact in the quality of care provided to patients in the ED. Furthermore, physicians in academic leadership roles have often achieved their position on the basis of their clinical expertise or scientific accomplishments rather than specific management training. Physicians may lack familiarity with basic QI techniques, but are often charged with maintaining and improving the quality of care, which involves changing the behavior of other physicians. This is a difficult task if physician-managers do not have the management tools to change structures and processes that impact the effective delivery of high-quality medical care. In response, many hospitals and academic medical centers have implemented QI training programs and are familiarizing physicians with these tools in hopes to give physician leaders the skills necessary to change the system and improve quality of care. 13, 14

Currently, there are no studies that examine the QI tools and techniques available to academic EPs. It is unclear how many EPs are being offered QI training, who is being offered the training (department chairs or practitioners) and which types of QI tools are being implemented. The purpose of this research is to assess academic emergency medicine (AEM) chairs perceptions of value, both realized and potential, of QI training programs.

METHODS

Study Design

The purpose of this project is to assess AEM chairs’ perceptions of value, both realized and potential, of QI training programs. No studies published to date have assessed academic EP perceptions of the value of QI training programs. To achieve this purpose, we developed a 20 item survey instrument to assess the percentage of AEM physicians who had received QI training, the type of training they received, their perception of the impact of this training on behavior, practice and outcomes, as well as any perceived barriers to implementing QI programs in the ED. This is a mixed-methods cross-sectional survey design using close-ended items with numerical responses and open-ended items on the same survey. 15 This mixed method survey design was chosen for triangulation purposes, to add breadth and scope to the project. 16

Study Setting and Population

The target population for this study is the chairs of academic EDs in the United States (U.S.). After obtaining Institutional Review Board approval, we gathered data from a sample of academic chairs of EM from the Association of Academic Chairs of Emergency Medicine (AACEM). This organization, comprised of the chairs of autonomous academic departments of EM in the U.S., currently has 85 members who are chairs of academic departments at Liaison Committee on Medical Education approved medical schools. We received a 69% (N = 59) response rate to the survey.

Study Protocol

Surveys were distributed to AACEM members at the annual meeting, as well as converted into an electronic form. To poll the chairs who did not attend the AACEM meeting, E-mails were sent to the AACEM list serve, asking members to participate in the survey chairs. Written informed consent of human subjects was waived by the authors’ Institutional Review Board.

Measurement

Survey items were created to address the specific aims of the study. Table 1 contains a list of specific aims and the corresponding survey questions created to address them. Two questions addressing demographic information were also included to control for length of practice in EM and size of ED.

Table 1

Survey questions and corresponding aims of the study

Specific Aim	Survey Questions
1. Investigate if academic ER physicians have access to QI training programs through their respective employers, and the extent of their participation in QI programs.	1. Has your hospital or department adopted a formal quality improvement program for physicians?
	□ Yes	□ No	□ Considering in the next 6 months
	3. How long ago was the quality program first implemented? _____ months □ NA
	4. Have YOU recently participated in any training programs for quality improvement?
	□ Yes	□ No
	5. How many days of training did you participate in the last 12 months?
	7. Approximately, how long has it been since you returned from training class? Weeks
	8. Since you have returned from the training, how many specific projects or processes have been analyzed by you or your team?
2. Examine the type and perceived effectiveness of QI program training received by academic ER physicians.	2. If yes, which of the following best describes this quality program:
	□ Lean/Toyota Production System
	□ Six Sigma
	□ PDCA
	□ Total Quality Management
	□ Not sure
	□ Other, please list _______________
	6. Who provided the training?
	□ Internal Trainers	□ External Consultants	□ Other
	9. Have you achieved the goals you expected from your training?
	□ Yes	□ No	□ Not Sure
	12. What specific impact would you say is most noticeable on your outcomes, if any?
	13. Have you measured, in concrete and quantifiable terms, this change in outcomes?
	□ Yes	□ No
	14. What best describes your feelings about the return on investment from quality training?
	□ Positive	□ Negative	□ Neutral/Not Sure
3. Determine if and how QI training changes the behavior or practice of academic ER physicians.	10. Which of the following best describes how you feel about behavioral or process changes in your practice of medicine based on the quality training your received?
	□ I will make changes sometime in the foreseeable future.
	□ I intend to make specific changes in the next 6 months.
	□ I will make specific changes next month.
	□ I have already made specific changes to my practice
	□ I have made changes and am working to maintain these changes.
	□ I have made changes which are now permanently engrained in my daily activities.
	11. What process changes to your own medical practice or department have you initiated?
4. Examine the overall perceptions of the impact of QI training, including potential value to practice and barriers to implementation.	15. Do you feel that quality programs could have a SIGNIFICANT positive or negative impact on your medical practice or the healthcare industry?
	□ Positive	□ Negative	□ Not Sure
	16. In your opinion, do quality programs have the greatest potential value in which of these areas (please choose and rank the top 2):
	Rank
	__Understanding and reducing medical errors
	__Improving specific clinical indicators
	__Cost savings or enhanced efficiency
	__Improving throughput or patient flow (e.g., reducing wait times)
	__Enhancing patient satisfaction
	__Standardizing physician care/treatment
	__Increased employee morale
	__Improving other administrative processes
	__Other _______________
	17. Do you feel that that the medical industry has done enough to improve quality of care?
	□ Yes	□ No	□ Not sure
Demographic questions	18. What is the size of your emergency department, in terms of:
	Number of physicians_______________
	Number of patients seen last fiscal year_______________
	19. How many years have you been practicing medicine? _______________

ER, emergency room; QI, quality improvement

Data Analysis

We used descriptive, correlation and qualitative analyses to determine the association between perceived impact of QI training, perceived change in practice and behavior, and type of training received. We used qualitative analyses to examine open-ended questions and analyzed themes of behavioral and practice changes analyzed to determine specific changes related to receiving QI training.

RESULTS

Overall, 59.3% of respondents report that their hospital has a formal QI program for physicians. A small percentage (3.4%) of respondents report that their hospital considers adopting a program in the next six months. The remaining respondents (37.3%) report that their hospital has no QI training program for physicians. Of the hospitals that have QI programs, the surveys indicate that they were implemented 20.6 months or a little less than two years ago. A majority (64.8%) of AEM Chairs who responded to our survey have not participated in a program for QI. Of the 19 respondents who participated in QI training, the average number of days spent in training was three, which did not include one outlier who reported over 30 days spent in training. On average, respondents reported it had been 18.4 weeks since returning from QI training, and that they had completed an average of 3.78 projects in that time period.

AEM chairs identified a variety of QI programs at their respective hospitals, but 22.5% chairs could not identify which type of QI program their hospital used. Figure 1 summarizes the types of QI programs identified by academic EPs.

Figure 1

Type of quality improvement program used at the hospitals of academic emergency medicine chairs.

PDCA, plan, do, check, act

Descriptive analyses show that the majority of respondents (63.2%) report that QI training was conducted by internal trainers. However, chi-square tests show that there is no statistically significant relationship between type of QI program implemented and internal or external trainers (χ2=4.638; p=0.914). Additionally, the type of QI program used by respondents had no impact on goals achieved by QI (χ2=12.382; p=0.260), whether or not changes in quality were measured (χ2=3.656; p=0.600), or perceptions about the return on investment for QI training (χ2=10.827; p=0.371). There was also no statistically significant relationships between who conducted the training and whether goals were achieved (χ2=4.886; p=0.299), changes were measured (χ2=2.825; p=0.244), or perceptions about return on investment for QI training (χ2=4.554; p=0.336). However, there was a statistically significant (χ2=14.383; p=0.006) relationship between if goals were achieved and AEM chairs perceptions about return on investment for QI training. If projects were successful, then the AEM chairs were significantly more likely to have positive perceptions about the return on investment of QI training. These findings did not vary significantly by years of practice or size of ED. When asked about the specific impact of QI training that was most noticeable on outcomes, AEM chairs reported (in the order of frequency) increased: 1 efficiency, 2 patient care quality and 3 morale and teamwork. However, 16% of respondents reported no noticeable outcomes. Increased efficiency included shorter lengths of stay in the ED, fewer patients leaving before treatment, decreased supply spending, and general increased physician productivity. Improved quality of care included fewer errors related to patient hand-offs, shorter wait times for critical procedures, and better nursing care. Improved morale and teamwork, both within the ED and between the ED and other departments, were also reported.

Figure 2 summarizes respondents’ plans to make changes to their behavior based on QI training.

Figure 2

Academic emergency medicine chairs plans to make changes to their behavior and practice based on the quality improvement training received.

When asked about specific changes initiated in their own practice or department, AEM chairs listed the following changes: decreased time from patient arrival to initial physician evaluation; decreased time from patient arrival to the completion of an electrocardiogram for patients with chest pain; improvement in compliance with Joint Commission core measures for pneumonia; increased compliance with airway management protocols; increasing the proper use of certain bedside tests (e.g. allowing a stool guaiac test to sit 2–3 minutes before reading); improving and standardizing end-of-shift patient hand-offs; improved compliance with Center for Medicare & Medicaid Services Physician Quality Reporting Initiative measures for stroke, myocardial infarction (MI) and pneumonia; implementation of rapid response systems for stroke and MI; creating special order sheets for pneumonia; improvements in teamwork in a “fast track” area; creation and implementation of streamlined admission and discharge processes; improvements in overall patient flow; improvements in team interactions; movement of patients to inpatient hallways to reduce ED crowding; development of rapid triage, bedside registration, and medication reconciliation processes; implementation of administrative “patient safety walk rounds”; development of a “Lean and Green” committee to improve efficiency and decrease environmental impact, creation of a process to improve the completeness of physician documentation, reduction of time from patient arrival to definitive care for patients with acute MI (“door to balloon time”); and increased follow-up for patients who elope from the department and “left without being seen” patients.

Correlation analyses show no statistically significant relationship between changes in behavior or practice and type of QI training received (χ2=21.867; p=0.643). However, there was a statistically significant (χ 2=21.659; p=0.017) relationship between behavioral changes and type of trainers. AEM chairs who received training from internal trainers were significantly more likely to have made changes and work to maintain those changes. Analysis of variance showed there were no statistically significant differences between changes in behavior based on number of weeks since returning from class (F-ratio=0.39; p=0.846), years of practice experience (F-ratio=1.78; p=0.197) or size of ED (F-ratio= 0.29; p=0.907).

When AEM chairs were asked if quality programs could have a significant positive or negative impact on their practice and the healthcare industry, 78.8% responded that the impact would be positive. Only 7.7% perceived that QI programs would have a negative impact, and the remaining 13.5% were not sure. AEM chairs also overwhelmingly (82%) responded that the medical industry has not done enough to improve quality of care, while only 6% disagreed, and 12% were unsure. Table 2 summarizes the rankings in the areas of greatest potential value of QI programs.

Table 2

Greatest potential value of quality improvement programs

	Rating average	Overall ranking
Understanding and reducing medical errors	1.69	1st
Improving patient flow/throughput	1.68	2nd
Improving specific clinical indicators	1.47	3rd
Standardizing physician care/treatment	1.38	4th
Increased employee morale	1.33	5th
Enhancing patient satisfaction	1.25	6th
Cost savings or enhanced efficiency	1.10	7th
Improving other administrative processes	1.00	8th

Correlation analyses showed no statistically significant differences in perceptions of QI program impact between physicians who had participated in training and those who had not (χ2=2.920; p=0.232). However, data did show a strong statistically significant (χ2=30.781; p=0.000) positive relationship between the belief that QI training programs could have a positive impact on the healthcare industry and the belief that the medical industry has done enough to improve quality of care. There was no statistical significance among overall perceptions of QI training, years of practice and size of ED.

DISCUSSION

In this study measuring the perceptions of use and value of QI training programs in AEM, we collected survey data from 69% of the 85 academic chairs in AACEM. This is the first study of their perceptions of QI training and value. The chairs reported that QI programs are relatively new to their EDs. This finding is somewhat surprising given that some QI programs such as TQM and PDCA have been used in hospitals for over 20 years and that a majority of U.S. hospitals have adopted some form of quality management program.17 Although most of their institutions offer training in one or more QI programs, a majority of chairs (64.8%) have not participated in any of these programs. This finding is consistent with recent research that found only about one-third of physicians have participated in system redesign efforts to improve the performance of the system in which they practice.18 Of the chairs who participated in a QI program, most received a brief exposure provided by internal trainers. Nonetheless, many reported that they were able to successfully improve some aspect of their clinical practice using the principles they learned.

The type of QI training (Six Sigma, Lean, TQM, PDCA) received and whether training was administered by external or internal trainers did not seem to affect the chairs’ perceptions of effectiveness. As might be expected, however, those who perceived that the goals of the training had been met were more likely to believe that their training program had been effective. Although this finding is not necessarily surprising, it is surprising that 41.2% of respondents who participated in training were not measuring changes in outcomes, given that Six Sigma, Lean, and TQM propose establishing goals and measuring performance in quantifiable terms. 19–20 This finding suggests that some QI programs are not being implemented correctly, and the lack of measurement of quantifiable outcomes and progress towards goals may affect perceptions of effectiveness of QI programs. Furthermore, the chairs reported that training received from internal trainers was more likely to impact their behavior and desire to continue maintain the changes that they had made. We speculate that the reason for this finding is because internal trainers are likely to serve as QI experts and role models and are more accessible than external trainers who leave after the training is complete.

We found that most chairs perceived that QI is important and has the potential for improving patient care in U.S. EDs. Further analysis of the perceptions of potential benefits by the chairs suggests that QI could help to understand and reduce medical errors and improve patient flow and cycle times, both critical measures of ED performance. When asked about specific changes made to their own practice in order to improve quality, chairs listed numerous implementations to improve quality. These changes are consistent with research that shows that QI programs are currently being used in EDs to improve throughput and reduce medical errors.21–25 Outcomes targeted for improvement include ED workload or volume, patient wait time, ambulance diversion, patient walkout rate, length of stay, patient satisfaction, triage of patients, turn-around time of other services such as laboratory and radiology, available beds, registration, discharge processes and staffing patterns. 21, 23–25

LIMITATIONS

There are several limitations to this study. First, the population of AEM chairs is relatively small. This limits our statistical analyses to chi-square and correlation analyses. Although we received a 69% response rate to the survey, results may have been biased. Chairs interested in QI efforts or who have received training may have been more likely to respond to the survey thereby creating a positive bias in favor of QI in the response. Additionally, the responses to the survey are opinions only and are not quantitative measurements of the programs or types of changes.

CONCLUSION

The number of EDs continues to decline while patient numbers keep increasing. The result is crowding, less patient interaction time for diagnosis and treatment, and the need for more efficient processes, practices and behaviors to improve core quality measures. Academic EDs will play a pivotal role in improving processes of care since they are generally larger facilities and treat more complex and diverse cases. This results in an increased need for EPs who can learn rapidly, and proactively improve processes of care. The implications of this research for physicians and administrators in EM are that QI efforts are perceived to have value. AEM chairs believe that QI programs were an effective way to drive needed improvements. Overall, the results suggest that there is a high level of interest in QI, but a general low level of adoption of training and implementation. Unfortunately, actual testing and demonstration of value has been minimal to date and there are very few EDs that have monitored and recorded quantifiable gains - thereby slowing the adoption of these methods. However, initiation and use of QI methods is especially important for chairs, who serve as leaders, mentors and role models to AEM physicians in the march towards healthcare quality.