Peer reviewing critical care: a pragmatic approach to quality management.

Critical care medicine frequently involves decisions and measures that may result in significant consequences for patients. In particular, mistakes may directly or indirectly derive from daily routine processes. In addition, consequences may result from the broader pharmaceutical and technological treatment options, which frequently involve multidimensional aspects. The increasing complexity of pharmaceutical and technological properties must be monitored and taken into account. Besides the presence of various disciplines involved, the provision of 24-hour care requires multiple handovers of significant information each day. Immediate expert action that is well coordinated is just as important as a professional handling of medicine's limitations.Intensivists are increasingly facing professional quality management within the ICU (Intensive Care Unit). This article depicts a practical and effective approach to this complex topic and describes external evaluation of critical care according to peer reviewing processes, which have been successfully implemented in Germany and are likely to gain in significance.

Introduction

Critical care medicine frequently involves decisions and measures that may result in significant consequences for patients. In particular, mistakes may directly or indirectly derive from daily routine processes. In addition, consequences may result from the broader pharmaceutical and technological treatment options, which frequently involve multidimensional aspects. The increasing complexity of pharmaceutical and technological properties must be monitored and taken into account. Besides the presence of various disciplines involved, the provision of 24-hour care requires multiple handovers of significant information each day. Immediate expert action that is well coordinated is just as important as a professional handling of medicine's limitations.

Intensivists are increasingly facing professional quality management within the ICU (Intensive Care Unit). This is highlighted by the Vienna declaration on ICU patient safety that was signed at the 2009 European convention of the European Society of Intensive Care Medicine (ESICM) [1]. It includes the commitment to actively pursue quality management within the setting of intensive care medicine. This article depicts a practical and effective approach to this complex topic and describes external evaluation of critical care according to peer reviewing processes, which have been successfully implemented in Germany and are likely to gain in significance.

Learning process in critical care: learn from mistakes, think within processes

ICU quality management refers to multiple professions and disciplines that cooperate in critical care medicine, with the parties involved being highly responsible for its successful implementation. Competences of the various professions and disciplines must be well coordinated and concentrated. Frequently, concepts of evidence-based medicine are insufficiently applied [2]. Concepts of evidence-based medicine may only be successfully realised once integrated into goal-oriented, constantly updated local standard operating procedures (SOP). It is the responsibility of quality management to ensure a reasonable balance between high expectations and reality. Sustainable quality management has been described by Donabedian and can be applied to intensive care medicine as well [3]:

Structure quality: What structural and organizational preconditions must be met? Is the available staff qualified and is the present technology adequate?

Process quality: Are core processes and standards of diagnostic and therapeutic procedures defined? Are they based on evidence?

Outcome quality: Are (therapy-) goals defined and is outcome-relevant data comprehensible? Are transparent outcomes available?

In 1997, the minimum requirements of an ICU have been defined and published by Ferdinande [4] on behalf of the ESICM, which are currently being revised. So far, only structural guidelines were defined for ICUs. Development and realisation of a structural standard (construction guidelines, technology, organizational aspects) of critical care is quite challenging, as current and prospective demands of modern critical care must be taken into account, e.g. the construction of an ICU must consider potential requirements that may result from increasing colonisation and infection with multiresistant strains [5]. Yet, structures are only the beginning when considering the quality of critical care.

Daily routine refers to processes that must be defined and described, in order to be regularly applied. Studies on patient safety and learning from errors (Sentinel-Events-Evaluation-(SEE) studies [6], [7]) have shown that particularly routine processes put ICU patients at risk, e.g. process chains of pharmaceutical preparation and application. The considerable amount of the literature on lung protective ventilation in patients with acute respiratory distress syndrome (ARDS) [8], [9], [10] or weaning strategies [11], [12], [13], [14] documents that there is a viable discussion in regard to core processes of critical care medicine. Due to the availability of a number of publications dealing with core processes of critical care (ventilation, sedation, circulatory support), expert committees developed national and international guidelines. Evidence levels of these guidelines increase as they are being continuously updated and refined. Examples are the guidelines for sedation, circulatory support after cardiac surgery, sepsis, or diagnosis and treatment of heart failure [15], [16], [17], [18]. However, guidelines do not warrant knowledge transfer into practice (implementation failure). Patients can only benefit from the progress of medicine once guidelines are being integrated into local standards and action is trained according to goal-oriented processes.

Implementation of evidence-based medicine on a daily basis remains the central issue in regard to the quality of medicine. It may initially appear as complex, particularly due to the challenges of the description of resources and processes and the requirement for a common definition of evidence-based standards within interdisciplinary teams.

The fact that the German Society for Anaesthesiology and Intensive Care Medicine (Deutsche Gesellschaft für Anästhesiologie und lntensivmedizin, DGAI) und the German Interdisciplinary Association of Intensive Care and Emergency Medicine (Deutsche Interdisziplinäre Vereinigung für Intensiv- und Notfallmedizin, DIVI) agreed on a common core data set of intensive care medicine in 2004 can be seen as an important progress in this issue [19]. This core data set did not only facilitate description and comparison beyond medical disciplines, it was the precondition for quality management on a larger scale allowing analysis of an immense amount of core data [20]. Yet, analysis of this core data set may be associated with elaborate data detection and analysis, since structures like automated data collection such as patient data management systems (PDMS) are not the nationwide standard. On the other hand, this core data set is the basis for performance-related critical care compensation within the German system of case-based lump sums [21]. Complex flat rates of ICU treatment are obtained from data of illness severity and treatment effort, which are both reflected by specific scores recorded during the entire ICU stay [22]. In order to calculate complex flat rates, the item “Glasgow Coma Scale” is removed from the Simplified Acute Physiology Score (SAPS II) [23], with 10 items of the Therapeutic Intervention Scoring System (TISS 28) [24] being added. Once reaching certain limits, the resulting sum scores lead to markedly improved compensation.

It can be of further advantage to record illness severity and treatment effort in combination, since studies from Scandinavia show that – compared to illness severity – treatment effort may decrease over time without a resulting decrease of treatment quality as reflected by risk-adjusted mortality [25]. This showed that treatment effort and cost do not directly correlate with qualitative outcomes of ICU treatment, meaning that more effort is not necessarily better. Preliminary data of the Charité indicates similar results during an observation period of 5 years.

When evaluating process effectiveness and efficiency in patients, patient benefit is most important. Discussions on the benefits from pulmonary artery catheters versus other advanced hemodynamic monitoring, enteral versus parenteral nutrition, or the general abandonment of tonometry are just a few examples for the critical questioning of benefits resulting from ICU processes. As evidence may change over time, standards must be updated and refined in regular intervals and are only valid for a limited period of time!

Focusing on the essentials: staff, processes, patients

Processes of intensive care medicine generally affect the staff as processes are usually executed from humans. Efficient and effective processes relate to efficient and effective staff. Process optimisation is a matter of employee organisation as about 2/3 of the expenditures spent in critical care refers to human resources. Billing of complex treatments of critical care within the German Diagnosis Related Groups (GDRG)-System structurally depends on the availability of qualified physicians and caregivers, which present one of the most important structural criteria within an ICU. 24-hour presence of critical care-experienced doctors and caregivers is expected [26]. The importance of 24-hour presence and hospital availability of a qualified and experienced critical care team as a fast back-up for urgent situations and acute complications has been investigated in detail by a major Australian study (ICU Liaison Nurse) [27]. ICU staffs practicing outcome-relevant procedures in critically ill patients have been identified as a statistically significant factor in regard to patient outcome improvement. A study conducted at the John’s Hopkins University showed that failure in providing punctual ICU treatment is associated with worse outcomes [2].

Which core processes should be highlighted due to evidence-based findings and which bedside processes influence patient outcomes? In order to answer these questions, it is necessary to systematically evaluate key indicators of outcome, that are detectable, reflect clinical settings, show validity and relate to patient outcomes. Systematic approaches in order to develop indicators that refer to critical care quality have been described extensively by Pronovost and colleagues, including associated difficulties [28]. Following the analysis and extensive discussion by an expert committee of the DGAI, key indicators of critical care were elaborated: Spanish indicators previously published were translated [29] and subsequently reduced to independent indicators by consensus. All of these key indicators significantly influenced ICU patient outcome in previous publications. Any indicator could be characterised based on quality of structure, process and outcome. On various ICUs of the Charité University Hospital, Kastrup et al. [30] identified indicators that displayed self-defined quality-reflecting targets of ICU core processes and showed that achievement of targeted goals led to shorter ICU length of stay, which is likely to be associated with outcome improvement. These so-called key performance indicators (KPI) are captured on a regular basis and allow for conclusions in regard to adherence to treatment standards and implementation of core processes.

Rothen and colleagues [31] observed a correlation between standardised ICU patient outcome (SMR = standardized mortality ratio) and standardised resource consumption. They observed that mortality was lower on ICUs with higher caregiver-to-ICU-patient ratios. Contrary, an analogue correlation could not be observed in regard to physicians! Of course, this result may be critically discussed due to methodological issues. Yet, caregivers work the closest with patients and accomplish the majority of ICU patient processes. Apart from a discussion regarding task delegation or demarcation of the various professions, analyses of patient processes present a central issue regarding ICU treatment quality. Process definition and management besides analysis of necessity and feasibility characterise the challenges of such patient process analyses.

Process management that targets the reduction and prevention of insecurities and conflict situations is an absolute necessity in order to investigate the increased risk of burnout within an ICU staff. Approximately 1/3 of ICU caregivers suffer from burnout syndromes according to a French study [32]. The influence of teamwork and team processes on patient outcome was shown in a recent publication [33]: input criteria were defined as team composition, team management, task structures, specific requirements and perspectives; task processes included communication, management, task delegation, coordination, as well as decision process; outcome criteria were defined as patient outcome (adverse events, mortality, length of stay, health related quality of life, end-of-life processes) as well as team outcome, which was assessed by satisfaction with team and job, team morale, stress, burnout, staff exchange, structurally describing links between different qualities and levels of quality.

Various aspects of quality must be considered and balanced in order to have enduring success within a system or a company. “Balanced score cards” ranking interaction of indicators within different levels of quality have been used within the economy and can be applied to health care systems as well [34].

Customer level: To what extent are we stake-holder oriented? (stakeholders of critical care refer to patients, staff, other disciplines or areas connected to the ICU)

Education level: In what way are goals or willingness to learn and advance present? Is knowledge passed on? Is there a learning system?

Process level: How well are processes defined and which have to be elaborated? Where are the weaknesses? Are standards being realised?

Financial level: Are we efficient and effective? Are resources and added value reasonably balanced? Do we consider the overall success?

Success is only likely to occur once these levels are applied in combination since none of them is viable alone.

Realising quality management within critical care

The primary intent of quality management is to target problematic fields within an institution and to get these under lasting control. Vagts et al. [35] may be right when questioning the intended purpose and reality of certifying intensive care medicine. In regard to the limitation of resources within medicine, it is always important to consider the balance between concrete expenditures and definite benefits. The need for more efficient tools of quality management within critical care is obvious. Which techniques can be used for this? The available literature describes a number of potential solutions, of which some provide concise solutions.

The easiest way to prevent mistakes or neglects is to work according to checklists. These have been used for decades within aviation and other disciplines and could be successfully applied to medicine by checking adherence to evidence-based standards off such lists [36]. Byrnes et al. [37] showed that adherence to 14 “best practice” goals of critical care could be increased to 99.7% after working with checklists. Once processes are standardised they can be monitored and controlled by generating report sheets for previously defined performance indicators. The initial goal should not be to achieve a maximum score but to target process optimisation reflected by improved key indicator scores. Goals should be realistic since unrealistic aims cause frustration and defence. Approaches of continuous quality improvement are described in detail within a review from Gallesio and colleagues [38]. Certainly, greater progress due to implementation of quality-improving measures is observed on ICUs with major quality shortcomings than on ICUs starting at higher levels due to previous efforts.

In addition to internal quality evaluation, the authors explicitly address external evaluation according to peer reviewing processes. Once quality-improving measures have been initiated and all ICU parties involved know both strengths and shortcomings of their ICU, external evaluation by peer review is suggested in order to achieve continuous improvement of quality.

ICU peer reviewing in Germany

In 2006, Baden-Württemberg [39] and Hamburg were the first to launch critical care medicine networks. In 2009, they were followed by Berlin/Brandenburg. Primary goal of such networks is to support the implementation of evidence-based medicine within daily routine and to improve both quality and balance between effectiveness and efficiency. One of their main tools is to peer review the critical care treatment provided to patients. Mutual visits of colleagues as well as learning from each other at eye level are the basis for such reviews. Peer reviews particularly highlight collegiality as well as interdisciplinary structures since they explicitly target improvement of communication between various professions and disciplines. In addition, participation of patients and/or their legal proxies in important decisions is structurally promoted.

In the meantime, peer reviews of critical care medicine have been standardised. Mutual responsibility, accordance with transparency, and commitment to absolute confidentiality towards third parties are characteristics of a professional external evaluation. When organising such peer reviews, written statements must be obtained prior to review visits, as reviewers must declare confidentiality and ICU-responsible management must approve. Chief physicians and caregivers of visited ICUs participate in the review process. The reviewer team consists out of two physicians specialised in critical care and one board-certified caregiver, with all reviewers employed at different hospitals. Reviewer assignment particularly takes into account that none of the participants involved is employed by a directly competing hospital. During the pilot stage, both the German Society for Anaesthesiology and Intensive Care Medicine (DGAI) and the Alliance of German Anaesthesiologists support peer reviews in terms of financing, staff and through the work within scientific working groups.

Standardised peer reviews focus on the systematic evaluation of the qualities of ICU structure, process and outcome. Prior to the visit, a list of the questions asked by the reviewers is provided to the ICU staff being visited, since it is in the interest of quality management that ICUs evlaluate themselves before being evaluated by external reviewers. How this happens and to what extent is left to the ICU expecting a visit.

Following the official introduction of participants from both sides, questions regarding visited ICUs are asked to the ICU staff in charge. Based on questionnaires used within clinically approved certification procedures and quality indicators of intensive care medicine, a review questionnaire applying Donabedian techniques was developed. It addresses all aspects of critical care medicine in a brief – yet concise manner (Figure 1 (Fig. 1)). Questions regarding the topics a) basics and organisation, b) staff, and c) patients target the quality of structures and processes. Questions in regard to quality of outcome are classified as either indicators of quality or aims of quality. The remaining part of the questionnaire refers to questions regarding the controlling and reporting systems. The amount of credits assigned depends on answer categories. Choices are “does not apply” (= 1 credit), “planned or in progress” (= 2 credits), “partially applies” (= 3 credits) und “applies” (= 4 credits). The maximum score is 420 credits resulting from approximately 100 questions. However, these scores should not be misunderstood as a grading but interpreted as landmarks in terms of prospective peer review evaluations.

Figure 1

Peer review questionnaire

Once the questionnaire completed, a joint inspection of the ICU follows in order to get to know ICU structures and perform bedside evaluation of adherence to “good practice”.

After this, reviewers and ICU staff in charge meet to share their impressions and ratings. Collegial ideas like learning from one another and self-improvement dominate the entire peer review. Within this context, it is irrelevant whether “peers” are employed in hospitals of different care levels. The aim is to evaluate an ICU based on its potential and its level of care provided. ICUs in hospitals providing maximum medical care may be as organised and structured as ICUs in hospitals providing basic medical care. The willingness to support each other in the evaluation and elaboration of potential problem solutions is explicitly expected from peer reviews.

After the closing meeting, the reviewer team meets one last time in order to write the final report, which is based on reviewer consensus. Strenghts, weaknesses, opportunities and threats are rated by a structured SWOT analysis, the questionnaire score being attached to this. This final report may serve as a landmark to the hospital staff. Potential resource shortages and special achievements are be pointed out.

Consequences resulting from review processes should be understood as a continuous improvement of quality. Besides, they offer suggestions to hospital management in terms of consulting. The final report will be handed out in person to the attending ICU physician, whereas the visited ICU department hands out their evaluation of the reviewer team.

In conclusion, the implementation of ICU peer reviews based on the common data set of the DGAI and DIVI offers the regional opportunity to realise evidence-based and locally adapted quality management, resulting in the formation of a German ICU communication platform that serves the best interest of patients, doctors and caregivers.

Notes

Conflicts of interest

The declarations of conflict of interest of all authors can be viewed on request.