Actual performance of mechanical ventilators in ICU: a multicentric quality control study.

Even if the performance of a given ventilator has been evaluated in the laboratory under very well controlled conditions, inappropriate maintenance and lack of long-term stability and accuracy of the ventilator sensors may lead to ventilation errors in actual clinical practice. The aim of this study was to evaluate the actual performances of ventilators during clinical routines. A resistance (7.69 cmH(2)O/L/s) - elastance (100 mL/cmH(2)O) test lung equipped with pressure, flow, and oxygen concentration sensors was connected to the Y-piece of all the mechanical ventilators available for patients in four intensive care units (ICUs; n = 66). Ventilators were set to volume-controlled ventilation with tidal volume = 600 mL, respiratory rate = 20 breaths/minute, positive end-expiratory pressure (PEEP) = 8 cmH(2)O, and oxygen fraction = 0.5. The signals from the sensors were recorded to compute the ventilation parameters. The average ± standard deviation and range (min-max) of the ventilatory parameters were the following: inspired tidal volume = 607 ± 36 (530-723) mL, expired tidal volume = 608 ± 36 (530-728) mL, peak pressure = 20.8 ± 2.3 (17.2-25.9) cmH(2)O, respiratory rate = 20.09 ± 0.35 (19.5-21.6) breaths/minute, PEEP = 8.43 ± 0.57 (7.26-10.8) cmH(2)O, oxygen fraction = 0.49 ± 0.014 (0.41-0.53). The more error-prone parameters were the ones related to the measure of flow. In several cases, the actual delivered mechanical ventilation was considerably different from the set one, suggesting the need for improving quality control procedures for these machines.

Introduction

Quality control and risk assessment, which are crucial to achieve the highest possible effectiveness of medical care,1 are particularly important in the intensive care unit (ICU) given the critical health status of the patients treated in this hospital setting.2,3 Among the different issues that should be addressed by quality control protocols in the ICUs, mechanical ventilation is especially relevant because this life-sustaining treatment must be carefully applied to strike a balance between insufficient ventilation and ventilation-induced lung injury.3

The first step to optimize mechanical ventilation is to use high-performance ventilators. Fortunately, currently available devices, using built-in sensors, microprocessors, and intelligent software, are able to adequately provide and monitor the target ventilation.4–7 The second step in mechanical ventilation quality control is to have personnel adequately trained to properly use the device accordingly to manufacturer instructions and to carry out an efficient servicing/maintenance of the equipment, ie, to ensure that the ventilators that have a satisfactory performance in the laboratory are also working adequately in their daily use in clinical ICU routines. To ensure correct functioning of mechanical ventilators, ICU personnel are subjected to specific training and these devices are subjected to well-defined periodic maintenance procedures and recalibration. However, the frequency and procedures are established by manufacturers, hospitals, and legal regulations, with occasional servicing procedures for repairs. However, it is remarkable that there are no published data on the overall effectiveness of all the above-mentioned factors, therefore reporting the actual performance of mechanical ventilation delivered to patients in the clinical ICU routines.

Accordingly, the aim of this work was to carry out a cross-sectional multicentric study to assess the real-life performance of ICU ventilators. We measured whether the ventilators ready to be used or in use on patients provided the target ventilation and whether the devices accurately monitored the ventilation variables. To this end, a custom test lung, including an analog lung model and reference sensors, was developed, tested, and connected to all the mechanical ventilators that were currently in routine use in four university hospital ICUs.

Materials and methods

Test lung

The actual performance of the ventilators was evaluated during mechanical ventilation of a custom-designed test lung (Figure 1). We tested the ventilator’s performance by connecting it to a reference lung instead of placing the sensors at the entrance to the endotracheal tube during patient ventilation to avoid breath-by-breath biological variability and the variation associated to the fact that different ventilator settings are used among patients in ICUs. The reference lung was connected to the ventilators through the bacterial filters used in the wards, and, depending on the manufacturer, they are characterized by a range of resistance of 1.2–1.5 cmH2O/L/s, and a dead space ranging from 60 mL to 63 mL. The device was built by connecting a mesh-type linear flow resistor of 7.69 cmH2O/L/s to an elastance model behaving as a purely elastic body with a static compliance of 100 mL/cmH2O. Linearity and reproducibility of the mechanical properties of these components have been tested accurately through bench evaluations over the range of operating variables.

Figure 1

Experimental set-up.

Sensors

A pressure transducer (DCXL30DS; Honeywell, Minneapolis, MN) and a Fleisch pneumotachograph (Sibel, Barcelona, Spain) connected to a second differential pressure transducer (DCXL01DS; Honeywell) were placed at the inlet of the test lung in order to accurately measure the actual pressure and flow provided by the ventilator.

A zirconia solid-electrolyte cell oxygen sensor (MWL-F, 0.1%–95% O2 ±1%; Fujikura Ltd, Tokyo, Japan) was connected immediately before the Y piece on the inspiratory line to measure the oxygen fraction in the inspiratory gas (FiO2) only when needed (see below).

The pneumotachograph was calibrated in room air with a 3L calibration syringe (Sibel). To correct for the different viscosity of the gas mixture delivered by the ventilators (FiO2 = 0.5), the calibration coefficient was corrected by a factor of 1.048.8 A water column was used to calibrate the pressure sensor. The oxygen sensor was calibrated using three different gas mixtures: room air (20.9% oxygen), pure nitrogen, and pure oxygen.

The values recorded were in ambient temperature and pressure either saturated with water vapor (ATPS) or in dry conditions (ATPD) depending if the ventilator was provided with a heated humidifier or not. In order to evaluate the amount of gas actually delivered to patients’ lung, the appropriate correction was applied to obtain the readings in body temperature and pressure, saturated with water vapor (BTPS) conditions by using the following equations:

where PA is the barometric pressure measured at the beginning of each experimental session; PVapB is the vapor pressure at body temperature (47 mmHg); PVapS is the partial pressure of water in saturated air in ambient conditions. TB is body temperature (310 K), and TA is ambient temperature measured for each experimental session.

Experimental procedure

All the ventilators available for patient use at the time of the study in the four ICUs participating in this study were tested: Hospital Universitario de Getafe, Madrid, Spain (A); Ospedale San Gerardo, Monza, Italy (B); Hospital Parc Tauli de Sabadell, Barcelona, Spain (C); Hospital Clinic, Barcelona, Spain (D). The test lung was carefully calibrated prior to any experimental session and its performance and potential leaks evaluated. The same test lung and equipment was used for all measurements.

All the ventilators were configured in order to apply volume-controlled mechanical ventilation with the following settings: tidal volume (Vt) = 600 mL, respiratory rate (RR) = 20 breaths/minute, inspiratory time = 1 second, expiratory time = 2 seconds, end-inspiratory pause = 10%, inspiration/expiration (I/E) ratio = 0.33, positive end-expiratory pressure (PEEP) = 8 cmH2O and FiO2 = 0.5. All the other parameters/settings, such as corrections for compressible volumes of ventilator circuits, and presence and type of humidifiers were set by the staff. Calibration procedures, if needed, were followed as for usual clinical practice in the ward. The ventilators were connected to the test lung through the breathing circuit and an antibacterial filter (employed at each ICU) and used to ventilate the test lung. After 30 seconds of ventilation to allow for stabilization, the signals from the transducers were digitized with a sampling rate of 10 Hz for 3 minutes and stored on a personal computer. During signal recording, the breathing pattern parameters displayed at the control panel of the ventilator were read, averaged, and registered. To avoid any alteration of the ventilators’ breathing circuits, during this recording the oxygen sensor was not connected to the test lung. Immediately after this first signal recording, the oxygen sensor was connected to the inspiratory line and its signal was recorded for 3 minutes after allowing at least 1 minute of ventilation.

Data analysis

From each recording, the flow signal was digitally integrated and any linear drift was corrected to obtain the lung volume signal (VL). As sensors were calibrated, the drift correction was used mostly to identify possible leaks in the circuits/connections. For each breath, inspiratory volume (Vti), expiratory volume (Vte), peak pressure (Ppeak), PEEP, and RR were computed and averaged for each recording providing a single data point. FiO2 was evaluated by averaging the oxygen sensor signal. BTPS correction was applied as necessary.

Differences between hospitals and models of mechanical ventilator were assessed by two-way analyses of variance (ANOVA; SigmaStat; Systat Software Inc., San Jose, CA, USA) considering the absolute value of the discrepancy between the figure set on the ventilator and the delivered values actually measured in the test lung. Similarly, the values of the parameters read at the ventilator panel were compared to the actual values measured by the test lung. To this purpose, we decided to average the readings provided by the ventilator during the trial. As it is not always clearly reported if a given ventilator is reporting independent values for each single breath or after filtering or smoothing procedures, considering an average reading over the recording period minimizes possible differences due to the data-processing strategy adopted by each model of ventilator.

Bland–Altman analysis was carried out for all the parameters between the measured values and the measures provided by the ventilators.

Results

The stability of the sensors of the test lung was confirmed by the maximum deviation of calibration coefficients from their average values (computed by considering all calibrations performed over a period of 3 months) being less than 3.8%, 1.4%, and 1.0% for flow, pressure, and oxygen sensors, respectively. A total of 66 ventilators were studied: 23, 22, 14, and 7 in centers A, B, C, and D, respectively. The models were: Draeger Evita 4 (EVITA4, 25 machines) (Drägerwerk AG & Co. KGaA, Lubecca, Germany), Siemens/Maquet Servo I (SERVOI, 16 machines) (Siemens AG, Munich, Germany; MAQUET GmbH & Co. KG, Rastatt, Germany), Siemens SV900C (SV900C, 12 machines), Siemens/Maquet Servo 300 (SERVO300, 7 machines), Draeger Evita XL (EVITAXL, 3 machines), Siemens SV900D (1 machine), Draeger Evita 2 (EVITA2, 1 machine) and GE Engstrom (GE, 1 machine) (GE, Fairfield, Connecticut, USA). Table 1 summarizes for each type of ventilator the modality of volume display, the activation of algorithms for the compensation for compressible volume, and the presence of heated humidifier (HH) in the circuit.

Table 1

	Volume data reported	Compressible volume correction	# of ventilators with HH filter	# of ventilators without HH filter
Siemens SV900D	ATP	Yes	0	1
Siemens SV900C	ATP	Yes	7	5
Maquet Siemens Servo 300	BTPS	Yes	2	5
Maquet Siemens Servo i	STPD 0°C	Yes	2	14
Drager Evita 2	BTPS	Yes	0	1
Drager Evita 4	BTPS	Yes	10	15
Drager Evita XL	BTPS	Yes	1	2
GE Engstrom	BTPS	No	0	1

Abbreviations: ATP, ambient temperature and pressure; BTPS, body temperature and pressure, saturated; STPD, standard temperature and pressure, dry; HH, heated humidifier.

The measured values of the breathing pattern indices delivered by the ventilator are reported in Figure 2. Although, on average, these values were very similar to the target settings, we found a remarkable intermachine variability. The ranges (min–max values) of the different variables were: 530–723 mL for Vti, 7.2–10.8 cmH2O for PEEP, 0.41–0.53% for FiO2, and 19.5–21.6 breaths/minute for RR. For these variables, 5, 7, 1 and 0 machines showed errors greater than 10%, respectively. The variables showing the greater variability were the volumes, with machine delivering up to 20% more and 11% less than the preset value of 600 mL, and the PEEP, with delivered values up to 35% more and −9% less than the preset value of 8 cmH2O.

Figure 2

Actual values of several breathing pattern parameters delivered by the machines for each ventilator included in this study.

Note: Different symbols refer to the different hospitals.

Abbreviations: BPM, breaths per minute; PEEP, positive end expiratory pressure.

The same variables are represented in Figure 3 grouped by ventilator model. In this case, the results are quite variable, with the variability within the same model being, for most variables, bigger that the variability between models.

Figure 3

Actual values of several breathing pattern parameters delivered to the test lung for each ventilator model.

Notes: Labels identify the following ventilator models: EVITA4: Draeger Evita 4 (25 machines); SERVOI: Siemens/Maquet Servo I (16 machines); SVC900C: Siemens SV900C (12 machines); SERVO300: Siemens/Maquet Servo 300 (7 machines); EVITAXL: Draeger Evita XL (3 machines); SV900D: Siemens SV900D (1 machine); EVITA2: Draeger Evita 2 (1 machine) and ENGSTROM: GE Engstrom (1 machine).

Abbreviations: BPM, breaths per minute; PEEP, positive end expiratory pressure; FiO2, fraction of inspired oxygen.

This is confirmed by the absence of statistically significant differences in the discrepancies between the preset and actual values between hospitals. Considering the different ventilator models, the two-way ANOVA analysis showed a significant difference in the errors between the ventilator models only for PEEP (P = 0.009) and RR (P < 0.001) values. The most relevant difference for PEEP errors (between SV900C vs SERVOI and vs EVITA4) was 10% and for RR (between EVITA2 vs SERVOI) was 7.6%.

Figure 4 shows the differences between values measured by the ventilator and the values measured at the test lung. In this case, there was a large variability between machines. The range (min–max values) of the percentage differences between machine reading and actual values were: −17%–33% for Vti, −70%–72% for Vte, −21%–13% for Ppeak, and −6.8%–5% for FiO2. For these variables, 6, 18, 25, 7, 0, and 0 machines showed errors greater than 10%, respectively. No significant differences were found between hospitals. Only Ppeak differed between ventilator models, showing significant statistical difference (P < 0.01). In Figure 5, cumulative distribution plots show the number of ventilators with a given accuracy value for Vt, PEEP, FiO2, and RR. Finally, Bland–Altman analysis plots for all the measured parameters are shown in Figure 6.

Figure 4

Percentage difference between values reported by the ventilator and the ones measured by the test lung for the different breathing pattern parameters considered in the study. Symbols are used as in Figure 1.

Abbreviations: Ppeak, peak pressure.

Figure 5

Cumulative percentage of ventilators as a function of their accuracy in delivering the preset inspiratory volume, PEEP, FiO2, and respiratory rate.

Abbreviation: PEEP, positive end-expiratory pressure; RR, repiratory rate; Vt, tidal volume.

Figure 6

Bland–Altman analysis graphs plotted for inspiratory and expiratory volumes, PEEP, FiO2, and respiratory rate.

Abbreviation: PEEP, positive end-expiratory pressure; BPM, breaths per minute; Ppeak, peak pressure.

Discussion

The aim of this study was to evaluate the ventilation delivered to patients in an ICU. In particular, we were wondering how the ventilator parameters prescribed by the attending physicians and programmed on the ventilator are actually delivered to patients. To this end, we created a test lung and test protocol to simulate a patient admitted to an ICU, therefore testing not only the efficacy of the device itself, but also of all the procedures that are defined and implemented to keep the devices working properly with time (maintenance, calibrations, effectiveness of repairs, etc). The most important finding was that, even if there was good overall performance shown by the ventilators, we found considerable performance variability with substantial errors in some individual cases, with some units showing errors greater than 10%, especially in the ventilation volumes. Even if these differences are unlikely to have major clinical relevance, the variability observed in our data underlines the importance of regular testing and maintenance procedures.

In contrast with previously published reports where the performance of different mechanical ventilator models was studied under well-controlled laboratory conditions7–9 on one single ventilator unit, in the present work we analyzed the actual performance of all ventilators during routine use within the ICU, with the aim of evaluating the reproducibility of mechanical ventilator performance in typical use.

Remarkably, the ventilators tested in this study were either the ones just disconnected from a patient or the ones ready for immediate use in new patients. It is also of note that neither the nursing/technical staff in care of ventilators nor the company/service in charge of ventilator maintenance were aware of the study. Accordingly, the performance results we obtained are representative of the way ventilators actually perform on patients in clinical practice.

The absence of noticeable systematic errors in the average ventilator performance suggests that the general maintenance procedure followed by the different centers was consistent. The finding that the level of errors found in ventilator performance did not depend on the hospital adds support to this interpretation. However, we found a considerable variability in the errors, mostly for volume- and Ppeak-related variables (Figure 4), strongly suggesting that either the frequency or the procedures of calibration/servicing need to be adjusted in order to avoid errors produced by any of the devices used in the ICU. Obviously, what is important for patient safety and medical care quality control is not the average of errors, but the individual figures.

These results that we found for these ICU devices are similar to the ones reported for home mechanical ventilation.10 However, given the more critical conditions of ICU patients and the greater complexity, costs, and servicing intervention of ICU devices, our results suggest that there is some room for further improvement in the performance of mechanical ventilators.

Limitations of the study

In this study, we only simulated one value of respiratory system impedance. Given that the performance of ventilators may depend on the mechanical load of the patient, the results we obtained do not cover all the range of potential cases found in clinical routine. However, the values of respiratory resistance and compliance that we implemented in the lung model, although reasonably representative of patients under mechanical ventilation, are in no way exceptional. In fact, greater mechanical impedance values are expected for a significant number of ICU patients subjected to mechanical ventilation because of severe respiratory diseases (eg, chronic obstructive pulmonary disease exacerbations, pneumonia, acute respiratory distress syndrome). It could be anticipated that in such instances, ventilator performance could be poorer than the one observed with the lung model used in this work.

Another limitation of this study is that, for the sake of comparison, we only tested one ventilator mode (controlled volume ventilation) and with only one set of ventilation parameters (eg, Vt, PEEP, f). Although both the investigated ventilator modes and settings are reasonably representative of mechanical ventilation of ICU, they do not cover the full possible spectrum. In particular, we did not test pressure-controlled ventilation and assisted ventilation. Accordingly, we were not able to test the real-life performance of setting, such as inspiratory and expiratory triggers, which have been shown to be particularly critical in laboratory studies carried out in controlled bench conditions instead of real clinical routines.

As it is not unusual that old machines are present in small number of units in an ICU, we kept the composition of the set of devices in use in the ward, therefore including also ventilators that were present as only one or two units, as this is what a patient admitted to that unit will receive. As a consequence, several ventilator models were represented by only small numbers, making it impossible to apply statistical analysis on performances of these units.

Finally, given the numbers of ventilator and ICU involved, we are not able to identify which factors between ventilator model, maintenance frequency and procedure, use and type of humidifier and filters, and unplanned repairs/servicing of the devices, constitute the major source of the variability we observed.

Conclusion

It is difficult to predict the clinical impact of the ventilator errors we found in this study. Accurate ventilator performance is crucial to provide the patient with the best ventilatory strategy. When setting mechanical ventilation, clinicians should be aware that the target tidal volume might be delivered with errors greater than 10%, even if in a minority of the cases.