Intelligent Quality Management 2 with IntraSpect™ technology for quality control of GEM® Premier™ 5000 blood gas analyzers- A novel application of the patient sample as its own control.

Background: Quality control (QC) in point-of-care (POC) testing has been greatly improved by automatic control processes, such as the Intelligent Quality Management (iQM®) technology found in GEM Premier blood gas analyzers (Werfen, Bedford, MA). The 2nd generation technology, iQM2, provides additional capabilities, notably the incorporation of IntraSpect software that monitors the response curves of individual tests to detect transient errors caused by micro-clots, micro-bubbles or any event that disturbs the sensor response during sample data acquisition. IntraSpect is a novel form of patient-based, real-time quality control (PBRTQC).
Methods: IntraSpect pattern recognition software monitors the last 15 measurements of each patient-response curve. Control limits for slope coefficients have been established from theoretical models and empirical data. Abnormal measurement behavior is flagged to identify transient errors that invalidate test results. Results from 1,013,391 patient samples were collected on 4,985 GEM Premier 5000 cartridges and 2,765 instruments in clinical use worldwide. Results and conclusions: Total pre-analytic and transient errors detected by IntraSpect were 1.91% worldwide. iQM2 with IntraSpect technology provides a unique control function detecting transient errors that would otherwise go undetected with traditional QC. Together with the statistical QC technology in iQM2, pre-analytic, analytic, and transient analytic errors are detected much faster-seconds versus hours-than by traditional statistical QC.

Introduction

A recent survey of point-of-care (POC) clinicians asked participants to identify the “practices and wants” that would be most helpful for improving the quality of POC testing [1]. The responses prioritized the following activities: more manufacturer-integrated quality and function checks; more in-house training by in-house staff; more training from manufacturers; more supervisor oversight; and more explicit directions from manufacturers. The need for manufacturers to support quality assurance and training activities is clearly critical for achieving high quality POC testing.

The strategic direction for manufacturers should be to provide more integrated controls and a more complete system for monitoring all potential failure modes. These capabilities would reduce the need for operator training and supervision of operations. An example of such a quality system is Werfen's iQM2, the second-generation Intelligent Quality Management system, for the GEM Premier 5000 blood gas analyzer. Nichols et al. [2] recently reported an extensive validation study of iQM2. The methodology for validating detection time was described in the validation of the first generation iQM, approximately 15 years ago [3]. Notably, the average times for detecting errors for each test are much quicker as compared with traditional statistical quality control (SQC) practices. This improved performance has also been demonstrated in field studies in four different institutions, as reported by Toffaletti et al. [4], in two different institutions, as reported by Mion et al. [5], as well as the more recent study in four different institutions, as reported by Nichols et al. [2].

A major improvement in iQM2 is the IntraSpect technology that monitors the measurement response curve of each patient sample. In addition to the analysis of Process Control Solutions (PCSs) that infer the quality of the analytical system before and after sample process, IntraSpect directly detects potential problems for each patient sample within the measurement cycle. During the sample measurement process, IntraSpect collects a series of readings which are then inspected using pattern recognition software to identify unusual sensor behavior due to transient errors. Transient errors are events that happened within the sample measurement due to sample preparation and presentation (e.g., micro-clots, micro-bubbles, interferences). Such errors can lack a recognizable signal, before or after sampling, and are random in nature – making them typically undetectable by traditional QC. Micro-clots and micro-bubbles are particular problems for systems that utilize whole blood samples and are particularly difficult to monitor via traditional, liquid control methods [6].

IntraSpect is a novel form of patient based real-time QC [PBRTQC] with integrated QC for samples. PBRTQC encompasses all types of patient-based control techniques including delta checks for individual patients, limit checks for individual samples, cross-check algorithms, such as anion gap, as well as a wide variety of population statistics such as Average of Normals (AoN). The use of PBRTQC is recommended by the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) Committee on Analytical Quality [7]. This committee has stated that “PBRTQC will become the mainstay of QC in laboratories once the profession sees the advantages of this form of process control, and manufacturers and middleware vendors provide the onboard capability” [8]. In that context, iQM2 provides an example of a manufacturer's implementation of a PBRTQC technique, along with automation of traditional SQC, and extensive system function checks. The application and performance of IntraSpect technology is described in this report.

Methods and materials

2.1 The GEM Premier 5000 system (Werfen, Bedford, MA) is a critical care analyzer that utilizes whole blood to perform tests for blood pH, pCO2, pO2, Na+, K+, Cl−, Ca++, glucose, lactate, total bilirubin, hematocrit, total hemoglobin, oxy-hemoglobin, deoxy-hemoglobin, oxygen saturation, carboxy-hemoglobin, and met-hemoglobin. The main components of the GEM Premier 5000 system are a multi-use disposable cartridge (GEM PAK), iQM2 active process control technology, and GEMweb® Plus Custom Connectivity, an automated web-based information management system. The GEM PAK includes all the required analytical components (sensors, solutions, sampler, optical cell, waste bag). iQM2 provides continuous QC monitoring before, during, and after sample measurement, analyzes five PCSs to monitor all the critical decision concentrations, and provides automatic error detection and correction of the system, complete with documentation of corrective actions.

2.2 Automated SQC was one of the breakthroughs with iQM. Analytic errors could be detected very quickly, as described in an early validation study [3], due to the unique application of SQC. Electronic drift limits, imposed on the sensors for various PCSs, correspond to statistical control limits that can be translated as individual control rules. Error detection capabilities can then be determined from theoretical power curves that describe the probabilities of rejection, as a function of the size of errors occurring. The size of the systematic error that is medically important can be calculated from the allowable total error for a test (e.g., CLIA criteria for acceptable performance) and the precision observed for the methods, which also determine the Sigma-metric (SM) for the test. The SMs for the GEM Premier 5000 with iQM2 are documented in the recent validation report by Nichols et al. [2]. The critical systematic error (ΔSEcrit) is related to the Sigma-metric by the equation SM = ΔSEcrit + 1.65. Given an SM of 6, it follows that the medically important SE is 4.35 times the observed SD. The probability of detecting medically important errors (Ped, probability of error detection) follows the methodology described originally by Westgard et al. [3]. Those probabilities can then be converted into average run lengths (ARL) to characterize the expected time for detection of medically important errors (ARLed = 1/Ped). Given the frequency of PCS analysis, ARLed can then be used to estimate the average time for detection of medically important errors. Obviously, the best QC performance will be obtained with the PCS tested most frequently, which is PCS B, analyzed every 2 min. For iQM2, Nichols et al. [2] have characterized the average detection times (ADT) using this approach as 2 min for most analytes, except sodium and glucose which require five and 21 min, respectively.

2.3 The iQM2 Total Quality Process begins with the installation of the GEM PAK on a GEM Premier 5000 system, first validating performance using the five PCSs, then monitoring patient sample analysis, using specific quality checks. These include mechanical, electronic, and fluidic checks, temperature checks, and sensor/CO-Ox checks using the PCSs to monitor change, along with pattern recognition to identify errors and interferences. IntraSpect technology provides real-time monitoring of each patient sample during the measurement cycle to provide additional information about the quality of measurement during the analysis of each patient sample. The process for employing all these checks is illustrated in Fig. 1.

Fig. 1

Description of measurement cycle for GEM Premier 5000 operation and identification of various system checks, IntraSpect monitor, and sensor checks.

The figure summarizes iQM2 Quality Control processes for each sample - before, during, and after - each sample measurement process on the GEM Premier 5000 system. Before - system and sensor functionality checks; During – system fluidic checks in aspiration and InstraSpect; After-sensor pattern checks to ensure sensor recovery after sample washing step and ready for next sample. Corrective action is automatically triggered when a sensor recovery after sample is detected by the iQM2 pattern recognition.

2.4 IntraSpect technology utilizes 15 readings during the last 15 s of patient sample measurements, to monitor the response of the sensor output. Pattern recognition software analyzes the readings for sensor response prior to reporting the patient result. The sample response, in millivolts readings, is fitted to a regression model. The regression equation characterizes the observed sample output and is used to estimate the value of the quantity and to identify abnormal sensor response. The behavior of measurement response curves for normal patient samples was characterized and established from historical clinical sample data collected from GEM Premier systems in the field. Unusual sensor responses are identified by the response shape and coefficients outside of acceptable limits.

Fig. 2 illustrates the expected response of a potassium sensor. The three lower responses represent the stable signals usually observed during the measurement process. The top response curve provides an example of a sample where the signal was disturbed by a factor significantly affecting its behavior – in this case due to a fluidic blockage during the sample aspiration, potentially caused by a blood clot in the sample. The affected sample reported an elevated potassium of 6.8 mmol/L when a value of approximately 4.0 mmol/L was expected. The expected value is based on the initial sensor output response and a retrospective sample analysis run in that same system.

Fig. 2

IntraSpect example response curves for K+ sensor.

Example of a sample affected by a transient event and detected by iQM2. The sensor signal is clearly a different pattern and flagged by IntraSpect (blue), as compared to sample signals of unaffected samples (red, grey and orange). . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

As shown in Fig. 2, the coefficients that, in this case, represent the slope of the signal, are close to zero for normal response curves. Once the nature of the responses for different sensors was determined, a library of coefficients was developed from several thousand samples of known concentrations. This library was then used to optimize the limits for the IntraSpect coefficients for each sensor. During this process, samples were manipulated to cover the reported range of the system for each measured analyte, as well as several known clinical conditions and other known preanalytical conditions (e.g., microbubbles, blood clots, interfering substances). The coefficients were tested and optimized, using data collected in several clinical locations, during the last steps of system development.

Fig. 3 shows the IntraSpect limits for potassium, where most of the coefficient values (slope) are close to zero, and the limits show only minor changes with the analyte concentration. This illustrates the model of “constant” limits that also applies to sodium, chloride, and calcium, though with some empirical modifications, based on the particular analyte, as shown in Fig. 4, parts A, B, and C, respectively.

Fig. 3

Distribution of slope coefficients for K+ sensor along with empirical control limits.

Representation of IntraSpect coefficients values against the analyte concentration and the associated thresholds (dotted lines) for flagging transient errors. Samples within the dotted lines are acceptable and outside are flagged or suppressed.

Fig. 4

Distribution of slope coefficients for various tests monitored by IntraSpect. Part (A) shows IntraSpect limits for Na+; (B) Cl−; (C) Ca++; (D) glucose; (E) lactate; (F) pO2; (G) pH; and pCO2.

IntraSpect coefficients values against the sample concentration for all tested sensors. Figures A to C show examples of constant coefficients. Figures D to H show the sensors where coefficients value (or sensor response) are variable to the concentration of measured analyte.

Fig. 4, part D shows the IntraSpect limits for glucose, where the limits vary with the measured glucose concentration. This “proportional” model applies to sensors with a diffusion component and also applies to lactate (part E). The curvature of the signal is related to the sample concentration. In addition, for glucose and lactate, the signals of these sensors are expected to increase during the sampling process until reaching a plateau (a flat area of the curve) that occurs when an equilibrium between the analyte concentration and the diffusion characteristics of the outer membrane has been achieved (besides other factors). There is no (expected) scenario where the sensor should show a significant decrease in signal during the sampling process. In fact, that event would indicate clear misconduct of the sensor performance, with potential for significantly impacting the analytical result.

Part F of Fig. 4 illustrates the IntraSpect limits for pO2. As expected on a sensor governed by a diffusion process, the signal is evolving during the data collection time and, therefore, a curved-based equation is used. These limits are influenced by the effects of the concentration of the measured analyte, with respect to the concentration of the same analyte in the PCS before the sample was run (in the case of pO2, 180 mmHg on PCS B). Therefore, the signal generated by samples with lower pO2 than PCS B, will create a downward-trend curve generating a coefficient lower than zero, while the opposite is expected for samples with pO2 higher than PCS B. In addition, more extreme coefficient values are expected at more extreme concentrations, leading to non-linear limits, as shown in Fig. 4. For completeness, Fig. 4 also provides the IntraSpect limits for pH (part G) and pCO2 (part H).

Once the control limits were established for the different sensors, the performance of iQM2 with IntraSpect technology was assessed from data collected on 4,985 GEM PAKs and 2,765 systems in clinical use worldwide. The control limits were generally less than half of the CLIA-allowable total error for the individual test, and data outside these limits showed that the errors exceeded the CLIA limits in 70% of the cases, as demonstrated by Mansouri et al. [9]. This assessment considered the errors detected on 1,013,391 patient samples tested in multiple settings.

Results and discussion

Table 1 summarizes the categories of errors, the errors detected by iQM2, and the prevalence of those errors. Total pre-analytic and transient errors detected by IntraSpect were 1.91% worldwide. Of the total errors observed, approximately half were transient errors detected by IntraSpect (0.80%).

Table 1

Types of errors detected, corrected, and documented in 1,013,391 patient samples, analyzed with 4985 GEM PAKS by 2765 GEM Premier 5000 instruments.

Preanalytical category	Type of error detected by iQM2	Worldwide Error Detection Prevalence (%)
Improper mixing anticoagulant	Micro-clots	0.79
Inadequate sample preparation and/orpatient-specifictreatment	Benzalkonium chloride	0.09
	Thiopental	0.02
	CO-Ox interferences	0.21
Transient errors	IntraSpect	0.80
Total		1.91

These errors correlated strongly with the detection of micro-clots caused by improper mixing and/or improper anticoagulant. Interferences due to benzalkonium chloride, thiopental, and optical problems from turbidity were observed to be smaller, but still significant, given the 1,013,391 total patient samples in this study.

The prevalence of the errors for the different tests is shown in Table 2. Chloride was highest at 0.39%, followed by pCO2 at 0.15% and ionized calcium at 0.14%. Due to the numerous factors that could trigger an IntraSpect error detection, such as fluidic and chemical factors, sensor membrane physical properties, or sample properties, the cause of these differences in error prevalence among the sensors are difficult to distinguish.

Table 2

Distribution of transient errors by analyte as detected by IntraSpect.

	Distribution of errors among analytes
	pH	pCO2	pO2	Na+	K+	Cl−	Ca++	Glu	Lac
Results affected (%)	0.02	0.15	0.03	0.10	0.05	0.39	0.14	0.06	0.07

When IntraSpect detects an error for a given analyte, the result for that particular analyte only is suppressed by the system. In combination with IntraSpect during-sample-measurement error detection, the totality of iQM2 processes is able to detect errors where the sensor response was affected before, during or after the sample [2,4,6].

The idea of using patient samples for QC has a long history. Beginning with Hoffman and Waid in the 1960s [10], a method of QC using the “average of normals” has been recommended. In the 1970s, the application of patient data was extended to red cell indices and “Bull's algorithm” [11] which has been widely implemented, possibly representing the most successful application of patient-based QC. In the 1990s, Westgard et al. [12] described how Average of Normals (AoN) could be designed to complement traditional SQC and utilized to measure run length. In the 2000s, interest in patient data QC was kept alive by Parvin [13] and Kazmierczak [14], but laboratories seemed occupied by the trend towards using risk management to design quality control plans. Then, since 2015, a wide variety of studies have been published on the performance of patient-based real-time QC by Bietenbeck [15], Fleming and Katayev [16], Ng et al. [17], Bietenbeck et al. [18], Liu et al. [19], van Rossum and van den Broek [20], and van Rossum and Kemperman [22]. Many of these investigators make up the IFCC PBRTQC group, recommending increased use of patient data for QC.

The IFCC group accepts that there will be a different design and validation models for the various applications of PBRTQC. The particular patient-based technique employed in IntraSpect makes use of the measurement response curve of an individual patient sample, in contrast to the focus on population statistics in most PBRTQC applications. As such, the IntraSpect application functions as a simple SQC model, where control limits are established based on the variation observed in measuring normal samples. Control limits are applied to monitor the shape of the response curve for each individual sample measured. Validation of performance, therefore, follows the principles applied for traditional SQC procedures but also relies on empirical observations of field data. There typically are sets of trial or training data used to establish control limits to minimize false rejections and maximize error detection. In the case of PBRTQC procedures, it is particularly critical to observe the actual rejection rates in practice to optimize performance [21].

While pre-analytical and transient errors are observed in approximately 1.9% of patient samples globally, they are still very significant when the volume of blood gas testing is considered. It is estimated that approximately 430 million POC blood gas tests will be performed globally, corresponding to 8.17 million patient samples with pre-analytic and transient errors. While these errors would be detected by iQM2 with IntraSpect, they would likely go undetected by traditional QC procedures. Detection of that small percentage of pre-analytic and transient errors still represents a significant improvement in the quality of blood gas testing and demonstrates the usefulness of the new IntraSpect technology in iQM2.

Conclusions

Transient errors caused by micro-clots, air bubbles, interferences, or any other event capable of disturbing sensor response, can be detected by monitoring the response curves of each individual patient sample. Studies demonstrate that such transient errors occur in 0.5–1.0% of patient blood gas samples and are likely undetected when using liquid control solutions and traditional quality control (QC). As recommended by the International Federation of Clinical Chemistry and Laboratory Medicine, the addition of patient data QC procedures can complement statistical QC and improve detection of clinically important errors.

Declaration of competing interest

JOW is a Werfen consultant. JC is a Werfen employee.