Absolute and trend accuracy of a new regional oximeter in healthy volunteers during controlled hypoxia.

BACKGROUND: Traditional patient monitoring may not detect cerebral tissue hypoxia, and typical interventions may not improve tissue oxygenation. Therefore, monitoring cerebral tissue oxygen status with regional oximetry is being increasingly used by anesthesiologists and perfusionists during surgery. In this study, we evaluated absolute and trend accuracy of a new regional oximetry technology in healthy volunteers.
METHODS: A near-infrared spectroscopy sensor connected to a regional oximetry system (O3™, Masimo, Irvine, CA) was placed on the subject's forehead, to provide continuous measurement of regional oxygen saturation (rSO2). Reference blood samples were taken from the radial artery and internal jugular bulb vein, at baseline and after a series of increasingly hypoxic states induced by altering the inspired oxygen concentration while maintaining normocapnic arterial carbon dioxide pressure (PaCO2). Absolute and trend accuracy of the regional oximetry system was determined by comparing rSO2 against reference cerebral oxygen saturation (SavO2), that is calculated by combining arterial and venous saturations of oxygen in the blood samples.
RESULTS: Twenty-seven subjects were enrolled. Bias (test method mean error), standard deviation of error, standard error of the mean, and root mean square accuracy (ARMS) of rSO2 compared to SavO2 were 0.4%, 4.0%, 0.3%, and 4.0%, respectively. The limits of agreement were 8.4% (95% confidence interval, 7.6%-9.3%) to -7.6% (95% confidence interval, -8.4% to -6.7%). Trend accuracy analysis yielded a relative mean error of 0%, with a standard deviation of 2.1%, a standard error of 0.1%, and an ARMS of 2.1%. Multiple regression analysis showed that age and skin color did not affect the bias (all P > 0.1).
CONCLUSIONS: Masimo O3 regional oximetry provided absolute root-mean-squared error of 4% and relative root-mean-squared error of 2.1% in healthy volunteers undergoing controlled hypoxia.

Maintaining adequate patient tissue oxygenation is of critical importance particularly in operating room and in intensive care settings. However, standard monitoring methods of systemic arterial and venous oxygen saturation may not represent the oxygenation state in peripheral tissues such as the brain.[1,2] Brain tissue is particularly susceptible to hypoxia, and oxygenation of the cerebral tissue has been found to be an important predictor of short- and long-term clinical outcomes. Cerebral hypoxia could be linked to neurological complications,[3] organ dysfunctions,[4,5] stroke,[6] and increased hospital length of stay.[7] Therefore, maintenance of adequate cerebral oxygenation may increase patient safety by preventing decreased cerebral perfusion and prolonged cerebral tissue ischemia.

Near-infrared technology-based regional oximeters became commercially available more than decades ago. Unlike traditional pulse oximeters, regional oximeters measure a mix of arterial, capillary, and venous blood in cerebral and peripheral tissue and do not depend on pulsatile flow.[8-10] Cerebral tissue hemoglobin oxygenation is estimated by transcutaneous measurement of the amount of light absorbed by hemoglobin in the cerebral cortex. This is achieved with a sensor comprising a near-infrared light source and a near-field and a far-field light detector. The near-field light detected is subtracted from the far-field light detected to calculate tissue oxygenation.[10] Historically, tissue oximeters have been used to capture trends in regional oxygenation rather than absolute values because no reference method for the validation of the measurements had been established.[11,12] More recently, oxygen saturation of blood from the jugular bulb vein and the radial artery in a ratio of approximately 70% to 30%, respectively, has been established as an acceptable reference for calculation of cerebral tissue hemoglobin oxygen saturation.[13,14]

Regional oximetry is being increasingly used by anesthesiologists and perfusionists during surgery.[15] Although overall performance of cerebral oximeters has improved significantly since they were first introduced, a recent study that compared 5 commercially available brain oximeters found significant accuracy variability between and within these devices, indicating a need for further improvements of the technology.[16]

Here we report the results of a clinical study to evaluate absolute and trend accuracy of a new regional oximeter (O3TM, Masimo, Irvine, CA) in healthy adult volunteers.

METHODS

Twenty-seven healthy adult volunteers were enrolled into an IRB-approved, prospective, unblinded laboratory study to evaluate a novel Masimo regional oximetry system (O3) during controlled hypoxia.

Procedures

After receiving informed written consent, subject demographic information (age, gender, weight, height, ethnicity, and skin pigmentation as determined by the Massey scale)[17]; medical history, and baseline vital signs (arterial blood pressure, heart rate, and baseline oxygen saturation) were recorded. A cerebral oximetry sensor connected to a regional oximetry system board was placed on the left side of the subject’s forehead. The system uses near-infrared spectroscopy, interrogating tissue by transmitting light of 4 different wavelengths through the tissue and processing the received light waveforms, to provide continuous measurement of regional saturation of oxygen (rSo2).

The level of oxygen within a subject’s blood was reduced in a controlled manner by altering the inspired oxygen concentration (Fio2) to achieve arterial oxygen saturation plateaus between 100% and 70%. Arterial saturation of oxygen was continuously monitored (Spo2) by a pulse oximeter (Radical 7 with R2-25 optical finger sensor, Masimo) and validated via intermittent readings from blood-gas analysis (ABL 800, Radiometer, Copenhagen, Denmark) at each plateau. To alter the Fio2, the protocol used a gas delivery system (Narkomed 6000, North American Drager anesthesia machine, Telford, PA), consisting of oxygen and nitrogen tanks, gas blender, and a small adult mouthpiece (VacuMed, Ventura, CA) placed in the subject’s mouth with lips tightly closed around it in a snorkel manner. Tight seal around the mouthpiece and a nose clip ensured that the atmospheric gases did not mix in the breathing circuit. A control computer continuously displayed end-tidal oxygen (eto2) and end-tidal carbon dioxide (etco2) pressure to perform controlled hypoxia while maintaining normocarbic conditions. At least 7 eto2 plateaus were targeted, providing stable Spo2 levels for at least 3 minutes at each step, followed by a final plateau of supplemental oxygen (Fio2 ≥50%) and return to room air.

Reference blood samples were taken from the radial artery and a jugular bulb vein catheter at baseline and at each plateau as well as room air to determine arterial oxygen saturation (Sao2), jugular bulb venous oxygen saturation (Sjvo2), and partial pressure of CO2 in the arterial blood (Paco2). A 24-gauge inner dilator (Cook Medical, Bloomington, IN, Micropuncture Introducer Set, MPIS-401-SST G-47940) was used for the cannulation and blood sampling from the internal jugular vein. In all subjects, the cannulation was guided by ultrasound visualization via Phillips Sonos 5500 echocardiography machine and a 8 MHz transthoracic probe, which ensured that the catheter was placed distally well past the entrance site of the facial vein into the internal jugular vein. (The roof of the jugular bulb can actually be felt with the small 24-guage inner dilator.) The blood sample from the jugular vein was aspirated with a 3-ml syringe at a slow rate, helped by the catheter’s diameter and length (10 cm), to avoid any mixing from the facial venous blood.

Electrocardiogram, heart rate, arterial blood pressure, Spo2, etco2, and eto2 were monitored throughout the study. To ensure normocapnic conditions, for example, the subject was not hyperventilating, Paco2 baseline was required to be ≥ 30 mmHg, and during the procedure the etco2 was maintained within ±4 mmHg of each subject’s baseline. The reference cerebral oxygen saturation (Savo2) was computed as Savo2 = 0.3 Sao2 + 0.7 Sjvo2, based on arterial and venous components in the cerebral blood volume.[14]

Data Analysis

Data were captured using a laptop computer running Automatic Data Collection software (ADC, Masimo, Irvine, CA) connected to the O3 system. Data analysis was performed using Matlab 2013 (The Mathworks, Natick, MA), R-Project, and MedCalc 12 (Medcalc, Ostend, Belgium) software. A priori power analysis was performed to estimate the required study population for an estimated test device performance.

Determination of absolute accuracy was based on the following statistics of the measurement error. Bias (defined as mean error; error = rSo2 − Savo2), standard deviation (SD) of error, and limits of agreement (bias ± 1.96 SD) were calculated using Bland and Altman analysis for repeated measures.[18] Standard error (SE) of bias and root mean square accuracy (ARMS = square root of the mean − squared error) were also computed. Mixed effects regression analysis accounting for multiple measures per subject and assuming independent covariance structure was performed to obtain a regression equation (slope, intercept, and SE of estimate) between rSo2 and Savo2.

Trend accuracy was determined in 2 ways. The first method is by calculating the root mean square of the relative error (individual sample error − mean subject error) as described previously.[10] Second, for an assessment of how rSo2 tracked changes in Savo2, sample-to-sample changes of rSo2 and Savo2 (denoted by ΔrSo2 and ΔSavo2, respectively) were analyzed by a mixed effect regression assuming independent covariance structure.

For both regression analyses (between rSo2 and Savo2, as well as between ΔrSo2 and ΔSavo2) the coefficient of determination (R2) was calculated as follows: The point estimates were derived from the mixed effects regression coefficients which account for repeated measures. To also account for repeated measures effect in the total residual error, subject-by-subject residual error was weighted by the number of observations in each subject. R2 was calculated using sum of squares of the resulting weighted residual error.

Finally, multiple linear regression analysis was performed to determine whether subject demographics or skin pigmentation affected device error (bias).

RESULTS

Twenty-seven subjects were enrolled into this prospective, unblinded clinical study. Four subjects were excluded due to early termination of the procedure, arterial CO2 pressure decreasing below the 30 mmHg threshold (or hyperventilation resulting in a wide range of CO2 level during desaturation), or improper proximal positioning of the internal jugular bulb catheter, leaving 23 subjects (age range 18–35 years, median 23 years, 12 males, and 11 females) included. Seventeen subjects (74%) had light skin pigmentation (Massey scale 1–5) and 6 (26%) dark pigmentation (Massey scale 6–10) (Table 1).

Table 1.

Summary Table of Subject’s Demographics

A priori power analysis allowing an alpha error of 5% and statistical power of 80% revealed that 22 subjects needed to be enrolled to detect an average per subject bias of 5% with an SD of 8% from the reference measurement, a range detected by other commercially released cerebral oximeters[10] and accepted by the Food and Drug Administration for regulatory findings.

Two hundred-ten reference blood samples were collected. Eight blood samples were excluded because hypoxia plateaus were not stable (Sao2 values differed by more than 3% within 1 plateau), leaving 202 separate Savo2 and rSo2 comparison datasets included in the analysis. Savo2 values ranged from 47% to 87% (Fig. 1).

Figure 1.

Distribution of oxygen saturation measurements (in percent) of all reference samples (Savo2) as measured by a blood gas analyzer.

Absolute accuracy analysis yielded a bias of 0.4%, SD of 4.0%, SE of 0.3%, and ARMS of 4.0%. The Bland–Altman limits of agreement were 8.4% (7.6%–9.3%) to −7.6% (−8.4% to −6.7%) (as shown in Fig. 2). Mixed effect regression analysis resulted in the equation rSo2 = 8.1 + 0.88 × Savo2 (95% confidence intervals [CIs] for the 2 coefficients were 5.2 to 11.1, and 0.84 to 0.92, respectively), and the SE of the regression (S) was 2.1% (Fig. 3). The coefficient of determination (R2) was 0.75 (95% CI, 0.69–0.81).

Figure 2.

Bland and Altman plot for determination of bias (test method error) and limits of agreement of both methods. Two hundred-two samples are plotted, along with lines for the bias (red line) and the upper and lower limits of agreement (bias ± 1.96 SD) (dashed black lines).

Figure 3.

Mixed effect regression analysis of rSo2 versus Savo2: scatter plot for 202 paired samples and the trend line representing the regression equation (black). Also shown are the identity line (45-degree line) (red) and the identity line ± ARMS (blue).

Trend accuracy analysis resulted in an ARMS of 2.1% with a mean of 0%, an SD of 2.1%, and an SE of 0.1% for the relative error. Comparison of directional sample to sample changes of both rSo2 measurements (ΔrSo2) and the reference Savo2 (ΔSavo2) by mixed effects regression analysis to account for multiple measures per subject resulted in the equation ΔrSo2 = 0.08 (−0.3 to 0.44) + 0.83 (0.8 to 0.87) × ΔSavo2 (Fig. 4). The coefficient of determination (R2) was calculated to be 0.90 (0.87–0.92).

Figure 4.

Mixed effect regression analysis for an assessment of trending performance of rSo2: scatter plot of 179 paired measurements for sample-to-sample changes, ΔSavo2, and ΔrSo2. Also shown are the line of identity (red) and the trend line representing the regression equation (black).

Multiple linear regression analysis for detection of confounding factors affecting device error (bias) revealed that age (P = 0.57) or skin color (P = 0.1) does not affect the device error (bias); however, there was a very weak correlation between gender and device error (P = 0.01, R2 = 0.03).

DISCUSSION

There is a strong clinical need for accurate tracking of rSo2 in peripheral tissues, particularly brain tissue during the perioperative phase to avoid ischemia.[19] A recent study compared the performance of 5 different commercially available regional oximeters in normal volunteers during hypoxemia. This evaluation demonstrated that currently available regional oximeters often perform with limited absolute accuracy, manifesting in an average ARMS of 9.1% for all 5 devices with a range of 4.28% to 9.68%.[10] We believe these findings demonstrate that enhancements to regional oximetry are desirable and would improve clinical confidence and clinical management with regional oximetry.

In this paper, we report the results of a prospective clinical study evaluating absolute and trend accuracy of a new regional oximeter (Masimo O3) for continuous, noninvasive measurement of rSo2. The Masimo O3 system has recently received the CE mark for the European Union and is currently pending Food and Drug Administration approval. Our findings revealed an ARMS for absolute accuracy of 4% and for trend accuracy of 2.1%. The rSo2 readings from the O3 correlated closely with the reference Savo2 measurements as shown by a correlation coefficient of 0.87. Trend accuracy calculation indicates that when individual bias in a subject is removed, the SD of the relative error is 2.1%. Figure 4 shows a comparison of directional changes between both measurements, with a correlation coefficient of 0.95, showing that Masimo O3 measurements also follow directional changes in the reference Savo2. These statistics indicate that Masimo O3 has an absolute root-mean-squared error of 4% and captures trends in cerebral oxygen saturation with relative root-mean-squared error of 2.1%.

We believe that proper methodology is crucial for successful development of a regional oximeter with good absolute and trending accuracy. To be able to calibrate the device accurately, it is very important to achieve stable plateaus lasting long enough to reach stable arterial saturation even with changing respiration rates.[10] Furthermore, placement of the jugular catheter and blood draws from it require care to avoid sampling extracranial venous blood, for example, from the superficial facial veins.

Reference cerebral tissue oxygen saturation is commonly assumed as a weighted sum of the arterial (A) and venous (V) oxygen saturations, which is kept constant (e.g., our analysis used the A/V ratio of 70/30). However, A/V ratio is not necessarily constant because both the cerebral blood volume as well as oxy- and de-oxy hemoglobin concentrations change in response to other hemodynamic variations. For example, Bickler et al.[16] reported that patients had different ratios of venous and arterial blood in the sensor field. Similarly, isocapnic hypoxia increases cerebral arterial blood flow and restricts venous outflow, with a concomitant increase in arterial blood volume relative to venous blood volume in the frontal cortex.[20] Such conditions and other factors altering arterial and venous ratio indicate that the reference oxygen saturation has an estimation error, which affects both calibration of the device as well as validation accuracy. Limitations of our study include the population of relatively young, healthy, adult volunteers, which may not reflect the performance in critically ill perioperative patients.

CONCLUSIONS

Valid accuracy studies of regional oximeters require precise data collection, blood sampling, and laboratory analysis methods. With the methods we used in this study, Masimo O3 regional oximetry provides an absolute root-mean-squared error of 4% and relative root-mean-squared error of 2.1% in healthy volunteers undergoing controlled hypoxia.

DISCLOSURES

Name: Daniel Redford, MD.

Contribution: This author helped design and conduct the study, and prepare the manuscript.

Attestation: Daniel Redford approved the final manuscript and attests the integrity of the original data and the analysis reported in this manuscript.

Conflicts of Interest: This author declares no conflicts of interest.

Name: Samata Paidy, MD.

Contribution: This author helped conduct the study, and review the manuscript.

Attestation: Samata Paidy approved the final manuscript and attests the integrity of the original data and the analysis reported in this manuscript.

Conflicts of Interest: This author declares no conflicts of interest.

Name: Faisal Kashif, PhD.

Contribution: This author helped design the study, collect and analyze the data, and prepare and review the manuscript.

Attestation: Faisal Kashif approved the final manuscript and attests the integrity of the original data and the analysis reported in this manuscript.

Conflicts of Interest: Faisal Kashif works in the Research and Development section at Masimo Corporation.

This manuscript was handled by: Maxime Cannesson, MD, PhD.