Portable, consumer-grade pulse oximeters are accurate for home and medical use: Implications for use in the COVID-19 pandemic and other resource-limited environments.

OBJECTIVE: To determine the correlation between 3 lightweight portable pulse oximeter devices compared to a standard wall mount pulse oximetry device.
METHODS: We performed a single-center, prospective, observational study of 4 pulse oximetry devices, 3 of which are commercially available to the public. A convenience sample of 200 emergency department (ED) patients with chief complaints of cardiopulmonary origin or a peripheral capillary oxygen saturation ≤ 94 percent were enrolled. Analysis of variance was performed to compare SpO2s and test characteristics of the 3 devices compared to control.
RESULTS: Although differences in measured SpO2s were observed (P < 0.001) across groups, the differences were small (mean differences ranged from 1.00% to 1.87%). The correlation between test devices and the control were high (r range 0.70-0.79). Although the test characteristics were not perfect, the devices did have good sensitivity using a cutoff value of 94% (sensitivity ranging from 90% to 92%), which improved with lower SpO2 cutoff values to 92% (sensitivity ranging from 96% to 97%).
CONCLUSION: The 3 commercially available devices were accurate enough to be clinically useful when compared to a hospital bedside monitor pulse oximeter. Consumer-grade portable pulse oximeters may be useful if overwhelming numbers of patients require oxygen saturation monitoring, such as during the COVID-19 pandemic.

INTRODUCTION

Background

Pulse oximetry (pulse ox) is an expedient and accurate tool to measure noninvasively the oxygenation status of any patient in whom this might be a clinical concern. It was developed during World War II by Glenn Allan Millikan, an American physiologist and mountaineer. The term “oximetry” is attributable to him. Pulse oximeters measure peripheral capillary oxygen saturation (SpO2) by measuring the difference in absorption of oxygenated versus deoxygenated blood at 2 different wavelengths (typically red light at 660 nm and infrared light at 940 nm). Oxygenated hemoglobin absorbs more infrared light, and deoxygenated hemoglobin absorbs more red light. This difference is measured by the diodes on the device and is used to calculate the SpO2. This tool can be placed on multiple spots on the body to obtain noninvasively an accurate measure of blood oxygenation and easily detect hypoxemia. , , , A finger probe is commonly used for measurements, but multiple other sites can be used with varying accuracy. , The use of pulse oximetry has been shown to reduce the need for more invasive measurements, such as an arterial blood gas. More recently, portable finger probes have been developed, and these allow oxygen saturation measurements to be obtained in a variety of environments and situations. , Small, portable pulse ox devices allow measurement of oxygen saturation in resource‐limited environments.

Importance

The consensus in the literature is that portable pulse oximeters are acceptable for use but may experience inaccuracies as a patient becomes increasingly hypoxemic. , , , , Pulse oximetry has several limitations that can cause inaccuracies in measurement: light‐emitting diode (LED) integrity, motion, hypotension, methemoglobinemia, carboxyhemoglobiemia, and anemia. , , A variety of portable pulse oximetry devices exist and range from inexpensive, commercially available to more expensive, health‐care‐oriented models (ranging from ∼$25 to over $300). These devices may also be useful in austere conditions or when the supply of medical‐grade pulse oximeters is exhausted. Determining the accuracy of affordable, consumer pulse oximetry devices could allow for a cost‐effective addition to the wilderness medicine field kit for both the professional and layperson. We found in our previous study that a variety of iPhone apps that purported to measure pulse oximetry were inaccurate and should not be used—even in austere conditions. Thus a true pulse oximeter is preferred.

The Bottom Line

The accuracy of consumer and medical‐grade home pulse oximetry devices has not been well studied. This study of 200 hypoxic emergency department patients demonstrated that three home pulse oximeters were accurate when compared to a standard emergency department monitor.

Goals of this investigation

This study assessed the accuracy of 2 consumer grade and 1 medical‐grade portable pulse oximetry devices as compared to a standard‐of‐care, wall‐mounted pulse oximetry device. We postulated that the portable devices would be equivalent and reliably measure oxygen saturation. We also postulated that these devices would provide accurate measurements in patients with some degree of hypoxia (ie, SpO2 ≤ 94%).

METHODS

Study design and setting

We performed a single‐center, open‐label, prospective observational study comparing the results of bedside pulse‐oximetry across 4 devices using a convenience sample. Two consumer‐grade and 1 medical‐grade portable pulse oximetry devices were compared to a standard emergency department non‐portable bedside unit (Figure 1). The 2 consumer‐grade models were Santa Medical SM‐165 (SM) and Walgreen's OxyWatch C20 (OxyWatch). The medical‐grade unit was a Nonin Onyx II 9550 (Onyx). The wall‐mounted hospital control unit used was an ED TRAM 451 pulse oximeter (General Electrics, New York City, NY). All portable units used in testing were obtained by University of Alabama at Birmingham (UAB) emergency and wilderness medicine faculty.

FIGURE 1

The 3 portable pulse oximeters used in this study. From left to right: Santa Medical SM‐165 (SM), Walgreens’ OxyWatch C20 (OxyWatch), and Nonin Onyx II 9550 (Onyx)

Selection of participants

The study population was a convenience sample of 200 adults presenting to the UAB ED with either hypoxia (SpO2 ≤ 94%), an acute exacerbation of chronic obstructive pulmonary disease or a chief complaint of chest pain or dyspnea. Screening was performed via the ED electronic medical record. Exclusion criteria was the presence of peripheral artery disease, anemia, or inability to give consent. No compensation was offered. Authors obtained informed consent and performed enrollment. The authors recording and performing the documentation were not blinded to the devices being used. Data were collected from July 2018 to January 2019.

Measurements

We recorded the demographic data, SpO2 measurements, the use of supplemental O2, and the reason for inclusion for each participant. Then SpO2 measurements were taken consecutively from the same finger using the 3 experimental pulse oximetry devices adjacent to the control unit. The control measurement was obtained with the bedside TRAM 451 pulse oximeter. The 3 portable device measurements (ie, Onyx, OxyWatch, and SM) (Figure 1) were obtained sequentially in a predetermined random order without delay between readings. A single oximeter reading was recorded as soon as each device was applied and showed a good waveform.

Statistical analysis

Descriptive statistics were performed. Counts with percentages, means with SDs and medians with interquartile ranges were calculated and presented, as appropriate. Differences between control and test groups SpO2 were calculated and presented with 95% confidence intervals (CIs). Mean SpO2s were compared across groups using an analysis of variance test. Bland‐Altman plots were created to evaluate agreement between the 3 tested devices and control. Pearson's correlation was performed between each test group and the control. Subgroup analysis of hypoxic patients was performed and planned a priori. A cutoff value of ≤ 94% was chosen to define hypoxia a priori. Sensitivities, specificities, positive predictive values, and negative predictive values were calculated. Sensitivity analysis using an SpO2 of 92% as a cutoff value to hypoxia was performed post hoc and is provided as a supplemental table. McNemar's test was performed to evaluate the diagnostic characteristics for sensitivity of hypoxia. The sample size was chosen based on a convenience sample of patients and authors' availability to perform enrollment. All analysis was performed using SAS software, version 9.4 (SAS Institute). This study was approved by the UAB Institutional Review Board.

RESULTS

A total of 200 patients were evaluated. The characteristics of the enrolled patients are shown in Table 1. The group was evenly distributed between white and African American patients and sex. The majority were not receiving supplemental O2 at enrollment. The distribution of patient complaints used to include patients is also shown. Note that the total is >100% because many patients had more than one complaint, for example, chest pain and shortness of breath. The distribution of SpO2 is depicted in Figure 2. The mean SpO2 varied across devices, both in the entire population as well as those determined to be hypoxic based on a control SpO2 value of <94%, is shown in Tables 2 and 3. Correlation between test groups and control SpO2 was high (Table 2). The degree of correlation is also illustrated for all the devices by the Bland‐Altman plots (Figures 3–5).

TABLE 1

Baseline population characteristics (N = 200)

Characteristic	Median (IQR; Range) or N (%)
Age in Years
Median (IQR; Range)	58 (51–69; 21–97)
Race N (%)
White	96 (48.5%)
African American	101 (50.5%)
Other	2 (1%)
Missing	1 (0.5%)
Sex N (%)
Male	108 (54%)
Female	92 (46%)
Receiving supplemental oxygen N (%)
Yes	68 (34%)
No	129 (64.5%)
Missing	3 (1.5%)
Receiving CPAP N (%)
Yes	4 (2%)
No	196 (98%)
Ventilator N (%)
Yes	0 (0%)
No	200 (100%)
Oxygen flow rate (LPM)
Median (IQR)	0 (0–2.25)
Oxygen saturation below 94% N (%)
Yes	60 (30%)
No	140 (70%)
Inclusion criteria
Shortness of breath	131(66%)
Chest pain	78 (39%)
Exac COPD	27(14%)
SpO2 ≤ 94%	17(9%)

CPAP, continuous positive airway pressure; Exac COPD, exacerbated chronic obstructive pulmonary disease; IQR, interquartile range.

FIGURE 2

SpO2 measurements obtained by each respective device are shown here. Each data point on each device represents the oxygen reading for one patient. The oxygen percentage data shown includes outliers as well as mean lines and markers. SM, Santa Medical SM‐165; OxyWatch, Walgreens’ OxyWatch C20; Onxy, Nonin Onyx II 9550

TABLE 2

Distribution of oxygen saturation and measured difference by instrument

	Measurement Instrument
Measure	Control	OxyWatch	SM	Onyx
Total patients (N)	200	198	200	200
Oxygen saturation (%)
Mean (SD)*	96.08% (3.84)	94.23% (4.70)	95.02% (4.49)	95.08% (3.90)
Min, Max SpO2	59%, 100%	59%, 100%	62%, 100%	68%, 100%
Pearson Correlation Coefficient (P value)	Ref	0.79 (< 0.001)	0.76 (< 0.001)	0.79 (0.001)
Difference in SpO2 (Control‐Measured; %)
Mean (95% CI)	Ref	1.87% (1.46–2.28)	1.07% (0.65–1.48)	1.00% (0.65–1.35)
Min, Max	Ref	−5%, 19%	−6%, 23%	−9%, 15%

CI, confidence interval; SM, Santa Medical.

*Analysis of variance: P = 0 < 0.001; F value = 15.85.

TABLE 3

Distribution of oxygen saturation and measured differences by instrument in patients with control SpO2 ≤ 94% (N = 60)

	Measurement Instrument
Measure	Control	OxyWatch	SM	Onyx
Measured SpO2
Mean (SD) a	92.15% (4.57)	90.25% (5.76)	91.37% (5.10)	91.60% (4.68)
Min, Max SpO2	59%, 94%	59%, 99%	62%, 98%	68%, 100%
Pearson's Correlation Coefficient (P value)	Ref	0.75 (0.001)	0.79 (0.001)	0.70 (0.001)
Difference in measured SpO2 (Control‐Measured)
Mean (95% CI)	Ref	1.95% (0.96–2.94)	0.78% (−0.04–1.60)	0.55% (−0.37–1.47)
Min, Max	Ref	−5%, 13%	−6%, 10%	−9%, 15%

CI, confidence interval; SM, Santa Medical.

Analysis of variance: P < 0.0001, F value = 14.49.

FIGURE 3

Bland‐Altman plots for Walgreens’ OxyWatch C20

FIGURE 5

Bland‐Altman plots for Nonin Onyx II 9550

Bland‐Altman plots for Santa Medical SM‐165

The more expensive, medical‐grade Onyx device showed a narrower distribution of SpO2 readings that more closely mirrored the control values (Figure 2). The Bland‐Altman plots also show this visually with fewer outliers in SpO2 readings for Onyx (Figures 3–5). The mean difference between the device readings and controls for the entire population was narrow–between 1% and 1.87% for all 3 devices (Table 2). Note that there were 2 missing measurements in the OxyWatch group (N = 198). There were several outliers on individual patients, with SpO2 readings varying from control as much as 23%. In the subgroup of patients with hypoxia (SpO2 ≤94%), the distribution of measurements is also narrow. The mean difference from control readings were 1.95% for OxyWatch, 0.78% for SM, and 0.55% for the Onxy device (Table 3). Although, there were statistical differences across groups, the clinical difference was less than 2% SpO2.

To evaluate the ability of each device to detect hypoxia, we classified the test results as either no hypoxia or hypoxia (SpO2 ≤94). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of each device are shown in Table 4. All 3 devices had sensitivities over 90%. The negative predictive value of all 3 devices was ≥80%. It should be noted that there were often small ranges of device readings that affected this grouping. That is, a control reading of 94% (defined as hypoxia) and a device reading of 95% (defined as no hypoxia) would be considered a false negative. We also performed a sensitivity analysis evaluating 92% as a cutoff SpO2 for hypoxia (Table S1). Although agreement was lower in all 3 test groups with the lower cutoff values, sensitivity for hypoxia improved to 96%–97%.

TABLE 4

Comparison of test devices to standard bedside oximeter for detecting hypoxia (SpO2 ≤94%)

	Measurement Instrument
	OxyWatch	SM	Onyx
Hypoxia (≤94%)
Sensitivity (95% CI)	92.17% (87.27–97.08)	90.70% (85.69–95.71)	92.06% (87.34–96.78)
Specificity (95% CI)	60.00% (49.59–70.41)	67.61% (56.72–78.49)	67.57% (56.90–78.23)
Positive predictive value (95% CI)	75.71% (68.61–82.82)	83.57% (77.43–89.71)	82.86% (76.61–89.10)
Negative predictive value (95% CI)	85.00% (75.97–94.03)	80.00% (69.88–90.12)	83.33% (73.90–92.76)
McNemar's Test (Sensitivity)‐P (Chi2‐DOF)	0.001 (14.53–1)	0.063 (3.46–1)	0.0165 (5.76–1)

Chi2, chi square; CI, confidence interval; DOF, degree of freedom; SM, Santa Medical.

Lastly, we calculated correlation coefficients for all 3 devices. These r values showed strong correlation with control readings. The data are shown in Tables 3–4. The correlation of the medical‐grade Onyx was the highest at 0.79, but it did not statistically differ from the other 2 devices. In the subgroup of patients with hypoxia, the SM device had the highest correlation coefficient at 0.789. The ranges were from 0.756–0.790 for all patient readings and 0.703–0.789 for the subgroup of patients defined as hypoxic.

LIMITATIONS

Although the devices were tested on 200 patients, our sample was one of convenience. These were not sequential patients. We performed the study when the authors were available to spend time in the ED. The choice of a cutoff value of 94% for SpO2 is somewhat arbitrary, though consistent with our previous study and the usual definition of normal SpO2 being between 95% and 100%. We also calculated sensitivity analysis using a cutoff of 92% and provided this as a supplemental table. This post hoc analysis may better reflect the performance of these devices for home monitoring of coronavirus disease 2019 (COVID‐19) patients where a cutoff of 92% has been used. The choice to allow our enrolling authors to screen patients with lower oxygenation was done to include a larger subset of these patients in order to test the device characteristics in this subset of patients. Hence, this was not an outcome variable but simply a cutoff level to allow subset analysis. Based on the limited focus of our data acquisition, it is difficult to determine the ultimate external validity of the study. There could have been moment‐to‐moment differences in pulse ox readings as the devices were placed sequentially on the patient's finger. To minimize this effect, we randomized the sequence of the device measurements. Nonetheless, an active treatment that was being instituted by the medical team caring for the patient might have improved the patient's saturation over the minute or 2 that it took to take 3 sequential measurements. We purposely included a significant subset of hypoxic patients, but these readings were taken under a controlled environment and by a trained observer, not under austere conditions or by a lay person. It is also possible that the utility of this device in austere conditions may be limited based on external factors (eg, excessive heat or cold). We did not explicitly exclude patients with hypotension, although patients who were critically ill did not represent a large subset of patients because they are often unable to provide consent, and screening/enrolling may have interfered with clinical care. Finally, this study was not blinded. The authors recording the data applied the devices in random order to the patient's finger and the data were collected by 2 separate authors (BM, JG). This minimizes but does not exclude observer bias.

DISCUSSION

Portable pulse oximeters bridge the gap between the hospital and out‐of‐hospital environment by providing a flexible and lightweight tool to both patient and health care practitioners. The oximeters tested provided an evaluation at 2 tiers of price. The SM and OxyWatch were the more affordable and commercially available devices. The medical‐grade Onyx device was the most expensive. Our study shows that although there are slight differences among the 3 devices tested, all performed relatively well compared to control in both patients with normal oxygen saturations and those with lower saturations. One third of our patient population had saturations at or below 94%. Our entire patient population were presenting to an ED with complaints that would make oxygen saturation an important vital sign. Unlike prior studies that relied on healthy volunteers, we analyzed these devices in a more clinically relevant population: ED patients at risk for hypoxia.

All three devices showed a narrow mean difference from control readings and had high sensitivity, for detecting hypoxia (SpO2 ≤ 94%). Caution must be used as based on the sensitivity alone as many as 10% (or 15% if the lowest 95% CI is used) of patients with hypoxia might be missed. However, using a lower and more likely clinically relevant cut‐off of 92% saturation, the sensitivity of all 3 devices was even higher—all 3 essentially 97% (99.6‐94.2). This is reassuring because these portable devices would likely be used as screening tools in the field where supplies may be more limited or as we postulate a tool to detect hypoxia in patients with possible COVID‐19.

We show that the 3 portable oximeters tested demonstrated a strong correlation to control for SpO2 readings.

All 3 devices had rare outliers of mean differences from control being >5%. These occurred less often with the Onyx device and had the lowest mean difference from control of 1%. However, whether this possible advantage outweighs its almost 10‐fold cost difference is questionable.

We undertook this study after our previous work showed that iPhone applications proved to be inaccurate. We were looking for tools for the clinician that were small, portable, and accurate and could be easily packed. An accurate device to measure oxygen saturation could be helpful as a fifth vital sign in an austere environment to assess a patient or victim having problems with respiration because of trauma, infection, or altitude.

We did not envision our results having applicability to a pandemic of a novel coronavirus (severe acute respiratory syndrome coronavirus 2) that has infected more than 30 million persons and killed nearly 1 million people worldwide as of this writing. The rapidity of spread of the virus and its lethality have overwhelmed healthcare systems worldwide. , , , This has resulted in shortages of medical materials like ventilators for patients in respiratory failure and personal protective equipment (masks, gloves, gowns) to protect health care practitioners. In its more severe form, the disease affects the lungs by causing viral pneumonia, hypoxia, respiratory failure, and death. The ability to measure oxygen saturation in hundreds or thousands of patients is an important determinant to guide such clinical decisions as the need for supplemental oxygen and, ultimately, mechanical ventilation.

Hospital systems have been overrun, and many areas have been or are expected to have more seriously ill patients than they can care for under usual practice. Many health systems have used atypical locations to care for these patients. China built several new hospital facilities within days. Hospitals across the United States have erected temporary spaces in ball fields or have used other spaces, like hotels, to care for the surge of patients. The ability to monitor patients in makeshift facilities is and will be important. Pulse oximetry plays a key role. Some patients have been given a small, portable pulse oximeter, like the ones we studied, to take home to monitor themselves so that they know when to seek a higher level of care. , ,

Although we did not study the entire spectrum of portable pulse oximeters, the devices selected were sensitive in detecting hypoxia using a 92% cutoff and had a strong correlation with standard ED oximetry in an ED population at risk for hypoxia. Further, the less‐expensive consumer grade models compare favorably to a costlier, medical‐grade device. This makes the inclusion of a pulse ox at home, in a medical bag, or a non‐traditional health care location, doable for most people.

Overall, there were statistically significant differences between the 3 devices tested; however, the differences were minor and unlikely to be clinically significant. All had sufficient degrees of correlation with the control device, >90% sensitivity, and greater than 80% NPV for hypoxia. Although these devices were not tested in an austere environment or during a global pandemic, we expect that they would perform well under such conditions. Our study provides evidence that these devices can accurately detect hypoxia and may be a useful tool for health care practitioners or patients during this pandemic. Futures studies should analyze whether these findings are applicable to a wider range of ED patients with features not present in our population (hypotension, mechanical ventilation, anemia). Further study in specific environments, such as extreme cold or altitude, might be needed to further elucidate their utility in austere environments. A study of these consumer grade devices in patients suffering with COVID‐19 infection would also be useful.

CONFLICTS OF INTEREST

None

AUTHOR CONTRIBUTIONS

Each author takes public responsibility for appropriate portions of the content and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.All authors made substantial contributions to conception and design, acquisition of data, and analysis and interpretation of data.All authors were involved in drafting the manuscript or revising it critically for important intellectual content. WS gave final approval of the version to be published.

Supporting Information

Click here for additional data file.