Publications

Background

Fingertip pulse oximeters are widely available, inexpensive, and commonly used to make clinical decisions in many settings. Device performance is largely unregulated and poorly characterised, especially in people with dark skin pigmentation.

Methods

Eleven popular fingertip pulse oximeters were evaluated using the US Food and Drug Administration (FDA) Guidance (2013) and International Organization for Standardization Standards (ISO, 2017) in 34 healthy humans with diverse skin pigmentation utilising a controlled desaturation study with arterial oxygen saturation (SaO 2) plateaus between 70% and 100%. Skin pigmentation was assessed subjectively using a perceived Fitzpatrick Scale (pFP) and objectively using the individual typology angle (ITA) via spectrophotometry at nine anatomical sites.

Findings

Five of 11 devices had a root mean square error (ARMS) > 3%, falling outside the acceptable FDA performance range. Nine devices demonstrated worse performance in participants in the darkest skin pigmentation category compared with those in the lightest category. A commonly used subjective skin colour scale frequently miscategorised participants as being darkly pigmented when compared to objective quantification of skin pigment by ITA.

Interpretation

Fingertip pulse oximeters have variable performance, frequently not meeting regulatory requirements for clinical use, and occasionally contradicting claims made by manufacturers. Most devices showed a trend toward worse performance in participants with darker skin pigment. Regulatory standards do not adequately account for the impact of skin pigmentation on device performance. We recommend that the pFP and other non-standardised subjective skin colour scales should no longer be used for defining diversity of skin pigmentation. Reliable methods for characterising skin pigmentation to improve diversity and equitable performance of pulse oximeters are needed.

Funding

This study was conducted as part of the Open Oximetry Project funded by the Gordon and Betty Moore Foundation, Patrick J McGovern Foundation, and Robert Wood Johnson Foundation. The UCSF Hypoxia Research Laboratory receives funding from multiple industry sponsors to test the sponsors’ devices for the purposes of product development and regulatory performance testing. Data in this paper do not include sponsor’s study devices. All data were collected from devices procured by the Hypoxia Research Laboratory for the purposes of independent research. No company provided any direct funding for this study, participated in study design or analysis, or was involved in analysing data or writing the manuscript. None of the authors own stock or equity interests in any pulse oximeter companies. Dr Ellis Monk’s time utilised for data analysis, reviewing and editing was funded by grant number: DP2MH132941.

Background

Retrospective clinical trials of pulse oximeter accuracy report more frequent missed diagnoses of hypoxemia in hospitalized Black patients than White patients, differences that may contribute to racial disparities in health and health care. Retrospective studies have limitations including mistiming of blood samples and oximeter readings, inconsistent use of functional versus fractional saturation, and self-reported race used as a surrogate for skin color. Our objective was to prospectively measure the contributions of skin pigmentation, perfusion index (PI), sex, and age on pulse oximeter errors in a laboratory setting.

Methods

We enrolled 146 healthy subjects, including 25 with light skin (Fitzpatrick class I and II), 78 with medium (class III and IV), and 43 with dark (class V and VI) skin. We studied 2 pulse oximeters (Nellcor N-595 and Masimo Radical 7) in prevalent clinical use. We analyzed 9763 matched pulse oximeter readings (pulse oximeter measured functional saturation [Spo2]) and arterial oxygen saturation (hemoximetry arterial functional oxygen saturation [Sao2]) during stable hypoxemia (Sao2 68%-100%). PI was measured as percent infrared light modulation by the pulse detected by the pulse oximeter probe, with low perfusion categorized as PI < 1%. The primary analysis was to assess the relationship between pulse oximeter bias (difference between Sao2 and Spo2) by skin pigment category in a multivariable mixed-effects model incorporating repeated-measures and different levels of Sao2 and perfusion.

Results

Skin pigment, PI, and degree of hypoxemia significantly contributed to errors (bias) in both pulse oximeters. For PI values of 1.0% to 1.5%, 0.5% to 1.0%, and <0.5%, the P value of the relationship to mean bias or median absolute bias was <.00001. In lightly pigmented subjects, only PI was associated with positive bias, whereas in medium and dark subjects bias increased with both low perfusion and degree of hypoxemia. Sex and age was not related to pulse oximeter bias. The combined frequency of missed diagnosis of hypoxemia (pulse oximeter readings 92%-96% when arterial oxygen saturation was <88%) in low perfusion conditions was 1.1% for light, 8.2% for medium, and 21.1% for dark skin.

Conclusions

Low peripheral perfusion combined with darker skin pigmentation leads to clinically significant high-reading pulse oximeter errors and missed diagnoses of hypoxemia. Darkly pigmented skin and low perfusion states are likely the cause of racial differences in pulse oximeter performance in retrospective studies.

It has long been known that many pulse oximeters function less accurately in patients with darker skin. Reasons for this observation are incompletely characterized and potentially enabled by limitations in existing regulatory oversight. Based on decades of experience and unpublished data, we believe it is feasible to fully characterize, in the public domain, the factors that contribute to missing clinically important hypoxemia in patients with darkly pigmented skin. Here we propose 5 priority areas of inquiry for the research community and actionable changes to current regulations that will help improve oximeter accuracy. We propose that leading regulatory agencies should immediately modify standards for measuring accuracy and precision of oximeter performance, analyzing and reporting performance outliers, diversifying study subject pools, thoughtfully defining skin pigmentation, reporting data transparently, and accounting for performance during low-perfusion states. These changes will help reduce bias in pulse oximeter performance and improve access to safe oximeters.

Background

Currently, no reliable method exists for continuous, noninvasive measurements of absolute cerebral blood flow (CBF). We sought to determine how changes measured by ultrasound-tagged near-infrared spectroscopy (UT-NIRS) compare with changes in CBF as measured by transcranial Doppler (TCD) in healthy volunteers during profound hypocapnia and hypercapnia.

Methods

Ten healthy volunteers were monitored with a combination of TCD, UT-NIRS (c-FLOW, Ornim Medical), as well as heart rate, blood pressure, end-tidal PCO2 (PEtCO2), end-tidal O2, and inspired O2. Inspired CO2 and minute ventilation were controlled to achieve 5 stable plateau goals of EtCO2 at 15-20, 25-30, 35-40, 45-50, and 55-60 mm Hg, for a total of 7 measurements per subject. CBF was assessed at a steady state, with the TCD designated as the reference standard. The primary analysis was a linear mixed-effect model of TCD and UT-NIRS flow with PEtCO2, which accounts for repeated measures. Receiver operating characteristic curves were determined for detection of changes in CBF.

Results

Hyperventilation (nadir PEtCO2 17.1 ± 2.4) resulted in significantly decreased mean flow velocity of the middle cerebral artery from baseline (to 79% ± 22%), but not a consistent decrease in UT-NIRS cerebral flow velocity index (n = 10; 101% ± 6% of baseline). Hypercapnia (peak PEtCO2 59.3 ± 3.3) resulted in a significant increase from baseline in both mean flow velocity of the middle cerebral artery (153% ± 25%) and UT-NIRS (119% ± 11%). Comparing slopes versus PEtCO2 as a percent of baseline for the TCD (1.7% [1.5%-2%]) and UT-NIRS (0.4% [0.3%-0.5%]) shows that the UT-NIRS slope is significantly flatter, P < .0001. Area under the receiver operating characteristic curve was significantly higher for the TCD than for UT-NIRS, 0.97 (95% confidence interval, 0.92-0.99) versus 0.75 (95% confidence interval, 0.66-0.82).

Conclusions

Our data indicate that UT-NIRS cerebral flow velocity index detects changes in CBF only during hypercarbia but not hypocarbia in healthy subjects and with much less sensitivity than TCD. Additional refinement and validation are needed before widespread clinical utilization of UT-NIRS.

Background

Universal access to pulse oximetry worldwide is often limited by cost and has substantial public health consequences. Low-cost pulse oximeters have become increasingly available with limited regulatory agency oversight. The accuracy of these devices often has not been validated, raising questions about performance.

Methods

The accuracy of 6 low-cost finger pulse oximeters during stable arterial oxygen saturations (SaO2) between 70% and 100% was evaluated in 22 healthy subjects. Oximeters tested were the Contec CMS50DL, Beijing Choice C20, Beijing Choice MD300C23, Starhealth SH-A3, Jumper FPD-500A, and Atlantean SB100 II. Inspired oxygen, nitrogen, and carbon dioxide partial pressures were monitored and adjusted via a partial rebreathing circuit to achieve 10 to 12 stable target SaO2 plateaus between 70% and 100% and PaCO2 values of 35 to 45 mm Hg. Comparisons of pulse oximeter readings (SpO2) with arterial SaO2 (by Radiometer ABL90 and OSM3) were used to calculate bias (SpO2 – SaO2) mean, precision (SD of the bias), and root mean square error (ARMS).

Results

Pulse oximeter readings corresponding to 536 blood samples were analyzed. Four of the 6 oximeters tested showed large errors (up to -6.30% mean bias, precision 4.30%, 7.53 ARMS) in estimating saturation when SaO2 was reduced <80%, and half of the oximeters demonstrated large errors when estimating saturations between 80% and 90%. Two of the pulse oximeters tested (Contec CMS50DL and Beijing Choice C20) demonstrated ARMS of <3% at SaO2 between 70% and 100%, thereby meeting International Organization for Standardization (ISO) criteria for accuracy.

Conclusions

Many low-cost pulse oximeters sold to consumers demonstrate highly inaccurate readings. Unexpectedly, the accuracy of some low-cost pulse oximeters tested here performed similarly to more expensive, ISO-cleared units when measuring hypoxia in healthy subjects. None of those tested here met World Federation of Societies of Anaesthesiologists standards, and the ideal testing conditions do not necessarily translate these findings to the clinical setting. Nonetheless, further development of accurate, low-cost oximeters for use in clinical practice is feasible and, if pursued, could improve access to safe care, especially in low-income countries.

Pulse oximetry is a standard of care during anaesthesia in high-income countries. However, 70% of operating environments in low- and middle-income countries have no pulse oximeter. The ‘Lifebox’ oximetry project set out to bridge this gap with an inexpensive oximeter meeting CE (European Conformity) and ISO (International Organization for Standardization) standards. To date, there are no performance-specific accuracy data on this instrument. The aim of this study was to establish whether the Lifebox pulse oximeter provides clinically reliable haemoglobin oxygen saturation (Sp O2 ) readings meeting USA Food and Drug Administration 510(k) standards. Using healthy volunteers, inspired oxygen fraction was adjusted to produce arterial haemoglobin oxygen saturation (Sa O2 ) readings between 71% and 100% measured with a multi-wavelength oximeter. Lifebox accuracy was expressed using bias (Sp O2 – Sa O2 ), precision (SD of the bias) and the root mean square error (Arms). Simultaneous readings of Sa O2 and Sp O2 in 57 subjects showed a mean (SD) bias of -0.41% (2.28%) and Arms 2.31%. The Lifebox pulse oximeter meets current USA Food and Drug Administration standards for accuracy, thus representing an inexpensive solution for patient monitoring without compromising standards.

Introduction

Pulse oximetry may overestimate arterial oxyhemoglobin saturation (Sao2) at low Sao2 levels in individuals with darkly pigmented skin, but other factors, such as gender and oximeter probe type, remain less studied.

Methods

We studied the relationship between skin pigment and oximeter accuracy in 36 subjects (19 males, 17 females) of a range of skin tones. Clip-on type sensors and adhesive/disposable finger probes for the Masimo Radical, Nellcor N-595, and Nonin 9700 were studied. Semisupine subjects breathed air-nitrogen-CO2 mixtures via a mouthpiece to rapidly achieve 2- to 3-min stable plateaus of Sao2. Comparisons of Sao2 measured by pulse oximetry (Spo2) with Sao2 (by Radiometer OSM-3) were used in a multivariate model to assess the source of errors.

Results

The mean bias (Spo2 – Sao2) for the 70%-80% saturation range was 2.61% for the Masimo Radical with clip-on sensor, -1.58% for the Radical with disposable sensor, 2.59% for the Nellcor clip, 3.6% for the Nellcor disposable, -0.60% for the Nonin clip, and 2.43% for the Nonin disposable. Dark skin increased bias at low Sao2; greater bias was seen with adhesive/disposable sensors than with the clip-on types. Up to 10% differences in saturation estimates were found among different instruments in dark-skinned subjects at low Sao2.

Conclusions

Multivariate analysis indicated that Sao2 level, sensor type, skin color, and gender were predictive of errors in Spo2 estimates at low Sao2 levels. The data suggest that clinically important bias should be considered when monitoring patients with saturations below 80%, especially those with darkly pigmented skin; but further study is needed to confirm these observations in the relevant populations.

Background

It is uncertain whether skin pigmentation affects pulse oximeter accuracy at low HbO2 saturation.

Methods

The accuracy of finger pulse oximeters during stable, plateau levels of arterial oxygen saturation (Sao2) between 60 and 100% were evaluated in 11 subjects with darkly pigmented skin and in 10 with light skin pigmentation. Oximeters tested were the Nellcor N-595 with the OxiMax-A probe (Nellcor Inc., Pleasanton, CA), the Novametrix 513 (Novametrix Inc., Wallingford, CT), and the Nonin Onyx (Nonin Inc., Plymouth, MN). Semisupine subjects breathed air-nitrogen-carbon dioxide mixtures through a mouthpiece. A computer used end-tidal oxygen and carbon dioxide concentrations determined by mass spectrometry to estimate breath-by-breath Sao2, from which an operator adjusted inspired gas to rapidly achieve 2- to 3-min stable plateaus of desaturation. Comparisons of oxygen saturation measured by pulse oximetry (Spo2) with Sao2 (by Radiometer OSM3) were used in a multivariate model to determine the interrelation between saturation, skin pigmentation, and oximeter bias (Spo2 – Sao2).

Results

At 60-70% Sao2, Spo2 (mean of three oximeters) overestimated Sao2 (bias +/- SD) by 3.56 +/- 2.45% (n = 29) in darkly pigmented subjects, compared with 0.37 +/- 3.20% (n = 58) in lightly pigmented subjects (P < 0.0001). The SD of bias was not greater with dark than light skin. The dark-light skin differences at 60-70% Sao2 were 2.35% (Nonin), 3.38% (Novametrix), and 4.30% (Nellcor). Skin pigment-related differences were significant with Nonin below 70% Sao2, with Novametrix below 90%, and with Nellcor at all ranges. Pigment-related bias increased approximately in proportion to desaturation.

Conclusions

The three tested pulse oximeters overestimated arterial oxygen saturation during hypoxia in dark-skinned individuals.