Approach for Propagating Radiometric Data Uncertainties Through NASA Ocean Color Algorithms.

Spectroradiometric satellite observations of the ocean are commonly referred to as "ocean color" remote sensing. NASA has continuously collected, processed, and distributed ocean color datasets since the launch of the Sea-viewing Wide-field-of-view Sensor (SeaWiFS) in 1997. While numerous ocean color algorithms have been developed in the past two decades that derive geophysical data products from sensor-observed radiometry, few papers have clearly demonstrated how to estimate measurement uncertainty in derived data products. As the uptake of ocean color data products continues to grow with the launch of new and advanced sensors, it is critical that pixel-by-pixel data product uncertainties are estimated during routine data processing. Knowledge of uncertainties can be used when studying long-term climate records, or to assist in the development and performance appraisal of bio-optical algorithms. In this methods paper we provide a comprehensive overview of how to formulate first-order first-moment (FOFM) calculus for propagating radiometric uncertainties through a selection of bio-optical models. We demonstrate FOFM uncertainty formulations for the following NASA ocean color data products: chlorophyll-a pigment concentration (Chl), the diffuse attenuation coefficient at 490 nm (K d,490), particulate organic carbon (POC), normalized fluorescent line height (nflh), and inherent optical properties (IOPs). Using a quality-controlled in situ hyperspectral remote sensing reflectance (R rs,i ) dataset, we show how computationally inexpensive, yet algebraically complex, FOFM calculations may be evaluated for correctness using the more computationally expensive Monte Carlo approach. We compare bio-optical product uncertainties derived using our test R rs dataset assuming spectrally-flat, uncorrelated relative uncertainties of 1, 5, and 10%. We also consider spectrally dependent, uncorrelated relative uncertainties in R rs . The importance of considering spectral covariances in R rs , where practicable, in the FOFM methodology is highlighted with an example SeaWiFS image. We also present a brief case study of two POC algorithms to illustrate how FOFM formulations may be used to construct measurement uncertainty budgets for ecologically-relevant data products. Such knowledge, even if rudimentary, may provide useful information to end-users when selecting data products or when developing their own algorithms.

INTRODUCTION

NASA has continually collected, processed, archived, and distributed global ocean color data since the launch of the Sea-viewing Wide Field-of-View Sensor (SeaWiFS) in 1997. This two decades-long multi-sensor data climatology continues to provide unprecedented synoptic-scale insight into near-surface oceanographic processes. Some of the satellite-derived variables, such as chlorophyll-a pigment concentration Chl (mg m−3), are considered as Essential Climate Variables (ECV) and are widely used by the oceanographic community to study phytoplankton ecology, marine biogeochemistry, and ecosystem responses to climate change (IOCCG, 2008;McClain, 2009; Franz et al., 2017).

Following formal definitions outlined in the Guide to Uncertainty in Measurement (JCGM, 2008), we can outline the objective of ocean color remote sensing as, to measure oceanographic quantities or measurands. We note that the measurement procedure involves a number of mathematical steps and assumptions that derive the measurand from sensor-observed top-of-atmosphere radiances. Thus, a derived ocean color data product is a result of measurement and should always be treated as an estimate of the measurand which has inherent uncertainty.

Quantifying uncertainty in derived ocean color data products (i.e.,measurands) is highly valuable, allowing end-users to: assess if datasets are fit-for-purpose, assess if observed temporal change is greater than uncertainty, assimilate uncertainties into climate models, and assess consistency among sensors (Maritorena et al., 2010; Gould et al., 2014). Additionally, a thorough understanding of uncertainty sources within a model may help guide the decisions of scientists when developing new satellite algorithms.

The measurement uncertainty (u), in an ocean color data product, y, can be expressed as the following: where u(y) represents uncertainties in y due to inherent inaccuracies/limitations in the algorithm (e.g., model coefficients), and u(y) represents uncertainties in y due to uncertainties in sensor-observed radiometry (data). In this paper we focus on u(y), that is, the propagation of radiometric uncertainties through bio-optical algorithms. For brevity, we shorten u(y) to u(y) throughout this paper unless otherwise stated.

For the ocean color community, much of our understanding of measurement uncertainty in derived data products is sourced from validation exercises using in situ datasets (Bailey and Werdell, 2006; Antoine et al., 2008; Melin, 2010; Mélin et al., 2016) or from Monte Carlo-type simulations (Wang et al., 2005). We note that advanced statistical methodologies have also emerged for predicting uncertainties in derived ocean color products (Moore et al., 2009; Salama et al., 2009; Jay et al., 2018). While validation studies remain critical for appraising the absolute skill of an ocean color algorithm, such datasets themselves have their own measurement uncertainty associated with in situ observations (including uncertainties associated with subpixel temporal/spatial/environmental variability). Monte Carlo-type analyses are particularly useful for understanding measurement uncertainty, however, these approaches can be computationally expensive and are impracticable to implement within pixel-by-pixel ocean color processing.

More recently, analytical first-order first moment (FOFM) methods have been proposed that can directly propagate radiometric uncertainty through an ocean color algorithm to estimate derived data product uncertainty (Neukermans et al., 2009; Salama et al., 2009, 2011; Lee et al., 2010; Maritorena et al., 2010; Lamquin et al., 2013; Qi et al., 2017). These approaches are based on the law of propagation of uncertainty according to JCGM (2008). A FOFM methodology benefits from being computationally efficient, thereby allowing it to be implemented in pixel-by-pixel ocean color data processing software (Lamquin et al., 2013). In addition, FOFM calculations can be used to estimate the relative contribution of individual sources to total measurement uncertainty.

Work presented here is the first comprehensive examination of methods that can be used to estimate uncertainties in NASA’s standard bio-optical data products. In this study we aim to demonstrate the feasibility of using a FOFM uncertainty framework to approximate ocean color data uncertainty in derived data products. The FOFM method, which itself is an analytical approximation, is first appraised by comparing FOFM-derived uncertainties with Monte Carlo-derived uncertainties. We demonstrate how this approach can be used as a method to check the correctness of FOFM calculations. Second, using FOFM propagation theory, we estimate uncertainty in derived ocean color products given spectrally-flat, uncorrelated relative uncertainties of 1, 5, and 10% in spectral remote-sensing reflectances, R (sr−1). We also consider spectrally-dependent, uncorrelated relative uncertainties in R published by Hu et al. (2013). Third, we consider how inclusion of covariances affect uncertainty estimates. A sample SeaWiFS scene of the Hawaiian Islands is used in this case study. Finally, we demonstrate how the FOFM approach may be used to estimate measurement uncertainty budgets. In our case study we consider two algorithms for estimating particulate organic carbon (POC; mg m−3), a key metric used to understand oceanic biomass and the carbon cycle.

In this work, we utilize a high quality in situ hyperspectral R dataset that can be spectrally subsampled to match the spectral characteristics of most existing and future ocean color sensors. This includes NASA’s Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission that is currently under development and will carry the first dedicated hyperspectral ocean color sensor.

DATA AND METHODS

Bio-optical Algorithms and Data Products

The NASA Ocean Biology Data Archive and Active Distribution Center (OB.DAAC) distribute a number of derived marine data products in two separate data suites: (i) the standard ocean color (OC) data product suite and, (ii) the inherent optical properties (IOP) product suite. The OC suite comprises established (legacy) ocean color data products that were developed during the SeaWiFS (1997–2010) and Moderate Resolution Imaging Spectroradiometer aboard Aqua (MODISA 2002–present) missions. The IOP suite comprises spectral component absorption and backscattering coefficients derived using the default configuration of the Generalized Inherent Optical Properties (GIOP) algorithm framework (Werdell et al., 2013). A selection of the OC suite and IOP suite products were used in this study (Table 1). More comprehensive detail of the bio-optical algorithms used to derive these data products and their associated uncertainties are given in Appendices A–E (Supplementary Material). We note that in this study the GIOP used a spectral subset of our R evaluation dataset (described in section Evaluation ) spanning 412–655 nm.

TABLE 1 |

Bio-optical ocean color data products.

Product name	Product suite	Symbol	Units	References
Chlorophyll-a pigment concentration*	OC	Chl	mg m−3	O’Reilly et al., 1998; Hu et al., 2012b
Chlorophyll-a derived from band ratio	–	ChlBR	mg m−3	O’Reilly et al., 1998
Chlorophyll-a derived from line height	–	ChlLH	mg m−3	Hu et al., 2012b
Diffuse attenuation coefficient at 490 nm	OC	Kd,490	m−1	Mueller, 2000
Particulate organic carbon	OC	POC	mg m−3	Stramski et al., 2008a
Normalized fluorescent line height	OC	nflh	mW cm−2 µm−1 sr−1	Behrenfeld et al., 2009
Absorption coefficient of total non-water components 443 nm	IOP	anw,443	m−1	Werdell et al., 2013
Absorption coefficient of phytoplankton at 443 nm	IOP	aϕ,443	m−1	Werdell et al., 2013
Absorption coefficient of colored dissolved and detrital matter at 443 nm	IOP	adg,443	m−1	Werdell et al., 2013
Particulate backscattering coefficient at 443 nm	IOP	bbp,443	m−1	Werdell et al., 2013

Note that NASA’s standard Chl product is a dynamic blend of ChlBR and ChlLH.

Modeling Bio-Optical Data Product Uncertainty

In this study we used the analytical law of propagation of uncertainty (JCGM, 2008) to propagate radiometric uncertainties through models used to derive bio-optical quantities. We follow the notation conventions outlined by JCGM (2008) where the uncertainty of a measured quantity, y, is denoted as u(y) and is the positive square root of the variance, u2(y). We note that y is derived from a model, f, of N input quantities, x. Following (JCGM, 2008), for uncorrelated input quantities, u2(y) can be calculated as: where, u(x) is the 1-σ uncertainty in the input quantity x. For our notation of spectral properties used in ocean color remote sensing, subscripts i correspond to wavelength. In this study, partial derivatives of target parameters were calculated analytically, however, these could also be computed numerically. For the situation where uncertainties of input quantities are correlated, Equation 2 is extended to: where u(x, x) = u(x, x) denotes the estimated error covariance associated with the quantities x and x. Comprehensive details of partial derivative calculations for each bio-optical algorithm in Table 1 are given in Appendices A–E (Supplementary Material).

Monte Carlo (MC) methods are routinely used to perform sensitivity analyses as well as quantify model output uncertainties (Refsgaard et al., 2007). In this study, we have utilized a MC approach to appraise FOFM calculations. As the partial derivative calculus within FOFM uncertainty estimates can be complex, we have used MC-to-FOFM comparisons as a means of checking calculations.

The MC estimates of uncertainties in this study were computed as follows: We note that the MC method captures non-linear effects and thus we cannot always expect direct agreement between and FOFM-derived u2(y). Indeed, even if a bio-optical model contains weak non-linearities and MC model input uncertainties are normally distributed, the number of MC iterations still needs to be suitably large for to agree with u2(y).

A given bio-optical model, f, that derives an output y, that is considered a function of n spectral remote sensing reflectance bands, R, is run 5,000 times.

Upon each iteration, each R is perturbed by a factor Δr, which is randomly sampled from a Gaussian distribution Δr, ∼N(0,u(R)), in which the mean is zero and the standard deviation, u(R), is known or assumed. No spectral correlations are assumed.

The MC simulation then generates a probability density function (PDF) for y. From the PDF, the mean value, and the standard deviation, σy, can be computed.

Evaluation R Dataset

To evaluate our FOFM uncertainty method, we used a dataset of high quality hyperspectral R spectra (N = 1124). Hyperspectral radiometric measurements were collected in situ during three different expeditions, representing a range of oligotrophic to mesotrophic waters: the SABOR experiment in the Gulf of Maine/North Atlantic/Mid-Atlantic coast (July–August 2014); AE1319 in the North Atlantic and Labrador Sea (August–September 2013); and NH1418 in the Equatorial Pacific (September–October 2014). A HyperOCR system (Sea-Bird Scientific) deployed on a tethered profiler in “buoy mode” was used to collect upwelling radiance, L (W m−2 µm−1 sr−1), and downwelling irradiance, E (W m−2), spectra during deployments lasting ∼5 min. During sample collection, the instrument was allowed to drift far enough from the boat to avoid the ship’s shadow.

The spectra were dark and tilt-corrected, and the upper and lower 25th percentile of the E spectra were removed from both E and L. The mean of the remaining spectra was used in subsequent analysis, providing one spectrum per deployment, and with uncertainties calculated as the standard deviation of the same spectra used to calculate the mean (N.B. uncertainties here represent only the experimental portion of the uncertainties, and calibration bias has not been accounted for). The L measurements were extrapolated to and across the air-water interface to obtain the water-leaving radiance, L (W m−2 sr−1), which were then used to calculate remote-sensing reflectance (R), defined as: The spectra were additionally corrected for Raman scattering following methods in Westberry et al. (2013), which was necessary to compensate for the scattering that water molecules themselves can contribute to L, especially at the blue wavelengths in very clear waters (McKinna et al., 2016). Finally, the R spectra were normalized to remove the angular effect of the sun position in the sky relative to nadir, following methods in Lee et al. (2011). For a more detailed description of the R calculations and processing, see Data and Methods section in Chase et al. (2017). All hyperspectral R used in this study are shown in Figure 1.

FIGURE 1 |

Hyperspectral remote-sensing reflectances (N = 1124) used in this study.

Finally, each hyperspectral R spectrum was spectrally sub-sampled. The resulting multiband R dataset had sixteen 10 nm-wide spectral bands centered on: 412, 425, 443, 460, 475, 490, 510, 532, 555, 583, 617, 640, 655, 665, 678, and 710 nm. This multispectral subset spanned the visible domain and included bands from both past and present NASA sensors (e.g., SeaWiFS and MODIS).

Radiometric Uncertainties

Spectrally Flat R Uncertainties

For NASA ocean color bio-optical algorithms, model input quantities are typically remote sensing reflectances, R (sr−1), which are derived from measured top-of-atmosphere radiances, L (W m−2 µm−1 sr−1), via atmospheric correction (AC) algorithms. Historically, a desirable science requirements for NASA ocean color missions has been R with relative uncertainty of 5% (spectrally flat) or less (Hooker et al., 1992; Hooker and McClain, 2000; McClain et al., 2004; PACE Science Definition Team, 2018). Whilst not directly representative of a true sensor (see section Spectrally-Dependent ), treating relative uncertainties in R as spectrally flat is still useful under circumstances where detailed knowledge of sensor performance characteristics is limited, such as during pre-launch scoping studies, to provide rudimentary uncertainty estimates. In this study we first consider 5% relative uncertainty in R to compare FOFM-to-MC calculations. We next use the FOFM method consider how spectrally flat relative uncertainties in R of 1, 5, and 10% impact estimated OC and IOP uncertainties. Note, we treat spectrally flat relative uncertainties in R of 1, 5, and 10% as spectrally uncorrelated.

Spectrally-Dependent R Uncertainties

We note that on-orbit uncertainties in L and R have previously been quantified for NASA’s SeaWiFS and MODISA missions (Eplee et al., 2007; Hu et al., 2012a, 2013; Angal et al., 2015). Whilst historically 5% has been the desired accuracy goal for R in the blue-green spectral range, work by Hu et al. (2013) reported that relative uncertainties of R for SeaWiFS and MODISA increase monotonically with wavelength, and that R relative uncertainty also varies as a function of Chl, or water-column optical complexity. To extend this study beyond spectrally flat relative uncertainties, we utilized the relative uncertainties for MODISA R estimated for the North Atlantic Ocean (see Table 2 of Hu et al., 2013). To estimate relative uncertainty for a given R spectra, we followed three steps: (i) linearly interpolate tabulated relative uncertainties to match the spectral resolution of our in situ R dataset, (ii) estimate Chl concentration using NASA’s standard OC algorithm, and (iii) linearly interpolate the spectrally tabulated relative uncertainties to estimate relative uncertainty for observed R based on the respective Chl concentration. Note, where estimated Chl exceeded 0.2 mg m−3 [beyond values reported by Hu et al. (2013)] we linearly extrapolated tabulated relative uncertainties. Figure 2 shows the spectral relative uncertainties in R [sensu Hu et al. (2013)] used in this study and how they vary with Chl concentration. Note, spectrally-dependent relative uncertainties in R computed as a function of Chl were treated as spectrally uncorrelated.

FIGURE 2 |

Relative uncertainties of R varying with Chl concentration. Original data taken from Hu et al. (2013) and interpolated to the multispectral resolution used in this study.

Spectrally-Correlated R Uncertainties

Our initial analyses treated R spectral uncertainties as uncorrelated, which in practice is an oversimplification. Indeed, AC algorithms utilize near-infrared bands to make assumptions about the contribution of atmospheric aerosols to L (Gordon and Wang, 1994; Bailey et al., 2010). Thus, R uncertainties are inherently spectrally correlated. While much work has been done to characterize radiometric uncertainties of NASA sensors used for ocean color (Eplee et al., 2007; Hu et al., 2012a, 2013), few studies have quantified off-diagonal elements of the variance-covariance matrices for top-of-atmosphere radiance, V, and remote sensing reflectances, V, respectively. We note that while beyond the scope of this work, parallel efforts are underway by the research community to derive pixel-by-pixel estimates of u(R) by propagating radiometric uncertainties through ocean color atmospheric correction algorithms (Gillis et al., 2018).

Recently, Lamquin et al. (2013) demonstrated a methodology to estimate V for MERIS data and propagate these through ESA’s clear water branch AC algorithm and into bio-optical data products. Critically, Lamquin et al. (2013) demonstrated that ignoring covariances can lead to overestimated data product uncertainties. In this study, using a similar methodology to Lamquin et al. (2013), we estimate V for SeaWiFS and then using a numerical approximation estimate V. A full description of this method can be found in Appendix F (Supplementary Material). We note that while our estimates of V are somewhat rudimentary, they are still useful for demonstrating the importance of including covariance terms in FOFM-based uncertainty estimates.

Satellite Data Processing

A SeaWiFS image of Hawaii captured on 1 December 2000 was used to demonstrate the FOFM methodology when applied to ocean color imagery. SeaWiFS Level-1 data was downloaded from NASA’s Ocean Biology Distributed Active Archive Center (NASA OB.DAAC) Level 1 and 2 Browser website (https://oceancolor.gsfc.nasa.gov/)[1]. Data were then processed from Level 1 to Level 2 using NASA Ocean Color Science Software (OCSSW). These processing steps include radiometric calibration, geolocation, and atmospheric correction. A prototype version of OCSSW code was used to compute u(Chl) using FOFM methodology where u(R) was estimated using an empirical methodology described in Appendix F (Supplementary Material).

RESULTS

Appraisal of Methodology

The MC methodology, while computationally expensive, was expected to give robust estimates of measurand uncertainties. Thus, MC outputs provided a benchmark to which the FOFM uncertainty estimates could be compared with for correctness. Direct calculations of FOFM uncertainties, u(y), were compared with MC output uncertainties, σy. To compare MC and FOFM calculations we used 5% spectrally flat relative uncertainty in R and computed the following comparison statistics: bias and Type II linear regression slope. When computing these statistics for the purpose of FOFM-to-MC comparisons, we assume that MC-estimated uncertainties were quasi-truth. We note that variables were log-transformed for these calculations following Seegers et al. (2018). Bias was computed as: where N = 1124 is the number of input spectra. Given that bias was computed using log-transformed variables, it becomes interpretable as multiplicative metrics (Seegers et al., 2018). We note that our bias calculations assume estimated OC and IOP uncertainties follow log-normal distributions; a property that is demonstrated later in the paper (e.g., Figures 4, 5).

FIGURE 4 |

(A–D) Histograms of derived Chl, K, POC, and nflh, respectively. (E–H) Histograms of FOFM-estimated uncertainties in derived Chl, K, POC, and nflh, respectively computed using 5% spectrally flat, uncorrelated uncertainty in input R. (I–L) Histograms of FOFM-estimated relative uncertainties in derived Chl, K, POC, and nflh, respectively. Note: FOFM-estimates of POC relative uncertainties in this example were invariant. Dashed curves represent MC results, solid blue bars represent FOFM results.

FIGURE 5 |

(A–D) histograms of derived a, a, a, and b, respectively. (E–H) histograms of FOFM-estimated uncertainties in derived a, a, a,443, and b, respectively, computed using 5% spectrally flat, uncorrelated uncertainty in input R. (I–L) histograms of FOFM-estimated relative uncertainties in derived a, a, a, 443, and b, respectively. Dashed curves represent MC results, solid blue bars represent FOFM results.

The MC and FOFM estimation of derived product uncertainties were in good agreement for the following OC products: K, POC, and nflh. This was indicated by slope and bias and statistics (Table 2) having values of, or near to, 1.0. However, regression statistics indicated Chl uncertainties derived using the FOFM method did not completely agree with the MC method (Table 2). To assess this discrepancy more closely, uncertainties in each component of the Chl algorithm were inspected, namely the band ratio (BR), line height (LH), and blended components. Regression statistics indicated that FOFM estimates of Chl product uncertainties did not agree well with MC values and were typically biased low by 27%, visualized further by the color-coded scatter plot in Figure 3A.

TABLE 2 |

Log-normal statistics comparing Monte Carlo (MC) and first-order first-moment (FOFM) uncertainty calculations for R with spectrally flat, uncorrelated 5% relative uncertainty.

	Derived product uncertainty
Product	Bias	Slope
Chl (all)	0.95	0.96
ChlBR	1.00	1.00
ChlLH	0.99	1.00
Chlblended*	0.73	0.72
Kd,490	0.99	1.00
POC	0.99	1.00
nflh	0.99	1.00
anw,443	0.99	1.00
aϕ,443	0.98	1.00
adg,443	0.98	1.00
bbp,443	0.99	0.98

Blended LH and BR Chl product span 0.134–0.165 mg m-3.

FIGURE 3 |

Scatter plot comparisons of data product uncertainties estimated from FOFM with those estimated from Monte Carlo (MC) simulations. (A–D) OC products Chl, K, POC, and nflh, respectively. Note that the scatter plot of Chl uncertainty is color coded with respect to the method use to derive the output product (line height: purple, band ratio: green, blended: yellow). (E–H) IOP products a, a, a, and b, respectively.

Derived uncertainties for IOP products generally agreed with MC simulations. Specifically, Table 2 shows FOFM estimates of uncertainties with respect to MC estimates for a, a, a, and b were biased low by 1%, low by 2%, low by 2% and, high by 2%, respectively. Slight disagreement between MC and FOFM estimates of u(b) can be visualized in Figure 3 when u(b) > 2.0 × 10−3 m−1. In addition, MC and FOFM estimates of u(a) showed slight disagreement when u(a) > 1.0 × 10−2 m−1.

These results demonstrate that while FOFM uncertainty calculations are computationally inexpensive, they serve as approximations only, especially in the case of Chl. Indeed, while FOFM-derived uncertainties can be expected to agree with MC-derived values for simple functions that vary linearly, it may not be unusual for FOFM-derived uncertainties to differ from MC-derived values; particularly when analyzing complicated non-linear problems (Putko et al., 2001; Mekid and Vaja, 2008). For example, with the IOPs we found slight differences in the order of 1% between MC and FOFM uncertainty estimates. For such mathematical functions, higher order methods such as Second Order First Moment (SOFM) methods may be useful, however, the added mathematical complexity may be prohibitive.

Uncertainties Estimated From in situ Radiometric Data

OC Product Uncertainties

Using the multispectral R evaluation dataset, uncertainties in derived OC products associated with 5% spectrally-flat relative, uncorrelated uncertainty in Rrs were computed. Figure 4 shows histograms of derived OC products, absolute uncertainties, and relative uncertainties. MC computations are summarized in Table 3, while FOFM computations are provided for comparative purposes in Table 4.

TABLE 3 |

OC products and associated uncertainties derived via MC method with 5%, uncorrelated relative uncertainty in R.

	Derived value		Absolute uncertainty		Relative uncertainty (%)
Product	Range	Median	Range	Median	Range	Median
Chl (mg m−3)	3.96 × 10−2−1.27	0.110	2.56 × 10−5-0.231	7.00 × 10−3	1.73–18.2	9.74
Kd,490 (m−1)	2.01 × 10−2−0.131	2.91 ×10−2	1.19 × 10−3−1.36 × 10−2	2.68 × 10−3	5.92–10.5	8.94
POC (mg m−3)	18.8–203.4	33.1	1.37–14.6	2.44	7.11–7.60	7.37
nflh (mW cm−2 µm−1 sr−1)	5.25 × 10−6−2.74 × 10−2	2.20 × 10−3	3.18 × 10−4-4.47 × 10−3	9.86 × 10−4	14.8–1.7 × 104	41.9

TABLE 4 |

OC products and associated uncertainties derived via FOFM method with 5%, uncorrelated relative uncertainty in R.

	Derived value		Absolute uncertainty		Relative uncertainty (%)
Product	Range	Median	Range	Median	Range	Median
Chl (mg m−3)	3.96 × 10−2−1.28	0.110	3.89 × 10−5-0.230	6.70 × 10−3	0.26–18.7	9.67
Kd,490 (m−1)	2.01 × 10−2−0.131	2.91 × 10−2	1.18 × 10−3−1.33 × 10−2	2.68 × 10−3	5.86–10.2	8.91
POC (mg m−3)	18.8–203.4	33.1	1.37–14.9	2.42	7.31
nflh (mW cm−2 µm−1 sr−1)	2.05 × 10−6−2.73 × 10−2	2.19E × 10−3	3.21 × 10−4−4.43 × 10−3	9.87 × 10−4	15.1–3.24 × 104	42.1

Relative uncertainties in POC computed using FOFM method were constant over the dynamic range.

The range of derived Chl confirmed that the dataset spans oligotrophic (0.04 mg m−3) to mesotrophic conditions (1.28 mg m−3) with a median value of 0.11 mg m−3. Values of u(Chl) span four orders of magnitude and have median values of 7.00 × 10−3 and 6.70 × 10−3 mg m−3 for the MC and FOFM methods, respectively. The relative uncertainties for Chl span a single order of magnitude and have median values of 9.74 and 9.67% for the MC and FOFM methods, respectively. Although the histogram of derived Chl in Figure 4 appears log-normally distributed, two distinct peaks are present; a low peak (ranging from 0 to 0.5 mg m−3) and a high peak (centered on 1.1 mg m−3). Since bio-optical properties are log-normally distributed in the ocean (Campbell, 1995), the peaks observed in the distributions of derived bio-optical variables are probably due to the limited size of the hyperspectral R dataset (N = 1124), that does not uniformly span the entire range of oceanic conditions (see Figure 1A in Chase et al., 2017).

The range of derived K spans an order of magnitude with a median value of 0.0291 m−1. The values of u(K) also span an order of magnitude with median values of 2.68 × 10−3 m−1 for both MC and FOFM calculations. The relative uncertainties for K span a single order of magnitude and have a median value of 8.94 and 8.91% for MC and FOFM calculations, respectively. The range of derived POC spans two orders of magnitude with a median value of 33.1 mg m−3. The values of u(POC) span an order of magnitude and have median values of 2.44 and 2.42 mg m−3 for MC and FOFM calculations, respectively. The relative uncertainties in POC have a value of 7.37 and 7.31% for MC and FOFM calculations, respectively. We note that the relative uncertainty in POC as computed by FOFM method exhibits no spread. For uncorrelated, spectrally flat relative uncertainties, u(POC)/POC is a function of u(R)/R and u(R)/R. Thus, when u(R)/R and u(R)/R are fixed (e.g., at 5%), u(POC)/POC is fixed. In practice, this will not always hold true, particularly when relative uncertainties in R are variable and spectrally dependent. We note that in Figure 4 the MC-derived relative uncertainties for POC are normally distributed over a narrow range centered on 7.37%.

The range of nflh spans three orders of magnitude with an MC-estimated median value of 2.20 × 10−3 mW cm−2 µm−1 sr−1. We note that direct calculations of nflh resulted in a median value of 2.19 × 10−3 mW cm−2 µm−1 sr−1. The values of u(nflh) span an order of magnitude with median values of 9.86 × 10−4 and 9.87 × 10−4 mW cm−2 µm−1 sr−1 for MC and FOFM calculations, respectively. The median relative uncertainty in nflh was 41.9 and 42.1% for MC and FOFM calculations, respectively (Figure 4). We note that the range of relative errors for nflh is very large (for MC calculations: 14.8–1.7 × 104%), and these should be interpreted with a caution. Low values of nflh, in the order of 1 × 10−6 mW cm−2 µm−1 sr−1, were derived from the evaluation dataset which in most likelihood would be beyond the detection limit of existing ocean color sensors. Further, while the absolute uncertainties associated with these low nflh values may also be small in magnitude, they can still manifest as large relative uncertainties.

IOP Product Uncertainties

Using the radiometric evaluation dataset, uncertainties in derived IOP products associated with 5% relative, uncorrelated uncertainty in R were computed following the methodology in Appendix E (Supplementary Material). Figure 5 shows histograms of derived IOP products, absolute uncertainties, and relative uncertainties. MC computations are summarized in Table 5 while FOFM computations are provided for comparative purposes in Table 6.

TABLE 5 |

IOP products and associated uncertainties derived using MC method with 5%, uncorrelated relative uncertainty in R.

	Derived value		Absolute uncertainty		Relative uncertainty (%)
Product	Range	Median	Range	Median	Range	Median
anw(443) (m−1)	9.40 × 10−3−0.127	0.0185	1.79 × 10−3−1.13×10−2	2.31 × 10−3	8.16–19.4	12.6
aϕ(443) (m−1)	5.80 × 10−3−9.43 ×10−2	9.60 × 10−3	1.63 × 10−3−9.68 × 10−3	2.04 × 10−3	10.0–29.2	21.4
adg(443) (m−1)	3.50 × 10−3−3.72 ×10−2	8.71 × 10−3	6.66 × 10−4−5.90 × 10−3	1.07 × 10−3	7.92–19.9	14.5
bbp(443) (m−1)	4.18 × 10−4−4.00 × 10−3	1.08 × 10−3	8.98 × 10−5−2.25E × 10−4	1.34 × 10−4	5.57–34.1	13.8

TABLE 6 |

IOP products and associated uncertainties derived using FOFM method with 5%, uncorrelated relative uncertainty in R.

	Derived value		Absolute uncertainty		Relative uncertainty (%)
Product	Range	Median	Range	Median	Range	Median
anw,443 (m−1)	9.42 × 10−3−0.127	0.0185	1.79 × 10−3-1.03 × 10−2	2.26 × 10−3	8.12–19.1	12.2
aϕ,443 (m−1)	5.86 × 10−3−9.45 × 10−2	9.63E-3	1.64 × 10−4−8.73 × 10−3	2.00 × 10−3	9.02–28.6	20.8
adg,443 (m−1)	3.51 × 10−3−3.70 × 10−2	8.73E-3	6.51 × 10−4−5.63 × 10−3	1.05 × 10−3	7.93–18.9	14.1
bbp,443 (m−1)	4.16 × 10−4−4.01×10−3	1.00E-3	9.00 × 10−5−2.11 × 10−4	1.33 × 10−4	5.25–34.1	13.9

The range of derived a spans two orders of magnitude with a median value of 0.0185 m−1. Values of u(a) span an order of magnitude with median values of 2.31 × 10−3 and 2.26 × 10−3 m−1 for MC and FOFM methods, respectively. The median relative uncertainty in a spans a single order of magnitude and has median values of 12.6 and 12.2% for MC and FOFM methods, respectively. The range of a, a, and b all span a single order of magnitude with median values of 9.6 × 10−3, 8.71 × 10−3, and 1.08 × 10−3 m−1, respectively. Absolute uncertainties in IOPs all span two orders of magnitude apart from u(a) which spanned a single order of magnitude. Highest relative uncertainties of all GIOP-derived products are for a (∼20%), whereas a, a, and b have relative uncertainties of similar magnitude that are all <15%.

Summary of MC and FOFM Comparisons

FOFM and MC estimates of OC and IOP uncertainties were generally in good agreement. This provides confidence that our FOFM analytical formulations were correct. However, FOFM-to-MC comparisons of Chl and IOP uncertainties, whilst similar in magnitude, exhibited a degree of scatter around the one-to-one line. We expect that these differences may be due to the MC method’s ability to handle non-linearity and discontinuities in the models more robustly than the FOFM approach. For example, the Chl model has several complex features such: switching between Chl and Chl, the Chl model’s selection of maximum band ratios, and the blending of Chl and Chl, which may not be fully captured by the FOFM method.

We thus found FOFM-to-MC comparisons to be useful as a “quick acceptability checking” of FOFM calculations. In practice, however, one should not always assume the two methods will closely agree as the MC model may handle non-linearities and discontinuities more robustly than the FOFM method. The FOFM and MC calculations also indicate that for normally distributed radiometric input uncertainties, the estimated output uncertainties for OC and IOP were log-normally distributed (as per Figures 4, 5). Such highly dynamic and variable nature of uncertainties in ocean color data products highlights the need for these estimates to be done on a pixel-by-pixel basis, rather than a single scene-wide estimate, further justifying the need for simplified, computationally inexpensive approach (i.e., FOFM).

We note that our FOFM uncertainty formulation for the GIOP currently does not consider uncertainty in spectral shape models [i.e., and ]. Indeed, we believe that this may be why there were some noticeable differences when comparing FOFM and MC methods, for example: when u(b) > 2.00 × 10−4 m−1 (Figure 3H). In a cursory study, we re-ran both FOFM and MC calculations with the shape models parametrized as spectral constants (i.e., having no uncertainties). This resulted in improved FOFM-to-MC comparisons (results not shown) and further highlighted how spectral shape uncertainties impact our FOFM uncertainty estimates. As part of future work, we thus plan to extend our current GIOP FOFM uncertainty formulation to include the spectral shape uncertainties. Additionally, we note that and , computed as functions of Chl and a red-green R ratio, respectively, are also correlated. Thus, an improved GIOP FOFM uncertainty formulation should also consider covariances between spectral shape models.

GIOP Model Misfit Uncertainties

In this analysis we used our high-quality evaluation R dataset to approximate GIOP model misfit uncertainties. Our assumptions in this exercise were: (i) the uncertainties in our R dataset are small, and (ii) the least squares residual of the optimal solution (model misfit) are thus due to an imperfect model.

In this analysis we first computed the error-covariance matrix, Errs, for each R observation as follows: (i) employ the Levenberg-Marquardt non-linear least squares optimization to iteratively find an optimal solution for the free variables x, x, and x which correspond to Chl concentration, a, and b, respectively (see Appendix E in Supplementary Material for further detail). We note that in the standard implementation of the GIOP, the cost function (Chi-squared) is unweighted. (ii) feed the optimal set of x, x, and x back in the forward reflectance model to compute a best-fit spectral sub-surface remote sensing reflectance, . (iii) calculate the spectral residual, ε, between and sensor-observed subsurface remote sensing reflectance. (iv) set the diagonal elements of Errs as the square of ε.

Next, by substituting Errs for Vrrs in Equation E13 the parameter error-covariance matrix, Ex, can be computed as: Where J is the Jacobian matrix of the forward model (see Appendix E in Supplementary Material for derivation). Finally, the estimates of parameter uncertainties due to model misfit were calculated as the square root of the diagonal elements of Ex. The model-misfit uncertainties are summarized in Table 7 and compared to parameter uncertainties due to Hu spectrally-dependent radiometric uncertainties (as per Table 6).

TABLE 7 |

GIOP model-misfit uncertainties estimated using the evaluation R dataset.

	Absolute uncertainty (m−1)		Relative uncertainty (%)		Difference between absolute data and absolute model misfit uncertainties* (%)
Product	Range	Median	Range	Median	Median
atw,443 (m−1)	3.88 × 10−4 −5.71 × 10−3	4.87 × 10−4	1.26–5.70	3.15	−77
aϕ,443 (m−1)	3.67 × 10−4−5.25 × 10−3	4.54 × 10−4	3.02–9.09	4.68	−77
adg,443 (m−1)	1.07 × 10−4−2.26 × 10−3	1.434 × 10−4	0.81–7.48	2.86	−86
bbp,443 (m−1)	2.94 × 10−5−2.17 × 10−4	5.22 × 10−5	1.57–9.58	4.52	−61

Differences between median absolute model uncertainties in this table and median absolute radiometric (data) uncertainties (column RU: Hu in Table 9).

We found that estimated GIOP model misfit uncertainties were 60–90% smaller than those imparted by radiometric uncertainty. Thus, by combining the two during pixel-by-pixel processing, it would be possible to more completely estimate u(y) for IOPs. However, we accept that our FOFM model-data misfit approach is approximate only and does not consider all uncertainties in the GIOP model formulation.

Comparing Product Uncertainties Due to Various Radiometric Input Uncertainties

In order to evaluate the impact of different R uncertainty values on derived product uncertainties, using the FOFM method we: (i) propagated spectrally flat, uncorrelated R relative uncertainties of 1, 5, and 10% through OC and IOP models, and (ii) propagated spectrally-dependent, uncorrelated u(R) through OC and IOP models by linearly interpolating/extrapolating tabulated data published by Hu et al. (2013), referred to as “Hu uncertainties” (see Figure 2). Summary results of this analysis are given in Tables 8, 9. As expected, introducing spectrally flat, uncorrelated R uncertainties of lower and higher value than the previously evaluated 5%, resulted in respectively, lower and higher uncertainties in data products, while the distribution of uncertainties kept the same shape as for the 5% run (Figure 6). For the product uncertainties derived using the “Hu R uncertainties,” both the shape of the distribution and median values changed from the 5% run (Figure 6). These results demonstrate the importance of considering spectral dependence in radiometric uncertainties. Notably, considering spectrally flat 5% relative uncertainties in R for a data product such as nflh, which utilizes red-end bands, may result in significant underestimation of likely data product uncertainties.

TABLE 8 |

Median OC data product uncertainties computed as relative uncertainties (RU) in R vary.

	Median absolute uncertainties				Median relative uncertainties (%)
	RU: 1%	RU: 5%	RU: 10%	RU: Hu	RU: 1%	RU: 5%	RU: 10%	RU: Hu
Product
Chl (mg m−3)	1.52 × 10−3	6.70 × 10−3	1.46 × 10−2	6.50 × 10−3	1.96	9.67	19.35	8.29
Kd,490(m−1)	5.37 × 10−4	2.68 × 10−3	5.36 × 10−3	5.07 × 10−3	1.78	8.91	17.8	17.3
POC (mg m−3)	4.84 × 10−1	2.42	4.84	4.38	1.46	7.31	14.6	13.1
nflh (mW cm−2 µm−1 sr−1)	1.97 × 10−4	9.87 × 10−4	1.97 × 10−3	4.47 × 10−3	8.41	42.1	84.1	197.6

TABLE 9 |

Median IOP data product uncertainties computed as relative uncertainties (RU) in R vary.

	Median absolute uncertainties				Median relative uncertainties (%)
	RU: 1%	RU: 5%	RU: 10%	RU: Hu	RU: 1%	RU: 5%	RU: 10%	RU: Hu
Product
atw,443 (m−1)	4.52 × 10−4	2.26 × 10−3	4.52 × 10−3	2.76 × 10−3	2.45	12.2	24.5	15.1
aϕ,443 (m−1)	4.00 × 10−4	2.00 × 10−3	4.00 × 10−3	2.42 × 10−3	4.15	20.8	41.6	23.8
adg,443 (m−1)	2.11 × 10−4	1.05 × 10−3	2.11 × 10−3	1.33 × 10−3	2.82	14.1	28.2	15.9
bbp,443 (m−1)	2.67 × 10−5	1.33 × 10−4	2.67 × 10−4	1.73 × 10−4	2.78	13.9	27.9	17.9

FIGURE 6 |

Upper row are histograms of derived OC data products uncertainties estimated using the FOFM method. Bottom row are histograms of derived IOP data product uncertainties estimated using the FOFM method. The four histograms in each subplot correspond to four different input u(R): spectrally flat R relative uncertainties of 1% (dashed black), 5% (blue), and 10% (orange) as well as spectrally dependent relative uncertainties taken from Hu et al. (2013) outlined in green dashed line.

Spectrally flat relative uncertainty in R (e.g., 5% in the blue-green region) is a commonly used accuracy goal for ocean color missions. However, we know from on-orbit data that sensors such as SeaWiFS and MODIS have largely not achieved their desired accuracy goals over the full spectral range (Hu et al., 2013), particularly at red wavelengths. In lieu of any knowledge of a sensor’s radiometric uncertainty characteristics (e.g., during design trade studies), one might decide to utilize desired relative radiometric accuracy goals to approximate ocean color data product uncertainties. However, our results have shown spectrally flat (5%) and spectrally-dependent (Hu) relative R uncertainties lead to different estimates of OC and IOP uncertainties. Indeed, for improved uncertainty estimates, we recommend the use of more representative spectrally-dependent u(R)/R, if known.

Application to Satellite Chlorophyll Image

The potential impact that spectrally-correlated uncertainties in R have upon ocean color data product uncertainties was evaluated using a scene of the southern Hawaiian Islands captured on 1 December 2000 (Figure 7). We have estimated on a pixel-by-pixel basis the covariance matrix of remote sensing reflectances, VRrs, as per the methodology described in Appendix F (Supplementary Material). Estimates of u(Chl) were then calculated both with- and without the off-diagonal terms in VRrs to demonstrate the impact of incorporating covariance terms (if known) when estimating uncertainties.

FIGURE 7 |

Derived data products for a SeaWiFS image of waters surrounding the Hawaii Islands captured on 1 December 2000. (A) Chl concentration derived using OCI algorithm, (B) u(Chl) computed with covariances included, (C) relative uncertainty in Chl computed with estimated R covariances included, (D) u(Chl) calculated without estimated R covariances included, and (E) relative uncertainty in Chl computed without estimated R covariances included.

The sample SeaWiFS Chl image (Figure 7A) shows that the clearest waters occurred southeast of Island of Hawaii (largest island) with two large eddies to the west. Regions of elevated Chl concentration are also visible along the northeast coast of the Island of Hawaii, and also adjacent to coastal waters of four islands (Maui, Lanai, Molokai, and Kahoolawe) to the northwest of Hawaii. Derived Chl ranges from 1.83 × 10−3 to 0.498 mg m−3 with a median of 0.066 mg m−3. When the off-diagonal terms in VRrs were considered, the estimated values of u(Chl) ranged from 1.30 × 10−3 to 0.075 mg m−3 with a scene-wide median of 5.20 × 10−3 mg m−3 (Figure 7B) and the relative uncertainties spanned 0.84–38.6% with a median of 7.89% (Figure 7C). When the off-diagonal terms in VRrs were not considered (i.e., set to zero), estimated values of u(Chl) ranged from 1.30 × 10−3 to 0.109 mg m−3 with a scene-wide median of 5.50 × 10−3 mg m−3 (Figure 7D) and relative uncertainties spanning 0.85–46.1 % with a median of 8.27% (Figure 7E). Note, these image statistics were computed with standard NASA level-2 quality control flags applied to remove the effect of: land, clouds, sun glint, atmospheric correction failure, product failure, and straylight contamination.

These results demonstrate how a FOFM method can be utilized in operational processing code to estimate uncertainties in derived bio-optical data products. The FOFM method was straightforward to implement within l2gen code and did not add any appreciable processing overhead. Whilst our estimation of VRrs is rudimentary (Appendix F in Supplementary Material), it allowed us to consider the covariance terms in the FOFM derivation of u(Chl). Critically, we demonstrated that the inclusion of off-diagonal covariance terms from VRrs led to lower estimates of both u(Chl) and u(Chl)/Chl when compared to the same calculations performed with off-diagonal elements of VRrs set to zero; a result consistent with findings of Lamquin et al. (2013). Additionally, this example was done with an operational processing code, demonstrating the easiness of implementing a FOFM method within day-to-day ocean color processing.

POC Algorithm Case Study

Recall from Equation 1, we broadly defined measurement uncertainty as having two sources: data uncertainty and model uncertainty. Throughout this paper we have focused heavily on deriving data uncertainties (i.e., propagation of radiometric uncertainty) which is useful if one is trying understand how a specific sensor’s noise characteristics may impact derived data product uncertainties. However, this information alone does not provide a complete picture of measurement uncertainty; model uncertainty also needs to be considered. We thus wish to demonstrate how with knowledge of model uncertainties one can draw more complete conclusions about biogeochemically-relevant data product uncertainties. As such, we present a case study in which we estimate POC measurement uncertainty for two different algorithms: (i) Stramski et al. (2008a) and (ii) Rasse et al. (2017).

Our motivation here is to solely demonstrate how one might develop algorithm uncertainty budgets (data and model uncertainty as per Equation 1) using a FOFM framework. Our calculations, however, are limited by: (i) the representativeness of our in situ R dataset which does not encompass all optical water-types found in the World’s oceans, (ii) our spectral u(R) values which are estimated from data published by Hu et al. (2013) for a MODIS-like sensor without co-variance terms, and (iii) our knowledge of model uncertainties, such as coefficients uncertainties, which is limited to those reported in literature and/or our best-guess estimates. We hence caution the reader should not use our reported numbers as a basis for algorithm selection.

POC Measurement Uncertainty Estimates

In this exercise, we performed rudimentary calculations to estimate measurement uncertainty budgets for two POC algorithms: (i) NASA’s standard POC algorithm (Stramski et al., 2008a) and (ii) the IOP-based model of Rasse et al. (2017). Conveniently for this exercise, both POC models have a power law formulation: where X in Stramski et al. (2008a) is a blue-to-green reflectance ratio (R/R, as per Appendix C in Supplementary Material) and the coefficients a and b have the values of 203.2 and −1.034, respectively. For the approach of Rasse et al. (2017) X is b, and the coefficients a and b have the values of 141,253 and 1.18, respectively. Note, in this case study we use GIOP-derived estimates of b, as inputs to the Rasse et al. (2017) model.

First, let us consider the model uncertainty component due to imperfect model coefficients. For both POC algorithms, with the coefficients a and b and their assigned uncertainties of u(a) and u(b), respectively, we can estimate the model variance for POC as: In the third term on the right-hand side of Equation 8, we set u(X) = 0 and u(X) = u(b) for Stramski et al. (2008a) and Rasse et al. (2017), respectively. We have also assumed the covariance of the coefficients a and b, which are determined by regression fit, is zero. For the Rasse et al. (2017) model, the reported model coefficient uncertainties u(a) and u(bpoc) are 45,534 and 0.046, respectively. For the Stramski et al. (2008a) model, values of u(a) and u(bpoc) were not reported. We did, however, estimate these model uncertainties by reanalyzing the original published dataset (Stramski et al., 2008b) and considering the likely uncertainty introduced by not accounting for the effect of filter pad absorption of POC (Novak et al., 2018). Following this cursory analysis (results not shown), we estimated u(a) and u(bpoc) for the Stramski et al. (2008a) model to be ∼2.20 and 0.015, respectively.

Next, we considered the data uncertainty component. The Stramski et al. (2008a) model’s data uncertainty FOFM calculus was formulated in Appendix C (Supplementary Material). For the Rasse et al. (2017) model, we first estimated u(b,). To do so, b, was calculated from GIOP-derived b as: The variance in b,470 due to data uncertainty was then estimated as: For this exercise, we used GIOP-derived values of u(b,470) and u(γ). The correlation between derived values of b,547 and γ was used to estimate the covariance term u(b,547, γ) as −1.64 × 10−6 m−1 nm−1. Using, the GUM methodology the variance in the Rasse et al. (2017) POC model due to data uncertainty was then estimated as: We finally estimated the measurement uncertainty budgets for both POC models using our R evaluation dataset and with Hu spectrally-dependent, uncorrelated radiometric uncertainties (results are shown in Table 10).

TABLE 10 |

Simplified random uncertainty budgets for two POC models.

Algorithm	Median derived value (mg m−3)	Median absolute uncertainty in mg m−3 (median relative uncertainty in %)
		Data	Model	Measurement
Stramski et al., 2008a	33.1	4.40	0.94	4.50 (16.6)
Rasse et al., 2017	37.8	6.96	17.30	18.6 (49.2)

Median absolute uncertainties and median relative uncertainties were computed using our Rrs evaluation dataset with Hu spectrally-dependent, uncorrelated radiometric uncertainties and basic knowledge of model coefficient uncertainty. We note that these data are intended to illustrate how one might formulate measurement uncertainty budgets. These data are not intended for algorithm comparison purposes.

In our rudimentary measurement uncertainty budget for the Stramski et al. (2008a) POC algorithm, we found the contribution of data (radiometric) uncertainty was larger than model uncertainty. Conversely, for the Rasse et al. (2017) POC algorithm, the contribution of model uncertainty was larger than data uncertainty. Whilst these POC algorithm uncertainty budgets may not be fully representative due to the assumptions we partook here, the exercise nonetheless demonstrates an important point: data and model uncertainties should both be considered if one wishes to use uncertainties as a means of benchmarking/comparing ocean color algorithms.

From an algorithm development perspective one can also use FOFM method to explore the relative contribution of individual uncertainty sources to the combined measurement uncertainty. We have graphically displayed the estimated component uncertainty contribution for each POC algorithm using pie charts (Figure 8). Such information may assist algorithm designers identify and minimize uncertainty sources within a model.

FIGURE 8 |

Pie charts demonstrate how individual uncertainty sources contribute to estimates of total measurement uncertainty. Here we consider: (A) a blue-green band-ratio POC algorithm and (B) an IOP-based POC algorithm. We note that these examples are intended to illustrate how one might visualize source contributions to measurement uncertainty. These plots are not intended for algorithm comparison purposes.

Summary of POC Case Study

Our brief example demonstrates the benefits of using the FOFM method for analytically estimating measurement uncertainty in POC. From an ecological perspective, this is particularly useful if one is trying to understand the variability in observed patterns, and distinguish real change from variation in uncertainty. Additionally, it allows for sensitivity analysis, thereby providing a guideline for improving model parameterization. The case study demonstrates how an uncertainty budget can provide additional information to end-users regarding data product quality, potentially informing algorithm selection, and/or guiding new algorithm development. Although ocean color algorithms are typically benchmarked based upon validation matchup metrics (Seegers et al., 2018), we expect model selection and development may be better guided by considering how data and model uncertainties manifest in derived data products.

This case study highlights a challenge if one wishes to compare/benchmark legacy ocean color algorithms based on their measurement uncertainty; one must have reasonable and complete knowledge of both data and model uncertainties to do so. Whilst we have demonstrated that it is possible to estimate and propagate random radiometric uncertainties using the FOFM framework, estimating model uncertainties remains a challenge. This is because model component uncertainties (e.g., model coefficient uncertainties) of legacy ocean color algorithms were not routinely reported. To address this, re-analysis of the structure of legacy ocean color algorithms using high quality bio-optical datasets, such as NASA’s bio-Optical Marine Algorithm Dataset (NOMAD; Werdell and Bailey, 2005), may be necessary. Without such knowledge, it remains a challenge to formulate complete measurement uncertainty budgets for legacy ocean color algorithms.

CONCLUSIONS

In this paper we demonstrated a FOFM-based method for estimating uncertainties in a selection of NASA OC and IOP products, namely: Chl, K, POC, nflh, a, a, a, and b, due to sensor-observed radiometric uncertainty. Using a high quality hyperspectral R dataset subsampled to our target wavelengths, we first appraised the FOFM methodology by comparing FOFM-derived uncertainty estimates with uncertainties estimated from MC simulations with an assumed relative spectrally flat, uncorrelated uncertainty in R of 5%. Our analyses showed that OC and IOP uncertainties estimated using the FOFM method generally agreed with MC simulations. Collectively, the FOFM-to-MC comparisons provided a basis for checking the correctness of the FOFM formulations, which are often algebraically complex. Further, we demonstrated that the FOFM formulation, which is computationally inexpensive, can be applied in routine pixel-by-pixel data processing for estimating uncertainties in derived ocean color data products.

This paper has primarily focused on propagating radiometric uncertainties through bio-optical models (u(y) in Equation 1). In practice, the combined measurement uncertainty in derived ocean color data products is expected to be larger once model uncertainties are included. In this study, we have broadly assumed that coefficients within the bio-optical algorithms themselves are errorless, which is not the case. Indeed, most coefficients in bio-optical algorithms have been derived empirically using in situ oceanographic datasets, which themselves have inherent uncertainties due to measurement method and environmental variability. The GIOP, for example, makes assumptions about spectral shapes of IOPs, utilizes an approximate forward reflectance model (Gordon et al., 1988), and employs a model to convert R, to r (Lee et al., 2002). Thus, there are a number of GIOP model components whose uncertainties, if characterized, may improve the overall estimate of IOP measurement uncertainty. Our case study of POC algorithms also highlighted how the addition of model (e.g., coefficient) uncertainties can further inform end-users, and may potentially guide algorithm development and/or selection.

Although this work represents a first step toward implementing pixel-by-pixel uncertainty estimates in NASA operational ocean color processing code, we recognize that continued effort is required. For example, strategies for quantifying uncertainties in look-up-table (LUT) based models, such as the two-band particulate inorganic carbon (PIC) algorithm (Balch et al., 2005) and bidirectional reflectance distribution function (BRDF) correction (Morel et al., 2002), are needed. Globally, there are a multitude of ocean color algorithms maintained by various researchers and/or institutes and formulating uncertainty estimates must be a collective effort. While the community continues to innovate new bio-optical algorithms, we strongly encourage model developers to characterize uncertainties as a matter of routine.

As we enter the hyperspectral world of PACE, it is credible to expect an evolutionary leap in remote sensing observation of ocean processes detailing, for example, phytoplankton diversity, physiological preferences, and ecology from space. This, parallel to the increase in computational power of the day-to-day data processing, will allow for more complex algorithms; algorithms which will need detailed evaluation of uncertainty budgets, to understand what is real, and what is hidden under the dashed line.