Revising the slant column density retrieval of nitrogen dioxide observed by the Ozone Monitoring Instrument.

Nitrogen dioxide retrievals from the Aura/Ozone Monitoring Instrument (OMI) have been used extensively over the past decade, particularly in the study of tropospheric air quality. Recent comparisons of OMI NO2 with independent data sets and models suggested that the OMI values of slant column density (SCD) and stratospheric vertical column density (VCD) in both the NASA OMNO2 and Royal Netherlands Meteorological Institute DOMINO products are too large, by around 10-40%. We describe a substantially revised spectral fitting algorithm, optimized for the OMI visible light spectrometer channel. The most important changes comprise a flexible adjustment of the instrumental wavelength shifts combined with iterative removal of the ring spectral features; the multistep removal of instrumental noise; iterative, sequential estimates of SCDs of the trace gases in the 402-465 nm range. These changes reduce OMI SCD(NO2) by 10-35%, bringing them much closer to SCDs retrieved from independent measurements and models. The revised SCDs, submitted to the stratosphere-troposphere separation algorithm, give tropospheric VCDs ∼10-15% smaller in polluted regions, and up to ∼30% smaller in unpolluted areas. Although the revised algorithm has been optimized specifically for the OMI NO2 retrieval, our approach could be more broadly applicable.

Introduction

Nitrogen oxides (NO = NO + NO2) are important species in both the stratosphere and troposphere, where they impact ozone chemistry and air pollution [Finlayson‐Pitts and Pitts, 2000; Seinfeld and Pandis, 2006]. Away from regions of tropospheric pollution, close to 90% of the NO2 total vertical column density (VCD) is found in the stratosphere, where it is approximately zonally symmetric and varies meridionally with season. In polluted tropospheric regions, the tropospheric column exceeds the stratospheric NO and comes from fossil fuel combustion, with lightning and bacterial soil emissions as secondary sources. Nitrogen dioxide (NO2) leads to photochemical ozone and aerosol production and is itself the subject of environmental regulation. NO2 VCDs have been retrieved from satellite‐based measurements since 1995, by Global Ozone Monitoring Experiment (GOME) [Burrows et al., 1999], GOME‐2 [Callies et al., 2000], Scanning Imaging Absorption Spectrometer for Atmospheric Chartography, on Envisat (SCIAMACHY) [Bovensmann et al., 1999], and Ozone Monitoring Instrument (OMI) [Levelt et al., 2006]. Each of these instruments uses the solar backscatter technique. All are in polar, Sun‐synchronous orbits, permitting, for some, nearly daily global NO2 measurements.

Satellite‐based estimates of NO2 columns have been widely used in studies of atmospheric chemistry, air quality, and climate (greenhouse gas trends and lifetimes), and in the evaluation of chemistry transport models (CTMs). In the stratosphere, satellite‐based NO2 observations have been used to study factors influencing ozone loss and recovery [e.g., Adams et al., 2013] as well as long‐term trends in stratospheric composition [e.g., Dirksen et al., 2011, Hendrick et al., 2012]. In the troposphere, satellite NO2 records show the sources [van der A et al., 2008; Lin et al., 2010; Ghude et al., 2013; Mebust et al., 2011; Mebust and Cohen, 2013], spatial patterns [Martin et al., 2003; Toenges‐Schuller et al., 2006; Russell et al., 2010; Hilboll et al., 2013a], and trends of NO emissions [Richter et al., 2005; Kim et al., 2006; Zhang et al., 2007; Boersma et al., 2008; McLinden et al., 2012; de Wildt et al., 2012; Russell et al., 2012; Duncan et al., 2013; Streets et al., 2013; Vinken et al., 2014], NO lifetimes [Schaub et al., 2007; Lamsal et al., 2010; Beirle et al., 2011], and the impact of population and economic activity on NO emissions [Lamsal et al., 2013].

NO2 VCDs for both the stratosphere and troposphere have been produced operationally by OMI since its launch in 2004 [Levelt et al., 2006]. Aura/OMI has an equator crossing time of approximately 13:45 in the ascending node, nearly global daily (until 2008) coverage, and a spatial resolution at nadir of 13 × 24 km2. Nitrogen dioxide slant column density (SCD), or the effective total column amount along the optical path, is retrieved using differential optical absorption spectroscopy (DOAS) [Platt and Stutz, 2006]. The NO2 vertical column density is then calculated as VCD = SCD/AMF, where the air mass factor (AMF) is estimated using radiative transfer calculations [e.g., Bucsela et al., 2013]. There are two independent sets of OMI operational NO2 products: OMNO2 (NASA) [Bucsela et al., 2008, 2013; Lamsal et al., 2014] and DOMINO (Royal Netherlands Meteorological Institute, KNMI) [Boersma et al., 2007, 2011], which are derived from the same SCD retrieval but use independent AMF computations and stratosphere–troposphere separation schemes. OMNO2 and DOMINO VCDstrat agree to within a few percent on a zonal mean basis, although Bucsela et al. [2013] showed that there are important differences in the spatial distribution of the stratospheric NO2 field. The current versions of the OMNO2 and DOMINO VCDtrop products (version 2) are also in generally good agreement [Bucsela et al., 2013].

Recent studies compared OMI VCD(NO2) to VCDs obtained from other instrumental measurements, both space based and ground based [Krotkov et al., 2012; Maasakkers et al., 2013; Belmonte Rivas et al., 2014; van Geffen et al., 2014], and suggest that the OMI‐estimated VCDs are greater than the others by as much as 30% over unpolluted regions. An almost constant, ∼−1.0 × 1015 mol cm−2 bias between the SCDs provided by DOMINO and an alternative QDOAS algorithm was noted by Anand et al. [2015]. All these studies suggest that the biased SCD(NO2) values are the likely source of the discrepancy in the final VCD(NO2) product.

In this study we address the long‐standing problem of the OMI SCD(NO2) bias by proposing a sequential DOAS SCD retrieval algorithm, applying it to the OMI data and highlighting some key results. The revision was made possible by an understanding of OMI's performance gained through more than 10 years of measurements, as well as a thorough analysis of fitting residuals. We show that the proposed approach radically reduces the known systematic OMI SCD(NO2) differences with respect to independent data sets. We also use the new SCD retrievals to examine the impact on both stratospheric and tropospheric VCD(NO2).

OMI and Operational NO2 Algorithm

The OMI detector is a two‐dimensional CCD array. The instrument optics are designed such that the spatial dimension of the detector is oriented across the orbit track with a 115° field of regard, while the other dimension records spectral information. The spatial dimension is divided into either 60 or 30 rows (after coadding), thus providing 60(30) effective cross‐track fields‐of‐view (FOVs; we call them rows in the technical discussion). Three separate detectors [Dobber et al., 2006], designated UV‐1, UV‐2, and VIS, have spectral coverage in the ranges 264–311 nm (δ λ = 0.63 nm), 307–383 nm (δ λ = 0.42 nm), and 349–504 nm (δ λ = 0.63 nm), respectively. The NO2 molecule has strong, broad, highly structured spectral features in the ∼350–510 nm range; hence, the VIS detector covers most of the wavelength range needed for the NO2 retrieval. Spectral measurements in the 60 cross‐track FOVs are made over 2 s intervals. This results in along‐track coverage of 13 km, and a cross track of ∼24 km for the near‐nadir FOVs. During each orbit a total of about 1640 exposures are recorded on the sunlit side of the Earth. The width of the swath is such that 15 orbits are required to observe the entire surface of the Earth.

An operational algorithm to derive VCD(NO2) is described in Bucsela et al. [2013] and Lamsal et al. [2014]. The algorithm runs in three separate, consecutive steps: (1) spectral fit, producing SCD(NO2) and SCD errors, (2) calculation of air mass factors and AMF errors, and (3) generation of vertical column densities using a stratosphere‐troposphere separation scheme.

Operational SCD Fitting Algorithm

The currently implemented SCD fitting algorithm [Boersma et al., 2002, 2007] closely follows the general concepts of DOAS [Platt and Stutz, 2006]. The earthshine spectra are converted to reflectances using a static solar reference spectrum, after a wavelength adjustment. The fixed‐temperature NO2, H2O, and O3 cross sections (each convolved with a parametrized slit function to match the OMI resolution), the single‐scattering air ring spectrum (RS) and a high‐order (currently, fifth‐order) polynomial function are simultaneously fitted to the reflectances, to produce the respective SCDs and RS amplitudes. The RS component is treated as a pseudoabsorber [van Geffen et al., 2014].

The Subsequent Steps: AMF Calculation and SCD‐to‐VCD Conversion

The retrieved SCDs are corrected for instrumental artifacts (in particular, for interrow systematic differences called stripes [see Dobber et al., 2008a]) using the stratospheric AMFs as described in Bucsela et al. [2013]. AMFs are calculated from tabulated, precomputed values of vertically resolved NO2 sensitivities (scattering weights) and a priori NO2 vertical profile shapes. The scattering weights are parametrized by three optical angles, the surface pressure and reflectivity at the FOV center. The surface pressure is determined from a digital elevation model, and the reflectivity is inferred from a climatology developed using OMI reflectance measurements [Kleipool et al., 2008]. The effect of clouds is modeled using the cloud radiance fraction and the effective cloud pressure taken from an independent OMI level 2 product [Acarreta et al., 2004]. The NO2 profile shape is selected from a monthly geographical climatology constructed from 4 years' simulations using the Global Modeling Initiative (GMI at 2°× 2.5° latitude‐longitude spatial resolution) chemistry and transport model [Strahan et al., 2007]. The stratospheric and tropospheric components are then separated following the method described by Bucsela et al. [2013].

In the following discussion, we apply the revised fitting algorithm (we call it Goddard SCD hereafter) to a representative sample of OMI L1B (calibrated radiances) data, convert the resulting SCDs into VCDs using the above‐mentioned procedure and compare the new estimates to the operationally produced (referred to as DOMINO V2 SCD) SCDs and VCDs.

The Proposed Sequential SCD Fitting Algorithm

While the currently implemented, operational algorithm simultaneously fits the ring pseudoabsorption spectrum, three absorbing chemical species, and up to six polynomial coefficients to the spectral reflectances, the proposed algorithm takes a step‐by‐step approach that we describe in this section. One of the advantages of the proposed approach is in its sequential, iterative parameter estimation, done in an order determined by the relative sizes of the spectral signatures. This potentially mitigates spurious dependencies, which could show up as correlations between the retrieved parameters. Such putative nonorthogonality of basis functions is recognized as a source of error in least squares parameter estimation [Anand et al., 2015]. Since these spurious dependencies could be, among other reasons, caused by presence of a structured instrumental noise, we pay specific attention to the procedures helping to reduce, if not altogether eliminate, the sources of such noise. In particular, we introduce an iterative, flexible wavelength adjustment procedure, as well as multistep correction of the noisy measurements, with a particular attention to the structured noise. Optimizing the approach, we use the fitting residuals as the most sensitive diagnostic. Any successful (i.e., robust and unbiased) SCD retrieval, performed either with a “conventional” DOAS algorithm, or under the proposed sequential DOAS approach, directly depends on a full account and optimal elimination of the interfering instrumental noise.

A flowchart of the proposed procedure is shown in Figure 1, while Table 1 compares some of the salient differences between the two algorithms. In the following section we only outline the major steps of the algorithm: the wavelength correction, the ring spectrum evaluation, and, finally, the sequential retrieval of the trace gas SCDs and evaluation of the SCD errors. The technical details and numerous performance evaluations are presented in the Appendix. In particular, in the Appendix we demonstrate the advantages of the flexible, multiwindow wavelength adjustment and discuss the benefits of the sequential SCD retrieval. We also provide the details of removal of the instrumental noise and calculations of the fixed‐pattern spectral residuals, finally comparing the algorithm's performance to the various metrics produced by an independent algorithm.

Figure 1

In the Input block we list the monthly mean solar irradiances (MSI), the radiances (wavelength, line‐of‐sight (row) and position (along‐orbit) dependent), the laboratory cross sections of NO2, H2O, and CHOCHO (X sections), the atmospheric (RS air) and liquid water (RS water) ring spectra convolved with the OMI slit function, the cloud fraction (CF) estimates provided by an independent retrieval from OMI observations. The preliminary estimates of the NO2, H2O and CHOCHO slant column densities (SCD‐1), as well as the spectral fixed‐pattern corrections (FPC) are used as additional input during the second‐pass SCD retrieval. RS denotes the amplitudes of the combined air and water ring spectrum.

Table 1

The Operational (DOMINO V2) and the Proposed (Goddard) SCD Fitting Parameters

	DOMINO V2a	Goddard
Wavelength range (nm)	405–465	402–465
Degree of polynomial	5, full spectral range	2 in 7 microwindows: (402‐410),(409‐418),
		(415–425), (424–434), (433–444), (438–453), and (451–465)
Solar spectrum	convolved high‐resolutionb	monthly averages of OMI irradiances
Retrieved SCDs	NO2, H2O, O3	NO2, H2O, and CHOCHO
Ring spectral reference	air, single scattering	air + water, multiscattering
NO2 reference spectrum	T = 220 Kc	T = 220 Kc
H2O reference spectrum	HITRAN‐2004	HITRAN‐2008

van Geffen et al. [2014].

Dobber et al. [2008b].

Vandaele et al. [1998].

Simultaneous Wavelength Adjustment and Ring Pattern Removal: The Microwindow Approach

In order to produce reflectances used in the SCD retrieval algorithm, we create monthly averages from the original daily OMI solar irradiance observations, thus striking a reasonable compromise between the high S/N required of the reflectances (individual daily solar records lack such S/N) and the dynamics of instrumental changes, mainly seen as gradual, long‐term wavelength shifts, as well as wavelength‐dependent changes in the OMI throughput [Marchenko and DeLand, 2014].

We choose the 402–465 nm spectral range for the NO2 retrieval, taking into consideration the strength of the NO2 spectral features. The wavelength range is limited on one end by the strong Solar CaII lines at wavelengths below 400 nm, and the broad, relatively intense O2‐O2 atmospheric absorption at 477 nm. Numerical simulations showed that the retrieved SCD(NO2) change by less than 5% with changes up to 10 nm in the placement of the retrieval window.

The earthshine radiance wavelengths are affected by instrumental thermal effects, as well as inhomogeneous illumination of the slit due to clouds. These effects can lead to pixel‐dependent wavelength shifts in the individual radiance spectra. This pixel dependence implies both the FOV‐sensitive wavelength shifts (clouds), as well as orbital changes in the prescribed wavelength grid for any particular FOV (mostly, thermal flexures, plus, to a far lesser extent, stray light). In the following procedure, we use the ring effect signatures on the measured radiances to estimate the relative wavelength offsets between the (monthly averaged) solar irradiance and the instantaneous earthshine radiance. This proves to be advantageous over the direct use of irradiances as a wavelength calibrator. In the particular OMI case, the ring spectrum may provide relatively better defined, high‐contrast, numerous spectral details, thus augmenting the method's sensitivity to the very small (∼0.01 pixel) wavelength shifts routinely recovered by the proposed approach. This is especially true for the critically important region around 420–440 nm, populated by prominent NO2 features. Moreover, since the RS amplitudes must be assessed and removed from the radiances, the use of the RS spectrum instead of irradiances allows us to combine the evaluation of the RS amplitudes and the inherently interrelated estimates of the wavelength shifts into a single iterative process. As the ring signal in the reflectances is easily an order of magnitude larger than the trace gas signal, any attempt to fit both the ring and trace gas SCD (as is done in the DOMINO V2 SCD fit) may introduce uncontrollable biases into the latter (see Appendices A1 and A3 for additional tests). Hence, it is desirable to remove ring signal prior to the SCD evaluation.

Our testing results conform to the conclusions of van Geffen et al. [2014], showing high sensitivity of the fitting residuals, as well as the retrieved SCDs, to the relatively small, ∼0.002 nm wavelength shifts (cf. OMI's ∼0.2 nm spectral sampling in the VIS channel), thus highlighting importance of the very accurate wavelength adjustments. To achieve such accuracy, we subdivide the NO2 fitting window into seven overlapping “microwindows” of various widths, from 8 to 15 nm. The microwindows enable flexible adjustments of wavelength‐dependent shifts between the radiance and irradiance spectra. More importantly, the use of overlapping, relatively short microwindows leads to robust evaluation of the RS amplitudes.

We determine the radiance‐irradiance wavelength shift in each microwindow, minimizing the spectral residuals with respect to the two parameters: the relative wavelength shift and RS amplitude. In the process of simultaneous evaluation of the wavelength shifts and the RS amplitudes, the low‐order polynomial (usually, n = 2) is used to adjust the reflectance so that the integral of the residual reflectance, over the spectral window, is unity. This is critical to avoiding errors in the subsequent trace gas fitting steps. That is, for subsequent steps in the algorithm, the reflectances are normalized to a wavelength‐dependent reference level. Hereafter we define the procedure of normalizing the reflectances to polynomials as “baselining”.

Once an optimal wavelength shift is found for each of the seven microwindows, the ring amplitudes are estimated one more time, checked for negative values and smoothed with a running mean (n = 3) filter. The ring patterns are removed, and in each microwindow the ring‐free reflectances are iteratively refitted with low‐order polynomials, n = 2. This iterative process provides a spectral reference level (“baseline”) that is very stable (usually to <0.01%) for the subsequent SCD retrieval.

The Iterative SCD Retrieval

The 402–465 nm wavelength window contains a highly structured NO2 absorption feature that is well suited to the retrieval of SCD(NO2). However, it also contains a relatively strong H2O absorption complex at 440–450 nm, and a generally much weaker glyoxal (CHOCHO) signal. The edges of absorption bands of O3 (>450 nm) and O2‐O2 (around 447 nm) also encroach on the window. However, they are relatively broad and featureless (considering the spectral span of the microwindows; this especially applies to O2‐O2), and they are effectively removed from the reflectances by the final baselining step of the microwindow approach described in the previous section. The same applies to the liquid water absorption feature [Peters et al., 2014].

We use the laboratory‐measured, high‐resolution, fixed‐temperature spectra for NO2 (T = 220 K) [Vandaele et al., 1998], H2O vapor (T = 280 K: produced by HITRAN08) [see Rothman et al., 2009], and CHOCHO (T = 296 K) [Volkamer et al., 2005]. These were convolved with wavelength‐ and row‐dependent slit functions appropriate to the OMI VIS channel [Dirksen et al., 2006]. There is no evidence for either long‐term secular changes or along‐orbit changes in the OMI slit function; hence, we apply the same set of the slit‐convolved reference spectra to all the available OMI data. We preserve the original wavelengths grids of the slit function‐convolved laboratory spectra, even if they far exceed the OMI wavelength sampling rate. This helps to reduce the inevitable errors introduced during the wavelength interpolation. Comparison of the implemented NO2 cross sections and those currently used by DOMINO V2 shows some relatively small deviations that could not cause the ∼30% SCD differences in the Goddard and DOMINO V2 SCD retrievals.

At OMI's spectral resolution, the highly structured absorption spectra of the trace gases are convolved with the relatively broad slit function. In general, this could result in significant nonorthogonality between the cross‐section functions. During a simultaneous retrieval of multiple SCDs, it is not possible to ensure that there is no mutual interference. For that reason, we elect to use a sequential, iterative approach that accounts for the trace gas features in turn.

The iterative Goddard SCD retrieval proceeds as follows: First of all, note that during the SCD evaluation we abandon the microwindow approach, since it completely served its purpose during the initial wavelength correction and baselining. Hence, we estimate SCD(NO2) using the normalized (baselined) reflectances in a truncated window (402–452 nm), excluding the 441.5–444 nm range (the strongest H2O absorption). We remove the NO2 signal from the reflectance at all wavelengths, and then estimate the H2O SCD using the reflectances in 440–449 nm. Finally, we estimate the CHOCHO SCD using the relatively short 433–459 nm window, centered on the most pronounced CHOCHO absorption features. All SCDs are independently evaluated by two algorithms (the multidimensional minimization Nelder‐Mead method and the nonlinear least squares Levenberg‐Marquardt fit; both described in Press et al. [1992]), with reflectances either in a linear (for the latter) or logarithmic form (for the former fit), and with the outcome cross checked for consistency. We use the SCDs estimates provided by the Levenberg‐Marquardt approach, unless this results in the unphysical, negative SCDs, in contrast to the positive SCDs from the Nelder‐Mead method. In such cases we revert to the latter. If both solutions turn negative, we provide their average and flag the result.

Applying the preliminary SCD estimates, we remove the trace gas absorptions from the normalized (“baselined”) reflectances which, at this point, could be regarded as fitting residuals; i.e., they should adhere to the baseline = 1.0. Using progressively tighter thresholds, initially set at ±5σ, we iteratively identify the low‐amplitude (<1%) instrumental noise and correct the corresponding pixels in the original, trace gas affected reflectances. These noise‐corrected reflectances are then subjected, one more time, to the SCD fitting, now using practically the whole 402–465 nm region for the final NO2 retrieval, minus the H2O active region 441.5–444.0 nm. The significantly broader NO2 window provides, on average, more stable solutions for the low‐value SCD(NO2).

After the second SCD evaluation, all the trace gas features are removed, once again, from the reflectances, and the resulting fitting residuals are evaluated for the presence of stable spectral patterns that arise primarily from undersampling (see more details in Chance et al. [2005]) of the OMI solar spectra. These patterns are averaged over large blocks of up to 250 consecutive orbital exposures. The fixed‐pattern correction is then applied to the radiances, and the iterative SCD retrieval process is repeated one more time (the “second pass” in Figure 1), now using the fixed‐pattern‐corrected radiances. The first‐pass SCD estimates are also used to rectify the reflectances prior to the second‐pass wavelength adjustment and ring spectrum removal procedure. After that, the final SCD estimates are computed.

SCD Errors

In the process of SCD retrieval the reflectances are cleaned of instrumental noise and finally adjusted for the fixed‐residual patterns. Removing, at the very last step, the spectral features of the trace gases from the reflectances, we find that the distribution of final fitting residuals is nearly normal (section A6). This allows us to calculate the SCD uncertainties from the shape of the χ 2 surface around the optimal SCD solution following the recipe from Press et al. [1992]. Because the fitting procedure disentangles the large‐amplitude rotational Raman patterns from the trace gas absorption features and implements a flexible estimation of wavelength shifts, the SCD uncertainties are determined with relatively small contamination from these effects, leading to considerably smaller SCD errors (see below).

Results and Discussion

To demonstrate the performance of the revised algorithm, we first analyze the NO2 retrievals produced by the DOMINO V2 and Goddard SCD algorithms, and then compare the new OMI NO2 retrievals with the time‐adjusted SCIAMACHY observations and NO2 simulation from the GMI CTM. To illustrate seasonal and latitudinal dependencies, the discussion focuses on four sets of SCD(NO2), centered on the solstices and equinoxes in 2005.

Comparison of the Operational DOMINO V2 and New Goddard SCDs

Below we show that the new Goddard‐retrieved NO2 columns are substantially reduced compared to the current, operational SCD estimates, and in better agreement with independent measurements and models. We attribute the larger values produced by the DOMINO V2 SCD fitting to a combination of factors: the wavelength adjustment based on a single wavelength shift assumed to apply over the entire wavelength range; the interference between the dominant ring spectrum patterns and the far weaker trace gas absorptions; and biases introduced by the high‐order polynomial fitting of the reflectances (see tests in the Appendices A1, and A3).

Figure 2 shows a single orbit #03610 from 20 March 2005. This particular orbit encompasses the highly contaminated suburbs of Beijing as well as relatively unpolluted open ocean regions. In general, over unpolluted regions the DOMINO V2 SCD algorithm produces substantially higher SCDs than does the Goddard SCD retrieval, which is consistent with the earlier conclusions by Krotkov et al. [2012] and Belmonte Rivas et al. [2014]. The Goddard − DOMINO V2 difference is smaller in polluted areas and larger in clean areas. The data from orbit 03610 are also presented, as a scatterplot, in Figure 3 (left), color coded according to the cloud radiance fraction (CRF). For relatively cloud‐free scenes (CRF < 0.5), it is apparent that the difference in the algorithms results primarily in an additive SCD shift, with some multiplicative effect. In cloudy circumstances (CRF > 0.5), the data are much more scattered. This is likely due to less precise SCD determination by either or both algorithms when clouds are present. For example, in the partially clouded FOVs the spatially inhomogeneous illumination of the Earth‐viewing aperture causes substantial wavelength shifts, which might not be accurately enough (i.e., ≲0.002 nm) estimated by the preprocessing algorithms. Either one or both of the SCD retrieval approaches may not fully apply the necessary additional wavelength corrections in such complex cases. This may result in less stable retrievals. One more factor may lead to the seemingly more robust DOMINO V2 SCD estimates. Specifically, we note the high correlation of the low‐value DOMINO V2 SCD NO2 retrievals and the ring‐scale estimates (section A3). Such interference presumably “stabilizes” the retrievals, artificially lessening the scatter, especially for the retrievals with the lowest SCDs. On the other hand, the DOMINO V2 shows systematically larger fitting residuals compared to the Goddard approach (the latter either with or without the fixed‐residual patterns), once we convert and scale the Goddard fitting residuals to the “root‐mean‐square error of fit” values provided by the DOMINO V2 output.

Figure 2

(left) SCD(NO2) from the DOMINO V2 SCD, (middle) Goddard SCD retrievals, and (right) their relative differences: OMI orbit #03610 from 20 March 2005.

Figure 3

(left) DOMINO V2 and Goddard SCD(NO2) retrievals for different cloud conditions: OMI orbit #03610 from 20 March 2005. (right) The (Goddard/DOMINO V2) ratios (black line) and (Goddard − DOMINO V2) differences (orange line).

There is another important aspect of the Goddard − DOMINO V2 difference. Figure 3 (right) clearly demonstrates that the difference cannot be treated as being SCD invariant. Converting the SCDs from Figure 3 to VCDs with AMFstrat, we find (not shown) that, in the low‐VCD (<4 × 1015 mol cm−2) domain explored in detail by van Geffen et al. [2014], the average offset comes close to −0.85 × 1015 mol cm−2, the value quoted by van Geffen et al. [2014], however, with a definite VCD‐dependent slope. Such offset reaches −1.5 × 1015 mol cm−2 for VCD>1016 mol cm−2. This suggests an interplay between some additive and multiplicative factors responsible for the SCD difference.

Striping

Since first light, Level‐2 OMI data products have been bedeviled by “striping,” where there are row‐to‐row mean biases that change from orbit to orbit. Such stripes are evident in Figure 2 (left; DOMINO V2 SCD). Figure 2 (middle; Goddard SCD) shows much smaller amplitudes of striping. Figure 2 (right; Goddard SCD − DOMINO V2 SCD) shows the magnitude and persistence of this effect on individual cross‐track rows along an orbit. Figure 4 summarizes, for the same data set (orbit 03610), the magnitude of the striping, computed as described in Bucsela et al. [2013]. While there is still some striping in the Goddard SCDs, it is much less pronounced than that found in the DOMINO V2 SCDs. In subsequent processing, to estimate the VCDs, we remove the stripes from SCDs applying the approach from Bucsela et al. [2013].

Figure 4

Striping amplitude in the DOMINO V2 and Goddard NO2 retrievals: OMI orbit #03610 from 20 March 2005.

Impact on Stratospheric and Tropospheric NO2 VCDs

In Figure 5, we compare OMI SCD(NO2) retrieved by the DOMINO V2 SCD and Goddard SCD algorithms over the entire globe on 20 March 2005. The features seen in the single‐orbit data are also evident in the global coverage: Compared to the DOMINO V2 SCD algorithm, the Goddard retrievals are consistently lower, by ∼10–40%, and are less affected by striping. The diminished striping and, more importantly, the smaller Goddard SCD fitting residuals should reduce the SCD errors and, consequently, the errors in the stratospheric and tropospheric NO2 columns. Figure 6 shows that the SCD(NO2) errors produced by the Goddard algorithm are significantly smaller than those produced by the DOMINO V2 algorithm, by ∼15–20%, even if we additionally account for the systematically smaller Goddard SCDs (Figure 3). It is also clear that the relative difference is a function of the SCD itself. The Goddard SCD errors in the high‐SCD areas (either polluted or observed at large solar zenith angles) do not exceed 5–7%, while the low‐SCD values (equatorial regions of oceans) bear substantially higher relative uncertainties, typically around 15–20%.

Figure 5

Daily, 20 March 2005 SCD(NO2) retrievals (top) for the Goddard algorithm, (bottom) with the relative difference, (Goddard − DOMINO)/D O M I N O.

Figure 6

SCD(NO2) errors for different SCDs (color coded: see the figure legend) and different months, with ±σ bars, to show the spread of the values.

To quantify how the observed SCD differences affect the derived stratospheric and tropospheric NO2 columns, we applied the operational OMI NO2 SCD‐VCD conversion algorithm [Bucsela et al., 2013; Lamsal et al., 2014] to both Goddard and DOMINO V2 SCDs. The algorithm calculates AMF and AMF errors, then generates VCDs from SCDs using a stratosphere‐troposphere separation scheme. Figures 7 and 8 show the comparison of stratospheric and tropospheric NO2 columns for 20 March 2005. As expected, the Goddard SCD retrievals yield consistently lower (∼30–40%) stratospheric NO2 columns in the tropics, with slightly less pronounced (∼20%) differences at high latitudes. These results agree with the findings of Krotkov et al. [2012] and Belmonte Rivas et al. [2014]. Since the stratosphere‐troposphere separation scheme uses NO2 observations from unpolluted and cloudy areas to estimate the stratospheric NO2 fields, the absolute differences in VCDtrop over those areas are expected to be small, as anticipated in Belmonte Rivas et al. [2014]. In the polluted regions, however, where VCDtrop constitutes a substantial fraction of the total NO2 column, the large change of SCDs also affects the tropospheric column NO2 retrievals, though these revisions look relatively smaller (Figure 9). Figure 10 explores the putative seasonal changes produced by the initial Goddard SCD‐DOMINO V2 SCD SCD biases. Predictably, both the VCDstrat and VCDtrop differences behave in a manner already revealed in Figures 7 and 8.

Figure 7

(top) Stratospheric VCD(NO2) estimated from Goddard SCD(NO2) for 20 March 2005. (bottom) The relative difference, (Goddard − DOMINO)/DOMINO.

Figure 8

(top) Tropospheric VCD(NO2) derived from Goddard SCD(NO2) for 20 March 2005. (bottom) The absolute difference: Goddard − DOMINO. The gray color represents cloudy areas with CRF > 50.

Figure 9

Binned relative differences, (Goddard − DOMINO)/DOMINO, of the tropospheric NO2 columns derived from the DOMINO V2 and Goddard SCDs, for 20 of March (blue), June (green), September (orange), and December (red). The shown data have Vtrop>1015 mol cm−2 for the relatively low cloud coverage, CRF < 50%.

Figure 10

Three day average differences (Goddard − DOMINO) binned at 0.25°×0.25° for the (left column) stratospheric and (right column) tropospheric VCD(NO2) for four representative months of the year 2005. The gray color in the right panel marks the cloudy areas (CRF > 50%).

Comparison to SCIAMACHY and Global Modeling Initiative Model

For evaluation with independent observations and a model, OMI data were compared with the total NO2 VCDs from SCIAMACHY [Bovensmann et al., 1999] nadir measurements and the German Aerospace Center (DLR) retrievals (version 5.02) [Lichtenberg et al., 2010] in Figure 11. We also compare the OMI data with NO2 simulated with the Global Modeling Initiative (GMI) CTM [Duncan et al., 2007; Strahan et al., 2007]. The GMI model incorporates a detailed chemical mechanism in the stratosphere [Douglass et al., 2004] and troposphere (based on the Harvard GEOS‐Chem model [Bey et al., 2001]) and includes up‐to‐date emissions, deposition, radiation, advection, and other important physical processes.

Figure 11

Seasonal comparison of NO2 Pacific zonal mean (180°W–140°W) total VCDs over 3 day periods in March, June, September, and December 2005, as indicated. Model output from the GMI model and data from three nadir retrievals are shown: SCIAMACHY (DLR nadir), OMI (DOMINO V2 SCD fitting), and OMI (Goddard SCD fitting). The model was sampled at 13:30, and the satellite data were corrected to 13:30 local solar time. Dots mark the mean VCD values, the boxes delimit the interquartile (25%, 75%) ranges and medians, and the whiskers (for the satellite data) show the 10%–90% ranges. The boxes are shifted slightly horizontally for clarity.

Stratospheric NO2 displays a well‐established diurnal variation, manifesting as a linear increase in NO2 during daylight hours [e.g., Bracher et al., 2005], with a similar effect in the total VCD over relatively unpolluted regions. In the following comparisons, we sampled the model at 13:30 time and used the normalized diurnal variation of NO2 in the GMI model to correct or adjust the OMI and SCIAMACHY data to 13:30. This allowed for comparison under similar photochemical conditions. In Figure 11 we plot the NO2 zonal means in 10° latitudinal bands sampled over the practically unpolluted Pacific Ocean regions (longitudes 180°W–140°W). We compare the total VCDs from 3 day periods in March, June, September, and December 2005. We show the GMI model output (sampled at 13:30, local solar time) along with the data from three independent retrievals: SCIAMACHY, OMI (DOMINO V2 SCD fitting), and OMI (Goddard SCD fitting). The OMI data were restricted to solar zenith angles below 85°, and radiative cloud fractions below 0.5. The substantially broader cross‐track spatial coverage of OMI was reduced to match the SCIAMACHY field of regard. The new VCD(Goddard SCD ) values are significantly lower and closely match both the model and SCIAMACHY averages. This is consistent with the works of Krotkov et al. [2012], Belmonte Rivas et al. [2014], and van Geffen et al. [2014]. The noticeably smaller spread of SCIAMACHY VCDs within latitude/month bins in Figure 11 could be due in part to SCIAMACHY's larger‐ground FOVs, which effectively averages out geophysical variability resolved by OMI. The inherent noise of SCIAMACHY measurements is also reduced by its comparatively higher signal‐to‐noise ratio.

Method Limitations

The problem of atmospheric trace gas retrievals from hyperspectral measurements presents many challenges to algorithm development. Some of these arise from the performance of the instrument. Those include various noise processes, uncertainties in the slit function characterization, stray light, wavelength shifts, and spectral undersampling. The trace gas SCDs are retrieved from reflectances, hence being exposed to inherent differences in the instrumental performance of the irradiance and radiance channels. Other difficulties are related to both uncertainties in the laboratory measurements and the dependence of monochromatic absorption cross sections on temperature, and, sometimes, pressure. Finally, there are challenges associated with constructing the SCD retrieval algorithm: atmospheric absorption is inherently nonlinear, and the spectral absorption cross sections, which serve as basis functions in the fitting procedure, are generally not mutually orthogonal; this gives rise to inaccuracies in the retrieved columns and spurious correlations between the retrieved SCDs of different species. Furthermore, the large disparity in the optical depths of different absorbers can lead to numerical stability issues when the absorbers are retrieved simultaneously in a multidimensional fitting algorithm. Some of these issues are generic, while others are closely tied to the performance of the particular instrument.

In the proposed optimized NO2 retrieval, we have sought to overcome many of the challenges just described. For example, we use a monthly median solar irradiance spectrum, which suppresses the instrumental noise in the daily OMI irradiances, and tracks with the real, physical spectral output of the Sun. We carefully register the wavelengths of the irradiance and radiance spectra. We do this in seven microwindows, which allows us to compensate for the wavelength‐dependent shifts between the wavelength grids of radiances and irradiances. We identify transient spikes—for example, those characteristic of cosmic ray signals—and remove them from the data. And we sequentially fit the spectral reflectances in order to overcome the problem of nonortnogonality of the basis functions of the three retrieved species (NO2, H2O, and CHOCHO), as well as of the RS. As a result of the sum of these measures, the reflectance residuals appear devoid of any coherent structures that would indicate either residuals of unretrieved species, or mutual interference between basis functions. The reflectance residuals are also normally distributed, to a very good approximation.

Nevertheless, it is possible that situations may arise in which the method's accuracy is compromised. For example, under conditions of very large absorber optical depth, their deeper spectral structures in the reflectances may be mistaken for noise and subjected to modification by one of the noise‐attenuating steps. This would result in a bias in the retrieved absorber column. When that biased absorber's spectral signature is subsequently removed from the reflectances, its residuals may alias into one of the other retrieved absorbers. At best, this event would lead to an anomalously large estimated uncertainty.

Though the removal of residual, along‐track coherent spectral patterns accounts for undersampling, it may not do so completely. No special undersampling correction is made beyond this, so it is possible that some signature of undersampling yet remains.

While the microwindows are optimized to take advantage of the structure in the RS, it is true that there is less such structure in some windows than in others. Thus, the wavelength offset may not be uniformly well determined in each of the windows. The process we use for merging the wavelength offsets between adjacent windows goes some way to stabilizing the solution to account for this. Still, there remains a possibility that inhomogeneities in the determination of the wavelength offset remain.

Our wavelength registration method, and, in particular, the selection of the microwindow boundaries, relies on a distinctively structured RS in the wavelength region where our target absorber has a favorable absorption band. In addition, in most cases, the effect of the RS is much larger than the optical depth of the absorber. To apply the overall algorithm to other atmospheric species, the wavelength registration step may need to be adapted to the situation in hand.

The gains provided by the flexible wavelength adjustment and sequential SCD retrieval could be offset, under specific circumstances, by the extremely high sensitivity of the SCD retrievals to the imposed wavelength shifts. This particularly applies to low‐SCD values coming from the unpolluted equatorial regions, as well as the areas with relatively higher SCDs, e.g., at the high northern latitudes in the equinox samples from Figure 11. In both cases the spread is mainly driven by the data coming from the partially clouded scenes. Such conditions may cause substantial wavelength shifts (see Voors et al. [2006] for more details). Could it be that the proposed algorithm results in less stable retrievals for the extreme cases of the inhomogeneous slit illumination? Such possibility prompts further targeted testing. However, this particular issue may affect the reported OMI results in a minor way, as only the data from almost cloud‐free areas (routinely, CF < 0.2–0.3) are used in the NO2 surveys.

Conclusions

The SCD retrieval approach redesigned and optimized for orbital OMI performance provides 10%–35% smaller SCD(NO2) estimates than the current operational DOMINO V2 algorithm. The new approach also results in less prominent striping and lower SCD errors. The ∼30% difference in SCD(NO2) leads to a similar‐magnitude decrease of the stratospheric VCD(NO2). We also see systematically reduced VCDtrop in the heavily polluted areas, by ∼10–15%. Less polluted regions show ∼20–30% decrease in VCDtrop.

Though the proposed algorithm was optimized for the NO2 retrieval in the 402–465 nm spectral range easily accessible in the OMI data, we have also been able to extend the approach to the NO2 spectral features in the 350–380 nm region and produce preliminary SCD(NO2) estimates. This spectral range is sampled by the Ozone Mapping Profiler Suite Nadir Mapper, flying on board the Suomi National Polar‐orbiting Partnership (NPP) satellite.