Frequency and causes of failed MODIS cloud property retrievals for liquid phase clouds over global oceans.

Moderate Resolution Imaging Spectroradiometer (MODIS) retrieves cloud droplet effective radius (re ) and optical thickness (τ) by projecting observed cloud reflectances onto a precomputed look-up table (LUT). When observations fall outside of the LUT, the retrieval is considered "failed" because no combination of τ and re within the LUT can explain the observed cloud reflectances. In this study, the frequency and potential causes of failed MODIS retrievals for marine liquid phase (MLP) clouds are analyzed based on 1 year of Aqua MODIS Collection 6 products and collocated CALIOP and CloudSat observations. The retrieval based on the 0.86 µm and 2.1 µm MODIS channel combination has an overall failure rate of about 16% (10% for the 0.86 µm and 3.7 µm combination). The failure rates are lower over stratocumulus regimes and higher over the broken trade wind cumulus regimes. The leading type of failure is the "re too large" failure accounting for 60%-85% of all failed retrievals. The rest is mostly due to the "re too small" or τ retrieval failures. Enhanced retrieval failure rates are found when MLP cloud pixels are partially cloudy or have high subpixel inhomogeneity, are located at special Sun-satellite viewing geometries such as sunglint, large viewing or solar zenith angles, or cloudbow and glory angles, or are subject to cloud masking, cloud overlapping, and/or cloud phase retrieval issues. The majority (more than 84%) of failed retrievals along the CALIPSO track can be attributed to at least one or more of these potential reasons. The collocated CloudSat radar reflectivity observations reveal that the remaining failed retrievals are often precipitating. It remains an open question whether the extremely large re values observed in these clouds are the consequence of true cloud microphysics or still due to artifacts not included in this study.

Introduction

The Moderate Resolution Imaging Spectroradiometer (MODIS) is a key sensor on board NASA's Terra and Aqua satellites. The unique spectral and spatial capabilities of MODIS enables the remote sensing of a variety of cloud properties on a global scale, from cloud masking [Ackerman et al., 1998], cloud top pressure, and thermodynamic phase [Menzel et al., 2006], to cloud optical and microphysical properties [Platnick et al., 2003]. The MODIS cloud products are widely used in a broad range of Earth system science applications. For example, the MODIS cloud optical thickness (τ), cloud droplet effective radius (r ), and derived cloud liquid water path (LWP) products are frequently used in studies of aerosol‐cloud interactions [e.g., Kaufman et al., 2005; Quaas and Boucher, 2005; Lebsock et al., 2008; Jiang et al., 2009; Quaas et al., 2009] and radiative effects of aerosols in cloudy‐sky conditions [Wilcox, 2012; Costantino and Bréon, 2013; Meyer et al., 2013; Yu and Zhang, 2013; Min and Zhang, 2014; Zhang et al., 2014; Meyer et al., 2015]. They have also been used for evaluating cloud parameterization schemes in climate models [Donner et al., 2011; Jiang et al., 2012; Kay et al., 2012; Pincus et al., 2012] and for studying cloud feedbacks [Zhou et al., 2013]. Given the wide usage of the MODIS cloud products, it is critical to evaluate and understand their quality and limitations. In this paper, we present a comprehensive analysis of the failed cloud τ and/or r retrievals for liquid phase clouds over ocean in the Collection 6 MODIS cloud product from Aqua (product name “MOD06”). Terra and Aqua MODIS cloud product is named as MOD06 and “MYD06,” respectively. Here we use the generic product name MOD06 for simplicity. The MODIS products are updated periodically to incorporate instrument calibration changes and significant algorithm enhancements, with the respective data processing streams and archives referred to as “Collections,” the most recent update being Collection 6 (C6) that includes several major updates in comparison with the previous Collection 5 (C5). A comprehensive user guide that summaries the major updates of the MOD06 product from C5 to C6 can be found in MODIS website [Platnick et al., 2014]. The theoretical basis for the τ and/or r retrievals can be found in [Nakajima and King, 1990; Platnick et al., 2003].

In the MOD06 algorithm, cloud τ and r are retrieved simultaneously from a pair of cloud reflectance observations, one from a visible, near infrared, or shortwave infrared (referred to collectively as “VNSWIR” hereafter) MODIS channel (i.e., 0.66 µm over land, 0.86 µm over ocean, and 1.24 µm over snow/ice) that has negligible water absorption and is sensitive primarily to τ, and the other from a shortwave infrared (SWIR) or midwave infrared (MWIR) channel (i.e., 1.6, 2.1, and 3.7 µm) that has strong water absorption and is sensitive to r [Nakajima and King, 1990; Platnick et al., 2003]. In practice, the retrieval is implemented through the use of precomputed look‐up tables (LUTs), an example of which is given Figure 1. These LUTs can be considered as the expected variability of cloud reflectances in the VNSWIR and SWIR/MWIR channels as a function of cloud τ and r under the plane‐parallel cloud assumption. For C6 MOD06, the range of τ and r for liquid phase clouds is 0.01 ≤ τ ≤ 150 and 4 µm ≤ r  ≤ 30 µm, respectively.

Figure 1

An example of the retrieval solution space for a liquid phase cloud over an ocean surface, assuming the solar zenith angle is 20°, the sensor zenith angle is 20°, and the relative azimuth angle is 0°.

Usually, the observed cloud reflectances (after necessary atmospheric corrections) lie inside the precomputed LUT solution space and a successful retrieval of cloud τ and r can be made by interpolating the immediate LUT points. However, it is possible (and in fact quite common as shown later) that the observations fall outside of the LUT solution space. In such cases, the retrieval is considered “failed” because no combination of τ and r within the above mentioned ranges can explain the observed cloud reflectances. At present a comprehensive analysis has not been undertaken that establishes the frequency and geographical distribution of these retrieval failures nor that addresses their potential causes (e.g., peculiarities of the algorithm and inherent limitations of the technique). Note that different causes of retrieval failure have very different implications. The failures due to retrieval uncertainties and artifacts arguably help to ensure the quality of the MOD06 product by implicitly excluding problematic pixels from the retrieval population. Conversely, if the r failures are due to SWIR/MWIR reflectances that imply smaller or larger r than the LUT range limits, and these do indeed represent real microphysics, exclusion of these observations from MOD06 would result in a sampling bias. It is therefore important to better understand the underlying causes of MODIS retrieval failures. Such an analysis is critical for end users of the MODIS cloud products and may also identify potential product improvements.

Motivated by the need for a better understanding of the failed MODIS cloud property retrievals, we present a systematic study of the failed MODIS τ and/or r retrievals for the liquid phase clouds over ocean in the MOD06 product. For the sake of simplicity, we shall refer to these clouds as marine liquid phase (MLP) clouds in this paper. We focus on the MLP clouds for a number of reasons. MLP clouds cover large areas of the world's ocean [Warren et al., 1988; Wood, 2012] and play a pivotal role in the radiative energy budget of the Earth‐Atmosphere system [Klein and Hartmann, 1993]. Furthermore, for remote sensing retrievals, the microphysical properties of MLP clouds are relatively less complicated than those of ice or mixed‐phase clouds, and the dark ocean surface is easier to model compared to bright land surfaces. Note that these factors do not necessarily lead to a low failure rate, but they do make the analysis of failure less complicated. In addition, several important updates to MOD06 have been made from C5 to C6 (see section 2.1 for details) that will help us characterize retrieval failures and their causes.

Our analysis of failed Aqua MODIS τ and r retrievals is aided by complementary observations from CALIOP (Cloud‐Aerosol Lidar with Orthogonal Polarization) on board CALIPSO (Cloud‐Aerosol Lidar and Infrared Pathfinder Satellite Observation) and the CPR (Cloud Profiling Radar) on CloudSat. Both CALIPSO and CloudSat operate within the A‐Train satellite constellation together with Aqua [Stephens et al., 2002], thus collocating their observations is relatively straightforward. As a passive sensor MODIS cannot reliably detect overlapping clouds and its cloud top phase retrieval algorithm may misinterpret cloud phase such that the wrong LUT may be used for τ and r retrievals. In this study, we use CALIOP observations collocated with Aqua MODIS retrievals to investigate the potential connection of τ and/or r retrieval failure to overlapping cloud condition and misinterpretation of cloud top phase. Furthermore, several recent studies suggest that the presence of drizzle in MLP clouds can have a significant influence on MODIS r retrievals, although the mechanisms underlying this influence are still being investigated [Kubar and Hartmann, 2009; Nakajima et al., 2010; Zhang, 2013]. In this study, we use the CloudSat CPR observations to understand the potential correlation between τ and/or r retrieval failure with drizzle in MLP clouds.

The rest of the paper is organized as follows: We will briefly describe the MODIS C6 cloud products, and CALIOP and CloudSat observations in section 2. A comprehensive analysis of the MODIS retrieval failure rates for MLP clouds will be presented in section 3. We investigate the potential causes for the retrieval failures in section 4 and summarize this paper in section 5.

Satellite Data

MODIS

The global retrieval failure rate analysis presented here is based on 1 year (2007) of the C6 Aqua MODIS products that include the 1 km geolocation product (MOD03), the cloud mask product (MOD35) [Ackerman et al., 1998], and the cloud top, optical, and microphysical property product (MOD06) [King et al., 1997; Platnick et al., 2003]. As previously stated, a retrieval failure occurs if the observed reflectances for a cloudy MODIS pixel lie outside of the LUT solution space, such that no combination of τ and r within the valid range can be found to explain the observation. In the previous C5 MOD06 product, little information was provided regarding the location of the observations with respect to the LUT, which makes it impossible to identify the causes of retrieval failures. For C6, a new Retrieval Failure Metric (RFM) data set provides additional information about retrieval failures. This RFM data set includes three parameters, namely, the optical thickness (τ RFM), the effective radius (r ) of the LUT point closest to the observations, and a cost metric indicating the relative distance of the observation from the closest LUT point.

Referring to Figure 1, retrieval failures can be classified into three major categories based on the relative location of the observation with respect to the LUT:

For observations with VNSWIR reflectance within the LUT solution space but SWIR/MWIR reflectance below the LUT (i.e., the grey region below LUT), implying that r is larger than the largest allowed value of 30 µm, r is set to 30 µm, and τ RFM is assigned the nearest LUT value (see Table 1). These retrieval failures will be referred to as “r too large” failures.

Table 1

Definition of the Three Major Types of MODIS Retrieval Failure Based on the Parameters Provided in the “Retrieval Failure Metric” SDS (Scientific Data Set) in the C6 MOD06 Product

Failure Type	Region in Figure 1	Retrieval Failure Metric SDS
		COT	CER	COST Metric
re too large	Lower	τ RFM = Nearest LUT COT	r e,RFM = 30 µm	≥0
re too small	Upper	τ RFM = Nearest LUT COT	r e,RFM = 4 µm	≥0
τ failure		Fill	Fill	Fill or Max

For observations with VNSWIR reflectance within the LUT solution space but SWIR/MWIR reflectance above the LUT (i.e., the grey region above LUT), implying that r is smaller than the smallest allowed value of 4 µm, r is set to 4 µm, and τ RFM is assigned the nearest LUT value. These retrieval failures will be referred to as “r too small” failures.

For observations with VNSWIR reflectance larger than that of the maximum allowed τ and SWIR/MWIR reflectance either above or below the LUT (i.e., the yellow regions to the right of the LUT), or VNSWIR reflectance smaller than that of the minimum allowed τ (i.e., the purple region), both r and τ RFM are set to fill values (i.e., −9999). These retrieval failures will be referred to collectively as “τ failures”.

Note that observations with VNSWIR reflectance larger than that of the maximum allowed τ, but SWIR/MWIR reflectance within the LUT solution space (i.e., the green region to the right of the LUT) are not considered retrieval failures and in MOD06 have valid r retrievals with τ assigned the maximum value. Table 1 summarizes the above retrieval failure classification and the associated RFM parameter values for τ RFM, r , and cost metric. Other types of retrieval failure are possible, though are found to be rare, accounting for less than 0.5% of all failed retrievals.

MODIS has three channels in the SWIR/MWIR, namely, 1.6, 2.1, and 3.7 µm, that are used for r retrievals (i.e., r , r , and r , respectively). We focus only on r and r in this study; however, since Aqua MODIS is known to have nonfunctional or noisy detectors in the 1.6 µm channel that make it difficult to compare the pixel‐level failure rate of r to those of r and r . A major algorithm update from C5 to C6 is how r and r are sampled and reported in the MOD06 product. In C5, r was reported as an independent product, while r was reported as its difference from r retrievals (i.e., r  − r ) such that the sampling of r was limited by the r retrieval success rate. For C6 this sampling bias is removed as all spectral r retrievals are independently reported along with their respective τ retrievals. A number of recent studies noted that the successful r and r retrievals can differ significantly [Nakajima et al., 2010; Painemal and Zuidema, 2011; Zhang and Platnick, 2011; Zhang et al., 2012]. This spectral difference is partly because the two bands have different sensitivity to cloud vertical structure and the presence of drizzle [Zhang and Platnick, 2011; Zhang, 2013] and partly because they are influenced to different degrees by retrieval uncertainties and artifacts (e.g., plane‐parallel bias induced by subpixel inhomogeneity, 3‐D radiative transfer effect) [Zhang and Platnick, 2011; Zhang et al., 2012]. The independent sampling of r and r enables us to analyze their failure rates separately. Note that for simplicity we shall refer to the spectral MLP cloud property retrievals by their respective SWIR/MWIR channel wavelengths, i.e., “2.1 µm” for the 0.86 and 2.1 µm combination and “3.7 µm” for the 0.86 and 3.7 µm combination.

In addition to reporting spectral r retrievals separately, the treatment of pixels determined to be “partly cloudy” by the Clear Sky Restoral (CSR) algorithm has also changed in C6. The purpose of the CSR algorithm was to identify cloud mask “cloudy” pixels that deviate from the assumption of a homogeneous overcast cloudy field of view and are thus considered poor retrieval candidates as they are expected to yield retrieval failures or suffer large uncertainties. As shown in Table 2, the CSR algorithm identifies two types of partly cloudy pixels, namely, pixels at cloud edge (CSR = 1) and pixels over ocean that are highly inhomogeneous according to the subpixel 250 m cloud mask and are likely partially cloudy (CSR = 3), as well as cloudy pixels that are not clouds but instead are likely thick aerosols, snow/ice surfaces, or sunglint (CSR = 2). CSR pixels are often a significant portion of those “cloudy” cloud mask pixels, particularly in broken cloud fields, and in C5 were “restored to clear sky” with τ and r assigned fill values regardless of retrieval success. The end result is an obvious sampling bias that often led to confusion among users because the population of pixels having valid r and τ retrievals is significantly smaller than that of cloudy pixels according to the cloud mask. For C6, cloud edge and inhomogeneous/partially cloudy pixels (i.e., CSR = 1,3) with successful r and τ retrievals are now retained and are reported together in partly cloudy (PCL) data sets separate from the standard overcast data sets; note that the CSR = 2 pixels, i.e., those that are likely not cloud, continue to be restored to clear sky. Readers interested in a detailed description of the CSR algorithm are referred to section 2.8 of the “User Guide for the Collection 6 Level‐2 MOD06/MYD06 Product and Associated Level‐3 Datasets”[Platnick et al., 2014].

Table 2

Definition of the MOD06 Clear Sky Restoral (CSR) Categories

	CSR Value	Note
Overcast	0
Cloud edge	1	One or more surrounding pixels are clear sky
Not cloudy	2	Cloud mask cloudy pixels likely to be thick aerosol, snow/ice surfaces, or sunglint
Partially cloudy	3	Subpixel 250 m cloud mask fraction 50% or less (over ocean only)

CALIOP

As a passive sensor, MODIS has only limited capability to identify overlapping clouds, which could be a potential issue causing retrieval failure. Furthermore, the MODIS cloud top phase retrieval algorithm may misidentify ice or mixed‐phase clouds as liquid phase clouds [Riedi et al., 2010], which can in turn also lead to retrieval failure. These potential causes of retrieval failures are investigated using collocated CALIOP observations. CALIOP measures the attenuated backscatter profile at 532 nm and 1064 nm, including linear depolarization information at 532 nm [Winker et al., 2009] and can identify multiple layers of clouds and aerosols as long as the lidar signal does not totally attenuate. Here the version 3.01 CALIOP level 2 1 km resolution cloud layer product (CAL_LID_L2_01kmCLay) is collocated with MODIS to provide overlapping cloud detection via the Vertical Feature Mask [Vaughan et al., 2009] to assess the impacts of overlapping clouds on the MODIS retrieval failure rates. The collocation is done by matching CALIOP profiles to the nearest MODIS pixels by minimizing the distance between their footprints on the surface [Cho et al., 2008]. This simple collocation algorithm does not account for the parallax effect caused by the difference in observing geometry between CALIOP and MODIS. But we compared our collocation scheme with a more rigorous one developed by [Holz et al., 2008] which accounts for the parallax effect using cloud height retrievals from CALIOP. The comparison results based on 1 month of data confirms that because most MLP clouds are low the parallax effect has a negligible impact on our failure retrieval analysis. The impacts of cloud phase classification errors are assessed using CALIOP's cloud layer phase retrieval [Hu et al., 2007] as an independent benchmark for the MODIS cloud top phase retrieval. It should be noted that differences in cloud top phase retrieval between CALIOP and MODIS can have various causes, e.g., information content difference, resolution difference, and algorithm design difference [Riedi et al., 2010]. Nevertheless, phase disagreement between CALIOP and MODIS indicates the scene is more complex in terms of thermodynamic or microphysical structure and therefore more prone to retrieval failure (see section 4.3). In addition, CALIOP has a 333 m cloud layer product (CAL_LID_L2_333mCLay), which is used in this study to identify broken clouds within 1 km MODIS pixels (see section 4.3).

CloudSat

The Cloud Profiling Radar (CPR) on board CloudSat is designed for the vertical profiling of clouds and precipitation [Stephens et al., 2002]. CPR operates at a frequency of 94 GHz, with an operational sensitivity of −30 dBZ, and has a horizontal footprint of 1.7 km along track by 1.3 km across track and a vertical resolution of 240 m [Tanelli et al., 2008]. Here we use CloudSat data, collocated with MODIS and CALIOP, to assess the potential impact of precipitation on MODIS retrieval failures. Similar to collocating CALIOP, we collocate CloudSat profiles with the nearest MODIS pixels by minimizing the distance between the surface footprints of the two.

A known issue of CloudSat is that the lowest two radar bins (about 1 km) above the ocean surface suffer from surface clutter contamination [Marchand et al., 2008]. Because of this issue, in addition to the minimum −30 dBZ sensitivity, CloudSat cannot detect a significant portion of low‐altitude, thin MLP clouds. Nevertheless, CloudSat is still able to detect a wide range of precipitation, including very light warm rain in thicker MLP clouds [Haynes et al., 2009; Lebsock and L'Ecuyer, 2011]. The CloudSat 2B‐GEOPROF product includes a cloud mask that identifies the location of hydrometeors in individual radar profiles over the instrument noise floor, as well as the radar reflectivity, expressed in dBZ, of the identified clouds [Marchand et al., 2008]. In particular, following previous studies we apply thresholds to the 2B‐GEOPROF radar reflectivity to identify collocated cloudy MODIS pixels that are likely to have precipitation, i.e., pixels with maximum column radar reflectivity larger than −15 dBZ [Haynes and Stephens, 2007; Lebsock et al., 2008; Kubar and Hartmann, 2009].

Global Failure Rate Analysis

Liquid Phase (MLP) Cloud Pixel Selection

The present analysis is limited to MODIS pixels over ocean labeled as “confident” or “probably” cloudy by the 1 km MOD35 cloud mask that are also identified as liquid phase by the MOD06 1 km cloud optical property thermodynamic phase discrimination algorithm (i.e., the Cloud_Phase_Optical_Properties data set); note that the MOD06 cloud optical and microphysical property retrievals are solar based and by definition daytime only. For this cloudy liquid phase pixel subset, successful retrievals are defined as those pixels having valid values for both τ and r ; fill values indicate a retrieval failure, in which case we check the aforementioned RFM data set to obtain information on failure type (see Table 1). For the year selected, 2007, the annual mean total cloud fraction over ocean from Aqua MODIS is 72.6%, out of which 52.8% (more than 35 billion 1 km pixels) satisfy the above criteria and are identified as MLP clouds. The rest includes ice phase or mixed‐phased clouds (31.1%); liquid phase clouds over sea ice (9.4%) and other complicated cases (6.7%). The global distribution of the MLP pixels based on simple aggregation is shown in Figure 2a. It should be noted that the MODIS sampling rate depends on latitude. Nevertheless, some regions known for high MLP cloud occurrence are clearly visible, for example, the NE Pacific stratocumulus deck off the coast of California. Some water bodies at high latitudes have low or even zero sampling rate because they are either covered by snow/ice (e.g., Hudson Bay, Baffin Bay, and Greenland Sea) and/or the Sun is too low (e.g., over Southern Ocean 60°S poleward) such that the solar zenith angle exceeds the daytime threshold applied in MOD06. About 73.5% of the MLP cloud pixels sampled in this year are overcast (CSR = 0) according to the CSR algorithm, 20.0% are cloud edge pixels (CSR = 1) and 6.5% are highly inhomogeneous/partially cloudy pixels (CSR = 3); note that pixels identified as not cloudy (i.e., CSR = 2) are by definition not included in the MLP cloud population. Thus, in C5 about 26.5% of the total marine liquid phase cloud population was discarded. However, including the partly cloudy (PCL) pixels in C6 does not necessarily yield a 26.5% increases in the marine liquid phase cloud retrieval population, because cloud edge and highly inhomogeneous/partially cloudy pixels are more likely to have retrieval failures compared to overcast pixels, as shown in the next section.

Figure 2

Global distributions of Aqua MODIS (a) MLP cloud pixels, (b) 2.1 µm failed retrievals, and (c) 3.7 µm failed retrievals for 2007. Color indicates pixel counts within 1° grid boxes.

Global Distribution of Failed Retrievals

For the daytime MLP cloud pixels sampled in 2007 by Aqua MODIS, the global failure rate of the 2.1 µm τ and r retrievals is 16.37%, a total that includes pixels with 2.1 µm failures only (8.96%) and those for which both the 2.1 µm and 3.7 µm retrievals failed (7.41%), i.e., “double failures” (see Table 3). The global failure rate of the 3.7 µm retrievals is 9.69%, including 2.28% 3.7 µm only failures and 7.41% double failures. The geographic distributions of failed pixels for 2.1 µm and 3.7 µm retrievals are shown in Figures 2b and 2c, respectively. The failure patterns shown here are notably different from the liquid phase cloud pixel distribution shown in Figure 2a. This suggests that retrieval failures are unlikely caused by random errors, in which case the distribution of failed retrievals would be correlated with the distribution of MLP cloud pixels. The geographic distribution of the 2.1 µm retrieval failure rate, defined as the ratio of the number of failed MLP retrievals to the total number of cloudy MLP pixels, including both overcast and PCL pixels, is shown in Figure 3a. Evidently, the failure rate is strongly dependent on cloud regime. For instance, the failure rate over the coastal stratocumulus regions, e.g., California, Peru, and Namibia, is significantly lower than that over the tropical trade wind cumulus cloud regions. It is interesting to note that although the middle‐ and high‐latitude ocean basins have a large number of MLP cloud pixels, their failure rates are rather low. The geographic distribution of the 3.7 µm retrieval failure rate for the combined overcast/PCL MLP cloud population is shown in Figure 3b. While the 3.7 µm failure rate is similarly dependent on cloud regime, its magnitude is generally lower than that of 2.1 µm.

Table 3

Failure Rates as a Function of CSR Category and Spectral Retrieval Combinationa

CSR Categories		Both Successful	2.1 Successful 3.7 Failed	3.7 Successful 2.1 Failed	Both Failed	Sum
	0: Overcast pixels	23,004,087,148	274,592,351	1,525,845,707	1,054,394,321	25,858,919,527
		(65.37%)	(0.78%)	(4.34%)	(3.00%)	(73.48%)
		88.96%	1.06%	5.90%	4.08%	100%
PCL	1: Cloud Edge	4,246,960,088	420,298,685	1,131,756,519	1,252,090,893	7,051,106,185
		(12.07%)	(1.19%)	(3.22%)	(3.56%)	(20.04%)
		60.23%	5.96%	16.05%	17.76%	100%
	3: Inhomogeneous pixels	1,377,957,225	107,837,359	495,261,719	300,037,340	2,281,093,643
		(3.92%)	(0.31%)	(1.41%)	(0.85%)	(6.48%)
		60.41%	4.73%	21.71%	13.15%	100%
Sum		28,629,004,461	802,728,395	3,152,863,945	2,606,522,554	35,191,119,867
		(81.35%)	(2.28%)	(8.96%)	(7.41%)	(100%)

Integer numbers denote MLP cloud pixel counts over 1 year of Aqua MODIS observations (2007). Numbers in parenthesis denote the percentages with respect to the total MLP pixel population. Numbers in bold denote the percentages with respect to the population of MLP pixels in each CSR category.

Figure 3

Global distributions of retrieval failure rates for all MLP cloud pixels (overcast + PCL) for (a) the 2.1 µm and (b) 3.7 µm retrievals. Also shown are the (c) 2.1 µm and (d) 3.7 µm failure rates for overcast pixels, and the (e) 2.1 µm and (f) 3.7 µm failure rates for PCL pixels.

Although the focus of this study is MLP clouds, to put the above results in a broader context, we have also calculated the failure rates for other types of clouds (e.g., liquid clouds over land and ice phase clouds over ocean and land), as well as based on Terra MODIS data. The failure analysis for other types of clouds, and for Terra, is shown in the supporting information. Overall, the failure rates in different categories are fairly similar, with slight differences between ocean and land and between liquid phase and ice phase. These differences will be studied in future research.

CSR Dependence

The rationale for discarding PCL pixels in C5 was the expectation that such pixels deviate, sometimes substantially, from the homogeneous overcast, plane‐parallel assumption used for the forward calculated LUTs, and are thus problematic for cloud retrievals (e.g., larger uncertainty and higher failure rate). As shown in Figures 3e and 3f and Table 3, the PCL pixels are indeed problematic as they have much higher failure rates than overcast pixels. The global failure rates of the 2.1 µm cloud edge (CSR = 1) and inhomogeneous/partially cloudy pixels (CSR = 3) are 33.81% (16.05% 2.1 µm only failure + 17.76% double failure) and 34.86% (21.71% 2.1 µm only failure + 13.15% double failure), respectively, or about 3 times the failure rate of overcast pixels (CSR = 0). The 3.7 µm retrievals have failure rates of 23.72% (5.96% 3.7 µm only failure + 17.76% double failure) and 17.88% (4.73% 3.7 µm only failure + 13.15% double failure) for cloud edge and inhomogeneous/partially cloudy pixels, respectively, or about 4 times the corresponding failure rate of overcast pixels. From a different perspective, overcast pixels contribute about 45% of all failed 2.1 µm retrievals while accounting for more than 70% of the total MLP cloud pixels. Conversely, the cloud edge and inhomogeneous/partially cloudy categories together contribute about 55% of all failed 2.1 µm retrievals even though they only account for less than 30% of the total MLP cloud population. A similar situation is also seen for the 3.7 µm retrievals.

Failure Types

Recalling Figure 1, there are three major categories of r and τ retrieval failures, i.e., (1) r too large failure, (2) r too small failure, and (3) τ failure. A detailed analysis of failure types, broken down by CSR category, for the 2.1 µm and 3.7 µm retrievals is given in Tables 4 and 5, respectively. Several key conclusions can be drawn from these tables. First, the r too large retrieval failure is the dominant failure type for both spectral combinations and for all three CSR categories. For the 2.1 µm retrievals, this failure type accounts for 83.83% of the entire failed 2.1 µm retrieval population. The r too small and τ failure types account for 8.45% and 7.54%, respectively. For the 3.7 µm retrievals, r too large, r too small, and τ failure types account for 63.49%, 23.77%, and 12.43% of the entire failed 3.7 µm retrieval population, respectively. Second, the two spectral combinations have nearly the same τ retrieval failure counts despite the large differences in other categories. This is not a coincidence. A further examination (see section 4.2) indicates that most of these τ retrieval failures, for both spectral combinations, occur over sunglint regions. Finally, compared with overcast and inhomogeneous/partially pixels, cloud edge pixels tend to have larger failure rates due to r too small and τ failures.

Table 4

Failure Type Analysis for the 2.1 µm Retrievalsa

CSR Categories		τ Failure: r e,RFM = −9999	re Too Small: r e,RFM = 4 µm	re Too Large: r e,RFM = 30 µm	Other Types	Sum
	0: Overcast pixels	143,653,712	116,699,600	2,314,800,640	1,894,739	2,577,048,576
		2.50%	2.03%	40.21%	0.03%	44.77%
PCL	1: Cloud edge	284,399,680	285,289,728	1,807,908,096	6,226,735	2,383,823,872
		4.94%	4.96%	31.41%	0.11%	41.41%
	3: Inhomogeneous	6,033,273	84,143,192	702,456,448	2,666,143	795,299,072
		0.10%	1.46%	12.20%	0.05%	13.82%
Sum		434,086,656	486,132,544	4,825,165,312	10,787,617	5,756,171,776
		7.54%	8.45%	83.83%	0.19%	100%

Integer numbers denote pixel counts. Percentages are with respect to entire population of failed 2.1 µm retrievals.

Table 5

Failure Type Analysis for the 3.7 µm Retrievalsa

CSR Categories		τ Failure: r e,RFM = −9999	re Too Small: r e,RFM = 4 µm	re Too Large: r e,RFM = 30 µm	Other Types	Sum
	0: Overcast pixels	133,531,600	235,715,616	957,177,536	1,295,643	1,327,720,320
		3.92%	6.92%	28.09%	0.04%	38.96%
PCL	1: Cloud edge	283,981,344	477,026,464	904,939,520	6,424,010	1,672,371,328
		8.33%	14.00%	26.55%	0.19%	49.07%
	3: Inhomogeneous	6,030,509	97,330,736	301,564,032	2,949,441	407,874,688
		0.18%	2.86%	8.85%	0.09%	11.97%
Sum		423,543,488	810,072,896	2,163,681,024	10,669,094	3,407,966,720
		12.43%	23.77%	63.49%	0.31%	100%

Integer numbers denote pixel counts. Percentages are with respect to entire population of failed 3.7 µm retrievals.

Potential Causes of Retrieval Failures

The above analysis revealed that a significant fraction of MODIS MLP cloud r and τ retrievals fail and that the rate of failure has a strong dependence on the spectral combination used for the retrieval (overall failure rates are 16.37% for 2.1 µm and 9.69% for 3.7 µm), cloud regime (e.g., higher failure rate over tropical trade wind cumulus regions than coastal stratocumulus regions), and the expected quality of the pixel (higher failure rate for PCL pixels than overcast pixels). Additionally, the r too large failure is the dominant failure type, accounting for about 60–90% of the failed retrievals depending on the spectral combination and CSR category. Much remains unexplained, however, primarily the physical (or otherwise) reasons for retrieval failures. The causes of retrieval failure are numerous and often interrelated, yet can be classified into three broadly defined categories:

Algorithm‐specific issues. Retrieval failures may be caused by issues specific to the algorithm itself, such as code bugs. More importantly, methodological choices such as spectral channel selection, thresholds, or the use of inadequate ancillary data, can yield retrieval failures either directly or indirectly via, for instance, cloud masking or thermodynamic phase errors.

Retrieval uncertainties and artifacts. There is a long list of factors known to cause uncertainties and artifacts in the retrieval, including cloud horizontal and vertical inhomogeneity and 3‐D radiative transfer effects, issues related to Sun‐satellite viewing geometry (e.g., sunglint, very low Sun, and special scattering angles), variation of pixel size with MODIS viewing angle, uncertainties related to overlapping clouds, and cloud model assumptions (e.g., assumed form of drop size distribution). Each of these factors can potentially lead to retrieval failure.

Inadequate LUT. It is possible that the r and τ ranges (4 µm ≤ r  ≤ 30 µm and 0.01 ≤ τ ≤ 150) of the operational LUTs do not adequately capture the natural variability of marine liquid phase clouds, i.e., it is possible that the observed too large or too small r is real.

Note that different causes of retrieval failure have very different implications. Failures due to retrieval uncertainties and artifacts arguably help to ensure the quality of the MOD06 product by implicitly excluding problematic pixels from the retrieval population, although this may lead to potential sampling issues. Conversely, if the too large or too small r failures do indeed represent real microphysics (i.e., the r range of the LUTs is too limited), exclusion of these observations from MOD06 introduces a sampling bias. It also provides the rationale for extending the range of the LUT to include large r values. Note that the 3.7 µm channel losses the sensitivity at r ~30 µm because of saturated absorption, but the 2.1 µm channel still has sensitivity to larger r >30 µm drops because of its weaker absorption [Zhang, 2013]. In addition, because 2.1 µm penetrates deeper into the cloud than 3.7 µm, if the different failure rates and different successful retrievals between 2.1 and 3.7 µm are in fact due to real cloud microphysics, they are then useful information for retrieving cloud vertical structure [Chen et al., 2007; Kokhanovsky and Rozanov, 2012]. It is therefore important to better understand the underlying causes of retrieval failures for better understanding the limitations of the MODIS product and for future improvement. Motivated by this, we have carried out a comprehensive investigation of the connection between MODIS retrieval failure rates and a variety of factors that can cause retrieval failures, including cloud inhomogeneity, sunglint, and precipitation.

Two caveats should be considered when interpreting the results presented here. First, an apparent correlation between enhanced failure rates and specific factors does not necessarily imply causation. Second, retrieval failures often coincide with multiple potential failure causes. For example, a cloud edge or inhomogeneous pixel that suffers from 3‐D radiative effects might also be located over sunglint. Likewise, an ice cloud over high‐latitude ocean where the Sun is low might be erroneously identified as a liquid phase cloud, in which case the enhanced 3‐D effect due to low Sun condition coupled with the cloud phase retrieval error could conspire to cause a failed retrieval. In such cases it is extremely difficult, if not impossible, to definitively determine the underlying cause of the retrieval failure.

Subpixel Cloud Inhomogeneity

A fundamental assumption of the MODIS r and τ retrievals is that for radiative transfer, a cloud pixel is horizontally homogeneous and independent of the influence of its surrounding pixels. When this so‐called plane‐parallel assumption is invalid, the observed cloud reflectance is subject to a variety of 3‐D radiative transfer effects [Davis and Marshak, 2010]. Because the 3‐D effects are not accounted for in the MODIS retrieval algorithm, the deviation of real cloud from the plane‐parallel assumption often leads to larger errors and uncertainties in the retrieved r and τ [Loeb and Davies, 1996; Várnai and Marshak, 2002; Marshak et al., 2006; Zhang and Platnick, 2011; Zhang et al., 2012].

One type of 3‐D effect, the plane‐parallel r bias, is worth special mention here. Recently, Zhang and Platnick [2011] found that because the LUTs are nonlinear, subpixel cloud inhomogeneity of τ can lead to overestimation of r [Zhang and Platnick, 2011; Zhang et al., 2012]. As demonstrated by the example in Figure 4, in some extreme cases the overestimation of r resulting from the plane‐parallel r bias may lead to retrieval failure. Here the point “P2” indicates a thin cloud with τ = 1 and r  = 25.5 µm, and the point “P1” a thick cloud with τ = 25 and r  = 25.5 µm. In Case 1, an inhomogeneous MODIS pixel is simulated assuming 70% of the pixel is covered by the thick cloud (P1) and 30% is covered by the thin cloud (P2). For the 2.1 µm LUT in Figure 4a, the reflectance of this pixel (0.7R  + 0.3R ) falls outside of the LUT leading to an r too large failure. Interestingly, the reflectance of the pixel is still inside the 3.7 µm LUT in Figure 4b, although the retrieved r  = 29 µm is significantly larger than the true value 25.5 µm [Zhang and Platnick, 2011; Zhang et al., 2012]. In Case 2, the thick cloud (P1) now covers 30% and the thin cloud (P2) covers 70%, and both the 2.1 µm and 3.7 µm retrievals fail.

Figure 4

An example illustrating how the plane‐parallel r bias can lead to failed r retrievals and why the (a) 2.1 µm retrieval is more prone to such failure than (b) 3.7 µm.

The hypothetical example in Figure 4 clearly demonstrates the possibility of the plane‐parallel r bias leading to r too large failures. Moreover, it also suggests the 2.1 µm retrievals are more prone to such failures than the 3.7 µm retrievals. To investigate potential plane‐parallel r bias‐induced failures in MOD06, we now examine the dependence of the MOD06 retrieval failure rates on subpixel cloud inhomogeneity. Here we use the inhomogeneity index H , defined as [Liang et al., 2009]: where SD[R(0.86 µm, 250 m)] and mean[R(0.86 µm, 250 m)] correspond to the standard deviation and mean of the measured 0.86 µm reflectances, respectively, of the sixteen 250 m resolution subpixels within the 1 km MODIS retrieval footprint. Defined in such a way, H generally increases with subpixel inhomogeneity. It is now included as a standard data set in the MOD06 product. It should be noted here that H has several limitations. It provides information only about the subpixel level inhomogeneity but has little sensitivity to some other 3‐D effects such as illuminating or shadowing by surrounding pixels. It provides information only in the visible and near infrared spectral regions but not in other wavelengths such as 2.1 µm or 3.7 µm. In addition, the H value for very thin clouds can be small, in which case it is no longer a useful index for quality or uncertainty because other factors, such as instrument noise and radiative transfer modeling errors become more prominent.

Retrieval failure rates as a function of H are shown in Figure 5 for both (a) overcast and (b) PCL MLP cloud pixels, as are the distributions of H (black lines) for each population. For overcast pixels the retrieval failure rates, primarily due to r too large failures (dash‐dotted lines), of the 2.1 µm (red lines) and 3.7 µm (blue lines) remain relatively flat when the value of H is between 0.01 and 0.3, and begin to increase toward both ends of H . In particular, when H is close to unity, the 2.1 µm and 3.7 µm failure rates are as large as about 40% and 20%, respectively. As seen in Figure 5b, the “U” shape variation of failure rate with H becomes more obvious for PCL pixels. For both overcast and PCL pixels, the enhanced failure rate at large H suggests that the plane‐parallel r bias is a contributing factor. It is not clear, however, why failure rate also increases at small H , especially in the case of PCL pixels. One hypothesis is that the very small H values (<0.01 for overcast pixels and <0.3 for PCL pixels) correspond to homogeneous but thin clouds, in which case failure rates are enhanced simply due to the low signal to noise ratio as mentioned above. The fact that all three types of failures increase at the small H end seems to support this hypothesis. Nevertheless, because the number of pixels with extremely small H is relatively small, we leave a deeper analysis for future research. We note, however, that Di Girolamo et al. [2010] also observed a small increase in the frequency of angular inconsistency between Multiangle Imaging Spectroradiometer‐observed radiance and that calculated using the MODIS‐retrieved cloud properties for very low values of H . They traced this to the occurrence of smoke overlying cloud, a scenario that invalidates the assumption used in the MODIS cloud property retrieval algorithm. Combining the overcast and PCL populations (not shown here) yields relationships similar to the overcast‐only results in Figure 5a because the overcast pixel population is substantially larger than the PCL population.

Figure 5

Global MODIS retrieval failure rates (colored lines) for MLP clouds as a function of subpixel inhomogeneity index H (black line) for a) overcast and (b) PCL pixels. Red and blue colors correspond to the 2.1 µm and 3.7 µm retrievals, respectively; note that τ retrieval failures are shown as a single green line since these failure rates are almost identical for both retrievals. Solid red or blue lines denote combined failure rates for all failure types. Dashed red or blue lines denote r too small failures, and dash‐dotted red or blue lines denote r too large failures.

Overall, the results in Figure 5 suggest that MOD06 r too large failures have a significant dependence on subpixel cloud inhomogeneity when H is large enough (H  > 0.3). Although it is difficult to definitively attribute this dependence to the plane‐parallel r bias, it is likely playing an important role as it provides a reasonable explanation for not only the increase of r too large failure rates with H but also the fact that the 2.1 µm retrievals are more prone to such failures than 3.7 µm. Nonetheless, the behavior of the PCL failure rates suggests that subpixel cloud inhomogeneity is not the only factor affecting MOD06 retrieval failures.

Sun‐Satellite Viewing Geometry

Some special Sun‐satellite viewing geometries pose challenges for passive cloud remote sensing methods. For instance, enhanced ocean surface reflection at sunglint angles often makes it difficult to distinguish clouds from the background surface signal. Furthermore, large viewing and/or solar zenith angles, associated with larger MODIS pixel sizes and the “bowtie effect,” longer atmospheric pathlength, and enhanced 3‐D radiative transfer effects, are known to cause large biases and uncertainties in cloud property retrievals [Loeb and Davies, 1996; Várnai and Marshak, 2002; Di Girolamo et al., 2010; Maddux et al., 2010; Grosvenor and Wood, 2014]. Lastly, the sensitivity to r at SWIR/MWIR channels is reduced at special scattering angles, such as the cloudbow and glory (see Figure 12 and related discussion).

Figure 12

(top row) Bulk scattering phase functions (black lines) for cloud droplets with different r and (bottom row) the corresponding phase function separation index (red line; see text for definition), for the (a) 2.1 µm and (b) 3.7 µm MODIS channels. Also shown are example LUTs near the cloudbow scattering angle illustrating the reduced sensitivity to r of the (c) 2.1 µm and (d) 3.7 µm retrievals (see text for details).

Figure 6 shows an example granule in which some failed retrievals coincide with sunglint that is clearly visible in the true color red, green, and blue (RGB) in Figure 6a. The sunglint angle θ glint, defined as the angle deviating from the specular reflection angle [Ackerman et al., 1998], is shown in Figure 6b. Both the 2.1 µm (Figure 6c) and 3.7 µm (Figure 6d) retrievals have a similar number of τ failed retrievals (pixels shaded yellow) over the sunglint region. These failures are probably due to errors in the cloud mask (i.e., “false cloudy” sunglint pixels) or inadequate representation of the ocean surface bidirectional reflectance distribution function by the Cox‐Munk approximation [Cox and Munk, 1954] used in the LUT forward radiative transfer calculations.

Figure 6

An Aqua MODIS granule, acquired on 2 May 2007, off the coast of the Baja California peninsula, showing τ retrieval failures due to sunglint. (a) RGB image, (b) 2.1 µm retrieval failures, and c) 3.7μm retrieval failures.

To investigate the statistical dependence of retrieval failure on sunglint condition, retrieval failure rate as a function of sunglint angle θ glint is plotted in Figure 7. Evidently, the failure rates for both overcast and PCL pixels are enhanced at small sunglint angles (θ glint < 20∘) where the Sun‐satellite viewing geometry is close to specular reflection. In addition, two features are worth attention. First, as expected the failures near specular reflection angles are largely τ retrieval failures. Second, the failure rates for PCL pixels over sunglint are almost twice as high as those for overcast pixels, a result consistent with the global failure rates in Figures 3 and 5.

Figure 7

Global MODIS retrieval failure rates for MLP clouds as a function of the sunglint angle θ glint for a) overcast pixels and b) PCL pixels. Color and line styles are the same as Figure 5.

Figure 8 shows retrieval failure rates as a function of viewing zenith angle (θ ). Note that the overcast (Figure 8a) and PCL (Figure 8b) 2.1 µm retrievals (red lines) have increasing failure rates with increasing θ when θ  > 50°. In contrast, the 3.7 µm failure rate has much weaker dependence on θ . The failed 2.1 µm retrievals at large θ are mainly r too large failures, a result consistent with recent studies that found MODIS r systematically increases with increasing θ from nadir toward the edge of scan [Horváth et al., 2014]. The apparent overestimation of r at large θ has been attributed to the combination of longer pathlengths that enhance atmospheric correction errors coupled with enhanced 3‐D radiative effects [Várnai and Marshak, 2007; Kato and Marshak, 2009; Liang and Di Girolamo, 2013; Horváth et al., 2014].

Figure 8

Global MODIS retrieval failure rates for MLP clouds as a function of the viewing zenith angle θ for (a) overcast pixels and (b) PCL pixels. Color and line styles are the same as Figure 5.

Figure 9 shows the MODIS retrieval failure rates as a function of solar zenith angle (θ 0). The failure rates for overcast pixels (Figure 9a) have an overall weak dependence on θ 0. In contrast, the failure rates of both spectral combinations for PCL pixels show significant and complicated variations with θ 0 in Figure 9b. Several interesting features are notable. First, both spectral combinations for PCL pixels have increasing failure rate with increasing θ 0 when θ 0 is larger than about 65°. This increase of failure rate is accompanied by an increase of τ failures and r too small failures and a decrease of r too large failures. The increase of τ failures is consistent with several previous studies that found MODIS τ retrievals show a systematic shift toward larger values with increasing solar zenith angle [Loeb and Davies, 1996; Seethala and Horváth, 2010; Grosvenor and Wood, 2014]. Likewise, the increase of r too small failures is consistent with a recent study by [Grosvenor and Wood, 2014] that found MODIS tends to underestimate r when θ 0 is large. In addition to the peak at large θ 0, the 2.1 µm PCL retrievals also have enhanced failure rates at θ 0 around 55°, an angle that is more likely to be associated with the special scattering angle (Θ) around the cloudbow (Θ ~135°). As shown later, the 2.1 µm retrieval failure rate is substantially enhanced around the cloudbow (see Figure 11). The 3.7 µm PCL retrievals have enhanced failure rates at θ 0 ~20° in addition to the peak at large θ 0; this peak is because a large fraction of sunglint pixels have θ 0 around 20°.

Figure 9

Global MODIS retrieval failure rates for MLP clouds as a function of the solar zenith angle θ 0 for (a) overcast pixels and (b) PCL pixels. Note that MOD06 τ and r retrievals require θ 0 < 81.4° [ Hubanks et al., 2008 ]. Color and line styles are the same as that in Figure 5.

Figure 11

Global MODIS retrieval failure rates for MLP clouds as a function of the scattering angle Θ for (a) overcast and (b) PCL pixels. Color and line styles are the same as Figure 5.

The effects of special scattering angles of water droplets, such as the cloudbow and glory, on retrieval failures can be seen more clearly in two granule case studies in Figure 10. The RGB image of the first granule, acquired by Aqua MODIS over the tropical southeast Pacific on 26 May 2007, is shown in Figure 10a. This granule is mostly covered by low‐level cumulus clouds. As shown in Figure 10b, a remarkably large fraction of 2.1 µm retrievals failed for this granule and are mostly r too large failures. A closer look of Figure 10b reveals an interesting U shape region with enhanced failure rate in the east half of the granule. This region corresponds to the cloudbow, i.e., scattering angles between 135° and 140° (Figure 10c). The RGB of the second granule, acquired by Aqua MODIS on 18 May 2007, over the northeast Pacific stratocumulus cloud region off the coast of the Baja California peninsula, is shown in Figure 10d. A belt with enhanced 2.1 µm retrieval failures (Figure 10e), mainly r too small failures, is visible near the center of the image, and in this case corresponds to the glory scattering angles between 174° and 180° (Figure 10f).

Figure 10

Examples of MODIS retrieval failure at special scattering angles. (a) RGB image of an Aqua MODIS granule acquired on 26 May 2007, over the southeast tropical Pacific; (b) 2.1 µm retrieval failures; and (c) scattering angle, with black contours denoting the cloudbow region. (d) RGB image of an Aqua MODIS granule acquired on 18 May 2007, off the coast of the Baja California peninsula; (e) 2.1 µm retrieval failures; and (c) scattering angle, with a black contour denoting the glory region.

Figure 11 shows the global MODIS retrieval failure rates as a function of scattering angle for the backscattering range (120°≤ Θ ≤ 180°). The 2.1 µm failure rates have two peaks at cloudbow angles around 135° and 143°, and a third at the glory angle near 177°. At these special scattering angles, the overcast 2.1 µm retrieval failure rate is around 10%–20%, and the PCL retrieval failure rate is considerably higher, up to 80%. The 3.7 µm failure rate has only one broad peak at cloudbow angles around 145°, with a second at glory angles near 177°.

Why are the special scattering angles so challenging for MODIS retrievals? It is probably due to the behavior of the bulk scattering phase functions (P 11) of water droplets at these special angles. For instance, the 2.1 µm P 11 of different r (Figure 12a) collapse on each other near 133°, 142°, and 177°, scattering angles coinciding with the high failure rate regions in Figure 11. To quantify this behavior, we define a phase function separation index (PS index) as the ratio between the mean value and the standard deviation of P 11 at a given scattering angle over all r in the MODIS LUT. Defined in this way, increasing convergence of P 11 yields large PS values. The PS index for the 2.1 µm channel (Figure 12a) has three distinct peaks and the 3.7 µm channel (Figure 12b) has two, each coinciding with the failure rate peaks in Figure 11. These are important features, because when the P 11 of different r converge, the sensitivity of the 2.1 µm or 3.7 µm channels to r is reduced, especially for optically thin clouds whose reflectances are dominated by single‐scattering events. This is illustrated in Figures 12c and 12d. In Figure 12c, two LUTs for the 0.86–2.1 µm retrieval are plotted, one (black line) corresponding to a scattering angle Θ = 135° (solar zenith angle θ 0 = 45°, viewing zenith angle θ  = 0°, and relative azimuth angle ϕ = 0°) and the other (dashed line) to a scattering angle Θ = 115° (solar zenith angle θ 0 = 45°, viewing zenith angle θ  = 10°, and relative azimuth angle ϕ = 0°). Clearly, the LUT near the cloudbow is narrower. As such, if the retrieval uncertainties from measurement or radiative transfer modeling are independent of scattering angle, then the narrower LUT near cloudbow makes retrievals more prone to failure because the uncertainty can “push” the otherwise successful observation out of the LUT more easily. Furthermore, if the uncertainty is random, both r too large and too small failures will be enhanced over the cloudbow angles and is indeed the case in Figure 11. The 0.86–3.7 µm LUT (Figure 12d) is similarly affected, though the 3.7 µm channel has a different cloudbow angle (Θ ~ 145°) than 2.1 µm.

Uncertainties due to Cloud Mask, Phase, and Overlap

The MODIS r and τ retrievals are dependent on successful cloud masking and cloud thermodynamic phase determination, and errors in these products are expected to translate into uncertainties and potential failures in r and τ retrievals. In addition, overlapping clouds may also be a potential reason for retrieval failure since the MOD06 LUTs assume single‐layer clouds only. In this section, we use the cloud products from CALIOP to assess errors in the MOD35 cloud mask and MOD06 cloud phase retrieval, to detect the occurrence of overlapping clouds, and to investigate whether and to what extent these factors affect MODIS retrieval failure rates. Note CALIOP only samples a very small subset of Aqua MODIS observations, thus sampling MODIS pixels only along the CALIOP track will exclude pixels in sunglint or with large sensor zenith angles. We have confirmed, however, that excluding pixels with sunglint angle <20° or θ  > 50° from the Aqua MODIS marine liquid phase cloud population yields very similar failure rates (13.83% for 2.1 µm and 8.27% for 3.7 µm) with those pixels along the CALIPSO track only (failure rates of 12.77% for 2.1 µm and 8.56% for 3.7 µm). This gives confidence that a failure rate analysis only along the CALIPSO track is representative of those off‐track pixels away from sunglint and large θ .

It should be noted that MODIS, CALIOP, and CloudSat cloud retrievals for the same cloud scene often differ from one other for many reasons. For example, the differences can be the result of different horizontal resolution as mentioned in section 2. In addition, MODIS, CALIOP, and CloudSat also have different sensitivities to the vertical structure of clouds. The CALIOP retrieval is only sensitive to cloud top. Cloud reflectance in the MODIS 0.86 µm channel is sensitive to the column‐integrated optical thickness, while reflectance in the 2.1 and 3.7 µm channels is sensitive only to the upper portion of the cloud [Platnick, 2000]. CloudSat can resolve the vertical structures of some MLP clouds but often misses many low and thin clouds. As a result, comparing cloud retrievals from MODIS, CALIOP, and CloudSat and understanding the differences are difficult tasks and beyond the scope of this paper. Here we simply use CALIOP and CloudSat as independent sources of information for classifying MODIS MLP retrievals.

MODIS MLP cloud pixels along the CALIPSO track are progressively categorized by potential retrieval failure causes using the CALIOP cloud layer products. A flowchart of this process is given in Figure 13. First, the cloudy MODIS pixels for which CALIOP 1 km layer product did not also detect a cloud are identified and set aside. These apparent MODIS cloud mask failures may in fact be clear‐sky scenes misidentified as cloud by the MOD35 cloud mask, or alternatively may be only partially cloud covered, with the portion viewed by CALIOP being clear. It might also be caused by the aforementioned parallax effect in viewing the same cloud with the two instruments at slightly offset times. Next, those pixels for which CALIOP found more than one cloud layer, i.e., overlapping cloud pixels, are identified and set aside, followed by those for which the CALIOP cloud phase retrieval is either ice phase or undetermined (i.e., phase disagreement). Finally, we check the 333 m resolution CALIOP cloud layer product. If one or more 333 m CALIOP top layers in a collocated 1 km MODIS pixel are not liquid phase cloud (e.g., clear‐sky or ice phase layer), then the MODIS pixel is identified as “broken cloud.” The remaining pixels are ostensibly “single‐layer” MLP clouds according to both MODIS and CALIOP. The single‐layer MLP clouds account for about 31% of the total along‐track failed MLP cloud retrieval population for the 2.1 µm and about 35% for 3.7 µm. We note that our single‐layer MLP clouds might still include the situation where a thick upper layer cloud opaque to CALIOP overlaps low‐level clouds.

Figure 13

Flowchart detailing the retrieval failure analysis methodology using collocated CALIOP and CloudSat observations.

The number of failed retrievals and failure rates for each category, with colors denoting contributions by each failure type, are shown in Figures 14a and 14c for the 2.1 µm retrievals, and Figures 14b and 14d for 3.7 µm retrievals, respectively. Note that no screening for overcast or PCL pixels is applied here. Predictably, the pixels that MODIS and CALIOP both agree are single‐layer MLP clouds have 2.1 µm and 3.7 µm failure rates that are significantly lower than the respective mean failure rates for all collocated MLP cloud pixels. Moreover, the pixels with cloud mask disagreement (roughly 8% of the total along‐track MLP cloud population) have rather large 2.1 µm and 3.7 µm failure rates, about 31% and 18%, respectively, both twice as large as the corresponding mean failure rates for all collocated MLP cloud pixels. A similar story is found in the “broken clouds” category, which has failure rates lower than the “cloud masking problem” and “phase problem” category, but higher than the single‐layer MLP category. These large failure rates are again a predictable result regardless of whether these MODIS pixels are in fact clear in which case retrieval failures are desirable, or are partially cloud covered, in which case they are likely identified as PCL pixels and, as shown previously, have enhanced failure rates.

Figure 14

MODIS retrieval failure pixel counts and rates for MLP clouds along the CALIPSO track. Subcategories are determined using the collocated CALIOP 1 km cloud layer product.

Similar to the cloud mask disagreement pixels, those with phase disagreement (about 5% of the total along‐track MLP cloud population) have 2.1 µm and 3.7 µm failure rates that are substantially larger than the respective mean failure rates for all collocated MLP cloud pixels. This result is not surprising given the dissimilar absorption properties of liquid water and ice in the SWIR and MWIR portion of the spectrum, differences that are in fact exploited within the MOD06 phase discrimination algorithm. Somewhat surprising, however, are the overlapping cloud pixels (about 14% of the total along‐track MLP population), which have 2.1 µm and 3.7 µm failure rates comparable to their respective mean failure rates for all collocated MLP cloud pixels. This result is nevertheless reasonable given the sensitivity of CALIOP for multilayer cloud detection. Because the lidar signal attenuates when τ is roughly 3 or larger, when CALIOP detects overlapping clouds the overlying cloud is likely to be thin cirrus. Thus, the fact these pixels are identified as liquid phase by the MOD06 phase discrimination algorithm indicates these scenes are radiatively similar to single‐layer liquid phase clouds, and identifying them as such may not in fact be problematic.

The single‐layer MLP population in Figure 14 is further filtered in Figure 15 to screen out inhomogeneous pixels, and pixels observed at cloudbow or glory or large solar zenith angles. These conditions account for 59% of all failed single‐layer MLP 2.1 µm (about 51% for 3.7 µm) cloud retrievals along the CALIPSO track.

Figure 15

Analysis of the potential failure reasons for the “single‐layer MLP cloud” population in Figure 14. Note that a pixel can be counted twice in failure reason classification. For example, a pixel can be inhomogeneous with H  > 0.3 and at the same time observed at the cloudbow angle.

Unexplained Failures

Thus far, the focus has mainly been on those failed retrievals that are potentially due to algorithm and retrieval artifacts. However, not all failed retrievals can be interpreted as retrieval artifacts analyzed above. Combining all the failure reasons in Figures 14 and 15 account for 88.2% of all failed 2.1 µm (84.1% for 3.7 µm) MODIS MLP cloud retrievals along the CALIPSO track (i.e., far left bar populations in Figure 14). The remaining (i.e., far right bar populations in Figure 15) does not have an apparent reason for failure (referred to as “unexplained failure” hereafter). These pixels are single‐layer water cloud pixels that are relatively homogenous and are not in special Sun‐satellite viewing geometries and have failures almost exclusively in the r too small or r too large categories. This raises an obvious question on whether the r range of the current MODIS LUT accurately reflects the natural variability of marine liquid phase clouds, the implication being that the extreme values of r associated with these pixels reflect the real cloud microphysics.

To achieve a better understanding of these unexplained failures, MODIS pixels along the CALIPSO track are categorized using collocated CloudSat maximum column radar reflectivity (dBZmax) observations. Specifically, these categories are “CloudSat‐undetected” clouds that cannot be seen by CloudSat because they either fall below the sensitivity of CloudSat CPR (−30 dBZ) or are within about 1 km to the surface and are therefore indistinguishable from surface clutter [Marchand et al., 2008]; nonprecipitating clouds having −30 < dBZmax < −15; and precipitating clouds having dBZmax > −15 dBZ. Figure 16 shows the 2.1 µm and 3.7 µm failed pixel counts (a and b, respectively) and failure rates (c and d, respectively) of the unexplained failure population from Figure 15 (replotted here as the far left bar) and the contribution of each CloudSat‐derived category (right three bars). Note that for the failure rates, the denominator in each category is the number of single‐layer liquid phase cloud pixels that are relatively homogenous, off the special Sun‐satellite viewing geometries, and have the required range of dBZmax. About 19%, 6%, and 75% of the 2.1 µm unexplained failure pixels along the CALIPSO/CloudSat track and 57%, 9%, and 35% of the 3.7 µm unexplained failure pixels are in the CloudSat‐undetected, “nonprecipitating” and “precipitating” categories, respectively. It is interesting to observe that almost all r too small failures are located in the CloudSat‐undetected category.

Figure 16

Analysis of the “unexplained retrieval failures” in Figure 15 using collocated CloudSat data.

On one hand, the results in Figure 16 seem to suggest that some failed MODIS retrievals might actually be connected to real cloud microphysics. The fact that r too small failures are almost exclusively in CloudSat‐undetected category agrees with the intuition that extremely small r values, if they were not retrieval artifacts, should most likely occur in those thin and low stratocumulus clouds that are difficult for CloudSat to see. It is also interesting to note that the precipitating category has significantly higher failure rate, almost exclusively r too large failures. This result is physically consistent with the expectation that precipitating MLP clouds have larger r , and are therefore more prone to r too large failures. These clouds either have a r that is indeed outside of the [4μm, 30μm] range or their reflectances are very close to the boundaries of the MODIS LUT and are pushed outside by random retrieval uncertainties. As discussed earlier, if these extreme values of r are proven to be real, retrieval failures would lead to sampling bias. In such case, it is reasonable to extend the LUT, in particular the 2.1 µm channel LUT which has the potential to retrieve r larger than 30 µm due to weaker absorption [Zhang, 2013].

On the other hand, we cannot completely rule out the possibility that the results in Figure 16 are still due to retrieval artifacts that have some correlation with CloudSat observations. For example, some studies suggest that drizzle processes can lead to open cellular structures in warm clouds and thereby increase cloud inhomogeneity [Wang and Feingold, 2009; Feingold et al., 2010]. Therefore, most r too large failures appearing in the precipitation category in Figure 16 might be due to the correlation between precipitation and cloud inhomogeneity. Note that, although we have screened out the H  > 0.3 pixels here, this filtering using H cannot identify pixels affected by other 3‐D effects, such as illuminating and shadowing by surrounding pixels. In addition, as mentioned earlier MODIS and CloudSat are sensitive to different parts of the cloud. The MODIS r retrieval is most sensitive to cloud top, while the maximum CloudSat radar reflectivity is often from cloud base. The microphysics connection between cloud top and cloud base in a precipitating cloud is complicated and not well understood. Although it is difficult to draw a firm conclusion, the analysis in Figure 16 reveals some most interesting pixels in failed retrievals that warrant future research.

Summary and Discussion

In this study, we carried out a comprehensive analysis of the failed cloud property retrievals in the C6 MODIS cloud product (MOD06) for marine liquid phase (MLP) clouds. The main findings from this study can be summarized as follows:

MODIS retrieval failure rates for MLP clouds have a strong dependence on the spectral combination used for retrieval (e.g., 0.86–2.1 µm and 0.86–3.7 µm) and cloud morphology (i.e., overcast versus partially cloudy (PCL)). Combining all CSR categories (CSR = 0,1, and 3), the 2.1 µm and 3.7 µm spectral retrievals have an overall failure rate of about 16% and 10%, respectively. The PCL pixels (CSR = 1 and 3) have significantly higher failure rates and contribute more to the total failure population than the overcast (CSR = 0) pixels.

The majority of failed retrievals are r too large failures, accounting for about 84% and 63% of the failed 2.1 µm and 3.7 µm retrievals, respectively.

The geographical distribution of failure rates has a significant dependence on cloud regime; failure rates are lower over the coastal stratocumulus cloud regimes and higher over the broken trade wind cumulus cloud regimes over open oceans.

Enhanced retrieval failure rates are found when MLP clouds have high subpixel inhomogeneity (see section 4.1), are located at special Sun‐satellite viewing geometries such as sunglint, large viewing or solar zenith angles, or cloudbow and glory angles (see section 4.2), or are subject to cloud masking, overlapping cloud, and/or cloud phase retrieval issues (see section 4.3).

More than 84% of all failed MLP cloud retrievals along the CALIPSO track can be attributed to at least one or more potential reasons mentioned above. The remaining failures could be either caused by other retrieval artifacts that are not studied in this paper (e.g., 3‐D effects not captured by H ), or possibly the consequence of true cloud microphysics, i.e., r is in fact very small or very large.

These findings have several implications. First, most retrieval failures occur in the situations known to be challenging for passive cloud remote sensing. This indicates that the performance of the current MODIS cloud retrieval algorithm is not inconsistent with our understanding gathered from previous studies. On the other hand, because these failures are the result of the inherent limitations of passive cloud remote sensing techniques, it remains difficult, if not impossible, to completely eliminate them even with a better understanding of their causes. Second, as demonstrated in this study, complementary information from other instruments, in particular CALIOP and CloudSat, are useful for understanding MODIS retrieval failures. In fact, some recent studies have already begun to explore the possibilities of using observations from other sensors to correct MODIS retrievals [Lebsock and Su, 2014].

Several important questions remain unanswered in this study. What are the cloud properties of those failed retrievals? Are they fundamentally different from those of successful retrievals? Note that the failed retrievals can account for about 30%–40% of the total cloud population in some regions dominated by broken cumulus clouds (see Figure 3). Exclusion of these failed pixels in the MODIS product might yield sampling biases over these regions if the failed pixels have systematically different cloud properties from successful retrieval pixels. Such potential biases merit further investigation.

Tables S1–S4

Click here for additional data file.