Uncovering the Large-Scale Meteorology That Drives Continental, Shallow, Green Cumulus Through Supervised Classification.

One of the major sources of uncertainty in climate prediction results from the limitations in representing shallow cumulus (Cu) in models. Recently, a class of continental shallow convective Cu was shown to share distinct morphological properties and to emerge globally mostly over forests and vegetated areas, thus named greenCu. Using machine-learning supervised classification, we identify greenCu fields over three regions, from the tropics to mid- and higher-latitudes, and establish a novel satellite-based data set called greenCuDb, consisting of 1° × 1° sized, high-resolution MODIS images. Using greenCuDb in conjunction with ERA5 reanalysis data, we create greenCu composites for different regions and reveal that greenCu are driven by similar large-scale meteorological conditions, regardless of their geographical locations throughout the world's continents. These conditions include distinct profiles of temperature, humidity and large-scale vertical velocity. The boundary layer is anomalously warm and moderately humid, and is accompanied by a strong large-scale subsidence in the free troposphere.

Introduction

Shallow convective cumulus (Cu) clouds, often referred to as fair‐weather clouds, are ubiquitous over the world's oceans and continents (Bony et al., 2004; Norris, 1998). These clouds impose a net cooling effect at the surface, since they reflect part of the incoming solar radiation while having far less influence on the outgoing longwave radiation (Boucher et al., 2013). Despite extensive research, shallow Cu still constitute an important factor in the uncertainty related to climate sensitivity and cloud feedback (Bony, 2005; Webb et al., 2006; Zelinka et al., 2020), with their space‐time organization and environmental conditions playing a potential major role regarding the latter (Bony et al., 2020; Nuijens & Siebesma, 2019; Vial et al., 2017).

There exists an extensive amount of literature on marine shallow Cu, that is, trade Cu, which are mostly confined to the tropical and subtropical oceans. In this marine environment, sea surface temperatures are relatively warm and stable, and moderate trade winds and large‐scale subsidence in the free troposphere (FT) prevail (Stevens et al., 2016). These conditions over the ocean exert a relatively weak and steady forcing which promotes shallow convection. In contrast, continental shallow Cu, are by far less studied. These clouds prevail in much more diverse locations over the world's continents. Continental shallow Cu experience a stronger and time‐dependent forcing stemming from the more significant diurnal cycle over land. They form at late morning, peak in the early afternoon, and dissipate before sunset, coinciding with the diurnal cycle of surface fluxes and convective boundary layer (CBL) development (Berg & Kassianov, 2008; Lenderink et al., 2004). Recently, a large subset of continental shallow Cu was shown to look similar, regardless of the clouds' locations around the world, and to share many cloud field properties, for example, cloud size distribution, cloud fraction (CF), organization patterns, and their tendency to form over forested and vegetated regions, thus termed greenCu (Dror et al. (2020), hereafter D2020). Despite their small sizes (∼1 km, Lamer and Kollias (2015)) and short lifetimes (∼30 min, Jiang et al. (2006)), greenCu form highly organized mesoscale‐sized patterns that sustain throughout the day (Dror, Chekroun, et al., 2021). Stable on one hand, these clouds show high‐sensitivity to local conditions such as land‐cover type and topography (Da Silva et al., 2011; Rabin et al., 1990), and also to mesoscale conditions, as shown for example, for the presence of smoke over the Amazon (Koren et al., 2004).

Yet, an understanding of the large‐scale meteorological conditions that allow greenCu to prevail over a wide variety of geographical environments, from the tropics through mid‐ and higher‐latitudes, is still missing.

To address this question, by exploiting machine‐learning (ML) classification tools, we are able to probe over 450,000 high‐resolution satellite images, over different land areas (the Amazon, central USA and Eastern Europe) and climatic conditions, spanning a period of 10 years. We produce a novel data set of ∼90,000 greenCu images, named greenCuDb (Dror, Silverman, et al., 2021) extracted during the northern hemisphere (NH) summer (June–July–August [JJA]), which is the dry season in the tropics, when shallow convection prevails. To train the neural‐network (NN) model, we benefit from the visual identification of greenCu patterns operated in D2020 for about 12,000 labeled cloud field images, and analyzed over a much shorter 2‐month time‐period than in the present work. As a main result of this study, the inspection of the meteorological and environmental conditions from ERA5 reanalysis (Hersbach et al., 2020) of the large greenCuDb data set reveals that the greenCu fields share very similar characteristics irrespectively of their geographical location.

Cloud Fields Classification

The need to obtain large data sets of high‐resolution, labeled cloud images is a key initial step in building‐up knowledge about the different clouds' properties. With the recent advances of ML methods for image analysis and classification (e.g., Ker et al. (2017); Kremer et al. (2017)) and dedicated platforms for the underlying algorithms, such clouds' data sets have been produced either from ground (Dev et al., 2015; J. Zhang et al., 2018) or satellite observations. Thus, various ML‐labeled satellite data sets have been obtained for different purposes: CUMULO (Zantedeschi et al., 2019) for learning cloud classes; Shallow Cloud (Rasp et al., 2020) to explore mesoscale organization of shallow clouds in the trades; CloudCast (Nielsen et al., 2021) for clouds' forecasting, and LSCIDMR (Bai et al., 2021) to infer weather systems from clouds. Each of these data sets differ in the clouds properties and goals they are after, and thus differ in their spatio‐temporal resolutions, the spectral bands of the images, the labels they annotate, and the area and time periods they span. The present study prolongs these recent efforts for clouds classification by producing a novel ML satellite data set, greenCuDb, which targets specifically shallow, organized convection over land, from high‐resolution MODIS images.

Training Data Set

The training stage of a ML model for supervised classification heavily relies on the availability of expertly labeled data, which in turn plays a key role to the model's performance. However, such labeled data sets are not always available. Here we benefit from such a data set to train our NN model, namely, the visually‐labeled images presented in D2020, based on high‐resolution (500 m) MODIS Aqua true‐color (RGB) images. The Aqua satellite crosses the equator at ∼13:30 local solar time, an ideal time to capture the peak of the evolution of continental shallow convection, since the surface fluxes and many of the CBL processes are maximal at that time (Brown et al., 2002; Y. Zhang & Klein, 2013). The images were taken over three 14° × 14° continental regions, which represent different climatic environments, including the Amazon in the tropics (17°S−3°S and 63°W − 49°W), through the USA in the mid‐latitudes (30°N−44°N and 96°W−82°W), to Europe in higher‐latitudes (46°N−60°N and 27°E−41°E). All regions feature shallow convection during JJA, and constitute greenCu hotspots. The 14° × 14° sized images were divided to smaller 1° × 1° domains, each comprised of 228 × 228 × 3 pixels, and visually classified to four different classes determined based on human expertise: (a) sparse (no clouds to a few greenCu), (b) greenCu (organized shallow Cu), (c) transition from shallow to deep convection (deeper and more clustered greenCu, hereafter transition), and (d) deep convective (large clouds that cover most or all of the domain, hereafter deep). Figure 1 shows a classic 1° × 1° sized image of each class. Note that other classes of clouds, such as shallow stratiform or cirrus, are not common over these regions during this season. In case of images that contain stratus, visible cirrus or smoke, the model is designed such that they will be labeled as deep convection. See for example, the low CFs shown for the deep class (especially over the Amazon), stemming from smokey images (Figure 1). The visual classification was performed over two months, July–August 2008, and resulted into a total of 12,146 labeled cloud field images (D2020): 2,762 sparse, 2,142 greenCu, 2,446 transition, and 4,796 deep convective images.

Figure 1

Model predictions: boxplot of cloud fraction (CF) per class and region (Amazon, USA, and Eastern Europe). In each box, the circle marks the median, the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers. Dashed horizontal lines indicate the mean 25th and 75th percentiles of greenCu CF used as lower and upper thresholds to filter the greenCu post‐processing. Bottom row shows, for each class, a representative 1° × 1° image.

Classification Model

The ResNet‐34 model (He et al., 2016b) that uses 34 layers of convolutional NN, widely used for image classification purposes (He et al., 2016a), is trained on the data set obtained from human labeling in D2020 to perform classification over a larger data set of satellite images. The model encoder applies a set of convolution operations (Conv2D) and nonlinear activation functions (ReLU) which are adjusted during the training to detect the meaningful features for a correct classification. For this task, the ResNet model implementation from the fastai python package (Howard & Gugger, 2020) is used with kernel size of 3 × 3 on images of 228 × 228 × 3 pixels, initialized with random weights and biases. To balance the data, the training data set is given by 1,500 randomly and uniformly selected labeled images within each class, out of which 20% are kept for validation. The trained model predicts greenCu with 90.0% accuracy (acc), and the recall (R), precision (P) and F1‐measure (F1) are 90.0%, 73.6% and 81.1%, respectively (see Supporting Information S1 for more details), which is suitable for the purposes of this study. For comparison, when the model was trained with the full data set from D2020: acc = 91.5%, R = 92.4%, P = 68.5% and F1 = 78.7%. To assess the model's predictions we compare morphological properties of the model‐labeled and the visually‐labeled images. The two datasets show good agreement and the four classes are consistent in both (Figure S1 in Supporting Information S1). The shift between the different classes is not sharp and can be ambiguous. For example, a sparse field can essentially be a forming or dissipating greenCu field, and a transition field may be deeper and/or clustered greenCu (D2020). Also, some images may have a mixed distribution of different classes. Therefore, confusions between (especially) the transition to greenCu are expected (Figure S2 in Supporting Information S1) and accounted for in the filtering steps described in Section 2.3.

The GreenCuDb Data Set

The classification task resulted in a total of 455,252 images, of which 21.65% (98,582) were labeled as sparse, 18.51% (84,282) as greenCu, 20.18% (91,851) as transition and 39.66% (180,537) as deep. Note that even‐though the model was trained with a balanced data set, it is still able to reproduce the imbalance that exists between the classes of the visual classification (e.g., the deep class contains about twice the amount of images as each of the other classes, both in the model‐labeled and in the visual data sets). To further improve our confidence in the model's attribution of a given cloud field to a specific class, we calculate the CF for the classified images using a cloud mask constructed specifically to detect greenCu (D2020). Figure 1 shows a monotonic increase in CF for the four classes. This increase is expected since the different classes essentially represent a continuous shift from almost clear sky (sparse) to an overcast state (deep). While there exist a natural spread in the distributions of CF for both classes and regions, the medians of each distribution are similar for all regions within the same class, and are well‐separated from the other classes, proving that the model successfully discriminates between the classes (see Figure S1 in Supporting Information S1 for other morphological properties).

Focusing on the greenCu, Figures 2a–2c show examples of such fields for each region. To avoid model false‐positive predictions (i.e., images wrongfully labeled as greenCu), we use the mean of the 25th (0.136) and 75th (0.291) percentiles of greenCu CF as the upper and lower limits to clean the greenCu data set (green shading and dotted gray lines in Figure 1). While this filtering affects greenCu CF distributions, it has a negligible effect on the distributions of other morphological and meteorological variables (Figures S1 and S3 in Supporting Information S1). Thanks to this strategy, we eliminate approximately 50% of the images but increase the reliability of the data set and thus, of the following analyses.

Figure 2

The greenCu share similar organizational patterns.Examples of 1° × 1° greenCu fields over the (a) Amazon, (b) USA, and (c) Eastern Europe. Observed Nearest Neighbor Cumulative Density Function (NNCDF) against Poisson NNCDF, and a boxplot of I org for the (d) Amazon, (e) USA, and (f) Eastern Europe.

Finally, the new greenCuDb combines high‐resolution imagery from three regions worldwide, during 10 JJAs (2003–2012). It consists 42,128 images, of which 15,727 are from the Amazon, 14,300 from the USA and 12,101 from Eastern Europe, scaling‐up the human analysis of D2020 by a factor of ∼20.

As evidenced from the images shown in Figures 2a–2c, greenCu throughout the world appear strikingly similar. Furthermore, D2020 have shown that beyond this similarity, these fields share many common properties, for example, their size distributions, the number of clouds in the field and the distances between them, and CF and cloud top height (CTH). However, it is the distinct and unique organization of greenCu that distinguishes these fields from the other cloud classes. Indeed, greenCu tend to organize into a regular pattern, taking often the shape of cloud streets. To quantify and characterize greenCu organization, we use the commonly applied organization index (I org; Weger et al. (1992); Tobin et al. (2012)), which compares the cloud field Nearest Neighbor Cumulative Density Function (NNCDF) to that of a randomly distributed cloud field (Poisson NNCDF, given by the Weibull distribution). Figures 2d–2f show the mean observed NNCDF against Poisson NNCDF for the greenCu in the different regions. And a boxplot of I org (the area under the NNCDF curve) for each region is shown to the right (see boxplot of I org for all other classes in Figure S1 in Supporting Information S1). The mean NNCDF curves are similar for all regions and show that greenCu fields deviate from randomness toward a regular (grid‐like) organizational pattern, with medians of I org = 0.418, 0.426, 0.424 in the Amazon, USA and Eastern Europe, respectively. This standard metric to measure clouds' organization against a random field (Weger et al., 1992) provides further evidence regarding the ability of the trained ML model to predict greenCu patterns. For a more general discussion regarding probabilistic metrics used in atmosphere science see Bröcker and Smith (2007).

Dependence of GreenCu on the Large‐Scale Meteorological Conditions

Next, we use the novel greenCuDb to gain insight on the large‐scale meteorological conditions associated with greenCu formation, and to understand how similar cloud fields are formed in such different geographical environments. To do so, we create composites for each class by assigning to each image the corresponding ERA5 reanalysis hourly products (Hersbach et al., 2020). We examine the mean vertical profiles of potential temperature (θ), specific humidity (q), relative humidity (RH), and the large‐scale vertical velocity (ω) of the different classes, the greenCu composites of each region, and the normalized deviations of the greenCu composites from the corresponding JJA climatological mean (Figure 3). The standard deviations of the mean profiles, and the climatologies are shown in Figures S4 and S5 in Supporting Information S1, respectively. The composites of the different classes show consistent behavior in terms of all inspected atmospheric profiles. The different classes are controlled by the interplay between the temperature, humidity and the large‐scale vertical velocity profiles. Going from the sparse to the deep class, we smoothly shift from a warmer and drier atmosphere, dominated by subsidence, to a colder and moister atmosphere at which the subsidence dominates at higher altitudes (greenCu and transition) or is non‐existent (deep).

Figure 3

Large‐scale meteorological conditions associated with the different classes (upper‐row), greenCu composites per region (middle‐row), and greenCu normalized anomalies from the June–July–August climatological mean (lower‐row) in the Amazon (black), USA (blue), and Eastern Europe (red). Mean vertical profiles of θ, q, RH, and ω (column‐wise, from left to right, respectively). The standard deviations are shown in Figure S4 in Supporting Information S1.

Focusing on the greenCu composites, we compare their characteristic profiles throughout the different regions (Figures 3e–3h). Although the θ‐ and q‐values differ between the regions, the greenCu RH profiles are remarkably similar. In terms of the large‐scale vertical velocity, all greenCu composites feature weak updraft near the surface, overlaid by a large‐scale subsidence, but the absolute values and the heights at which ω changes its sign differ. In the Amazon, the large‐scale updraft as well as the overlaying subsidence share moderate values. In the USA and Eastern Europe, the updraft is weaker and confined into the lower CBL, near the surface, and turns into a weak to moderate subsidence throughout the CBL, to finally adjust to a strong subsidence in the FT (altitudes above 800 mb). The CBL depth (seen here as the height at which RH values sharply decrease) is shown to be greater for the Amazon than for the other regions, in agreement with D2020. The authors of D2020 showed that greenCu CTH in the Amazon is larger compared to the USA and Eastern Europe (Figure 5 in D2020).

Although the Amazon, USA, and Eastern Europe regions belong to different climate zones (Köppen, 1923), and generally exhibit different large‐scale circulations, the climatologies of the three regions reveal that during JJA these environments share stable large‐scale conditions, such as warm, humid CBLs (Figure S5 in Supporting Information S1). These stable conditions result in the Amazon from the prevalence of the South Atlantic Subtropical High (Nobre et al., 1998) as the Intertropical Convergence Zone reaches its northward displacement (∼10°) in JJA. In the USA and Eastern Europe, it is related to the weakening of the extratropical storm track over mid‐latitudes and its migration poleward during NH summer (Parker et al., 1989; Whittaker & Horn, 1981).

To gain insight on the key players discriminating each greenCu composite from the local climatology, we calculate the variable's normalized deviations from their corresponding JJA climatologies by subtracting the climatological mean and dividing by the climatological standard‐deviation (Figures 3i–3l). Since greenCu are more frequent over the Amazon during JJA (D2020), larger anomalies are expected in the USA and Eastern Europe. The anomalies indicate that the large‐scale conditions favoring greenCu formation are obtained by (a) a warmer CBL (near the surface) and FT compared to the climatologies (Figure 3i). (b) A drier profile (negative anomalies), especially near the surface and in the FT in the USA and Eastern Europe, and a moister CBL (positive anomalies), and drier FT in the Amazon (Figures 3j and 3k). And (c) an increase in the magnitude of the large‐scale subsidence relative to the climatologies, mostly pronounced in the FT (δω > 0 in all regions, becoming larger as going from the Amazon to the USA to Eastern Europe, Figure 3l). The drier FT and stronger subsidence (compared to the climatology) in all regions inhibit greenCu's development into deeper clouds such as those of the transition and/or deep class.

Summary and Discussion

By means of visual classification, continental, shallow clouds were shown to share many common properties: their sizes, CF, the patterns they form, and their preferred formation over forests and vegetated area, named greenCu (D2020). These results allow us to: (a) scale‐up the human analysis of D2020 by producing a novel machine‐learned data set of greenCu images, and this way (b) characterize the large‐scale meteorological conditions that drive greenCu and their very existence in such diverse environments throughout the world's continents.

We use morphological properties such as CF as metrics to evaluate the model's predictions and show that it is able to successfully discriminate between four consistent classes: sparse, greenCu, transition to deep convection, and deep convection (Figures 1 and S1 in Supporting Information S1). We then restrict greenCu CF, and establish the new greenCuDb. It contains 42,128 high‐resolution greenCu images over three regions from the tropics (Amazon), to mid‐latitude (USA), and higher‐latitudes (Eastern Europe), over a period of 10 seasons (JJAs, 2003–2012). One of greenCu's most prominent features is their distinct cloud field organization. It is shown here to be similar throughout the regions and to exhibit a regular (grid‐like) structure (Figure 2), in agreement with Dror et al. (2020), Dror, Chekroun, et al. (2021).

Using greenCuDb together with ERA5 reanalysis we show that greenCu profiles of potential temperature, specific humidity and RH and large‐scale vertical velocity are distinguished from those of the other classes, and that greenCu form under similar meteorological conditions regardless of their geographical location. These conditions include a distinct combination of temperature, humidity and large‐scale vertical velocity throughout the CBL and the FT: the CBL is anomalously warm and just moist enough to allow cloud formation. It is capped by large‐scale subsidence throughout the FT which inhibits the shallow clouds from transitioning to deep convection. A slightly drier profile with stronger subsidence will result in greenCu dissipation (i.e., sparse‐like conditions), and a colder, moister profile with weaker subsidence will result in greenCu clustering and deepening (i.e., transition/deep‐like conditions).

We believe that the findings of this study, by improving the detection and understanding of shallow Cu, can benefit short‐term forecasting over land and estimations of Earth's energy balance. Specifically, the insights regarding the large‐scale meteorological conditions propitious to greenCu formation are useful for the parameterization of such clouds in General Circulation Models as well as advance our understanding in low cloud feedback. The question of how such cloud fields and their feedback effects will respond to changes in the meteorological conditions within a warming climate, remains however open.

Supporting Information S1

Click here for additional data file.