The Illumination of Thunderclouds by Lightning: 2. The Effect of GLM Instrument Threshold on Detection and Clustering.

Lightning is measured from space using optical instruments that detect transient changes in the illumination of the cloud top. How much of the flash (if any) is recorded by the instrument depends on the instrument detection threshold. NOAA's Geostationary Lightning Mapper (GLM) employs a dynamic threshold that varies across the imaging array and changes over time. This causes flashes in certain regions and at night to be recorded in greater detail than other flashes, and threshold inconsistencies will impose biases on all levels of GLM data products. In this study, we quantify the impact of the varying GLM threshold on event/group/flash detection, flash clustering, and gridded product generation by imposing artificial thresholds on the event data taken from a thunderstorm with a low instrument threshold (∼0.7 fJ). We find that even modest increases in threshold severely impact event (60% loss by 2 fJ, 90% loss by 10 fJ) and group (25% loss by 2 fJ, 81% loss by 10 fJ) detection by suppressing faint illumination of the cloud top from weak sources and scattering. Flash detection is impacted less by threshold increases (4% loss by 2 fJ), but reductions are still significant at higher thresholds (35% loss by 10 fJ, or 44% if single-group flashes are removed). Undetected pulses cause individual flashes to be split and severely impact the construction of gridded products. All of these factors complicate the interpretation of GLM data, particularly when trended over time under a changing threshold.

Introduction

Cloud‐to‐Ground (CG) strokes detected by optical or Radio Frequency (RF) sensors are only a small part of the larger lightning “tree” that extends throughout the cloud. Lightning activity includes a variety of CG and in‐cloud phenomena that radiate across a vast range of energies and frequencies (Cummins & Murphy, 2009). What parts of the flash are resolved depends on the sensitivity of the instrument and the portion of the electromagnetic spectrum it measures. Both VHF‐band RF instruments and optical sensors are capable of mapping major portions of the lightning tree (Nag et al., 2015; Peterson et al., 2018; Rison et al., 1999) and detecting powerful emissions from CG strokes (Koshak, 2010; Light, Suszcynsky, & Jacobson, 2001). RF or optical sensors with low sensitivities may only detect the particularly‐energetic strokes or similarly‐powerful in‐cloud events (Jacobson & Light, 2012), while the most sensitive instruments will be able to map nearly the full extent of the lightning tree—which can cover hundreds of kilometers (Lang et al., 2017; M. Peterson et al., 2020; M. J. Peterson et al., 2020).

Optical sensors have the additional issue that the lightning emissions—regardless of power—will be significantly modified by absorption and scattering in the cloud medium between the source and satellite (Brunner & Bitzer, 2020; Koshak et al., 1994; Light, Suszcynsky, Kirkland, & Jacobson, 2001; Peterson, 2020b; Thomson & Krider, 1982). Cloud regions that are particularly opaque to lightning signals (either from large optical depths or a composition/geometry that favors diffuse reflection off cloud sides over transmission through the medium) can prevent the detection of even powerful optical lightning signals from space – which can be seen as anomalies in the radiance data (Peterson, 2020a). Opaque clouds lead to poor Detection Efficiencies (DEs) for instruments like the Geostationary Lightning Mapper (GLM: Goodman et al., 2013; Rudlosky et al., 2019) in certain types of storms with problematic precipitation structures (Bitzer, 2019; Rutledge et al., 2019; Said & Murphey, 2019; Thomas, 2019).

Attenuation of the lightning signals by the cloud medium is not an issue for RF detectors. Thus, optical lightning sensors like GLM have potentially more to gain from optimizing for sensitivity compared to other types of lightning instruments. Scattering is particularly problematic for pixelated optical detectors compared to photodiode detectors, as the dispersed signal may be divided between pixels. This likely contributes to discrepancies between instruments noted in van der Velde et al. (2020). Any improvements in sensitivity from lowering the detection threshold will allow the instrument to recover flashes that are obscured below optically thick clouds, while also resolving more of the lightning tree in all flashes. The primary concern in lowering the threshold is a dramatic increase in solar artifacts (Peterson, 2020c) and noise events. However, the potential benefits of such an optimized threshold for GLM's diverse collection of operational products have not been fully quantified.

The GLM threshold affects how much of the flash can be detected from space, and it also limits the amount of thundercloud illumination that is measured from orbit. Scattering interactions allow optical lightning sources to illuminate the cloud scene far beyond the physical dimensions lightning flash. The most powerful groups detected by the Lightning Imaging Sensor (LIS: Blakeslee et al., 2020; Christian et al., 2000) and GLM encompass 10,000+ km2 of the cloud top (Peterson et al., 2017)—and include large areas of lower clouds that can transmit low‐altitude lightning emissions or reflect high‐altitude lightning emissions to provide a “shortcut” path to the satellite compared to traversing the full optical depth of convective cloud (Peterson, 2019b; M. Peterson et al., 2020; M. J. Peterson et al., 2020). Much of this neighboring cloud illumination is sufficiently dim to quickly fall below the higher thresholds.

This is the second part of our thundercloud illumination study. In Part 1 (Peterson et al., 2021a), we examined how the positions and geometries of optical sources affected GLM measurements of cloud illumination. In Part 2, we shift our focus from the emitter to the GLM instrument and quantify the effects of the GLM threshold on event/group/flash detection, flash clustering, and gridded product generation. We take the Colombia thunderstorm from Part 1 that had a low GLM threshold of ∼0.7 fJ, impose artificial thresholds on the event data between 1 fJ and 10 fJ, and then recompute the GLM cluster features (Mach, 2020) and gridded products using only the events above the imposed threshold. Changes in lightning rates and flash characteristics are discussed, along with how these changes affect the products downstream.

Data and Methodology

Coincident observations by GLM and regional ground‐based Lightning Mapping Arrays (LMAs: Rison et al., 1999) from two thunderstorms of interested are described in Part 1. In Part 2, we will use only the GLM data that were collected from these thunderstorms. The following sections describe the GLM data and our approach for imposing artificial thresholds on the GLM products.

The GOES‐16 Geostationary Lightning Mapper

GLM is a staring imager that records cloud top illumination in a narrow spectral band around the 777.4 nm Oxygen emission line triplet at 500 frames per second (Goodman et al., 2013). Transient changes in cloud top illumination characteristic of lightning are detected by subtracting the estimated background brightness of a given pixel from the pixel energy integrated during a 2‐ms integration frame, and then comparing the remaining signal energy with the current local instrument threshold. If the recorded energy is greater than the threshold, the instrument will trigger and report an event at that pixel. These pixel events are then clustered into higher‐level features that describe lightning—including “groups” (which approximate lightning pulses) and flashes (Goodman et al., 2010). Groups are defined as clusters of contiguous events on the GLM imaging array during the same integration frame, while flashes are clusters of groups in close temporal (330 ms) and spatial (16.5 km) proximity to one another. The clustering algorithm has been found to not be very sensitive to changes in these clustering thresholds (Mach, 2020). This cluster data is then used to construct GLM gridded products such as Flash Extent Density (FED), Average Flash Area (AFA), and Total Optical Energy (TOE) (Bruning et al., 2019).

The GLM data produced and distributed by NOAA is also subject to limitations within the Lightning Cluster Filter Algorithm (LCFA: Goodman et al., 2010), which must run in an operational setting with strict latency requirements. To ensure timely data, the LCFA imposes arbitrary limits on group and flash clusters that prevent them from becoming too large or complex. These thresholds (101 events per group, 101 groups per flash, and a maximum flash duration of 3 s) are rather low compared to even LIS flashes (Peterson et al., 2017), and cause the most exceptional cases of lightning (i.e., M. Peterson et al., 2020; M. J. Peterson et al., 2020) to be artificially split into multiple pieces that are flagged as a “degraded” quality in the operational GLM data. To recover these flashes Peterson (2019a), Peterson (2019b) developed methods that can be applied in post‐processing to repair the flash cluster data and generate science‐quality GLM data. This data is available at Peterson (2021a). As in Part 1, we use the repaired data here rather than the operational LCFA data.

Approximating GLM Thresholds and Imposing Artificial Thresholds

While the threshold at each pixel is not specified for each GLM event, threshold values can be estimated from the lowest‐energy events reported by GLM (Figure 1a in Peterson & Lay, 2020). If GLM can detect events from a thunderstorm down to 1 fJ, then the threshold must be somewhere below (and probably close to) 1 fJ – otherwise dimmer events would be reported. These estimates allow us to examine relative differences in threshold from storm to storm and from region to region, and to trend thresholds over time. However, minimum event energy is not a perfect approximation to the threshold – especially at the pixel level. Radiative transfer across the cloud scene affects the energy distribution of recorded events. Cloud regions that do not produce lightning still can be illuminated by very bright lightning pulses from a nearby thunderstorm (i.e., Peterson et al., 2017). If these are the only optical pulses that generate GLM events in a particular pixel, then the minimum reported energy will be quite high compared to pixels in the nearby thunderstorm core. In the storm core, meanwhile, at least some of the lightning pulses will be rather dim, producing events near the actual threshold.

Figure 1

Timeseries of geostationary lightning mapper minimum event energy during a thunderstorm over (a) Colombia near the satellite subpoint where thresholds are generally low and (b) Colorado where thresholds are known to be relatively high. Minimum event energies are computed for every pixel, and the minimum (red), mean (dark blue) and maximum (light blue) values over each region are reported.

In the absence of an optical phenomenon that is known to significantly raise the local GLM threshold (an example being sustained illumination from solar glint: Peterson, 2020c) and locations along the borders between Real Time Event Processors (RTEPs) (Cummins, 2021), the actual threshold should not be substantially different between neighboring cloud regions that are subject to the same background illumination. Thus, the minimum event energies over a flash‐ or convective‐scale region are expected to be a better approximation for the nominal threshold throughout that region than the minimum energies reported at each pixel. We use the flash minimum event energy per flash as our threshold proxy in this study while recognizing that it is not a perfect representation of the local instrument background.

Artificial thresholds above the approximate local instrument threshold are applied to the GLM detections by collecting all the event data from the Colombia thunderstorm, removing detections that are less energetic than the chosen artificial threshold, and then reconstructing the derived GLM data products from the events that exceed the new threshold (for example, the meteorological imagery in Bruning et al., 2019). We consider artificial thresholds between 1 fJ and 10 fJ in this study, which correspond to typical GLM thresholds (Figure 1a in Peterson & Lay, 2020; Cummins, 2021) over the regions across the Americas where lightning is most common (i.e., Figure 1 in Rudlosky et al., 2019, Figure 5 in Peterson, 2019a).

Figure 5

Geostationary lightning mapper total optical energies (color contour) and progression of groups over time (line segments) from a long horizontal lightning flash under (a) no imposed threshold, (b) a 3 fJ threshold, (c) a 6 fJ threshold, and (d) a 9 fJ threshold. Flash sections that are split at higher thresholds are indicated as disconnected line segments with a unique color and listed index for each split flash.

The removal of dim events and groups under higher thresholds can cause single flashes to become split if the remaining groups/events no longer meet the LCFA clustering criteria. While higher thresholds reduce the number of flashes detected by GLM, flash splitting artificially increases the number of flashes that would be reported. As these split flashes do not represent physically distinct phenomena, we examine the effect of this splitting separately from the loss of GLM detections. Section 3.2 examines the consequences of event/group/flash losses under each threshold using the original cluster data that best captures the physical development of each flash. Then, Section 3.3 examines how flash splitting modifies these results by constructing new cluster data using the remaining events at each artificial threshold.

Results

GLM Thresholds in Colorado and Colombia Thunderstorms

GLM is known to have a relatively high threshold in parts of Colorado (Cummins, 2021). To test this assertion and quantify differences in threshold between the Colorado and Colombia thunderstorms, we construct timeseries of minimum GLM event energies in Figure 1 for the Colombia (Figure 1a) and Colorado (Figure 1b) thunderstorms. As in the timeseries in Figure 1 (Colombia) and Figure 3 (Colorado) in Part 1, times are listed relative to 00:00 UTC on the first day of the storm (11/1/2019 for Colorado and 7/1/2019 in Colorado). To permit direct comparisons with the full set of statistics presented in Part 1, we likewise only report thresholds within the LMA data domains (latitude/longitude boxes near the center of the LMA network) here. The red timeseries in Figure 1 show the minimum GLM event energy for any pixel within the LMA data domain during each 15‐min interval, and approximate the lowest threshold experienced by any flash in the region. The dark blue timeseries, meanwhile, average the minimum event energies across all pixels with lightning in the domain, and the light blue timeseries show the maximum value of minimum event energy in any pixel. Horizontal lines are drawn to show constant energies from 1‐5 fJ (solid) as well as 10 and 100 fJ (dashed).

Figure 3

Histograms of geostationary lightning mapper (a) group energy and (b) group area under each imposed threshold. Only groups whose maximum event energies exceed 10 fJ are considered.

We noted in Part 1 that there were periods of time during the Colorado storm where lightning activity was observed above the warmer clouds surrounding deep convection, but not in the thicker convective clouds, themselves. This indicates that the sample of lightning measured by GLM will be biased towards the brighter flashes and the minimum pixel energy in these peripheral regions is not expected to be the best representation of the actual GLM threshold. Throughout the timeseries in Figure 1b, the lowest event energies (red) range from <2 fJ to 5 fJ, while the maximum pixel values of minimum event energy reach 70 fJ—causing average pixel values to range from 5 to 10 fJ. The minimum event energy also varies according to time of day, with the red curve starting at 5 fJ in the late afternoon and largely decreasing to below 2 fJ after nightfall. Examining changes in flash count and Total Optical Energy (TOE) after imposing artificial thresholds between 1 fJ and 10 fJ indicates that the overall effective threshold for the Colorado thunderstorm was between 3 and 4 fJ, as this is the point where notable changes in the GLM detection totals begin to occur.

Thresholds in the 3–4 fJ range are still relatively high for GLM. The Colombia case (Figure 1a) is expected to have a low threshold due to its proximity to the satellite subpoint. Indeed, the lowest event energies (red curve) during the most active storm period (6–11 UTC) are universally below 1 fJ. As with the Colorado thunderstorm, certain pixels have greater minimum event energy values (light blue curve) up to 30 fJ, and these impact the overall domain mean (blue curve). However, the overall effective GLM threshold for the region is inferred to be <1 fJ, and probably close to the ∼0.7 fJ average for the red curve during this period.

The Effect of GLM Threshold on Event/Group/Flash Detection

Artificial thresholds of between 1 fJ and 10 fJ are imposed on the GLM event data from the Colombia thunderstorm to determine how many of the original flashes are removed by increasing the threshold. Figure 2 shows how event count (a), group count (b), flash count (c), and TOE (d) change under each imposed threshold. Solid lines include all of the GLM data, while the dashed lines do not consider data from single‐group flashes that are removed by the current version of the LCFA (Rudlosky & Virts, 2021). As minimum event energy values were close to 1 fJ in Figure 2e, there is little difference between no imposed threshold (0 fJ) and a 1 fJ threshold in any of the plots. However, increasing the threshold to 2 fJ severely impacts event detection (Figure 2a). By a 4 fJ threshold (comparable to the Colorado case), 70% of the original events have been eliminated, while 90% of events are missed under a 10 fJ threshold.

Figure 2

Geostationary lightning mapper (GLM) (a) event, (b) group, and (c) flash counts, and (d) total optical energies from the Colombia thunderstorm under artificial thresholds ranging from 0 fJ (original instrument thresholds) to 10 fJ. Solid curves indicate all GLM flashes while dashed curves only consider multi‐group flashes that would not be removed by the LCFA.

The loss of these dim events under higher thresholds affects group and flash detection, TOE, and the characteristics of the remaining groups and flashes. Overall group counts (Figure 2b) are reduced by 25% under a 2 fJ threshold, 55% by 4 fJ, and 81% by 10 fJ. TOE (Figure 2d), meanwhile, is reduced by 12% by 2 fJ, 29% by 4 fJ, and 53% by 10 fJ. The TOE values in Figure 2d accumulate all events from the Colombia thunderstorm, but TOE is also reported as a gridded product. The severe losses in TOE at the storm level between a 1 fJ and 10 fJ threshold are the first indication that changing thresholds will become important when trending GLM grids.

Of the four parameters considered in Figure 2, flash count (Figure 2c) is the key metric for GLM performance, and it is least impacted by thresholds changes. Only 4% of the original flashes (solid line) are lost by increasing the threshold to 2 fJ, 15% by 4 fJ, and 35% by 10 fJ. Many of these flashes are reduced to a single group, however, and would not be reported by the LCFA. Removing these flashes (dashed line) increases the overall losses to 7% by 2 fJ, 20% by 4 fJ, and 44% by 10 fJ. Still, these losses are small compared to the total numbers of missed events and groups and the reductions in thunderstorm TOE in Figure 2, indicating that the primary effect of an increased threshold is the loss of flash detail and the extent of cloud illumination that can be resolved by GLM.

Figure 3 examines the remaining groups that do not fall completely below threshold and plots histograms of group energy (Figure 3a) and group footprint area (Figure 3b) (as a percent of the original group area/energy) under thresholds ranging from 1 fJ to 10 fJ. For each threshold, the horizontal bins in Figure 3 sum to 100%. Percentiles are also tracked between thresholds with line overlays. While these groups are still resolved by GLM, their appearance is significantly modified under the increased thresholds. The median group energy declines to two‐thirds of the energy of the original group, while the median group area is reduced to one‐fourth of the original group area. However, not all groups are affected in the same way, leading to a broad range of possible energy reductions under higher thresholds. While some groups lose virtually none of their original energy by 10 fJ, others lose 95%.

The group area distribution in Figure 3b shows that the loss of faint events at higher thresholds causes the group area to be substantially eroded. While quantization from an integer number of illuminated pixels limits the possible values and causes percentages that correspond to rational numbers (i.e., 25%, 50%) to stand out, the distributions still demonstrate that group area is more sensitive to threshold effects than group energy. This is because most of the events that comprise a group are rather dim compared to the brightest event in the group. A point source within a cloud may consist of a single bright event in the pixel over the source with a surrounding ring of dim events. In this case, there will be one brighter event, and then eight dim events in the ring. Thus, the dim pixels in a group will far outnumber the bright pixels. Even increasing the threshold slightly to 2 or 3 fJ severely impacts the detection of peripheral dim events, and the median group area is reduced by half while the median group energy only declines by 12%. Under the highest thresholds, groups might only contain the single brightest event from the original group.

To determine what the loss of dim events does to flash characteristics, Figure 4 repeats the analysis from Figure 3 at the flash level. Flashes must have at least one event above the maximum threshold (10 fJ) to be considered, and all groups that fall below the threshold will not contribute to the flash energy (Figure 4a) or flash area (Figure 4b). The key difference between the group level (Figure 3) and the flash level (Figure 4) characteristics is the notable lack of flashes that are virtually unchanged from the original threshold. Even moderate thresholds of 2–3 fJ are sufficient to erode much of the flash energy and at least some of the flash area.

Figure 4

As in Figure 3, but for geostationary lightning mapper flashes rather than groups.

The reductions in flash energy are more severe than group energy losses because flashes are comprised mostly of small dim groups offset by a few bright pulses (Peterson et al., 2018). While individually dim, the total energy from these pulses has a significant impact on the overall flash energy ‐ and they are removed entirely under these higher thresholds alongside the dim portions of the brighter groups. The flash area reductions, meanwhile, are comparable to the group area reductions that we saw previously in Figure 3. This is probably not a coincidence, as flash area is often determined by the single largest group in the flash (Peterson et al., 2017). Lateral propagation only plays a central role in determining flash area once the group separation exceeds the scale of these brighter individual GLM groups.

The Effect of GLM Threshold on Flash Clustering

The complete loss of dim groups and erosion of brighter groups under an increased threshold poses a challenge for GLM lightning mapping and flash clustering. Flash structure is mapped by tracking the faint localized discharges that occur along the developing branches of the flash. As these events start to fall below threshold, the GLM maps resolve flash structure using an increasingly‐smaller number of points – adding uncertainty to the path that was taken by the flash through the cloud. Eventually, the removal of these dim events will reach a point where the remaining events become separated in space and time beyond the thresholds used by the GLM clustering algorithm (Goodman et al., 2010). This causes the original single lightning flash to be split into multiple flash features that represent different illuminated portions of the lightning tree.

Figure 5 demonstrates how increasing the threshold affects clustering using the case of a long horizontal lightning flash from the Colombia thunderstorm. The flash is mapped with a color contour representing the TOE from only the flash in question, and with a line segment overlay connecting subsequent groups in the flash. If the original flash becomes split into multiple flash features, each of these split flashes will be assigned a different color for the group line segment overlay. Index numbers for each split flash are also drawn in its assigned color. The flash is plotted under the original threshold in Figure 5a. Most of the group activity in the flash occurred along its southern flank, while a linear branch can also be noted extending to the northeast. TOE values were reasonably‐high over most of the flash footprint (i.e., >100 fJ) ‐ but note that these are summed over all events, which can mask large numbers of events that might be removed under the higher thresholds.

Figure 5b removes all events below a 3 fJ threshold and then reclusters the flash using the remaining events. While the extent of the flash derived from its group‐level structure is reduced under this higher threshold due to the loss of groups at the ends of the southern and northeastern branches, the overall flash structure is mostly intact. However, the large distance between some of the groups (i.e., long straight lines in Figure 5b) ‐ particularly along the northern branch ‐ signify that the group separations are approaching the limits of the clustering algorithm (16.5 km, 330 ms). The clustering algorithm still combines these groups into a single flash at this point because GLM clustering depends on the separation of events within a group and not the separation of group centroids that are depicted by the line segments. Still, the large group spacing indicates that we have limited information about how the northern branch of the flash developed, and further increasing the threshold is likely to result in splitting.

Figure 5c increases the threshold up to 6 fJ, causing in the first split section from the original flash. Removing the events below this threshold prevents most of the development of the northern branch from being resolved. The branch is still evident as a contiguous feature in the TOE plot, but the individual groups and their constituent events that would be detected by GLM are too far apart in space and time to meet the GLM definition of a flash. Thus, the collection of groups at the far end of the branch are split into a distinct flash feature (depicted with blue line segments and assigned an index of 2) from the primary flash (colored black with the original index of 1). Continuing to increase the threshold to 9 fJ (Figure 6d) splits the northern branch of the flash further into a third central piece (colored red with an index of 3). Thus, we have the primary flash (black), the larger central split flash (red) and the original split flash at the end of the branch (blue) – which has been reduced to a single point and would not be reported by the current version of the LCFA.

Figure 6

Fractions of flashes that become split after imposing an artificial threshold categorized by original flash extent.

Flash splitting artificially increases thunderstorm flash counts. Long horizontal flashes like the case in Figure 5 are particularly problematic because their lateral development makes them prone to being broken into multiple small pieces (as we saw in Figure 5d) rather than two roughly equal sized segments. Moreover, as these flashes occur outside of the convective core where flash rates are generally low to begin with, any splitting will noticeably alter the local flash rate (or FED). The scale of this problem is demonstrated in Figure 6 by quantifying the frequency of flash splitting at each flash size and imposed threshold. Flash splitting is not a severe issue for convective‐scale flashes (i.e., ∼10 km in size), as <10% of flashes are split at any threshold. However, when flashes grow to 50‐km, more than half are split at the higher thresholds (i.e., >3 fJ) and nearly 80% of the largest flashes (100+ km) at the highest thresholds (5–10 fJ) are split.

It is not immediately clear how much the threshold‐based splitting will change the overall flash rate for a given thunderstorm because the larger flashes that are frequently split are far less common than the convective‐scale flashes that remain intact at higher thresholds. To assess the impact of splitting on the flash rates from the Colombia thunderstorm, Figure 7a counts the number of reclustered flashes at each threshold and compares this number to the original flash counts from Figure 2c. As before, we consider both the case of all flashes (solid lines) and multi‐group flashes (dashed lines). When the imposed threshold is near the instrument threshold (i.e., 0 fJ – 1 fJ), the original (black curves) and reclustered (blue curves) flash counts are nearly identical. However, imposing a 2 fJ or higher threshold causes the reclustered curves to increase beyond the original flash count curves. All of the curves in Figure 7a decrease at higher thresholds as whole flashes fall below the threshold, but the separation in the curves remains fairly constant. Figure 7b quantifies this by computing the ratios between the reclustered flash counts and the original flash counts at each threshold. Flash splitting artificially increases the overall flash rate from the Colombia thunderstorm by 6% for all flashes and 3% for multi‐group flashes under a 2 fJ threshold, 9% and 6% under a 4 fJ threshold, and 12% and 7% under a 10 fJ threshold. These increases partially counteract the 35% (all flashes) and 44% (multi‐group flashes) losses in flash count over the same threshold range that we discussed with Figure 2c. The negative offset from flash splitting masks the amplitude of the flash rate reduction at higher thresholds, making it seem as though GLM is not missing as many flashes as it actually is.

Figure 7

The effect of flash splitting on flash rates. (a) The total number of original flashes (black) and flashes that have been reclustered to account for splitting (blue) at each imposed threshold. (b) The ratio of reclustered flashes to the original flash count at each threshold. Solid lines include all flashes while dashed lines only consider multi‐group flashes.

The Effect of GLM Threshold on Gridded Product Generation

The GLM threshold affects the gridded products generated from flash cluster data by combing the effects discussed in the previous sections. These include:

Removing below‐threshold events at the periphery of the flash/group footprints reduces the spatial extent of features in the gridded data

Eliminating below‐threshold events modifies the flash characteristics represented in the grids – both in the original sample of flashes, and following the threshold‐based artificial flash splitting

Removing below‐threshold events/groups/flashes modifies the sample of lightning used to generate the grids ‐ introducing a bias towards the more prominent flashes that can survive the removal of below‐threshold events (including the LCFA removal of single‐group flashes)

While the degree to which the above effects impact the GLM gridded products differs among the products, it generally depends on whether the grid is generated by summing/averaging flash characteristics (i.e., TOE, FED, AFA, Average Flash Extent, Average Flash Duration) or by looking at their minimum values (i.e., Minimum Flash Area: MFA). Grids based on maximum flash characteristics are not considered here but would more closely resemble the total/mean grids than the minimum grids. We elect to discuss two representative grids that demonstrate these effects – AFA and MFA – and provide the remaining grids as Supporting Information S1. All of these grids are generated using only multi‐group flashes that would not be removed by the LCFA.

AFA grids are shown in Figure 8 for imposed thresholds between 0 fJ and 10 fJ, with an increment of 2 fJ between panels. Note that an exponential scale is used to capture the large dynamic range of GLM flash areas, so even slight changes in color represent a notable difference in flash size. When no artificial threshold is imposed, flash sizes within the primary storm feature at the center of the image range from 600 km2 at its center to over 5,000 km2 at its northwest edge. This behavior is largely due flashes of all energies illuminating the convective core of the thunderstorm while the brightest flashes can also illuminate neighboring clouds ‐ but long horizontal flashes can also contribute to larger flash areas in these non‐convective regions.

Figure 8

Average flash area grids generated from (a) the original geostationary lightning mapper data and the reclustered data under artificial thresholds of (b) 2 fJ, (c) 4 fJ, (d) 6 fJ, (e) 8 fJ, and (f) 10 fJ.

Imposing a threshold of even 2 fJ (Figure 8b) removes much of the illumination around the edge of the thunderstorm feature, while causing all AFA values to decrease. The largest AFA values are around 3,000 km2 while the convective core sees its first pixels in the 300–600 km2 range. Increasing the threshold further from 4 fJ up to 10 fJ (Figures 8c–8f) continues these trends: the thunderstorm feature becomes smaller while the AFA values continue to decline. What was initially a region of small flashes surrounded by larger flash areas under a <1 fJ threshold is reduced to a cluster of small flashes by 10 fJ with only a few pixels of increased flash area on the northwestern flank to indicate the larger flashes from the original grid.

These changes in AFA are modest compared to grids that examine minimum values like MFA. Figure 9 shows the MFA grids at each threshold level during the same thunderstorm snapshot as Figure 8. Removing dim GLM events has a greater effect on MFA because only the brightest events in the groups that comprise a given flash might exceed the imposed threshold. Thus, while the initial MFA grid in Figure 9a might resemble an amplified version of the AFA grid in Figure 8 that emphasizes the small flashes in the convective core and the larger flashes that illumine its periphery, minimum flash sizes quickly fall off across the thunderstorm feature as thresholds are increased beyond 2 fJ (Figures 9c–9f). By a 10 fJ threshold, nearly half of the gridpoints within the feature correspond to flashes consisting of only 1‐pixel GLM flashes, and the remainder are <200 km2 in size. In reality, these optical emissions are probably smaller but limited to the size of a GLM pixel (64 km2 in Colombia) if the optical source is centered within a GLM pixel or 2–4 pixels (128–256 km2) if the optical source is located at a pixel boundary.

Figure 9

As in Figure 8, but for minimum flash area grids.

These variations in AFA and MFA with threshold demonstrate the challenge of trending GLM gridded products over time. Thresholds usually change from day to night and, as we saw in Figure 1b for the Colorado case, these changes can extend over multiple femtojoules of event energy (Cummins, 2021). This is further complicated by threshold differences across the GLM imaging array (i.e., between RTEPs). As storms move and develop over time, the characteristics of their flashes will change, driving trends in the GLM gridded products. But these trends will be confounded by any changes in local threshold that occur over the same period. To mitigate threshold biases in gridded product trends, it is necessary to construct grids that remain consistent over the life cycle of the storm in question. Imposing artificial thresholds at the highest threshold values encountered by the storm of interest, as we have done here, is one way of accomplishing this. However, this comes with the cost of losing much of the flash detail that is required to measure the flash characteristics being trended.

Conclusion

This second part of our thundercloud illumination study focuses on the effect that the GLM instrument threshold has on GLM data products. To quantify threshold effects, we consider a thunderstorm that occurred over Colombia with a low instrument threshold (∼0.7 fJ), impose artificial thresholds over the range of 1 fJ to 10 fJ, and then examine how each type of GLM data product is modified by these threshold changes.

The primary effect of the threshold‐based changes to the GLM products is the loss of faint events that are present under lower thresholds. Losing the below‐threshold dim events erodes the footprints of GLM groups and flashes until they fall completely below the higher threshold and go undetected. As flashes are comprised primarily of dim events and groups offset by a few energetic pulses, increasing the instrument threshold has a greater impact on event and group detection than on flash detection. Imposing a 2 fJ artificial threshold on our thunderstorm case decreases the event count by 60% compared to the original data, while imposing a 10 fJ threshold removes 90% of the original events. Meanwhile, 25% of groups are removed with a 2 fJ threshold and 81% are lost under a 10 fJ threshold.

Threshold effects on flash detection are complicated by the issue that losing dim events and groups can result in flash splitting, as the remaining events and groups exceed the space and time thresholds employed by the GLM clustering algorithm. Of the original GLM flashes in the Colombia thunderstorm, 4% fall completely below a 2 fJ threshold and 35% are eliminated by a 10 fJ threshold. The current version of the GLM LCFA also removes single‐group flashes. Filtering out these flashes increases the threshold‐induced losses to 7% by 2 fJ, 20% by 4 fJ, and 44% by 10 fJ. At the same time, flash splitting artificially increases the overall flash count by 6% at 2 fJ and up to 12% by 10 fJ and the multi‐group flash count by 3% at 2 fJ and up to 9% by 10 fJ. While the Colombia thunderstorm exemplifies how the GLM threshold impacts clustering, we expect that the amplitudes of the GLM event/group/flash rate reductions will vary between thunderstorms. The prevalence of lightning sources near the cloud top in this storm (see Part 1) and the larger reductions reported in Figure 9 of Cummins (2021) suggest that it might be an optimistic case. Further work is needed to produce general statistics for GLM event/group/flash rate reductions with threshold.

Gridded products generated from GLM flash cluster data are also severely impacted by a combination of missed/split flashes and reductions in the flash/group footprints at higher thresholds. We consider the AFA and MFA products as representative of total/mean/maximum products (AFA) and minimum products (MFA) and examine how they change under the imposed thresholds. The size of the GLM feature describing the illuminated thunderstorm decreased under higher thresholds, as illumination around the periphery of the storm core from distant bright/large flashes quickly falls below threshold. Flash sizes within the storm core also generally decreased due to smaller flashes losing portions of their footprints and flash splitting. The key difference between AFA and MFA is the scale of this reduction in area. As MFA examines the minimum flash area, flashes comprised of a few events just above the threshold can report flash areas corresponding to just 1 or 2 GLM pixels. This is problematic for non‐convective storm regions that produce large lightning flashes, as these long horizontal flashes are more prone to splitting at higher thresholds than small convective‐scale flashes.

These results demonstrate the importance of considering the context surrounding GLM detections – the configuration of the cloud scene, the corresponding instrument threshold, the location and time of day, etc. ‐ when interpreting GLM data. This is especially necessary when accumulating data from a diverse collection of lightning flashes (for example, when generating gridded products) or trending GLM data over time. Changes in the situational context (for example, spatial/diurnal changes in threshold) can have a considerable impact on the results.

Future work in Part 3 (Peterson et al., 2021b), will leverage these results to show how the altitude of the source within the cloud can be estimated from group‐level cloud illumination metrics. Finally, Part 4 (Peterson & Mach, 2021) will evaluate volumetric meteorological and thundercloud imagery derived from GLM data.

Supporting Information S1

Click here for additional data file.

Movie S1

Click here for additional data file.

Movie S2

Click here for additional data file.

Movie S3

Click here for additional data file.

Movie S4

Click here for additional data file.

Movie S5

Click here for additional data file.