Invited Perspective: Impact of Exposure Measurement Error on Effect Estimates-An Important and Neglected Problem in Air Pollution Epidemiology.

Given that the health effects of air pollution have clear policy implications, questions have been raised on biases that may influence health effect estimates and consequently modify the estimated population risks. Measurement error (ME) in exposure is a ubiquitous characteristic of air pollution epidemiological studies. The bias that may be caused by exposure ME on the effect estimates has attracted limited attention to date. However, quantifying the impact of ME and developing correction methods for the potentially biased effect estimates will lead to better assessment of the health burden of air pollution exposures. Several studies addressing this issue are based on simulated data where the “true” effect is taken as known and the consequences of using a “surrogate” estimate subject to ME are investigated.

The paper by Wei et al.[1] addresses this issue, investigating the bias from ME in long-term estimates of fine particulate matter [PM in aerodynamic diameter ()] in residential areas of study participants on the mortality effect estimates. This paper has several exceptional methodological features and is an excellent addition to the limited relevant literature. The authors used a sophisticated spatiotemporal model to predict concentrations at ZIP Code level, and estimated the error based on of the ZIP Codes that have a monitoring site. The authors evaluated 162 scenarios based on the observed errors and, additionally, on the assumption that the error is either 2- or 3-fold larger. It includes linear and nonlinear concentration–response functions (CRFs) with either 100% Berkson (where the surrogate measurements have smaller variability than the true measures) or 100% classical error (where the surrogate measurements have larger variability than the true measures). They concluded that ME biased the effect estimates toward the null, and larger error led to larger bias, but the magnitude of the bias was small.

We would like to note that some aspects of the paper by Wei et al.[1] are unique and not common in many situations and lead to a smaller reported magnitude of bias, as described below.

ME is defined as the difference between true and surrogate exposure measures.[2] It is evident that the estimation of ME depends on what is defined as the true measure. In Wei et al.,[1] as in several other cases,[3] the true measure is defined as the modeled ambient concentration at the cohort member’s residence. In other situations, the true measure may be defined as the personal exposure to pollutants from ambient sources or even total personal exposure from any source.[4] The definition of true has obvious implications for the assumed or estimated ME.[5]

Butland et al.[6] reported on the magnitude of bias in effect estimates according to the correlation coefficients between surrogate/modeled and true measures and their variance ratios. These quantities reflect, the type of error: Berkson-type (variance ratios ) or classical-type (variance ratios ), and, also, reflect the variance of the errors, given that the errors are defined as the difference between the two exposures. They assessed the impact of ME from spatiotemporal model predictions on the bias in effect estimates of both short- and long-term exposures and concluded that in the majority of cases the bias is toward the null (i.e., the effects are underestimated). Specifically, for variance ratios of modeled vs. true values of 1–2 (implying a more classical ME) and correlation coefficients of 0.5–0.9, the bias toward the null can be even . In contrast, it can be small and negligible, with a variance ratio of and correlation coefficients of . On the other hand, the direction of the bias may be away from the null, with a variance ratio of 0.5 (implying a large Berkson error) and high correlations.

What variance ratios and correlation coefficients between surrogate and true exposure estimates are observed in real-world situations? Wei et al. reported variance ratios between 0.97 and 1.03.[1] In other reported work assessing typical modeling strategies, variance ratios differed depending on the exposure model. For example, variance ratios for background sites for exposure to nitrogen dioxide ranged from 0.76 to 1.38 for short-term exposures (with associated correlation coefficients ) and from 1.5 to 11.6 for long-term exposures (correlation coefficients from 0.44 to 0.97)[7]; variance ratios for exposure to PM in aerodynamic diameter () ranged from 0.37 to 0.97 for short-term exposures (correlation coefficients from 0.71 to 0.95) and 0.51 to 1.18 for long-term exposures (correlation coefficients from 0.13 to 0.39).[8] The bias in the effect estimates associated with variance ratios farther from 1, and the associated correlations, in these prior studies were much larger than that found by Wei et al.[1] It can also be observed that investigating short-term exposures, where the observed correlations are higher, variance ratios are rarely found, thus the upward bias is only rarely observed. It should be noted that the ME problem gains further complexity in studies of multi-exposures.[9,10]

All error categories classified according to error source—such as spatial misalignment, time–activity patterns, or model parameter specifications[5]—contain elements of classical- and Berkson-type errors, but the proportion of each type and the determining factors have not been studied adequately. The consequences of this mixture of error types should be evaluated in future work, as Wei et al.[1] also acknowledge.

In conclusion, exposure ME under some outstanding modeling situations in locations with relatively dense monitoring can produce minor bias in the effect estimate, mostly toward the null. However, under many of the usual circumstances found in studies of air pollution epidemiology, ME can bias the effect estimate toward the null to a substantial degree. In very rare and unusual situations, the bias can be directed away from the null. The magnitude and impact of the ME assessed in every situation depends on the way ME is defined, and it is thus important to clarify the definitions and their implications. The magnitude of bias depends on the correlations and variance ratios of surrogate and true exposure–concentration values, leading to the necessity to report these parameters. Further, ME in multi-exposure evaluations, and its impact on the effect estimates, has been the focus of very few studies to date, although it is crucial for quantifying independent exposure effects. It is important to quantify ME under various possible scenarios and in various locations, assess the factors that affect it, and work further on valid and generalizable correction methods.[11,12] To date the main correction methods that have been developed are regression calibration and simulation extrapolation (SIMEX). These are based either on external validation samples from the same populations or on extrapolating the results obtained. They have been used to correct mainly for additive classical ME, which might be an oversimplification of the true error types.[11,12] In addition, ME correction studies are limited in number and the methods depend on the local situation. In the absence of studies investigating transferability of the correction factors, results obtained cannot be generalized.