Transferability and generalizability of regression models of ultrafine particles in urban neighborhoods in the Boston area.

Land use regression (LUR) models have been used to assess air pollutant exposure, but limited evidence exists on whether location-specific LUR models are applicable to other locations (transferability) or general models are applicable to smaller areas (generalizability). We tested transferability and generalizability of spatial-temporal LUR models of hourly particle number concentration (PNC) for Boston-area (MA, U.S.A.) urban neighborhoods near Interstate 93. Four neighborhood-specific regression models and one Boston-area model were developed from mobile monitoring measurements (34-46 days/neighborhood over one year each). Transferability was tested by applying each neighborhood-specific model to the other neighborhoods; generalizability was tested by applying the Boston-area model to each neighborhood. Both the transferability and generalizability of models were tested with and without neighborhood-specific calibration. Important PNC predictors (adjusted-R(2) = 0.24-0.43) included wind speed and direction, temperature, highway traffic volume, and distance from the highway edge. Direct model transferability was poor (R(2) < 0.17). Locally-calibrated transferred models (R(2) = 0.19-0.40) and the Boston-area model (adjusted-R(2) = 0.26, range: 0.13-0.30) performed similarly to neighborhood-specific models; however, some coefficients of locally calibrated transferred models were uninterpretable. Our results show that transferability of neighborhood-specific LUR models of hourly PNC was limited, but that a general model performed acceptably in multiple areas when calibrated with local data.

Introduction

Land-use regression (LUR) models have frequently been used to estimate traffic-related air pollutant (TRAP) exposures for epidemiology studies.[1−11] Pollutants modeled using LUR include NO2, PM2.5, particle number concentration (PNC), black carbon (BC), and volatile organic compounds (VOCs).[1−6,12−22] LUR models of TRAP are developed by regressing measurements from dense monitoring networks or mobile monitoring against spatial covariates including distances to or densities of land uses (e.g., road networks, topography, and land cover).[2,3] To improve short-term TRAP estimates, temporal variables including central-site measurements, wind speed and direction, temperature, atmospheric stability, and hourly traffic intensity have been incorporated into LUR models.[5,13,17−24]

LURs should transfer reasonably well between areas with similar land use, meteorology, and pollution source characteristics; however, site-specific models typically outperform transferred models because local predictors and their relationships to emissions may depend on location (e.g., due to differences in fleet composition and emissions, and in the built environment).[1,25−27] For example, when an annual average NO2 LUR model for Huddersfield, U.K. was transferred to four other U.K. cities, and predictions were scaled based on local monitors, the slope between predictions and measurements ranged from 0.48 to 1.04 and the R2 ranged from 0.51 to 0.76.[4] In contrast, this model did not capture spatial variability and overpredicted measurements when transferred to Hamilton, Canada.[1] Models for other pollutants (e.g., PM10, NO) have been less transferable than those for NO2, possibly because monitor locations were selected based on NO2 variation, and other pollutants vary on different spatial scales.[25,27,28] Further study is needed of how methodological artifacts (e.g., study area size, availability, and comparability of GIS data, or monitoring methods), interpretability of covariates, and differences in pollution source or dispersion characteristics contribute to poor transferability.[3,26,27]

Less work has been done regarding generalizability of regional models to local areas. Regional models for annual average NO2 in Europe performed similarly to local models, except in southern Europe (Turin and Rome, Italy; Athens, Greece; Barcelona, Spain; Marseille, France) where poor performance of regional models applied to smaller areas was attributed to heterogeneity in NO2 emissions and concentrations, topography, meteorology, and other factors.[28] Models developed for large areas within Switzerland (170 km2) and The Netherlands (6000 km2) also performed poorly relative to city-specific models.[11,29] More generalizable regression models could be developed if greater emphasis were placed on developing models with variables applicable to multiple locations than on maximizing R2.[3]

This paper focuses on ultrafine particles (UFP; <100 nm in aerodynamic diameter) near roadways. UFP are present in high concentrations in motor vehicle exhaust emissions[30−33] and living near major roads is associated with increased risks of cardiovascular and pulmonary disease.[34−36] LUR models of PNC (a proxy for UFP) have been developed for broad urban areas including Amsterdam,[14] Basel,[24] New Delhi,[37] Vancouver,[38] and near-roadway neighborhoods in U.S. cities,[5,15,56] but we could find no studies addressing the generalizability or transferability of PNC models. Differently formulated built environment variables in these PNC models suggest that city-specific models may be necessary.[39]

Our objectives were to (1) develop 4 hourly neighborhood-specific LUR models of PNC in and around Boston (MA, U.S.A.) and examine their transferability considering neighborhood-specific calibration; and (2) develop a Boston-area (BA) model using pooled neighborhood data and test its generalizability by applying it to the individual neighborhoods. The models were developed using consistent data sources for all neighborhoods to minimize the impacts of methodological differences on the model comparisons. This research was part of the Community Assessment of Freeway Exposure and Health (CAFEH) study, a community-based participatory research study of the relationship between UFP and cardiovascular disease in adults.[8] The four neighborhood models are being used to assign ambient hourly PNC estimates at residential addresses.[8,9]

Methods

Data Collection

Four neighborhoods in the metropolitan Boston area near Interstate 93 (I-93; 1.5 × 105 vehicles per day in all seasons[40]) were studied: Somerville, Dorchester/South Boston (referenced as “Dorchester”), Chinatown, and Malden (Figure 1). The areas included both mixed residential-commercial areas (Somerville, Dorchester, and Malden) and a highly urban area with tall buildings, street canyons and multiple highways (Chinatown). Somerville and Dorchester contained both near-highway (<400 m) and urban background (>1000 m) areas; Chinatown (near-highway) and Malden (urban background) were paired because they have demographically similar populations and Chinatown was too small to contain a background area. We did not identify significant nonroad UFP sources (e.g., industry, energy generation, shipping) in any of the study areas. Diesel vehicles contributed ∼3.8% of highway traffic and <5% of local traffic in all of the study areas.[41,42] More detailed descriptions of the study areas are available elsewhere.[5,8,43,44]

Figure 1

Annual average PNC predicted by the land use regression models. The left panel shows PNC predictions from the Boston-area model. Annual average PNC predictions from the (a) Somerville, (b) Malden, (c) Chinatown, and (d) Dorchester neighborhood-specific models are shown in the right panels. The Boston Globe stationary monitoring station is marked with a black star.

Mobile monitoring with the Tufts Air Pollution Monitoring Laboratory (TAPL) was conducted in Somerville between September 2009 and September 2010, in Dorchester between September 2010 and July 2011, and in Chinatown and Malden between August 2011 and July 2012 (Supporting Information, SI, Table S1).[43,44] The impacts of nonsimultaneous monitoring in the study areas (i.e., interannual variation in TRAPs measured at an EPA monitoring station) were small compared to seasonal and diurnal differences in PNC, and were therefore assumed to not play an important role in model differences.[43] Monitoring was conducted under a wide range of conditions at different times between the hours of 04:00 and 22:00 on 34–46 days per neighborhood distributed across all seasons (∼21–70 h per season) and days of the week.[43] This was more monitoring than suggested by Van Poppel et al.,[45] who concluded that 3–16 h of mobile monitoring per season sufficiently characterized spatial PNC variation in their neighborhoods. On each monitoring day the TAPL was driven over a fixed route in one neighborhood for 2–6 h at <20 m/s (72 km/h; mean and median = 5 m/s = 18 km/h). PNC was monitored at 1-s intervals using a butanol condensation particle counter (Dp,50 = 4 nm; CPC 3775, TSI, Shoreview, MN) and matched to locations with a Garmin V GPS unit. In addition, continuous monitoring for model performance evaluation was conducted with a second CPC (identical to the one in the TAPL) at the Boston Globe site ∼20 m east of I-93 in Dorchester between March and May 2011 (Figure 1).

The CPC used for mobile monitoring was manufacturer-calibrated at the start of the study in September 2009 and again in July 2011, and the CPC used at the Globe site was received from TSI in March 2011. Side-by-side measurements by these CPCs differed by <3%.[44] PNC measurements were censored for flow rate errors (2% of observations) and for potential self-sampling of TAPL exhaust when the TAPL speed dropped below 1.4 m/s (5 km/h; ∼14% of observations, mainly during complete stops at intersections).[44] Using the Particle Loss Calculator, we estimated combined inlet and tubing particle losses of <10%.[46] GPS coordinates >20 m from the centerline of the nearest road (due to poor GPS reception in street canyons) were moved to the monitoring route centerline using ArcGIS 10.1 (ESRI, Redmond, CA; 6% of data in Chinatown only).[43]

One-second PNC measurements were assigned spatial variables using ArcGIS. GIS variables (e.g., road type, road features including width and curb type, and distance and direction from I-93 or other major roads) were obtained from MassGIS.[47,48] Distances from major intersections with estimated average vehicle delays ≥20 s were also calculated for Chinatown.[49]

Because higher-resolution covariate data were not available, each one-second PNC measurement was assigned hourly meteorological and traffic values using SAS version 9.3 (SAS Institute, Inc., Cary, NC). Hourly wind speed and direction (7.9 m above ground level) and temperature (2 m above ground level) measurements for development of all models were obtained from Logan International Airport.[50] Hourly traffic volume and average speed on interstate highways were provided by the Massachusetts Department of Transportation (stakeholder.traffic.com). Neighborhood-specific real-time traffic and wind data were not available.

Model Building and Testing

Neighborhood Model Building

Neighborhood-specific log–linear regression models for PNC in Somerville, Dorchester, Chinatown, and Malden were built and tested using methods summarized in Table 1 and described elsewhere.[5] We developed hourly models of PNC so that estimates could be matched with the time-activity patterns of the CAFEH study population. Variables were developed by visual inspection of the functional form of the relationship with the logarithm of PNC, ln(PNC), and included in the models if p <0.05 and R2 increased by >1%. When multiple variable forms improved the R2, the one with the greatest degree of physical interpretability and consistent interpretations across neighborhoods was selected. All 133 variables considered are listed in SI Table S2. For each neighborhood, k, model calibration was performed using ln(PNC) measurements to estimate regression coefficients (β) for a unique set of explanatory variables, X. Model predictions of ln(PNC) using the coefficients β and variables X were generated for each measurement point to evaluate the model. Surface maps of predicted PNC were generated to assess spatial and temporal trends.

Table 1

Summary of Alternative Land Use Regression Models and Their Evaluation Criteria

model	model buildinga	predictionsa	observed ln(PNC)b	predicted ln(PNC)b	test	question addressed by the model
neighborhood	Yk = βk*Xk	Zkk = βk*Xk	Yk	Zkk	Leave-one-day-out cross-validation	How similar are neighborhood-specific PNC models for exposure assessment?
direct transfer	Yk = βk*Xk	Zjk = βk*Xjk	Yk	Zjk vs Zkk	RMSE and correlation	Were site-specific models better than directly transferred models?
calibrated transfer	Yj = βj*Xjk	Wjk = βj*Xjk	Yj	Wjk vs Zkk	RMSE and correlation	Were site-specific models better than calibrated transferred models?
Boston-area (pooled data)	Yp = βp*Xp	Zpp = βp*Xp	Yp	Zpp	Leave-one-day-out cross-validation	Could a general model including all of the neighborhoods perform well overall?
Boston-area (applied to individual neighborhoods)	Yp = βp*Xp	Zjp = βp*Xjp	Yp	Zjp vs Zpp	RMSE and correlation	Were the models generalizable? Were site-specific models better than the BA model?
Boston-area with neighborhood-specific calibration	Yj = βjp*Xjp	Wjp = βj*Xjp	Yj	Wjp vs Zpp	RMSE and correlation	Was the locally calibrated BA model better than the BA model with calibration from all neighborhoods?

Model building is a procedure of using known ln(PNC), Y, values to estimate the values of regression parameters, β, for a given set of explanatory variables, X. Subscripts refer to a neighborhood used to develop a particular model (k), a new neighborhood where a model is applied (j), and pooled data or variables (p). Predicted ln(PNC) using original calibration (Z) or a new calibration for testing (W) have two subscripts: the neighborhood where the model is applied followed by the neighborhood where the model was developed. For example, the calibrated transfer model-building step predicts ln(PNC) in a new neighborhood (Y) from β fit using data from the new neighborhood and X, explanatory variables selected from model building in neighborhood k but with values from neighborhood j. The predictions for the calibrated transfer model, W, use the same β and X from the model building step.

Predicted ln(PNC) was compared to observed ln(PNC) to test the model performance.

Models were evaluated using adjusted-R2, root-mean-square error (RMSE), variance inflation factors (VIF), and leave-one-day-out cross-validation. Preferred models had higher R2 and lower RMSE than less preferred models, and had VIF <5 for all variables. Leave-one-day-out cross-validation, one of many evaluation methods,[51] was used to test whether individual monitoring days substantially influenced predictions. Cross-validation was conducted by calibrating each model nCV times, iteratively excluding the data 1 day at a time from monitoring days 1 to nCV and then predicting ln(PNC) for each excluded day.[19] Adjusted-R2, model RMSE, and RMSE of model predictions were evaluated for each iteration. All modeling was performed in R 3.0.1.[52]

Model Transferability

Transferability, the extent to which ln(PNC) models for one neighborhood can be applied to others, was evaluated in two ways: (1) direct transfer of model parameters X with regression parameters β to a new neighborhood j, and (2) recalibration of regression parameters (β) using observations in neighborhood j and transferred explanatory variables X. Transferability was tested by comparing models built for one neighborhood and applied to a second neighborhood to the model built specifically for the second neighborhood. The extent of transferability was measured by RMSE and R2 between predictions and measurements. Neighborhood-specific models were considered to have a better fit than directly transferred models if correlations were higher and RMSE were lower for neighborhood-specific models. While transferring models could introduce errors given differences in local traffic and source distance-direction relationships, the proposed procedure was informative for understanding the magnitude and nature of the errors.

Model Generalizability

Generalizability (i.e., the performance and adaptability of a model when applied to new conditions while maintaining the same basic set of explanatory variables) was tested using a Boston-area (BA) model that considered all model parameters used in the neighborhood models. Regression parameters in the BA model were fit using pooled data from monitored neighborhoods. The BA model was cross-validated following the procedure in Section 2.2.1. Generalizability was evaluated by comparing performance of the BA model to neighborhood-specific models. Neighborhood models were considered better than the BA model if the neighborhood-specific predictions had lower RMSE and higher R2 than the BA model predictions. Like transferability, generalizability of the BA model was also tested with neighborhood-specific calibration.

Results and Discussion

Neighborhood Models

Model Comparison

Hourly PNC estimates were aggregated to annual averages to facilitate comparison of spatial variation across the neighborhoods. Annual average PNC ranged from 6300 to 47 000 particles/cm3, with higher PNC and greater variation predicted near I-93 and major roads (Figure 1). While there was considerable temporal and spatial variability in each neighborhood, the predicted annual average PNC differences were generally larger between neighborhoods than within neighborhoods. These modeling results are consistent with the measurements in these neighborhoods (e.g., SI Figure S1).[43] Some variables were common to all neighborhoods and reflective of general physical predictors of pollutant levels (e.g., temperature and wind speed), while other variables reflected local conditions and source patterns (e.g., specific wind direction terms, distance-decay gradients; Table 2).

Table 2

Multivariate Neighborhood-Specific and Boston-Area Land Use Regression Models for ln(PNC)a

	Somerville		Dorchester		Chinatown		Malden		Boston-area
model adjusted R2	0.42		0.35		0.23		0.31		0.26
variable	coeffb	SEc	coeffb	SEc	coeffb	SEc	coeffb	SEc	coeffb	SEc
(intercept)	10.677	0.011	9.844	0.014	10.209	0.014	7.012	0.029	10.618	0.004
spatial variables
within highway corridord	0.244	0.006	0.292	0.014	NA	NA	NA	NA	0.219	0.005
on a major roadd	0.208	0.005	0.132	0.003	NA	NA	0.102	0.006	0.108	0.003
upwind of I-93d	–0.192	0.005	–0.449	0.004	NA	NA	NA	NA	–0.247	0.002
upwind of nearest major roadd	NA	NA	NA	NA	NA	NA	–0.012	0.005	–0.047	0.002
distance upwind of I-93, km	–0.213	0.006	–0.204	0.004	NA	NA	NA	NA	–0.247	0.001
distance downwind of I-93, km	–0.464	0.007	–0.626	0.005	–0.373	0.014	NA	NA	–0.314	0.001
distance from nearest major road, km	–0.230	0.014	NA	NA	NA	NA	NA	NA	–0.362	0.009
distance from Dorchester Ave, km	NA	NA	–0.642	0.006	NA	NA	NA	NA	NA	NA
distance from major intersection, kmf	NA	NA	NA	NA	–0.964	0.020	–0.267	0.008	NA	NA
meteorology
temperature, °C	–0.037	0.000	–0.012	0.000	–0.008	0.000	–0.007	0.000	–0.0192	0.0001
humidity, %	NA	NA	NA	NA	0.002	0.000	NA	NA	NA	NA
wind speed (U), m/s	–0.182	0.002	–0.179	0.001	–0.071	0.001	–0.113	0.002	–0.100	0.001
cosine of wind direction relative to I-93	–0.029	0.003	NA	NA	NA	NA	NA	NA	NA	NA
square of cosine of wind direction relative to southeast	0.820	0.007	NA	NA	NA	NA	0.804	0.008	NA	NA
Eastd,e	NA	NA	–0.228	0.007	NA	NA	NA	NA	NA	NA
N-ENEd,e	NA	NA	0.438	0.007	NA	NA	NA	NA	NA	NA
Westd,e	NA	NA	0.336	0.007	NA	NA	NA	NA	NA	NA
sine of wind direction	NA	NA	NA	NA	0.347	0.004	NA	NA	NA	NA
wind direction ±15° from airport and downtown Bostond	NA	NA	NA	NA	NA	NA	NA	NA	0.400	0.003
traffic and day of the week
low traffic (<7000 vph)d	–0.103	0.006	NA	NA	NA	NA	NA	NA	–0.204	0.003
congestion (<64 km/h)d	0.181	0.005	NA	NA	NA	NA	NA	NA	0.022	0.003
volume on I-93, 1000 vph	NA	NA	0.138	0.001	0.012	0.001	0.177	0.002	NA	NA
Mondayd,g	0.398	0.010	NA	NA	0.496	0.008	1.823	0.015	0.297	0.003
Tuesdayd,g	0.569	0.008	NA	NA	0.373	0.006	1.645	0.014	0.521	0.003
Wednesdayd,g	0.530	0.008	NA	NA	0.379	0.006	1.457	0.013	0.501	0.003
Thursdayd,g	0.579	0.008	NA	NA	0.773	0.006	1.109	0.013	0.359	0.003
Fridayd,g	0.239	0.011	NA	NA	0.793	0.006	1.107	0.015	0.559	0.004
Saturdayd,g	0.504	0.008	NA	NA	0.018	0.006	0.459	0.016	0.043	0.004
Weekdayd,g	NA	NA	0.080	0.003	NA	NA	NA	NA	NA	NA

Variables in the model are statistically significant (p ≤ 0.001). Temporal variables are input on an hourly basis. NA = not applicable for this model.

Coeff is the coefficient estimate. The full model is the intercept plus the sum of products of the coefficients and their variable values.

SE is the standard error in the coefficient estimate.

These variables are categorical variables. All other variables are linear variables.

The wind categories for Dorchester are defined as Variable or Calm (reference), N-ENE (337.5°–67.5°), East (67.5°–180°), and West (180°–337.5°).

Major intersections are defined as either intersections with average vehicle delay of 20 or more seconds (Chinatown) or intersections adjacent to transit stations (Malden).

The reference for day of week is Sunday when all days are included individually or weekend days when only weekday vs weekend is included.

Mobile source proximity variables were important in each neighborhood (Table 2). Distance upwind and downwind from I-93 were most important in Somerville. In Dorchester, distance to Dorchester Avenue (a major surface road) had a larger coefficient than distance to I-93, likely because the section of I-93 in Dorchester was below grade and had a 3-m-high noise barrier on the west side.[43] These factors combined to reduce transport of UFP from I-93 and into the study area. In Chinatown, distance from major intersections and distance downwind from I-93 were the main spatial factors affecting PNC, and any effect of distance from the below-grade section of a second major highway (I-90) was masked by the stronger effect of major intersections. As expected, Malden (urban background) had negligible influence from I-93, with the most influential spatial variable being distance from a transit station near the monitored area. PNC gradients downwind of I-93 varied by neighborhood; the coefficient for distance to I-93 in Somerville was 25% smaller than in Dorchester, and the coefficient for Chinatown was 20% smaller than in Somerville. Diesel locomotives on the railroad tracks immediately adjacent to I-93 may have inflated this coefficient in Dorchester. In contrast with Zhu et al, who reported stronger PNC gradients in winter than in summer in Los Angeles,[53] we did not observe seasonality in the relationship between ln(PNC) and distance from I-93. Road type and distance from the nearest major road were important in all models except for Chinatown, where the entire monitoring route was on major roads.[48] Other road features were relatively uniform within neighborhoods and not significant in the models.

Linear functions for temperature and wind speed were used in each neighborhood model (Table 2). All models had negative coefficients for temperature with some variability across neighborhoods (0.8–3.7% decrease in PNC per °C). There was ∼18% decrease in PNC per m/s increase in wind speed in Somerville and Dorchester. In comparison, the wind speed coefficient was smaller in Malden (11.3%), likely because Malden was far from major sources, and in Chinatown (7.1%), where decreased natural ventilation in street canyons may have reduced dispersion of PNC.[54]

Each model included wind direction and traffic on I-93, but with different functional forms because different transformations resulted in the greatest neighborhood-specific improvement in adjusted-R2. The cosine of wind direction relative to the southeast was used for unidentified sources affecting Somerville and Malden, while wind direction categories (i.e., variable or calm, east, north to east northeast, or west) were used in the Dorchester model. Both the Chinatown and Somerville models included sinusoidal functions of wind direction relative to I-93. Linear functions of traffic volume on I-93 were used in Chinatown, Malden, and Dorchester; however, traffic categories (congested, typical or low volume) were used in Somerville because the relationship of ln(PNC) with traffic volume was nonlinear.[5] The models for Somerville, Chinatown, and Malden incorporated day of week, while Dorchester only differentiated between weekdays and weekends. Time of day was not used because it was correlated with more physically interpretable variables, and because PNC measurements were not available for all 24 h.

Model Evaluation

The neighborhood models explained 42% of the ln(PNC) variability in Somerville, 35% in Dorchester, 23% in Chinatown, and 31% in Malden (Table 2), consistent with similar intraurban spatial-temporal regression models of PNC (SI Table S3).[15,20,24,55,56] Because there was little variability in adjusted-R2, RMSE, or prediction RMSE under leave-one-day-out cross-validation, we concluded that our models were adequately powered and robust (Table 3). A factor contributing to the generally low R2 of the neighborhood models was that changes in emissions and dispersion of UFP occurred at time-scales smaller than those of covariates.[5,57] All variance inflation factors were <4 (SI Table S4).

Table 3

Leave-One-Day-out Cross-Validation of Neighborhood-Specific and Boston-Area Land Use Regression Models of ln(PNC)

model	nCV (ntotal) daysa	adj-R2b	RMSEb	prediction RMSEb
Somerville	39 (43)	0.42 ± 0.01	0.64 ± 0.007	0.67 ± 0.21
Dorchester	31 (35)	0.35 ± 0.01	0.63 ± 0.007	0.63 ± 0.21
Chinatown	45 (46)	0.23 ± 0.01	0.69 ± 0.006	0.75 ± 0.26
Malden	33 (34)	0.32 ± 0.01	0.76 ± 0.009	0.86 ± 0.30
Boston-area	153 (158)	0.26 ± 0.003	0.74 ± 0.002	0.73 ± 0.27

Monitoring was conducted on ntotal days and cross-validation was possible for nCV days. Leave-one-day-out cross-validation (LOO) was performed by removing 1 day of measurements at a time, so there are nCV cross-validation models, each of which was built on ∼10 000 one-second PNC observations.

Each leave-one-day-out cross-validation result is reported as mean ± standard deviation. The LOO adjusted R2 and RMSE are for the model developed on the training data set with 1 day removed. Prediction RMSE was calculated as the error in hourly predictions for each point in each testing data set that consisted of the day that was removed.

Dorchester model predictions followed trends in PNC measurements at the Globe site and were moderately well correlated with the measurements; however, the model underestimated PNC for times of highest concentrations (mainly morning rush hours; SI Figures S2–S4). The Chinatown model performed relatively poorly, possibly due to street canyons, which may have trapped local mobile emissions and increased the GPS measurement uncertainty.[54] We previously showed that removing localized PNC spikes in the Somerville model increased the R2 from 0.49 to 0.56 and decreased the coefficient for on major road,[5] consistent with observations of other researchers that LUR model error is generally larger for higher concentrations due to increased measurement variability.[51] While one study found that averaging just two 15 min measurements made on different days could increase the R2 of PNC regression models from 47% to 72%,[20] we did not do so because between-day variability was larger than within-day variability.

Spatial autocorrelation for each neighborhood was measured by Moran’s I for the ln(PNC) measurements and model residuals. Although the autocorrelation in the residuals was less than in the actual measurements (reduced by ∼63% for the Boston-area model) Moran’s I remained significant (SI Table S5). We were unable to quantify the extent of temporal autocorrelation because the relationship between consecutive measurements was affected by movement of the TAPL.[5] PNC lag terms were not included in any of the models because they would have replaced predictive temporal variables (e.g., temperature).

Model Transferability

Models directly transferred from one neighborhood to another generally performed poorly in terms of R2, slope, and intercept of the simple linear regression between the measured ln(PNC) and model predictions (Table 4). Except for Malden (>2000 m from I-93) and Chinatown (<1000 m from I-93), all predictor variable ranges were similar in the four neighborhoods (SI Table S1); therefore, extrapolation outside the predictor variable calibration ranges was not an important factor affecting transferability. The transferred models predicted the correct order of magnitude of PNC, but did not capture the concentration contrasts, particularly for the highest concentrations (SI Figures S5–S8). Inclusion of overly specific temporal variables (e.g., wind direction) for each neighborhood may have contributed to poor direct transfer of the models. The transferability of the neighborhood models was similar to what others have reported for annual average models of urban NO2 and PM10 in European and North American cities.[1,25−27]

Table 4

Evaluation of Performance of Neighborhood-Specific and Boston-Area Land Use Regression Models of ln(PNC) when Directly Transferred to Somerville, Dorchester, Chinatown, and Malden

		area appliedb
modela	statisticc	Somerville	Dorchester	Chinatown	Malden
Somerville	SLR	0.42x + 5.97d	0.15x + 8.62	0.19x + 8.10	0.15x + 8.01
	R2	0.42	0.04	0.09	0.12
	RMSE	0.64	0.89	0.83	0.88
Dorchester	SLR	0.20x + 8.31	0.35x + 6.65d	0.12x + 9.29	–0.03x + 9.73
	R2	0.12	0.35	0.06	<0.01
	RMSE	0.82	0.63	0.81	1.10
Chinatown	SLR	0.12x + 9.02	0.05x + 9.78	0.23x + 7.91d	0.01x +9.62
	R2	0.07	0.01	0.23	<0.01
	RMSE	0.83	0.82	0.69	1.01
Malden	SLR	0.25x + 6.75	0.09x + 8.63	0.27x + 6.72	0.31x + 6.62d
	R2	0.09	0.01	0.10	0.31
	RMSE	1.30	1.17	1.16	0.76
Boston-area	SLR	0.25x + 7.67	0.24x + 7.72	0.18x + 8.48	0.12x + 8.51
	R2	0.30	0.25	0.13	0.16
	RMSE	0.71	0.70	0.74	0.84

Each row represents a neighborhood-specific or Boston-area (BA) model.

Each column represents the neighborhood where the measurements were predicted.

The reported statistics are SLR = simple linear regression between predictions and measurements formatted as (slope)x + (intercept), R2 = R2 from SLR, RMSE = root-mean-square error between measurements and predictions. Values are bold for R2 > 0.2 and slope >0.2.

Cells on the diagonal represent the performance of models when applied to the neighborhood where they were developed.

Transferred models with explanatory variables from the original neighborhoods and recalibrated coefficients estimated the measured ln(PNC) nearly as well as the original models and much better than transferred models without calibration (Table 5 and SI Tables S6–S9; Figures S5–S8). The performance of both calibrated transferred and directly transferred models—as measured by R2 and RMSE—was generally highest when applied to Somerville followed by Dorchester, Malden, and Chinatown in that order. This reflects greater spatial and temporal contrasts in pollution patterns in Somerville and Dorchester than Malden and Chinatown and greater spatial-temporal variability in PNC emissions in Chinatown relative to the range of available explanatory variables. Some terms in the calibrated transferred models had signs opposite from those expected a priori (e.g., increasing PNC with distance from I-93 when calibrating the Somerville model for Malden; SI Table S6), indicating that improved performance of the calibrated transferred models was at the expense of physical interpretability rather than true transferability. The fraction of R2 lost by calibrated transferred PNC models relative to neighborhood-specific PNC models was similar to that of calibrated transferred annual average models of NO2, another constituent of motor vehicle exhaust.[4,28] Therefore, transferability of regression models of traffic-related air pollutants reflected the extent of similarity of local sources and physical surroundings in the different neighborhoods.

Table 5

Evaluation of Performance of Neighborhood-Specific and Boston-Area Land Use Regression Models of ln(PNC) when Locally Calibrated in Somerville, Dorchester, Chinatown, And Malden

		area appliedb
modela	statisticc	Somerville	Dorchester	Chinatown	Malden
Somerville	R2	0.43d	0.34	0.21	0.29
	RMSE	0.64	0.65	0.70	0.77
Dorchester	R2	0.39	0.35d	0.19	0.21
	RMSE	0.66	0.63	0.71	0.81
Chinatown	R2	0.32	0.24	0.23d	0.27
	RMSE	0.70	0.68	0.69	0.79
Malden	R2	0.40	0.25	0.21	0.31d
	RMSE	0.65	0.67	0.70	0.76
Boston-area	R2	0.40	0.31	0.20	0.27
	RMSE	0.65	0.67	0.71	0.78

Each row represents a neighborhood-specific or Boston-area (BA) model.

Each column represents the neighborhood where the measurements were predicted.

The reported statistics are R2 = R2 and slope from the simple linear regression between predictions and measurements, RMSE = root-mean-square error between measurements and predictions. Note that the slope is equal to the R2 because for a simple linear regression conducted on the same unit space, the relationships between a random variable (containing both systematic and random components) and prediction (containing the variance of the systematic components) are expected to equalize the slope and the square of a correlation coefficient, which is R2. Values are bold for R2 > 0.2.

Cells on the diagonal represent the performance of models when applied to the neighborhood where they were developed.

Model Generalizability

The Boston-area model predicted the same general trends as the neighborhood-specific models, although with less intraneighborhood variability (Figure 1). The BA model performed better than the transferred models but not as well as the neighborhood-specific models or the calibrated transferred models (Tables 2–5 and SI Table S10; Figure 2). Calibrating the BA model in each neighborhood moderately improved both R2 and RMSE (Table 5 and SI Table S10). All coefficients in the BA model had signs as expected a priori and were statistically significant (p < 0.001). Likewise, each coefficient was within the range of those in the neighborhood models (Table 2). The BA model included variables for temperature, wind speed, distance and direction (relative to wind) from I-93 and major roads, day of week, and traffic congestion. The sector and cosine wind direction variables that were included in neighborhood-specific models were replaced in the BA model with a single categorical variable for high concentration directions that predicted 40% higher PNC when the wind was coming from the direction of Logan Airport and downtown Boston. While we defined this variable according to whether the wind direction was within ±15° of the airport and downtown Boston, the variable should be interpreted as descriptive and not able to discern among sources.

Figure 2

Predicted (PRED) vs measured (MEAS) ln(PNC) (particles/cm3) for all four areas using the Boston-area model calibrated using data from all neighborhoods (transfer PRED, top) and recalibrated with only neighborhood-specific data (Recal PRED, bottom). Shaded contours enclose 25, 50, 75, and 96% of the data.

A more general model does not necessarily require large sacrifices in either R2 or model prediction accuracy. The BA model’s generalizability was greater than expected given the improvement of site-specific models over general models in most previous LUR generalizability studies.[11,25,26,29] Our results are consistent with one recent European study that showed similar performance of regional NO2 and PM2.5 models relative to city-specific models.[28] These results suggest the generalized model may be appropriate for other east-coast American cities with highways, particularly if it were calibrated with local data. Further, developing the BA model resulted in a unified wind direction variable that could potentially improve future transferability by replacing neighborhood-specific variables.

General Discussion

We developed four site-specific neighborhood-scale (0.5–2.3 km2) spatial-temporal PNC regression models that work reasonably well in their respective neighborhoods. These models captured the temporal effects of temperature and wind, but not nucleation or precipitation events due to their low frequency in our data set. We provided a methodology to measure the transferability and generalizability of regression models. We applied these methods to the Boston-area neighborhoods, showing limited direct transferability but reasonably good generalizability and calibrated transferability.

Unmeasured, large-scale interneighborhood differences were likely factors in the poor transferability of our models. That one variable could describe some of the largest wind direction effects (i.e., downwind of Logan airport and downtown Boston) suggests that model transferability was hindered by incomplete characterization of sources outside of the neighborhoods. However, our small study area reduced the effect of some methodological artifacts in models of larger areas—for example, equivalence of the definition of land use variables in different areas[25,26]—and minimized potential differential effects of regional background pollution levels that were observed in other model-transferability studies (e.g., the ESCAPE study in Europe[28]). Our models also benefitted from detailed street network data.

Finding equivalent traffic and meteorological data sets for all locations can be challenging. The lack of detailed real-time neighborhood-specific traffic and wind data for the small neighborhoods in our study may have limited the transferability of the models, particularly when predicting PNC spikes due to traffic and localized wind effects.[5] For future models, the parametrization of traffic could be improved by collecting data on the diurnal and weekly traffic trends on local roads (not just highways). Including other interneighborhood differences in highways and other streets, buildings, and roadside structures (e.g., elevation, street canyons, and shape) may also improve model transferability and generalizability.[39,54,59−62]

Our findings suggest several potential methods to enhance the development of transferable or generalizable PNC LUR models in near-highway neighborhoods. Continuously operated urban or regional background monitoring stations could complement mobile monitoring campaigns to allow for better temporal resolution of local and regional source contributions. Distributed meteorological measurements within study neighborhoods would allow for improved parametrization of microscale meteorological effects.

The neighborhood models had limited transferability; however, our success with the BA model leads us to conclude that it could be calibrated and transferred to other neighborhoods in the Boston area and perhaps other cities. General models like our Boston-area model may be useful to reduce the monitoring effort needed to estimate PNC over areas with neighborhoods that have broadly similar features including vehicle fleet characteristics, dispersion characteristics, and land use.