Methodological issues in descriptive environmental epidemiology. The example of study Sentieri.

BACKGROUND: Descriptive epidemiology identifies associations between environmental exposures and health effects that require results from methodologically stronger studies before causation can be considered.
OBJECTIVE: To critically review the methodology and results of Sentieri, a descripitive study on residence in areas with one or more industrial source of pollution.
METHODS: We systematically reviewed the literature quoted by Sentieri for the selection of health effects of nine types of pollution sources of a-priori interest. We also reviewed and meta-analyzed the results of the first report of Sentieri, that analyzed mortality in 44 polluted sites (PS), and 17 causes of deaths during 1995-2002.
RESULTS: Among 159 study results quoted by Sentieri, 23.9 % were supportive of an association between residence near a pollution source and a health effect, 30.2 % were partially supportive, 10.7 % were not supportive, and 35.2 % were not relevant. Among 653 standardized mortality ratios for associations between PS-specific pollution sources and causes of death, 14.4% were significantly above 1.02, and 9.0% were significantly below 0.98. Among 48 meta-analysis, seven were significantly above 1.0, including five on exposure to asbestos.
CONCLUSIONS: Sentieri exemplifies the limitations of descriptive environmental epidemiology studies, in which most hypotheses have limited prior support, most results do not show associations, data on potential confounders and other sources of bias are not available. Such studies tend to replicate well-known associations and occasionally can identify critical situations requiring more investigation, but cannot be used to infer causality either in general or in specific circumstances.

Introduction

Environmental epidemiology addresses the impact of environmental exposures on human health and investigates how naturally occurring or synthetic risk factors may predispose to or protect against outcomes such as diseases, injuries, developmental abnormalities, or death (61). A feature of a large proportion of environmental epidemiology studies is the use of aggregated data, typically according to geographic areas or temporal periods. Such data are often available from administrative or surveillance programs, which have not been established for research purposes. Establishing causal links between specific environmental exposures and complex, multifactorial diseases and conditions is a challenging endeavor and requires stronger evidence than the one provided by studies based on aggregated data (63).

The project Sentieri, whose acronym stands for Mortality Study of Residents in Polluted Sites (PS), is a descriptive ecologic study, aimed at analyzing mortality and other outcomes among residents in PS of national interest for environmental remediation, in order to confirm previously reported associations (71). PS are located in the proximity of active or dismissed industrial plants, incinerators, hazardous waste landfills or natural sources of pollutants, and are defined according to the boundaries of one or more municipalities, the smallest administrative units in the country. The Authors of Sentieri analyzed 44 PS comprised in the “National Environmental Remediation Programme”. They are located in 17 Italian regions; 21 of them are situated in the North, 8 in the Center and 15 in the South, comprising a total of 298 municipalities (Appendix Table 1). For each PS, the Authors of Sentieri collected data on level of contamination from national and local environmental remediation programs, and developed a classification scheme of PS based on the main sources of environmental pollution. In a series of reports, the Authors analyzed data on mortality, cancer incidence and hospital admissions for the whole set of PS or some subsets (71, 72, 19, 73, 74, 75, 101).

Table 1.

Pollution sites included in the study Sentieri

Pollution site	Sources of environmental exposure	Exposure circumstance	Region	Number of municipalities
Industrial area of Val Basento	Chemical plant, asbestos	Ch, As	Basilicata	6
Industrial area of Porto Torres	Chemical plant, petrolchemical plant, power plant, harbor, waste landfill	Ch, PR, PP, Ha, WL	Sardinia	2
Mount Vesuvius area	Asbestos, waste landfills	As, WL	Campania	11
Sacco river	Chemical plant	Ch	Lazio	9
Balangero	Stone quarry, asbestos, waste landfills	Mi, As, WL	Piedmont	2
Bari	Asbestos	As	Apulia	1
Lower Chienti river	Shoe industry	Ch	Marche	5
Biancavilla	Quarry, asbestos	Mi, As	Sicily	1
Bolzano	Aluminum and magnesium plant	Ch	South Tyrol	1
Brescia	Chemical plant, waste ladfills	Ch, WL	Lombardy	3
Brindisi	Chemical and petrolchemical plant, power plant, harbor, waste landfills	Ch, PR, PP, Ha, WL	Apulia	1
Broni	Asbestos	As	Lombardy	1
Casale Monferrato	Asbestos	As	Piedmont	48
Cengio and Saliceto	Dye production industry, waste landfill	Ch, WL	Liguria	32
Cerro al Lambro	Waste landfills	WL	Lombardy	2
Cogoleto	Plant for the production of sodium dichromate, waste landfill	Ch, WL	Liguria	2
Crotone, Cassano and Cerchiara	Chemical plant, waste landfills	Ch, WL	Calabria	3
Emarèse	Quarry, asbestos, waste landfills	Mi, As, WL	Aosta Valley	1
Falconara	Chemical plant, refinery, power plant	Ch, PR, PP	Marche	1
Fidenza	Chemical plant, urban and special waste landfills	Ch, WL	Emilia Romagna	2
Gela	Chemical plant, petrolchemical plant, refinery, waste landfills	Ch, PR, WL	Sicily	1
Lakes of Mantua	Chemical plant, petrolchemical plant, harbor, waste landfills	Ch, PR, Ha, WL	Lombardy	2
Lagoon of Grado and Marano	Plant for the production of cellulose, dock	Ch, Ha	Friuli Venezia Giulia	6
Domizio-Flegreo littoral	Waste landfills	WL	Campania	77
Livorno	Refinery, harbor	PR, Ha	Tuscany	2
Manfredonia	Chemical plant, waste landfills	Ch, WL	Apulia	2
Massa Carrara	Farmaceuticals, petrolchemical plant, steel plant, harbor, asbestos, waste landfills, incinerator	Ch, PR, IS, Ha, As, WL, In	Tuscany	2
Milazzo	Refinery, steel plant, power plant	PR, IS, PP	Sicily	3
Orbetello	Chemical fertilizer production plant	Ch	Tuscany	1
Pieve Vergonte	Chemical plant, waste landfill	Ch, WL	Piedmont	3
Pioltello and Rodano	Chemical plant, waste landfill	Ch, WL	Lombardy	2
Piombino	Chemical plant, steel plant, thermoelectric power plants, harbor, industrial landfills (hazardous waste)	Ch, IS, PP, Ha, WL	Tuscany	1
Pitelli	Chemical plant, power plant, harbor, asbestos, coal-fired power station storage area	Ch, PP, Ha, As, WL	Liguria	2
Priolo	Chemical plant, petrochemical plant, refinery, harbor, asbestos, waste landfills	Ch, PR, Ha, As, WL	Sicily	4
Sassuolo and Scandiano	Ceramic processing	Ch	Emilia Romagna	6
Serravalle Scrivia	Exhausted oil regeneration plant	Ch	Liguria	2
Sesto San Giovanni	Steel plant, waste landfill	IS, WL	Lombardy	2
Sulcis, Iglesiente and Guspinese	Chemical plant, mines, waste landfill	Ch, Mi, WL	Sardinia	39
Taranto	Refinery, steel plant, harbor, waste landfill	PR, IS, Ha, WL	Apulia	2
Terni	Steel plant, waste landfill	IS, WL	Umbria	1
Tito	Chemical plant, steel plant, asbestos, waste landfill	Ch, IS, As, WL	Basilicata	1
North Trento	Chemical plant	Ch	Trentino	1
Trieste	Chemical plant, refinery, steel plant, harbor	Ch, PR, IS, Ha	Friuli Venezia Giulia	1
Venice (Porto Marghera)	Chemical plant, petrochemical plant, refinery, harbor, waste landfills	Ch, PR, PP, Ha, WL	Veneto	1

Ch, chemical industry; PR, petrochemical industry and oil refinery; IS, iron and steel plant; PP, power plant; Mi, mining; As, asbestos industry; Ha, harbor; WL, waste landfill; In, incinerator.

Given the prominence of Sentieri study in the national public health community, we aimed at assessing whether the project was optimally designed for its objectives, was carried out as it should, and its findings were being interpreted correctly. Our review therefore systematically addresses the methodology used in Sentieri to assess environmental exposure and to select associations to be confirmed (71), and its application to the first report on mortality during 1995-2002 (72), and discusses the validity of its results.

Methods

The aim of the Sentieri study, as stated by its authors, is “to describe and assess the health status of populations in PS, with respect to mortality, in order to help identifying priorities in interventions of environmental remediation aimed at the prevention of pathologies caused by environmental exposures” (71). Sentieri is unique among descriptive studies on health effects of environmental pollutants in small areas in the sense that it was based on a literature review aimed at identifying a priori specific associations between circumstance of exposure to sources of environmental pollution and health outcomes, and to classifiy the strength of the underlying evidence, in order to confirm these associations in an analysis of mortality in Italian PS (71). The authors of Sentieri conducted a series of literature reviews on nine exposure circumstances that were present in the PS under study: chemical plants (Ch), petrochemical plants and oil refineries (PR), iron and steel plants (IS), power plants (PP), mining operations (Mi), harbors (Ha), plants using asbestos and other mineral fibres (As), waste landfills (WL) and incinerators (In). The choice the of death to include in the study was apparently based only on epidemiologic data, and was based, according to the authors, on “the results of previous Italian investigations in the areas at risk and the most recent epidemiologic evidence on the effects of sources of environmental exposures identified in Sentieri for which a cause-effect relationship is suggested but not proven” (71).The review also considered other causes of the diseases and conditions under study, such as air pollution, tobacco smoking, alcohol drinking, socio-economic status and occupation. The authors analyzed 63 causes of death (54 in all ages, 3 in infant aged 0-1 year, and 6 in children agead 0-14 years). For each association between environmental exposure and health outcome the authors classified the strength of the evidence in three categories:

- sufficient to conclude on the presence of a causal association;

- limited but not sufficient to suggest the presence of a causal association;

- inadequate to evaluate the presence or absence of a causal association.

Pirastu et al. (72) applied this exposure scheme to the data on mortality during 1999-2005. For each PS, results were reported for (i) overall mortality and mortality for major causes of death, (ii) causes of death with sufficient or limited evidence of causality, (iii) other causes with statistically significant increase in mortality. For exposure-disease combinations with at least three observed deaths, results were reported as sex-specific standardized mortality ratios (SMR) with their 90% confidence intervals (CI), using the population of the respective region as reference. SMR were also reported after adjustment for a deprivation score based on level of education, unemployment, proportion of rented households, and residential density, measured at the level of municipality (40).

Our critical review addressed these two reports of Sentieri. First, we checked the methodology adopted by the authors of Sentieri to identify the outcomes for the detailed analysis (71). Two authors (CC, NF) reviewed indipendently the studies quoted by the authors of Sentieri to support their classification of sufficient or limited evidence, i.e., studies showing or suggesting an association between the exposure circumstance under consideration and one or more outcomes. For each exposure-outcome combination retained in Sentieri, we classified the results of each study - after consensus with a third author (PB) in case of disagreement - in four categories:

supportive (S): the results support the interpretation of the authors of Sentieri in showing an association (relative risk [RR]>1 & p<0.05 in the primary analysis);

partially supportive (PS): the results are only partially consistent with the interpretation of the authors of Sentieri (RR>1 & p>0.05, or RR>1 & p<0.05 only either in a secondary analysis, e.g., by gender, age group, outcome subgroup, or under special circumstances, e.g., after environmental accidental release);

not supportive (NS): the results do not support the interpretation of the authors of Sentieri (RR<1);

not relevant (NR): no results were reported on the association between exposure and outcome, or results were reported only for unspecific exposure circumstances, e.g., residence near an industrial complex comprising multiple types of industries, exposure to traffic pollution.

In the second step of the analysis, we abstracted the sex-specific (considering children as separate sex) SMRs and 90% CI presented in report on 1995-2002 mortality (72) for the associations in each relevant PS between specific exposure circumstances and causes of death, whose evidence was classified by the authors as sufficient or limited (see above). We escluded results with less than five observed deaths, for which SMR and CI were not reported. We did not consider results of major causes of death, nor results that were not classified of interest a priori.

We classified the results of individual PS in five categories according to the magnitude of the SMR and the level of statistical significance: (i) SMR <0.98 and p <0.05; (ii) SMR <0.98 and p >=0.05; (iii) SMR >=0.98 and SMR <=1.02; (iv) SMR >1.02 and p >=0.05; (v) SMR >1.02 and p <0.05. We tested whether the distribution of results in the five categories was symmetric (i.e., same number of significant and non-significant ‘positive’ and ‘negative’ results) by applying a χ2 test after excluding the central (null) category.

For each association between exposure circumstance and health outcome with results for at least three PS, we performed a random-effects meta-analysis [DerSimonian & Laird, 1986] of the sex-specific SMR across the relevant PS. We quantified the heterogeneity between PS-specific results using the I2 test (44). We report meta-relative risk (RR) and their 95% CI.

Results

Evaluation of the associations between specific exposure circumstances and health outcomes

As mentioned above, Sentieri considered nine exposure circumstances: chemical industry, petrochemical industry, iron and steel industry, power plants, mining, harbor, asbestos and other mineral fibres, hazardous waste landfills and incinerators. Table 1 summarizes the evaluations made by the authors of Sentieri of the evidence linking residence near each of these sources of exposure and health effects. For two combinations (mining and pleural cancer and asbestos and pleural cancer) the evidence was considered sufficient to conclude for a causal relationship; for 38 additional combinations – which were not mutually exclusive since some health effects overlapped, such as respiratory diseases and acute respiratory diseases – the evidence was considered limited.

Table 1

Evaluation of the strength of the evidence on exposure to circumstances of environmental pollution and health effects in Sentieri (71)

	Exposure circumstance
Health effect	Ch	PR	IS	PP	Mi	As	Ha	WL	In
Gastric cancer	L								L
Colorectal cancer	L
Liver cancer									L
Lung cancer		L		L		L			L
Pleural cancer					S	S	L
Soft tissue sarcoma									L
Ovarian cancer						L
Lymphohematopoietic neopl.									L
Non Hodgkin lymphoma									L
Respiratory diseases	L	L	L	L			L
Acute respiratory diseases		L	L	L
Chronic respiratory diseases			L
Asthma	L	L	L	L			L
Congenital malformations		L						L
Perinatal conditions	L	L						L
Respiratory diseases (children)		L	L
Asthma (children)	L	L	L	L

L (light shading): limited evidence; S (dark shading): sufficient evidence

Ch, chemical industry; PR, petrochemical plant and refinery; IS, iron and steel plant; PP, power plant; Mi, mining; As, asbestos industry; Ha, harbor; WL, waste landfill; In, incinerator

In the following paragraphs we summarize the assessment of the reviews of literature performed by the authors of Sentieri. A detailed review of the individual studies is included in Appendix 1.

Chemical industry

The review of the literature on six health effects potentially associated with residence near a chemical industry included 20 studies, providing a total of 27 results (Table 2). Only one result was supportive of an association, and three additional results were partially supportive. A total of 22 results were not relevant to test the hypothesis of a role of residence near a chemical industry on various health effects.

Table 2

Associations between residence near a chemical industry and selected health outcomes

Study	Gastric cancer	Colo-rectal cancer	Respiratory diseases	Asthma	Perinatal conditions	Asthma (children)
Clapp et al., 2005 (17)	NR	NR
US EPA, 2000 (87)		NR
Dahlgren et al., 2003 (21)			PS	NS		NR
Fung et al., 2007 (35)			NR	NR		NR
Kordysh et al., 2005 (53)			NR
Ware et al., 1993 (91)			PS*	PS*		S
Bobak et al., 1999a (10)					NR
Bobak et al., 1999b (11)					NR
Bobak et al., 2001 (12)					NR
Dejmek et al., 2000 (22)					NR
WHO, 2002 (28)					NR
Jedrychowski et al., 2004 (47)					NR
Kallen et al., 2000 (49)					NR
Kanitz et al., 1996 (50)					NR
Klotz et al., 1999 (52)					NR
Perera et al., 1998 (68)					NR
Ritz et al., 2000 (82)					NR
Ritz et al., 2002 (83)					NR
Rylander et al., 1995 (85)					NR
Sram et al., 2005 (86)					NR

S, supportive; PS, partially supportive; NR, not relevant (see text for details)

* results from studies of children applied to adults

Petrochemical industry and oil refinery

The review of the literature on six health effects potentially associated with residence near a petrochemical industry and oil refinery included 17 studies, providing a total of 36 results (Table 3). Results of studies included in the reviews of lung cancer, respiratory diseases (either overall, acute or chronic), asthma in adults, respiratory diseases in children and asthma in children were supportive or partially supportive of an association. In the case of perinatal conditions, one of the three studies included in the review supported the association. The review of congenital malformations included two studies conducted by the same researchers whose results supported the hypothesis of an association, in particular for hypospadias (7,8).

Table 3

Associations between residence near a petrochemical industry or oil refinery and selected health outcomes

Study	Lung cancer	Respiratory diseases	Acute respiratory diseases	Asthma	Respiratory diseases (children)	Asthma (children)	Perinatal conditions
Belli et al., 2004 (5)	PS
Bhopal et al., 1998 (6)	PS
Edwards et al., 2006 (25)	NR
Gottlieb et al., 1982 (39)	PS
Pirastu et al., 2007 (70)	NS
Yang et al., 2000 (99)	PS
Yang et al., 1999 (98)	PS
Bowler et al., 2002 (15)		PS	PS	PS		NR*
Ware et al., 1993 (91)		PS*		PS*		S
Wichmann et al., 2009 (92)		PS*	PS*	PS*	S	S
Yang et al., 1998 (97)		PS*	PS*		PS
Loyo-Berrios, 2007 (56)						S
Lin et al., 2004 (55)							NR
Oliveira et al., 2002 (65)							PS
Xu et al., 1998 (95)							NR

S, supportive; PS, partially supportive; NS, not supportive; NR, not relevant (see text for details)

* results from studies of children applied to adults (or vice-versa)

Iron and steel industry

The review of the literature addressed four health effects in adults and two in children potentially associated with residence near an iron and steel industry and included seven studies, providing a total of 25 results (Table 4). Seven of these results concerned diseases in adults but were derived from studies in children. Fifteen results, including six on adults derived from studies in children, supported (fully or partially) the interpretation of the authors of Sentieri. The remaining results were not relevant to the health effects under consideration.

Table 4

Associations between residence near an iron and steel industry and selected health outcomes

Study	Respiratory diseases	Acute respiratory diseases	Chronic respiratory diseases	Asthma	Acute respiratory diseases (children)	Asthma (children)
Bhopal et al., 1998 (6)	NR		NR	NR		NR
Cara et al., 2007 (16)	PS*	PS*			S
Forastiere et al., 1994 (33)					NR
Lewis et al., 1998 (54)	PS*	PS*		NR	PS	NR
Petrela et al., 2001 (69)	NR
Pope et al., 1991 (76)	S	S	PS	PS	PS	S
Wilhelm et al., 2007 (93)	PS*	PS*		NS*	PS	NS

S, supportive; PS, partially supportive; NS, not supportive; NR, not relevant (see text for details)

* results from studies of children applied to adults

Power plants

The review of the literature on four health effects in adults and one in children potentially associated with residence near a power plant included eight studies, providing a total of 21 results (Table 5). Two of the results supported the interpretation of the authors of Sentieri, and four were partially supportive, while the remaining 15 results were not relevant to the possible effects of residence near a power plant, including five results of studies in children that were applied to adult diseases.

Table 5

Associations between residence near a power plant and selected health outcomes

Study	Lung cancer	Respiratory diseases	Acute respiratory diseases	Asthma	Asthma (children)
Parodi et al., 2003 (66)	PS
Fano et al., 2006 (29)		NR		NR	NR
Forastiere et al., 1992 (32)		PS*	PS*	S*	S
Forastiere et al., 1994 (33)		NR*	NR*	NR*	NR
Karavus et al., 2002 (51)		PS	NR	NR*
Aekplakorn et al., 2003 (1)				NR*	NR
Henry et al., 1991 (43)				NR*	NR
Peled et al., 2005 (67)				NR*	NR

S, supportive; PS, partially supportive; NS, not supportive; NR, not relevant (see text for details)

* results from studies of children applied to adults

Mines

Residence near a mine was linked to pleural cancer. Four studies were quoted, all of which supported the association (14, 41, 64, 78).

Asbestos and other mineral fibers

The review of effects of residence near an asbestos or other mineral fibers plant included seven papers: three were not supportive, because they concerned three health effects, each providing a separate result (13, 30, 37). Two studies on pleural cancer were supportive of the hypothesis of an association (58, 62) and one study each on lung and ovarian cancer, respectively, were either partially supportive (79) or not relevant (80).

Harbors

The review of effects of residence near a harbor concerned three health effects and included four papers, providing a total of six results. Four of these results were partially supportive of an association between residence near a harbor and respectively pleural cancer (45, 62) and either respiratory diseases or asthma (2, 4).

Hazardous waste landfills

The review of the literature on two health effects in children potentially associated with residence near a hazardous waste landfill included 13 studies, providing a total of 17 results (Table 6). Four results supported the interpretation of the authors of Sentieri (two on congenital malformations and two on perinatal conditions), four results were partially supportive, while five results did not support the interpretation; the remaining four results were not relevant to the research question.

Table 6

Associations between residence near a hazardous waste landfill and selected health outcomes in children

Study	Congenital malformations	Perinatal conditions
Croen et al., 1997 (20)	PS
Dolk et al., 1998 (23)	PS
Dolk et al,. 2003 (24)	NR
WHO, 2002 (28)	NS	NS
Geschwind et al., 1992 (37)	S
Goldberg et al., 1995 (38)	NR	NR
Marshall et al., 1997 (60)	NS
Porta et al., 2009 (77)	S	PS
Vianna et al., 1984 (88)	NR
Vrijheid et al., 2000 (90)	PS	NS
Elliot et al., 2001 (27)		S
Johnson, 1999 (48)		S
Rushton et al., 2003 (84)		NS

S, supportive; PS, partially supportive; NS, not supportive; NR, not relevant (see text for details)

Incinerators

The review of the literature on health effects in children potentially associated with residence near a incinerator comprised six non-independent neoplasms. The review included 10 studies, providing a total of 16 results (Table 7). Eight results supported the interpretation of the authors of Sentieri of an association with liver cancer, lung cancer, soft tissue sarcoma and non Hodgkin lymphoma), and two additional results partially supported the interpretation.

Table 7

Associations between residence near an incinerator and selected health outcomes

Study	Stomach cancer	Liver cancer	Lung cancer	Soft tissue sarcoma	Lymphohe-matopoietic neoplasms	Non-Hodgkin lymphoma
Franchini et al., 2004 (34)	NS
Rushton, 2003 (84)	NS
Porta et al., 2009 (77)		S	S	S	PS	S
Elliott et al., 1996 (26)			S
WHO, 2007 (94)				NS	NS	NS
Zambon et al., 2007 (100)				S
Biggeri et al., 2005 (9)				PS
Comba et al., 2003 (18)				S
Floret et al., 2004 (31)				NS
Viel et al., 2000 (89)				S

S, supportive; PS, partially supportive; NS, not supportive (see text for details)

Overall assessment

A total of 159 results were quoted by the authors of Sentieri to support the evaluation of sufficient or limited evidence for an association between residence near nine circumstances of environmental pollution and 17 health outcomes. Among them, 38 results (23.9 %) supported the interpretation of the authors, 48 (30.2 %) offered partial support, 17 (10.7 %) were relevant but did not support the interpretation of the authors, and 56 (35.2 %) were not relevant to the research question.

Review and meta-analysis of the results of the mortality analysis

The report on 1995-2002 mortality (72) reported 653 sex-specific SMRs for associations between exposure circumstances and causes of death classified by the authors as having sufficient or limited evidence of causality. Their distribution is reported in Table 8, and their distribution according to magnitude of the SMR and level of statistical significance is shown in Figure 1. A total of 59 SMR (9.0%) were lower than 0.98 and statistically significant, 223 (34.1%) were lower than 0.98 but not statistically significant, 63 (9.7%) were between 0.98 and 1.02 (and none was significantly different from 1.0), 214 (32.8%) were higher than 1.02 but not statistically significant, and 94 (14.4%) were significantly higher than 1.02. Most results in the latter category concerned chemical industry, petrochemical and oil refinery, mining and asbestos plants. The p-value of the χ2 test for symmetry of the distribution in the five categories shown in Figure 1 was 0.04. Since results on pleural cancer contributed a large proportion of positive and significant results, reflecting the well-known increase in this neoplasm among residents near asbestos plants (59), we repeated the analysis excluding the results for pleural cancer: the p-value of this analysis was 0.12. The cumulative distribution of the log-transformed SMR is shown in Figure 2, confirming the symmetric distribution of the results, with the exception of a few elevated SMR in the top part of the distribution.

Table 8

Number of SMRs in the first report of Sentieri (72), by exposure circumstance and outcome

Outcome	Ch	PC	IS	PP	Mi	As	Ha	WL	In	Total
Gastric cancer	61								2	63
Colorectal cancer	62									62
Liver cancer									2	2
Lungcancer		24		12		2			2	58
Pleural cancer					3	17	21			41
Soft tissue sarcoma									2	2
Ovarian cancer						9				9
LHP									2	2
Non Hodgkin lymphoma									2	2
Respiratory diseases	62	24	16	12			24			138
Acute respiratory diseases		24	15	12						51
Chronic respiratory diseases			16							16
Asthma	46	22	11	10			20			109
Congenital malformations		12						24		36
Perinatal conditions	27	12						23		62
Respiratory diseases (child)
Asthma (child)
Total	258	118	58	46	3	46	65	47	12	653

Ch, chemical industry; PC, petrochemical plant; IS, iron and steel plant; PP, power plant; Mi, mining; As, asbestos industry; Ha, harbor; WL, waste landfill; In, incinerator; LHP, lymphohematopoietic neoplasms

Figure 1

Distribution of SMR in the report of Sentieri (72) by exposure circumstance, magnitude of the SMR and level of statistical significance

Figure 2

Cumulative distribution of logarithms of SMR in the report of Sentieri (72)

We conducted 48 meta-analysis of the sex-specific SMR, by outcome and exposure circumstance. Results are reported in Table 9. We did not conduct any meta-analysis for residence near incinerators, since these results were reported for only one PS: among the six outcomes, increased SMR (with p<0.05) were reported for liver cancer in both sexes and lung cancer in men.

Table 9

Meta-analyses of SMRs of sex-specific specific associations between exposure circumstances and selected outcomes (72)

Exposure circumstance and outcome	Men					Women
	N	RR	95% CI	p-het.	I2%	N	RR	95% CI	p-het.	I2%
Chemical industry
Gastric cancer	31	1.01	0.95 - 1.08	<0.001	54.0	30	0.97	0.91 -1.04	0.045	32.7
Colorectal cancer	31	1.01	0.97 - 1.05	0.40	4.4	31	1.02	0.97 -1.07	0.035	34.0
Respiratory diseases	31	0.98	0.90 - 1.06	<0.001	92.7	31	0.95	0.89 -1.02	<0.001	80.6
Asthma	23	1.04	0.85 - 1.26	0.073	31.8	23	1.02	0.85 -1.24	0.12	26.8
Perinatal conditons*	27	1.06	0.96 - 1.19	0.17	20.5
Petrochemical plant
Lung cancer	12	1.10	1.04 - 1.17	<0.001	81.4	12	1.09	0.98 -1.21	<0.001	89.3
Respiratory diseases	12	0.98	0.89 - 1.07	<0.001	89.9	12	1.04	0.94 -1.14	<0.001	89.3
Acute respiratory diseases	12	1.08	0.91 - 1.28	<0.001	85.9	12	1.06	0.85-1.27	<0.001	89.0
Asthma	11	0.80	0.60 - 1.08	0.07	41.1	11	0.91	0.74 -1.11	0.41	3.1
Congenital malformations*	12	1.07	0.95 - 1.20	0.37	7.5
Perinatal conditons*	12	1.07	0.92 - 1.25	0.07	40.1
Iron and steel plant
Respiratory diseases	8	1.09	0.99 - 1.20	<0.001	83.7	8	0.99	0.89 -1.10	<0.001	77.8
Acute respiratory diseases	8	1.11	0.92 - 1.33	<0.001	76.4	7	1.02	0.86 -1.21	<0.001	76.8
Chronic pulmonary dis.	8	1.07	0.91 - 1.25	<0.001	87.7	8	0.97	0.88 -1.07	0.16	33.7
Asthma	6	0.80	0.54 - 1.19	0.20	31.3	5	0.88	0.67 -1.16	0.61	0
Power plant
Lung cancer	6	1.04	1.01 - 1.08	0.60	0	6	1.08	0.92 -1.27	0.03	59.6
Respiratory diseases	6	0.94	0.82 - 1.07	<0.001	85.8	6	1.03	0.87 -1.23	<0.001	89.6
Acute respiratory diseases	6	0.94	0.66 - 1.33	<0.001	90.4	6	1.05	0.77 -1.44	<0.001	90.9
Asthma	5	0.83	0.57 - 1.20	0.45	0	5	0.97	0.72 -1.31	0.99	0
Asbestos industry
Lung cancer	10	1.10	1.01 - 1.19	<0.001	82.7	10	1.03	0.91 -1.17	0.01	58.4
Pleural cancer	9	3.28	1.84 - 5.82	<0.001	96.5	8	2.67	1.13 -6.30	<0.001	95.6
Ovarian cancer						9	1.01	0.92 -1.11	0.92	0
Harbor
Pleural cancer	12	1.67	1.36 - 2.04	<0.001	74.1	9	1.32	1.07 -1.63	0.82	0
Respiratory diseases	12	1.01	0.92 - 1.10	<0.001	89.4	12	0.98	0.89 -1.08	<0.001	87.3
Asthma	10	0.81	0.60 - 1.08	<0.001	41.6	10	0.90	0.73 -1.10	0.38	6.6
Waste landfill
Congenital malformations*	24	1.01	0.95 - 1.08	0.74	0
Perinatal conditons*	23	0.98	0.89 - 1.09	0.04	37.0

N, number of studies; RR, relative risk; CI, confidence interval; p-het, p-value of test for heterogeneity; I2, I2 test for heterogeneity (44)

* Results apply to children of both sexes

The distribution of the 48 meta-analysis by magnitude of the summary RR and the level of statistical significance is shown in Figure 3. In 11 meta-analysis (23%) the summary RR was lower than 0.98, and in 22 (46%) it was higher than 1.02. In seven meta-analysis (15%) the summary RR was statistically significant: petrochemical industry and lung cancer in men (RR=1.10), power plant and lung cancer in men (RR=1.04), harbor and pleural cancer in men (RR=1.67) and women (RR=1.32), mines and pleural cancer in men (RR=3.28) and women (RR=2.67) and mines and lung cancer in men (RR=1.10). Most meta-analyses showed a high level of heterogeneity between PS-specific results: in 30 out of 48 (63%) the p-value of the test for heterogeneity was lower than 0.05.

Figure 3

Distribution of meta-analyses by magnitude of summary RR and level of statistical significance

Discussion

The study Sentieri is an ambitious attempt to develop a systematic tool for the surveillance of potential health effects of residence near industrial sources of pollution. It comprised a novel methodological approach based on reviews of the literature to select a priori exposure-disease associations of potential interest. These associations formed the core of the analysis of mortality in Italian PS. However, our critical review identified important limitations in its design, that bore on the validity of its results. The reviews performed by the authors of Sentieri comprised a large proportion of irrelevant and negative results (47% of all quoted results). While it is probably true that for several of these associations few high-quality studies were reported in the literature, this does not justify relying on results that are not (or only partially) relevant. It is not surprising therefore that the results of the mortality analysis were in most part null. However, given that the purpose of the investigation was targeted surveillance, the fact that relatively few clear signals were identified, may not be necessarily a problem.

It is important to note that we did not evaluate the quality and validity of the studies quoted by the authors of Sentieri, but only whether the results supported the interpretation made by the same authors. In particular, in several studies that were classified as supportive or partially supportive of an association between residence near an industrial source and a health effect, potential confounding and other sources of bias were not addressed. Similarly, we did not try to evaluate the overall evidence underlying each of the associations selected by the authors of Sentieri, including studies published after 2011. However, we reviewed the literature to identify relevant systematic reviews and meta-analyses, that are summarized in Supplementary Table 1 (only for associations with limited or sufficient evidence in Sentieri, excluding asbestos). The studies reported in Supplementary Table 1 offer limited or no support to the hypothesis of an association between the exposure circumstances and the outcomes addressed by Sentieri.

The only health effect clearly identified in Sentieri was an increased in mortality from pleural cancer in the PS with sources of asbestos exposure. An increased risk of pleural mesothelioma in residents near asbestos plants has been shown in numerous studies and in meta-analyses (59). Besides replicating this well-established association, the contribution of Sentieri to the scientific knowledge is limited. In addition plueral cancer, only three exposure-outcome associations included in the analysis yielded a significant result in the meta-analysis, and they all concerned lung cancer in men: residence near petrochemical industry (RR=1.10), power plant (RR=1.04), and mines (RR=1.10). The lack of a corresponding result in women detracts from their credibility in terms of causality of the environmental exposure. Alternative explanations include a confounding effect by employment in the same or other industries and by tobacco smoking, which would both be more prevalent among men. In fact, given the strong association between tobacco smoking and lung cancer, even modest differences in the habit between the male population of the PS and that of the region, used as reference, would explain the effect attributed to residential exposure. Using the indirect method to adjust for the confounding effect of tobacco smoking proposed by Axelson and Steenland (3), under the assumption of a distribution of tobacco smoking in the referent population equal to that of Italian men (46) (55% never smokers, 25% former smokers [RR=3.9] and 20% current smokers [RR=9.0; (36)], a RR in the order of 1.10 would be explained by an increase of 3% among former and current smokers, and a corresponding decrease of 6% in never smokers. Results of studies from Italy suggest that uncontrolled confounding by smoking in occupational studies can explain difference in risk of lung cancer in the order of 10-30% (81).

The high level of heterogeneity detected in the meta-analyses of results for associations between specific exposure circumstances and outcomes suggest that other factors, which vary across PS, are responsible for the mortality in the studies areas, rather than the exposure to the industrial sites under study. These other determinants of mortality do not appear to be adequately controlled by the deprivation score used in Sentieri.

A further consideration is the fact that the large number of analyses might have generated statistically results by chance. Assuming independence between the SMR (a weak assumption because common causes are likely to influence results in the two sexes, or in SIN with similar socio-economic characteristics), 5% of SMR results were expected to be statistically significantly different from 1.0. This corresponds to approximately 16 positive and 16 negative SMR, compared to 59 significantly decreased and 94 significantly increased SMR observed in the analysis. In other words, approximately one fourth of negative results and one sixth of positive results can be simply explained by chance. An additional concern is the choice of the authors to report 90% confidence intervals, that may be justified in the context of an exploratory study, but not when the results aim at confirming previously reported associations, as in the case of Sentieri.

We assessed the specificity of the reviews conducted by the authors of Sentieri, which was 53% if we combine supportive and partially supportive results. Conversely, we did not systematically assess the sensitivity of the reviews, i.e., whether any relevant study was ignored. However, we selected the association between residence near a petrochemical industry and risk of lung cancer as case study for sensitivity. A literature search revealed at least two studies that reported relevant results and were published before 2010 (42, 96): the results of both studies did not show an increased risk of lung cancer.

Descriptive epidemiology studies, in particular when they are based on ecologic-level data on exposure and outcome (63), are considered a tool for generation of hypotheses that need to be tested in more rigorous studies that are based on information collected at the individual level and include provisions for protection from bias, and its quantification. Based on these premises, results of descriptive studies cannot contribute to causal inference (63). The study Sentieri does not appear to be an exception, and the small number of identified associations reinforces the inappropriateness of the study design to test specific causal hypotheses. While the selection of association of interest might be considered an improvement over the agnostic approach of typical descriptive studies, we showed that the validity of this process was jeopardized by the inclusion of irrelevant studies, and the lack of consideration to negative studies. In addition, other limitations of descriptive studies apply to Sentieri, including lack of appropriate adjustment for potential confounders, such as tobacco smoking, overweight and obesity, and diet, as well as exposure misclassification from lack of consideration to other sources of pollution, such as traffic. Results of Sentieri should therefore be interpreted as exploratory according to the commonly adopted criteria for descriptive studies, and adequate to report potential problems in individual PS. Like any other descriptive epidemiology study, Sentieri should not be used to identify specific associations between environmental exposures and health outcomes, and even less to establish their causal nature.