Ozone exposure and health effects: a protocol for an umbrella review and effect-specific systematic maps.

INTRODUCTION: Ambient ozone exposure may be adverse to health. Since the reported associations between ozone and health effects are heterogeneous and the underlying pathways are indistinct, the overall relationship remains unclear. Only a few overall syntheses of the evidence regarding ozone and health effects are available to date. METHODS AND ANALYSIS: We plan to summarise the current evidence on ozone-related health effects systematically. First, to identify the possible associations between ambient ozone exposure and health outcomes, we will conduct an umbrella review. PubMed, Web of Science and grey literature will be searched for systematic reviews on exposure to ambient ozone and any possible health endpoints published before 31 May 2019. Data selection and extraction will be carried out by one reviewer, and a second reviewer will check the agreement of a sample of the studies. The methodological quality of the eligible systematic reviews and level of evidence regarding ozone and every specific health effect will be evaluated. Second, for each of the identified effects with a high level of evidence, comprehensive information retrievals will be conducted, considering both epidemiological and experimental studies. The study selection and data mapping will be carried out by one reviewer and checked by the second reviewer. We will summarise the information of the filtered epidemiological and experimental studies to conduct several systematic maps presenting the currently available evidence for the specific health effect. Because the association between ozone exposure and chronic obstructive pulmonary disease (COPD) is relatively well investigated, we will at least conduct one systematic map of ozone and COPD. ETHICS AND DISSEMINATION: No ethical approval is required for this study. The completed umbrella review and systematic maps will be considered for publication and presentation. We will additionally upload the relevant data to publicly accessible online databases. PROSPERO REGISTRATION NUMBER: CRD42019123064. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To the best of our knowledge, this protocol is the first one for an umbrella review or a systematic map focusing on ozone exposure and health outcomes.

The umbrella review will provide a comprehensive outline of the currently available systematic reviews on ambient ozone exposure and health outcomes, and an epidemiological evidence reservoir accessible to the public.

The systematic map of a specific ozone-related health effect would be the first of this kind to synthesise evidence from both epidemiological and experimental studies.

Since the umbrella review will include only published systematic reviews, the quality of evidence will, thus, depend on the included reviews, and some latest original high-quality studies may be neglected.

Publication bias and quality of the original studies might affect the evidence synthesised by a systematic map.

Introduction

Ambient air pollution has been considered a leading cause of the global burden of disease (GBD).1 According to the GBD study, air pollution was estimated to account for 4.9 million deaths (95% uncertainty interval (UI) 4.4–5.49 million) and 147.0 million (132.0–162.0 million) disability-adjusted life years (DALYs) in 2017.2

Although the majority of the air pollution-related burden is attributed to particulate matter (PM) in the GBD studies,2–4 the burden from ambient ozone pollution is also alarming. In 2017, ambient ozone exposure caused 472 000 (95% UI 177 000–768 000) deaths and a loss of 7.4 million (2.7–12.0 million) DALYs.2 In addition, considering climate change, global warming and increased emissions of ozone precursors, the long-term ambient ozone is expected to increase in concentration.5 Thus, ozone-related mortality might grow in the future.6

The association between ozone and chronic obstructive pulmonary disease (COPD) is regarded to be in line with the basis of evidence rules by the GBD study group,2 and COPD is, thus, the only one ozone-related health outcome included in the GBD study 2017.2 A lack of evidence for other effects is largely explained by ozone is under investigated compared with PM, and because based on the hierarchy of evidence,7 the conducted epidemiological observational studies are not considered as a high quality.

Regarding the evidence, a systematic review is a standard method for summarising and analysing available evidence on health issues. Systematic reviews are regarded to be of high evidence hierarchy and of low risk of bias.7 Currently, as systematic reviews have synthesised a growing number of original studies, more evidence is accumulating. From these reviews, we can find out ozone exposure linked to different kinds of health effects, not only to the respiratory system but also to the cardio-cerebrovascular,8 9 central nervous system10 11 or mental health.12 13 Consequently, the possibility of tracing evidence up to date has been overwhelmed by the rapidly increased number of systematic reviews.14

Remarkably, an umbrella review, systematically reviewing previously published systematic reviews, could generate a higher level of the hierarchical evidence7 and has attracted increasing research attention. This trend can be identified by the number of newly published umbrella reviews15 16 and protocols17–19 on various topics. An umbrella review would be a feasible way to outline the associations between ozone exposure and health effects.

On the other hand, biological mechanisms that possibly lie behind the associations reported by epidemiological studies are relatively unclear—although exposure to ozone has been postulated to be associated with adverse health effects via oxidative stress and inflammatory response.20 21 More and more efforts are trying to bridge the gap between epidemiological associations and biological relevance, yet they are largely restricted to descriptive discussion of the results from relevant experimental studies22 23 or narrative review of studies with a similar setting.24 There still is an absence of any systematic syntheses of evidence on a specific ozone-related health effect.

Therefore, a systematic map25 26 or a systematic evidence map,27 which uses systematic search and strategical selection but seeks no evidence synthesis, is an appropriate emerging method to comprehensively summarise and catalogue the ozone-related broad and miscellaneous evidence from original experimental studies, as well as original epidemiological studies.

On this background, we aim to perform an umbrella review on ambient ozone exposure and health effects by systematically reviewing existing systematic reviews. We will additionally supplement those significant ozone-effect pairs filtered from the umbrella review with original epidemiological and experimental evidence, to provide a more comprehensive picture on ozone and health. This study will (1) identify exposure-related health effects of ambient ozone by an umbrella review and (2) integrate current available epidemiological studies, and cross-reference the effects with experimental studies, by conducting systematic maps.

Methods and analysis

This project has been registered on the International Prospective Register of Systematic Reviews (https://www.crd.york.ac.uk/prospero/) on 24 April 2019.28 The present protocol was developed and modified in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 checklist29 and the guidance on systematic maps published by the Collaboration for Environmental Evidence.30 31

Patient and public involvement

This study will have no patient or public involvement and will collect no primary data.

Study design

The project will be divided into two processes. The first process will be an umbrella review. In other words, a systematic review to identify the published systematic reviews on the association between ambient ozone exposure and any health endpoints. We will further grade the evidence of ozone-effect pairs based on parameters from the systematic reviews with meta-analysis.32 33 The second process will be the development of systematic maps for the specific health effects with high evidence levels identified by the umbrella review. For example, if COPD were to be identified as an ozone-related health effect with a high evidence level, we would conduct a systematic map of ozone exposure and COPD. Every systematic map will contain all of the published original epidemiological studies, as well as the currently available experimental studies of the specific health effect.

Umbrella review

In our registered protocol,28 we specified that the first process will contain three search paths: a systematic review of systematic reviews, of the burden of disease studies and of relevant study reports. The current protocol has been renamed ‘systematic review of systematic reviews’ to ‘umbrella review’, and contains no systematic review of the burden of disease studies. The detailed table of PRISMA-P is listed in the Supplementary (online supplementary table S1).

The process of the umbrella review will be guided by the PRISMA guideline.34 Given no agreed method for conducting an umbrella review,7 our methodology will be performed in accordance with previously published umbrella review on risk factors (eg, dietary factor, lifestyle, medical history and socioeconomic status) and health effects.32 35–37

Eligibility criteria

Types of participant

The general human population will be considered, regardless of age, sex, race, region, as well as health states.

Types of exposure (intervention)

The exposure is ambient ozone. Indoor and occupational exposure will not be considered.

Types of comparator

Comparisons are varied across studies. Generally, the comparison is between the participants or periods with lower levels of ambient ozone exposure and the more highly exposed populations or periods.

Types of outcome

Any possible health effects, such as symptoms, conditions, diseases, morbidity, mortality, considering both the long-term or chronic effects and the short-term or acute effects, will be our outcomes of interest. Additionally, several possible indirect indicators, such as restricted activity days, days with symptoms or hospital admissions, will be involved where applicable.

Inclusion criteria

We will include the systematic reviews that investigated ambient ozone exposure and health effects, published before 31 May 2019, written in English or German.

Exclusion criteria

We will not include articles, abstracts, dissertations or letters. Cell or animal studies and botanical studies will be excluded. The papers about indoor or occupational ozone exposure and clinical studies on ozone therapy will not be considered.

Information source and search strategy

Databases to be searched will include PubMed and Web of Science. Combinations of both free terms and Medical Subject Headings (MeSH) connected with ‘ozone’ and ‘systematic review’ or ‘meta-analysis’ will be used for search (online supplementary table S2).

Reference lists of included studies will be searched manually for potentially relevant papers. Additionally, grey literature, such as reports from relevant institutes, will be searched on their websites. A list of relevant institutes (online supplementary table S3) is adopted and updated from a previous project.38

Study management and selection

The assessed studies will be imported to EndNote V.X8 software. After deduplication, the selection process contains two stages: an initial screening of the titles and abstracts based on the aforementioned criteria and followed by a second screening of the full texts of the papers filtered by the initial screening. The entire process will be illustrated by a PRISMA flow chart.

One member of the reviewer team (TZ) will independently conduct the study selection. A second reviewer (NSM) will check the agreement of a randomly selected sample of the studies (at least 10%). The strength of the agreement will be calculated by the Kappa score. Further disagreements will be determined through consensus by a third member (JH).

Data extraction

A predesigned Microsoft Excel table39 will be revised and used for extracting data from the selected studies (online supplementary file 2). The extracted information will include the first author, year of publication, journal, type of study (systematic review with or without meta-analysis), search results (name of database, date of search, number of hits), key information of included studies (the overall population–exposure–comparator–outcome (PECO) or population–intervention–comparator–outcome (PICO) statement) and the summarised ozone-related results of the included studies, as well as the method of quality assessment and/or risk of bias used by the studies. When the data of interest are incomplete, the corresponding author will be contacted for acquiring additional information. Furthermore, for the involved systematic reviews with meta-analysis, data on the total number of cases, effect estimates (eg, OR or risk ratio), confidence interval (CI), p value, as well as results about heterogeneity (eg, the I2 statistic, the Q value and the associated p value) and results on publication bias (eg, Egger test) will be recorded. One member of the reviewer team (TZ) will extract the information. The second reviewer (NSM) will check the agreement of a randomly selected sample of the studies (more than 10%).

Study assessment

We will adopt ‘a measurement tool to assess systematic reviews 2’ (AMSTAR2) criteria40 to evaluate the methodological quality of included systematic reviews and meta-analyses. The AMSTAR2 contains 16 items in total, and 7 out of them are considered as critical domains. The assessment by AMSTAR2 generates no score but an overall rating of the review, that is, high, moderate, critically low and low. A detailed description of AMSTAR2 can be accessed elsewhere.40

Data analysis

For the included systematic reviews, we will qualitatively describe their main results in a summary table. The table will contain the first author, year of publication, key information of included studies (the overall PECO or PICO statement), the summarised ozone-related results and the results of AMSTAR2. Moreover, information on the total number of cases, effect estimates, CI, p value, heterogeneity and publication bias will be listed for the systematic reviews with meta-analysis.

Apart from the descriptive information, quantity analysis will be conducted for meta-analysis where applicable. In case two or more studies present overlapping datasets of original studies on the same health effect, we will retain the study with the largest datasets.32 A random effect p value of a meta-analysis will be mainly adopted. A prediction interval41 indicating the heterogeneity between studies and estimating the uncertainty of effect estimate in a future new study will be calculated based on the extracted data of the identified meta-analysis.

A standard formula for a prediction interval based on k studies can be obtained as

Where is a point estimate of the heterogeneity variance, is the percentile of the t-distribution with k−2 degrees of freedom.41

In order to compare the different effect estimates across studies, we will recalculate the various effect estimates into ORs.33 39

Finally, a classification of levels of evidence (table 1) summarised by the previously published umbrella reviews35–37 will be used and updated to grade the ozone–health effect pairs. The health effects classified to have classes I–III evidence will be considered to have a high level of evidence.

Table 1

Levels of evidence

Evidence	Class	Requirement
Convincing	Class I	Number of cases >1000, meta-analysis p value <10−6, between study heterogeneity I2 <50% and 95% prediction interval excluding the null; no detected publication bias
Highly suggestive	Class II	Number of cases >1000, meta-analysis p value <10−6, the largest study with a statistically significant effect and class I criteria not meet
Suggestive	Class III	Number of cases> 1000, meta-analysis p value <10−3 and class I or II criteria not meet
Weak	Class IV	Meta-analysis p value <0.05 and classes I–III criteria not meet
Non-significant	Null	meta-analysis p value> 0.05

Synthesis of results

The results of this umbrella review will include the flow chart of the literature search process, the aforementioned summary table, and a synthesis table for meta-analysis containing the results of the quantitative analysis. We anticipate discussing the overall descriptive results of the assessed systematic reviews, and the classification of levels of evidence regarding ozone–health effect pairs.

Systematic map

Based on the results of the umbrella review, the second process of the project will focus on the identified ozone-related health outcomes with high evidence levels (class I, II or III). However, given COPD is widely investigated and included already in the GBD study,2 we will conduct a systematic map of ozone and COPD regardless of the confidence level stemmed from the umbrella review.

Unlike the registered protocol,28 we revised the second process of this project. The updated systematic map will cover both the published epidemiological studies and experimental studies. The current PRISMA-P table in Supplementary (online supplementary table S4) is adopted from the RepOrting standards for Systematic Evidence Syntheses42 and the journal Environment International.31

A revised PRISMA reporting guidelines and checklist43 will be used to guide the process of systematic maps. The planned systematic maps will mirror the guidelines established by the National Toxicology Program/Office of Health Assessment, Development and Translation (OHAT)44 as well.

Types of PECO or PICO

A general PECO or PICO statement for systematic maps is listed in table 2.

Table 2

PECO or PICO statement for systematic maps

Element	Description
Populations	Any human (epidemiological studies or human exposure studies), or animal, or ex vivo/in vitro studies using organs, tissues, cells, or cellular components, for example, cell-free receptor binding assays (experimental or toxicological studies)
Exposures or Intervention	Ambient ozone exposure in epidemiological studies or ozone exposure in experimental or toxicological settings
Comparator	Study populations or periods (person-time) exposed to a lower level of ozone (epidemiological studies) or the control groups (experimental or toxicological studies) than the more highly exposed subjects, periods (person-time) or groups
Outcomes	COPD and any health effects with high evidence levels identified in the umbrella review

COPD, chronic obstructive pulmonary disease; PECO, population–exposure–comparator–outcome; PICO, population–intervention–comparator–outcome.

Inclusion and exclusion criteria

The studies included in systematic maps should contain original research investigating ozone exposure and the health effects with high levels of evidence identified by the umbrella review. The epidemiological, experimental or toxicological studies in line with the PECO or PICO statement will be included. Articles on clinical ozone therapy and botanical studies will be excluded. Reviews, letters or conference abstracts will not be involved in systematic maps.

We will have no requirements on publication dates for a systematic map. Articles published in the English language will be focused on.

The search will be conducted in PubMed and Web of Science, using both free terms and MeSH terms for ozone and the identified health effects. Reference lists of included articles and systematic reviews on the same topic identified by the umbrella review will be searched manually for potentially relevant papers.

Since the results of the umbrella review will provide outcomes of systematic maps, we currently cannot report the entire search strategy. However, regarding the case of COPD, a general strategy would be “(ozone OR O3) AND (“chronic obstructive pulmonary disease” OR COPD)”.

One member of the review team (TZ) will conduct the study selection. The second reviewer (NSM) will check the agreement of a sample of the studies (more than 10%). A third member (JH) would be involved in the case of disagreement. The assessed studies will be imported to EndNote V.X8 software. After deduplication, we will upload the data to Health Assessment Workspace Collaborative (HAWC, https://hawcproject.org/) for management and screening. HAWC is an online open-source platform providing a transparent method for study selection, data extraction, data assessment, evidence synthesis and data visualisation.45 46

Data coding strategy

After the full-text screening, a coding tool will be designed and updated to extract and record data from the included studies. Based on the OHAT protocol,44 45 extracted information could involve: study identification information (first author, year of publication), type of study (epidemiological or animal, or ex vivo/in vitro study), key information of included studies (the overall PECO or PICO statement), as well as the summarised conclusions of a study. Additional information about conflict of interest, funding statements and acknowledgements will be extracted likewise. Similarly, one reviewer (NSM) will check the other’s (TZ) work with a sample of the studies (more than 10%).

Study assessment and data mapping

For the systematic maps, no assessments or analyses will be considered.

For a specific health effect, a systematic map will present evidence assessed from epidemiological, human exposure, animal and ex vivo/in vitro studies separately. A table will be prepared for narratively mapping the extracted data. We will also present the result by a display in Tableau Public (https://public.tableau.com/en-us/s/). An interactive map contains the above-mentioned information in the table will be easily and freely accessed online.47

We plan to develop systematic maps for each health effect identified via the umbrella review that is supported by a high evidence level. The results of a systematic map will include a flow chart recording the process of study selection, a table presenting the general information and the online display. Regarding a determined ozone-related health effect, we will discuss the available evidence, current paucity and future suggestions.

Pilot study

Information retrieval for the umbrella review

We conducted a preliminary search for the umbrella review. Our search strategy yielded 1867 hits on 31 May 2019. After deduplication and a primary screening, this number was reduced to 100 (figure 1).

Figure 1

Flow chart illustrating the preliminary literature search and first-step study selection of systematic reviews on ozone exposure and health outcome (published prior to 31 May 2019).

Search string of a systematic map on COPD

We conducted a presearch for the systematic map on COPD in PubMed with the string “(ozone[tiab] OR O3[tiab]) AND (“chronic obstructive pulmonary disease”[tiab] OR COPD[tiab])” on 22 September 2019. There were 254 hits.

Ethics and dissemination

Since this study will only collect and analyse data from published and accessible studies, approval from an ethics committee is not required.

The umbrella review and systematic maps will be disseminated in reports and peer-reviewed journals, and if applicable, will also be presented at relevant conferences. The data of systematic maps would be publicly available at the website of HAWC and Tableau Public.