Metaplot: a novel stata graph for assessing heterogeneity at a glance.

BACKGROUND: Heterogeneity is usually a major concern in meta-analysis. Although there are some statistical approaches for assessing variability across studies, here we present a new approach to heterogeneity using "MetaPlot" that investigate the influence of a single study on the overall heterogeneity.
METHODS: MetaPlot is a two-way (x, y) graph, which can be considered as a complementary graphical approach for testing heterogeneity. This method shows graphically as well as numerically the results of an influence analysis, in which Higgins' I(2) statistic with 95% (Confidence interval) CI are computed omitting one study in each turn and then are plotted against reciprocal of standard error (1/SE) or "precision". In this graph, "1/SE" lies on x axis and "I(2) results" lies on y axe.
RESULTS: Having a first glance at MetaPlot, one can predict to what extent omission of a single study may influence the overall heterogeneity. The precision on x-axis enables us to distinguish the size of each trial. The graph describes I(2) statistic with 95% CI graphically as well as numerically in one view for prompt comparison. It is possible to implement MetaPlot for meta-analysis of different types of outcome data and summary measures.
CONCLUSION: This method presents a simple graphical approach to identify an outlier and its effect on overall heterogeneity at a glance. We wish to suggest MetaPlot to Stata experts to prepare its module for the software.

Introduction

Meta-analysis is a useful statistical technique for combining multiple quantitative data. This method is usually used in the context of a systematic review of the literature. However not all systematic reviews include meta-analysis to estimate an overall summary measure (1). Since the results in meta-analysis are combined from different studies, which have not followed a common protocol, heterogeneity is usually a major concern in meta-analysis (2). Indeed, heterogeneity will always exist whether or not we are able to detect it (3). Thus, it is crucial to understand the limitations of meta-analysis beside its benefits and the importance of detecting sources of variability across studies (4). We should keep in mind that dealing with heterogeneity is not straightforward (2).

Although there are some statistical approaches for assessing variability across studies, we aim to present a new approach to heterogeneity using MetaPlot that investigate the influence of a single study on the overall heterogeneity.

Material and Methods

We developed a two-way graph for Stata statistical software, which can be considered as a complementary graphical approach for testing heterogeneity. We named this graph “MetaPlot”. This graph consists of a two axes (x, y) with “reciprocal of standard error” (1/SE) or “precision” on x axis, “Higgins’ I2 statistic” with its 95% confidence interval (CI) on y axis. In this approach, first, the overall I2 statistic is computed. Then, by omitting one study in each turn, I2 statistic is recomputed to investigate the effect of each study on the overall heterogeneity. Thus, there are N+1 estimates of I2 statistic. Finally, these computed numerical data are plotted on Meta-Plot in order of studies’ precision, therefore, by having a first glance at the graph; one can predict immediately to what extent omission of each study may affect the total heterogeneity

Results

In this article, we used the same dataset (5) used by Higgins et al. (6) and we used Stata 9 statistical program for calculation of all numerical data presented in the graph (Fig. 1).

Fig. 1:

Meta-Analyses of Residential Exposure to Electromagnetic Fields and Childhood Leukemia[; MetaPlot delineates I2 and Q statistics against precision

The green curve shows I2 variations across studies with 95% CI [vertical red lines]. “Overall” on the extreme right side of the graph indicates the overall I2 value without omitting any study from the analysis. Precision on x-axis (blue curve) helps to distinguish the size of each trial. By glancing at the graph, one can easily investigate to what extent omission of a single trial may affect the overall heterogeneity and which trial has the most influence. Indeed, MetaPlot determines how much I2 would be changed if a specific trial is omitted from the analysis. We can also identify easily the “outliers” and their effects on the variability across studies.

In this example, the overall I2 estimate without omission of any study from the analysis is 69% with 95% CI: 26%, 87%. In this meta-analysis, “Wertheimer” study acts as an outlier. If this study is omitted from the analysis, the I2 statistics would decrease to 31% with 95% CI: 0%, 73%which indicates non-significant heterogeneity between studies.

Discussion

This method is a simple way to identify an outlier and its effect on overall heterogeneity. It is possible to implement this graph for meta-analysis of different types of outcome data (e.g. binary, continuous or time to event) and different types of summary measures (e.g. odds ratio, risk ratio, rate ratio or hazard ratio).

In conclusion, MetaPlot is a visual complementary approach for testing heterogeneity which investigates the influence of a single study on overall heterogeneity. This method shows graphically as well as numerically the results of an influence analysis, in which I2 statistic with 95% CI is computed omitting one study in each turn. It is possible to implement this graph for meta-analysis of different types of outcome data as well as summary measures. We wish to suggest this method to Stata experts to prepare its module for the software.

Ethical Consideration

All Ethical issues (such as informed consent, conflict of interest, plagiarism, misconduct, co-authorship, double submission, etc) have been considered carefully.