Authors' reply to the letter to editor "Acid suppressant use in association with incidence and severe outcomes of COVID‑19: a systematic review and meta‑analysis".

To the editor:

We would like to appreciate your opinions and comments by Prof. Gou-Fu Li and Guo Yu with respect to our systematic review of acid suppressant use and the incidence and severe outcomes of COVID-19. We have replied on a point-by-point basis, with emphasis especially on aspects of our choice of statistical and methodological evaluation, and indicate that it is not necessary to change the conclusions we have drawn.

First, regarding the exclusion of non-peer-reviewed (“unpublished”) studies, “Cochrane Handbook for Systematic Reviews of Interventions” by Higgins and Green had made it clear that the inclusion of data from unpublished studies can introduce bias and the studies that can be located might be an unrepresentative sample of all unpublished studies. In addition, unpublished studies might be of lower methodological quality than published studies [1]. Before analyzing only published studies, we did perform a meta-analysis including unpublished studies in advance. These unpublished studies included three case–control studies [2, 3, 4] and two retrospective studies [5, 6]. This analysis including these unpublished studies yielded primary results similar to those after including only published studies. Specifically, a significant association between PPI use and severe outcomes of COVID-19 was observed (OR/RR = 1.47, 95% CI, 1.20–1.80, I2 = 75.7%) and there was no significant association between PPI use and incidence of COVID-19 (OR/RR = 1.10, 95% CI, 0.77–1.56, I2 = 84.6%). Moreover, no publication bias was found in our meta-analysis on the effect of PPI use on adverse outcomes as evidenced by the symmetrical Begg’s funnel plots and a P-value of 0.57 for the Egger’s test. We note to avoid misunderstanding that the time of analysis including unpublished studies was March 2021. In addition, while acknowledging the urgency of COVID-19, it is reasonable to exclude unpublished studies from the time of our search considering the points mentioned above.

Additionally, the leave-one-out method used as sensitivity analysis is no longer recommended in Cochrane systematic reviews. While meta-analyses increase the statistical power and the precision of ascertaining the studied effect, excluding any single study most likely will change the pooled effect estimate numerically, potentially significantly. Furthermore, inclusion of repeated analyses using a leave-one-out approach increases the likelihood of observing a statistical significant by chance alone due to Type I error inflation [7].

Finally, although several of the unpublished studies have now been published in peer-reviewed journals, as noted in the letter to the editor, they were unpublished at the time the manuscript was submitted, we could not include them in our study. Furthermore, two of the papers [8, 9] indicated in the letter that have been withdrawn after strict peer-review did not examine the impact of PPI or H2RA use on COVID-19 outcomes.

We acknowledge that the Newcastle–Ottawa Scale (NOS) used to rate the methodological quality of observational studies included in our meta-analysis has not yet been published in a peer-reviewed journal [10]. However, the NOS is one of the most commonly used tools for evaluating the methodological quality of studies in a systematic review [11]. Furthermore, use of an unpublished tool to examine the quality of included studies and inclusion of published studies in a meta-analysis are two separate issues.

In the entire manuscript, indeed we reported odds ratio/relative risk (OR/RR)s but erroneously described these as HRs, and this should be changed in the entire manuscript.

Finally, the standard Cochrane method clearly indicates that meta-regression can be implemented for categorical variables with multiple levels by implementing dummy variables with a referent category [1]. Examples of this can be found in the tables presented in several meta-analyses [12, 13]. Though not specifically stated in our Methods, we did use dummy variables for the application of meta-regression.