Non-peer-reviewed data, effect measures, and meta-regression analysis on proton pump inhibitor use and COVID-19.

The debate on the use of proton pump inhibitors (PPIs) and COVID-19 outcomes remains, even after the presentation of our comprehensive meta research [1]. Kim et al. [2] should be commended for their updated meta-analysis reassuring the positive association of current PPI use and severe outcomes of COVID-19, which is consistent with our findings [1]. We agree that the updated meta-analysis by Kim et al. [2] looks much better than another four updated (or just repeated) meta-analyses in many aspects, such as additional subgroup analyses and meta-regression, specified in the Discussion section and their Supplemental Table 5. However, we believe that the discussion on non-peer-reviewed studies (or “unpublished data”) in their review [2] warrants careful clarifications. Moreover, the effect measure and meta-regression used in the meta-analysis by Kim et al. [2] should be clearly characterized.

Instead, we do not think the exclusion of preprint studies, particularly for those on COVID-19 during the early phase of this pandemic, should be viewed either as an advantage or a limitation. First, most non-peer-reviewed studies assessing associations between PPI use and COVID-19 outcomes which were presented initially as preprints have now been published in peer-reviewed journals [3-8], without dramatic changes in the clinical data collected retrospectively or prospectively. Second, several studies which had gone through strictly peer-review processes have been retracted because of unreliable data [9, 10]. Third, the inclusion of preprint studies would not introduce additional biases into meta-analyses if a subgroup analysis on peer-reviewed status (peer-reviewed studies vs. non-peer-reviewed studies), sensitivity analysis restricting to peer-reviewed studies, or relevant leave-one-out analysis is performed. Last, the Newcastle–Ottawa Scale (NOS) [11] used by Kim et al. [2] to rate the methodological quality of observational studies included in their meta-analysis has not yet been published in a peer-reviewed journal, which do not compromise the importance and utility of NOS in meta-analyses at all.

Regarding the effect measure selected in the meta-analysis, Kim et al. [2] should have specified it more clearly and reasonably. The extraction of adjusted odds ratio (OR) was stated in the Data extraction section, whereas hazard ratios (HRs) rather than ORs were claimed to be pooled in the Data synthesis Sect. [2] It should be clarified whether Kim et al. [2] excluded studies reporting data as OR only instead of HR or whether ORs were converted to HRs. If the former is true, the rationale for the exclusion of those studies with ORs should be provided. If the latter is true, it should be clearly described what kind of method under what assumptions did Kim et al. [2] employ for such conversions.

Additionally, Kim et al. [2] reported results of meta-regression for both categorical variables (such as “research location,” “administration time of PPIs,” “active use of PPIs,” or “geographical region”) and continuous variables. Unfortunately, the method on meta-regression for categorical variables was not described. Namely, it was not clear how did such meta-regression implement for categorical variables with multiple levels, e.g., geographical region. It seems that the meta-regression in this paper might not follow the standard Cochrane method which specifies a nominated reference subgroup and estimates the differences between the nominated reference subgroup and each non-reference subgroup by using dummy variables which can only be given values of 0 or 1 [12]. Instead, there was no a nominated reference subgroup in their meta-regression analysis of geographical region stratified by North America, Europe, and Asia.

Although we applaud for the updated meta-analysis by Kim et al. [2] we believe strongly that these points necessitate clear clarifications.