Bayesian inference for asymptomatic COVID-19 infection rates.

To strengthen inferences meta-analyses are commonly used to summarize information from a set of independent studies. In some cases, though, the data may not satisfy the assumptions underlying the meta-analysis. Using three Bayesian methods that have a more general structure than the common meta-analytic ones, we can show the extent and nature of the pooling that is justified statistically. In this article, we reanalyze data from several reviews whose objective is to make inference about the COVID-19 asymptomatic infection rate. When it is unlikely that all of the true effect sizes come from a single source researchers should be cautious about pooling the data from all of the studies. Our findings and methodology are applicable to other COVID-19 outcome variables, and more generally.

INTRODUCTION

Meta‐analyses are commonly used to summarize information from a set of independent experiments, observational studies or sample surveys. Doing this may strengthen inferences when there are deficiencies in the individual studies such as small sample sizes. Methodology for combining findings from repeated research studies has a long history and, in particular, meta‐analyses have become very popular over the past 30 years. From an online search for “books meta‐analysis” we found 49 books. Thus, it was natural that early in 2020 several meta‐analyses were conducted (and subsequently published) about infection rates from the novel coronavirus. Looking at several early review papers we were concerned whether the meta‐analyses were carried out in an appropriate manner. Even after careful evaluation to include only studies thought to be comparable, there may be subsets of the collection of studies where the true (subset) effects are very different. If this is so, pooling the data from all of the studies may result in misleading conclusions. Borenstein et al. add: “If the variation is substantial, then we might want to shift our focus…Rather it should be on the fact that the … effect differs from study to study. Hopefully, it would be possible to identify reasons … that might explain the dispersion.”

In this article, we consider three Bayesian methods that have a more general structure. One can use these methods to check the validity of the more standard approaches by investigating whether the set of true effect sizes come from a common source. If the assumptions underlying the standard approaches are not met, our proposed methodology will lead to more appropriate inferences.

From five review papers we selected for further analysis several studies that have different features. In each of these cases the objective is to make inference about the asymptomatic infection rate.

Please note that we are not evaluating specific meta‐analytic methods. Our concern is about appropriate aggregation of possibly disparate data.

Buitrago‐Garcia et al. explain the importance of a review: “Accurate estimates of the proportions of true asymptomatic and presymptomatic infections are needed urgently because their contribution to overall SARS‐CoV‐2 transmission at the population level will determine the appropriate balance of control measures. If the predominant route of transmission is from people who have symptoms, then strategies should focus on testing, followed by isolation of infected individuals and quarantine of their contacts. If, however, most transmission is from people without symptoms, social distancing measures that reduce contact with people who might be infectious should be prioritized, enhanced by active case‐finding through testing of asymptomatic people.” Referring to a narrative review report (Oran and Topol ) that presents a range (over studies) of 6% to 96% for the proportion of individuals positive for SARS‐CoV‐2 but asymptomatic, the authors point out the need for a careful review.

Standard meta‐analyses typically assume that the true effect sizes, come from a common source. Even after including only those studies thought to be comparable, may be composed of distinct subsets, each with a different underlying distribution. This seems likely for some of the reviews, for example, the 79 rates in Buitrago‐Garcia et al. that range from 0.01 to 0.92. To make appropriate inferences the three Bayesian methods have a more general structure than that assumed in a standard meta‐analysis. The principal method, termed uncertain pooling, is flexible in that it can identify distinct subsets of : For example, for subsets there would be true effect sizes, . Then, pooling the data from all of the studies may lead to misleading inferences. This methodology will also indicate when true effect sizes have a common source, thus leading to an appropriate inference. The more general structure should ensure greater concordance of the data with our model than with a more restricted model. A better fitting model should lead to better inference. Specifically, only similar studies will be combined. It is not surprising that there is strong statistical evidence that in three of the four data sets that we analyze (Section 4) the true effect sizes do not come from a single source. Then the analyst should be cautious about combining the data from all of the studies.

For a general discussion of Bayesian methods for meta‐analysis see Schmid, Carlin and Welton. Borenstein et al. is a basic treatment of fixed and random effects models for a meta‐analysis while Rice, Higgins and Lumley re‐evaluate fixed effect(s) meta‐analysis.

Section 2 has brief descriptions of the data sets that we analyze, together with background information. The methodology is introduced in Section 3 while the results are summarized in Section 4. Section 5 has a brief summary and an extension that accommodates study level covariates with notes about the availability of covariates in the reviews we investigate.

STUDY DESCRIPTIONS

This section has brief descriptions of the meta‐analyses that we have analyzed together with some background information. The definitions of asymptomatic infection rate and the conditions required for including individual studies in the meta‐analysis differ considerably, and are too detailed to present all of them in this article.

The first paper that we considered was by He et al. Using data from six studies they obtain estimates of the asymptomatic infection rate, noting that these measures differ considerably over the six studies, and explaining that this may be due to “differences in data collection, sample size, and the conditions.” Since the information from one of the six studies is inconsistent with that in the other five studies we include only the latter in our analyses. As seen in Table 1 the sample proportions range from 0.22 to 0.78 with little or no clustering.

TABLE 1

Sample effect sizes, SEs, and posterior summaries from the basic uncertain pooling and DPmeta methods and binomial‐beta model for five COVID‐19 studies of the asymptomatic infection rate

	InvBeta prior for δ^2				DPmeta
Study	Effect	PostMean	SE	95% Cred Int	M=1/5	M=5
1	0.310	0.337	0.128	(0.139, 0.644)	0.292	0.348
2	0.565	0.582	0.103	(0.354, 0.766)	0.655	0.573
3	0.217	0.224	0.045	(0.144, 0.322)	0.246	0.238
4	0.667	0.663	0.061	(0.535, 0.779)	0.701	0.662
5	0.783	0.776	0.032	(0.706, 0.837)	0.743	0.769
		Reversible jump MCMC
Study		PostMean		95% Cred Int
1		0.273		(0.145, 0.566)
2		0.650		(0.445, 0.780)
3		0.233		(0.152, 0.325)
4		0.682		(0.547, 0.780)
5		0.759		(0.689, 0.827)

The first meta‐analysis concerning only the asymptomatic coronavirus disease rate is He et al. In their Section 1 the authors note the importance of studying the asymptomatic rate and that this rate is not “well characterized.” They conduct meta‐analyses for all 41 studies and five subsets. The rates are markedly heterogeneous (proportions from 0.02 to 0.75 and total numbers of cases from 4 to 44 627), suggesting concern about the aggregation. Our analysis is for the children subgroup, with eleven studies. As seen in Table 2, the sample proportions range from 0.11 to 0.57 while the standard errors (SEs) range from 0.01 to 0.16. Unlike the first meta‐analysis, there is some apparent clustering.

TABLE 2

Sample effect sizes, SEs, and posterior summaries from the basic uncertain pooling and DPmeta methods and binomial‐beta model for eleven COVID‐19 children studies of the asymptomatic infection rate

	InvBeta prior for δ^2				DPmeta
Study	Effect	PostMean	SE	95% Cred Int	M=1/6	M=6
1	0.129	0.132	0.012	(0.109, 0.157)	0.139	0.133
2	0.158	0.157	0.027	(0.114, 0.220)	0.142	0.155
3	0.530	0.526	0.046	(0.436, 0.613)	0.507	0.516
4	0.278	0.278	0.074	(0.134, 0.489)	0.231	0.271
5	0.129	0.156	0.060	(0.066, 0.308)	0.144	0.159
6	0.500	0.481	0.125	(0.247, 0.682)	0.497	0.455
7	0.571	0.516	0.132	(0.282, 0.736)	0.501	0.482
8	0.154	0.203	0.100	(0.065, 0.519)	0.163	0.192
9	0.200	0.243	0.126	(0.082, 0.557)	0.188	0.221
10	0.111	0.210	0.104	(0.037, 0.554)	0.173	0.197
11	0.556	0.487	0.165	(0.191, 0.753)	0.488	0.449
		Reversible jump MCMC
Study		PostMean		95% Cred Int
1		0.133		(0.109, 0.159)
2		0.157		(0.112, 0.215)
3		0.523		(0.430, 0.612)
4		0.272		(0.131, 0.489)
5		0.153		(0.084, 0.257)
6		0.490		(0.274, 0.658)
7		0.524		(0.276, 0.726)
8		0.174		(0.084, 0.378)
9		0.223		(0.088, 0.526)
10		0.171		(0.071, 0.411)
11		0.518		(0.266, 0.724)

The third data set is a subset of six of the eleven studies in He et al. These six studies were chosen to illustrate properties when there is considerable separation. From Table 3, there are two apparent clusters with cluster proportions of about 0.13 and 0.55; overall, the SEs range from 0.01 to 0.16.

TABLE 3

Sample effect sizes, SEs, and posterior summaries from the basic uncertain pooling and DPmeta methods and binomial‐beta model for six COVID‐19 children studies of the asymptomatic infection rate

	InvBeta prior for δ^2				DPmeta
Study	Effect	PostMean	SE	95% Cred Int	M=1/6	M=6
1	0.129	0.132	0.012	(0.109, 0.157)	0.135	0.132
2	0.158	0.150	0.027	(0.113, 0.211)	0.137	0.152
5	0.129	0.143	0.060	(0.066, 0.260)	0.138	0.152
6	0.500	0.521	0.125	(0.307, 0.708)	0.501	0.461
7	0.571	0.548	0.132	(0.343, 0.747)	0.504	0.492
11	0.556	0.537	0.165	(0.265, 0.769)	0.495	0.463
		Reversible jump MCMC
Study		PostMean		95% Cred Int
1		0.132		(0.109, 0.157)
2		0.150		(0.109, 0.200)
5		0.142		(0.082, 0.220)
6		0.528		(0.345, 0.705)
7		0.536		(0.342, 0.716)
11		0.546		(0.359, 0.741)

Buitrago‐Garcia et al. has several important features, that is, the authors consider rates associated with both asymptomatic and presymptomatic cases and only include studies that document follow‐up and symptom status at the beginning and end of follow‐up or modeling. Their meta‐analyses are based on 79 studies, summarized for the entire set and seven subsets. The 79 rates are markedly heterogeneous (proportions from 0.01 to 0.92 and total numbers of cases from 2 to 1012). Our analysis is for the screening subgroup with seven studies, noted by Buitrago‐Garcia et al. as being of special interest. Here, the proportions range from 0.17 to 0.50 while the SEs range from 0.04 to 0.35.

For additional background we give the criteria that He et al. used to select the studies that they used in their meta‐analyses. Then we summarize features of the eleven children study, from He et al., that we have analyzed. He et al. searched two databases, PubMed and Embase, following the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guideline. They included the following items: “COVID‐19” and analogous phrases and “Asymptomatic.” They included “articles reporting a specific number of asymptomatic infection cases in confirmed COVID‐19 patients, information describing the epidemiological and clinical features of COVID‐19.” There is no evidence of a risk‐of‐bias assessment or consideration of a sufficient follow‐up period. Byambasuren et al. identify these as characteristics essential when making decisions about which studies to include in a meta‐analysis. Since He et al. is the first meta‐analysis to make inferences about the asymptomatic infection rate, one may conjecture that they were motivated to publish their results quickly.

Of the eleven studies of children five papers are published in Chinese, so there are only summaries in English. Only three of the eleven papers give the age distribution of the children, although most give the mean age and some give the range. Seven of the papers give the sex distribution. In five studies all cases were associated with a single hospital while four studies summarized the results from many hospitals. There was no information for two studies. Several papers noted that most of the patients had a history of close contact with adults with COVID‐19.

METHODOLOGY

A common assumption in situations where combining data is plausible is: For the are independent where , the are known, is the number of studies and is the number of replicates. Note that all of the analyses of COVID‐19 data that we consider make assumptions like (1).

Uncertain pooling

The first method, uncertain pooling, is based on Malec and Sedransk and Evans and Sedransk. Since this method may be unfamiliar, we describe it in some detail. They showed that a prior for can be selected to reflect the beliefs that there are subsets of such that the in each subset are “similar”, and that there is uncertainty about the composition of such subsets of Let be the total number of partitions of the set Denote a particular partition by , let denote the number of subsets of in the th partition , and let denote the set of study labels in subset for For our analyses , and 11 with , and respectively. For other values of the total number of partitions of an ‐element set is given by the Bell number, Recent work (eg, Dahl et al. ) proposed using prior information to place a prior on the set of partitions This will increase the complexity of the computations but avoid the need to consider the partitions explicitly.

To specify a prior for first condition on Malec and Sedransk and Evans and Sedransk assume that there is independence between subsets, and within the are independent with Also, the are mutually independent with Conditioning on the variances above (but suppressing them in our notation), and letting leads to the following expected results. Defining and letting and where

Inference about includes uncertainty about the value of that is, where the notation is simplified by using integration rather than summation for

To evaluate (6) we need One must be careful about specifying how the because the models corresponding to the partitions have different numbers of parameters. We use a method described in Section 3 of Evans and Sedransk that postulates little prior, relative to sample, information about the and is invariant to changes in the scale of . Let and be the Kullback‐Leibler information about . With prior and letting the subject to = constant The factor in the exponent, a consequence of the limit process just described, is the usual within sum of squares from a conventional, weighted, analysis of variance. Now, is likely to decrease as increases, for example for a new partition of obtained by creating subsets of the existing . Since increases as decreases, it is helpful to have the second term, , that penalizes partitions with larger values of .

For our analysis we take and write Inference for is made using  (6) and (7) with where the conditional posterior moments of are given in (3) and (4).

Our analyses will indicate whether the true effect sizes come from a common source. If so, then using a standard meta‐analysis will provide appropriate inference. If not, several alternatives should be considered, as discussed below.

If a prior evaluation indicates that one of the studies, , can be regarded as a gold standard we can consider the posterior distribution corresponding to study to be the object of inference. Then, using the posterior expected value for illustration, where is defined in (3). Thus, inference for is a function of together with data from the other studies as determined by the form of (3), and, critically, by the likelihood associated with the set of subsets, , containing study See Evans and Sedransk for additional details and an application to a notable study of the effect of using aspirin by patients following a myocardial infarction.

Otherwise, one must rely on substantive evaluation to decide whether any distinct subsets identified in the analysis should be analyzed separately, for example, by separate standard meta‐analyses. If there are no covariates that can distinguish the subsets, then the distribution of the true effect for study is a mixture distribution with unknown probabilities associated with the components.

If there are distinct subsets and a single analysis is presented it is important to include a credible interval for the overall true effect. Taken together with the presence of distinct subsets, a very wide interval for the overall true effect would be a strong indication that the single summary rate is not informative.

Dirichlet process mixture

An alternative to the uncertain pooling method is to use a Dirichlet process mixture (DPM), one of the most popular nonparametric Bayesian methods. This methodology, presented in detail in sections 2.1 and 2.2 of Müller et al., is outlined below, paraphrasing the text in Müller et al. Suppose that there is an observed i.i.d. sample For Bayesian inference a prior probability model on is needed. Starting with the basic Dirichlet process (DP), let and , the base distribution, be a probability measure defined on the sample space . A DP with parameters , is a random probability measure defined on which assigns probability to every measurable set The DP is an infinite‐dimensional analog of the finite‐dimensional Dirichlet prior. In particular, . If , the precision parameter, is large is highly concentrated around ; as the process is essentially

With DP is a discrete measure, so it is typical to extend the DP to DPM, by using a mixture over a simple parametric form such as a pdf. Let be a finite dimensional parameter space. For each , let be a continuous pdf. Given a probability distribution defined on , a mixture of with respect to has the pdf This mixture model can be expressed as an equivalent hierarchical model, especially relevant for our application, that is, with .

For our analysis we have used the R function DPmeta from the package DPpackage: see Jara et al. for additional details.

The model in DPmeta, for fixed, is (9) and (10) with the pdf of a random variable, and

The (independent) hyperparameters are The uncertain pooling method requires only that one specify a prior distribution for and By contrast DPmeta requires substantial prior input, that is, values for and We have concerns about the sensitivity of inferences to some of the choices of distributions in DPmeta and possible over‐fitting since there are more quantities to be specified than in the uncertain pooling method. Moreover, our analyses include data from only five, six, seven, and eleven studies. So, we have omitted the specification and made inferences for a selected set of values of as suggested by Escobar. Also, we omitted the step, , and replaced and with their maximum likelihood estimates.

Binomial‐beta model and reversible jump Markov chain Monte Carlo

As noted by a reviewer, a limitation of our uncertain pooling method is that the sample SEs are assumed to be known, as is typically done. There is contemporary research (Yao et al. ) that models both the sample mean and the log of the sample SE. However, they assume a bivariate normal distribution for these two statistics, a questionable assumption for our (binomial) case. An alternative that we have investigated assumes a binomial likelihood together with beta and uniform prior distributions in a hierarchical model. The limitation, here, is that one must use Reversible Jump Markov Chain Monte Carlo (RJMCMC) for the computations, and implementation is substantially more difficult than the method presented in Section 3.1.

Recall that the methodology used in Section 3.1 is based on Evans and Sedransk who use a constraint in the limit process to overcome the problem that the partitions, , have different sizes, that is, that varies with Without this adjustment would not be invariant to changes in the scale of the outcome variable, RJMCMC addresses the problem of varying by introducing additional random variables that enable the matching of parameter space dimensions across the partitions. For a concise outline of the RJMCMC method see Gelman et al. The pioneering paper (Green ) provides the theoretical background for RJMCMC, but also includes, as an example, an application to the uncertain pooling methodology. We have used this model and RJMCMC to analyze the data in the four tables in Section 4.

Assume independent responses, , with Within

The group mean parameters, , are drawn independently from while log (log , log Finally, .

Then the joint distribution of all the variables is with where the 1 in the last expression is the pdf of the (assumed) uniform distribution for

Section 6.2 of Green gives the full conditionals for and The step involving a possible move from partition to a new partition is much more complicated. Green uses a process that jumps between partitions making only the changes of splitting a group (a birth) and combining two groups (a death). There is an algorithm to select the groups to split and merge. Then births are attempted with probability and deaths with probability Jumping to a new partition requires a change in the vector since its length must increase or decrease by one unit. Several steps are required to develop the associated proposal, finally leading to a complicated acceptance probability.

For the pool‐all partition Hamza, van Houwelingen, and Stijnen suggest with and As an alternative it may be feasible to adapt this to the specification in (11), and implement it using RJMCMC.

RESULTS AND DISCUSSION

Our inferences are for the asymptomatic rates, that is, population proportions, and Tables 1, 2, 3, 4 use this representation. That is, in Tables 1, 2, 3, 4 we present the sample proportions and the SEs. The posterior means and intervals from basic uncertain pooling (Section 3.1) and summaries from DPmeta (Section 3.2) and RJMCMC (Section 3.3) are for the population proportion. However, for the computations for basic uncertain pooling and DPmeta we have used two different representations of in (1), that is, is the sample proportion, , and is the sample log odds, . While we only present results for the latter representation, our findings are similar for both choices. From (1), is replaced by As is typically done in applications such as this, for example, DerSimonian and Laird, we have replaced with an estimate from the sample. For each of the meta‐analyses we give in Table A1 the number of asymptomatic cases and observations for each of the component studies together with references where these data can be found.

TABLE 4

Sample effect sizes, SEs, and posterior summaries from the basic uncertain pooling and DPmeta methods and binomial‐beta model for seven COVID‐19 screening studies of the asymptomatic infection rate

	InvBeta prior for δ^2				DPmeta
Study	Effect	PostMean	SE	95% Cred Int	M=1/6	M=6
1	0.500	0.377	0.353	(0.119, 0.838)	0.310	0.300
2	0.500	0.389	0.250	(0.148, 0.782)	0.310	0.307
3	0.333	0.332	0.136	(0.156, 0.568)	0.310	0.300
4	0.167	0.226	0.068	(0.088, 0.385)	0.302	0.253
5	0.273	0.288	0.067	(0.173, 0.416)	0.307	0.285
6	0.397	0.382	0.057	(0.283, 0.498)	0.318	0.351
7	0.297	0.300	0.038	(0.229, 0.380)	0.308	0.296
		Reversible jump MCMC
Study		PostMean		95% Cred Int
1		0.363		(0.178, 0.779)
2		0.376		(0.192, 0.761)
3		0.323		(0.172, 0.511)
4		0.260		(0.118, 0.378)
5		0.297		(0.190, 0.406)
6		0.360		(0.266, 0.481)
7		0.302		(0.232, 0.377)

TABLE A1

Observed numbers of asymptomatic cases and observations ()

Study	Table 1, a	Table 2, b	Table 3, b	Table 4, c
1	4/13	94/728	94/728	1/2
2	13/23	27/171	27/ 171	2/4
3	18/83	61/115		4/12
4	40/60	10/ 36		5/30
5	130/166	4/31	4/31	12/44
6		8/16	8/16	29/73
7		8/14	8/14	41/138
8		2/13
9		2/10
10		1/9
11		5/9	5/9

Table 1: Effect sizes and SEs are given in figure 4 of He et al. (mislabelled as case fatality rate), but the must be obtained from the background papers: 1. Nishiura et al. ; 2. Kimball et al. ; 3. Song et al. ; 4. Serra ; 5. Day.

Tables 2 and 3: are given in figure 3 of He et al.

Table 4: are given in figure 1 of Buitrago‐Garcia et al.

For uncertain pooling inference for is made using (6). To start, evaluate the right side of (7) for then standardize by dividing the individual terms in the grid by their sum. This provides an approximation for Then select a random sample of size from the normalized values of For each selection, sample from For the logit case, transform to at each step. Starting with a grid for and with a very large range for , we reduced the space to make the 2D‐grid sampler faster. Specifically, we retained 99.2% of the probability associated with the extensive grid. We generated values of for the eleven study case and for the other cases. Finally, note that approximations for the marginal posterior distributions, that is, and can be obtained directly from the grid approximation of .

As noted in Section 1, our study was motivated by a meta‐analysis by He et al. that included early studies of the COVID‐19 asymptomatic infection rate. He et al. carried out a standard meta‐analysis using a normal‐based random effects model with the effect sizes as the outcome random variable and assuming fixed SEs. Since there is considerable variation in the effect sizes it is prudent to be cautious and investigate whether the true effect sizes are from a single source.

Our analysis starts with the basic uncertain pooling method (Section 3.1) and DPmeta (Section 3.2). We summarize the results using the binomial‐beta model and RJMCMC (Section 3.3) at the end of this section. For the basic uncertain pooling method we assumed, a priori, that all 37 partitions have equal probability, that is, For the prior on , independent of , we used two distributions:

For a standard random effects model, Gelman recommends using a half‐Cauchy prior distribution for While our situation is very different, that is, many partitions and weighting depending on (not ), we have adopted this suggestion in (a) by transforming the half‐Cauchy pdf to obtain the Inverse Beta (InvBeta) pdf of Previous research has shown that there are benefits to having the prior distribution for concentrated near 0, leading to the choice in (b) with . In the following we present results only for (a) as those for (b) are similar.

As described in Section 2, a complete specification of DPmeta requires estimates of many hyperparameters. This seems inappropriate for meta‐analyses such as these. So, we have replaced and with their maximum likelihood estimates, thus eliminating the need to specify the other hyperparameters. We have adopted the suggestion of Escobar to use a small set of values for , that is, , typically augmented by a value for much smaller than and one larger than

We start by discussing the meta‐analysis that motivated our investigation, that is, He et al. We analyze the data (sample proportions, numbers of observations) from five of their studies, omitting one (Mizumoto et al. ) whose estimates are based on extensive modeling. As such, Buitrago‐Garcia et al. “considered …[this] study …separately, because of the different method of analysis,” and He et al. excluded it entirely. Note that the numbers of asymptomatic cases and observations are included in Table A1.

In Table 1 we present for each of the five studies the sample effect size (sample proportion, ), posterior expected value of the true effect size, standard error (SE), and 95% credible interval for the true effect size. Here, SE = where is the total number of observations. From DPmeta, there are the posterior expected values of the true effect size corresponding to and 5 (a good representation of the six values we used in our analysis). The remaining columns give the posterior means and credible intervals obtained by using the Reversible Jump MCMC method. We summarize our results from the basic uncertain pooling method and DPmeta first, adding brief comments about the results obtained by using the RJMCMC method at the end of this section. In general, the results are consistent.

The posterior probability, , associated with the pool‐all partition, is minuscule, that is, Now suppose that we consider the probability that there is any single large cluster. That is, we include all partitions where and for where for The sum of these probabilities is , a very small quantity.

A standard way (Gelman ) to assess the likelihood that the true effect sizes come from a common source is to assume the pool‐all model, , and evaluate the posterior predictive ‐value using a standard (Gelman, (6.4)) discrepancy measure. With overall effect define where is now the sample log odds, is as in (12), and is . The discrepancy measure, is based on the pool‐all model, as defined in (1) and (2). Then the posterior predictive ‐value is with denoting a replication from the pool‐all model. For these data the ‐value is , showing that the observed data are not concordant with the pool‐all model. These results show that it is highly unlikely that the five true effect sizes come from a single source.

Another way to analyze these data is to construct a similarity matrix. For each pair of studies, and , the similarity matrix gives the posterior probability that and are in the same cluster. We present in Figure 1 (top) a visual representation of the similarity matrix using the basic uncertain pooling method (Section 3.1) while Figure 1 (bottom) is the corresponding similarity plot using the RJMCMC methodology in Section 3.3. The study numbers are given on both the and axes and there is a legend showing, for each cell, the value of the pairwise probability. Figure 1 (top) shows limited strong clustering, {2,4}, while Figure 1 (bottom) shows some additional clustering, {1,3}, {2,4} and {4,5}. Thus, limited aggregation seems appropriate. This may not be surprising since the He et al. review only included observations from the very early part of the COVID‐19 pandemic.

FIGURE 1

Cell () gives the categorized posterior probability that studies and are in the same cluster. (Top) Basic uncertain pooling; (bottom) RJMCMC

From Table 1 the 95% credible intervals are very wide, a further indication of considerable uncertainty. Moreover, a 95% credible interval for the overall true effect size, , is . This wide interval is a strong indication that a single summary value such as the posterior mean of would not be informative.

The results from applying DPmeta provide additional insight. When is small, for example, = 1/5, the posterior expected values show considerable clustering, that is, {2,4,5} and {1,3}, similar to those seen in Figure 1(bottom). This clustering is not surprising since choosing a very small value of means that there is very limited sampling from the parametric centering measure . Using a moderate choice, = 5, the five posterior expected values are close to those obtained using the basic uncertain pooling methodology as are the 95% credible intervals (intervals are not shown in Table 1).

The results in Table 1 and Figure 1 suggest that one should not pool the data from these five studies. The next review that we consider, He et al., also shows the same issue, that is, questionable pooling of the data from all of the eleven studies. However, in this case, our analysis provides evidence of considerable clustering. Looking at the characteristics of the eleven studies could reveal the reasons for this clustering, and the direction to take to make appropriate inferences.

The analysis presented below uses a subset of the data in He et al., namely the asymptomatic infection rate in eleven studies of children. The results are summarized in Table 2 and Figure 2, analogous to Table 1 and Figure 1.

FIGURE 2

Cell () gives the categorized posterior probability that studies and are in the same cluster. (Top) Basic uncertain pooling; (bottom) RJMCMC

From Table 2 and Figure 2 it is apparent that there are several distinct subsets. Further investigation could reveal features that separate these subsets, leading to advances in understanding the differences in the asymptomatic rates.

The posterior probability, , associated with the pool‐all partition, is minuscule, that is, As described for the He et al. study, the sum of the posterior probabilities associated with partitions having only a single cluster and 0, 1, 2, 3, 4, or 5 singleton subsets is . This result suggests that it is unlikely that the eleven true effect sizes come from a single source. Moreover, using the discrepancy measure in (13), the posterior predictive ‐value is , showing that the observed data are not concordant with the pool‐all model. Finally, a 95% credible interval for the overall true effect size, , is . This interval, together with the clustering, is a strong indication that a single summary value such as the posterior mean of would not be informative.

From Figure 2 the most likely cluster is {3,6,7,11}, while the next most likely one is {1, 2, 5, 8, 9, 10}. In this case, there is considerable clustering but it does not extend to all eleven studies. Without additional evidence this analysis suggests conducting separate standard meta‐analyses for the two large subsets.

In Table 2 the posterior expected values for the individual studies in cluster {3, 6, 7, 11} are, essentially, an average of the corresponding four sample effect sizes, but reduced in magnitude. This reduction reflects the contributions from the data from the other seven studies.

The clustering seen in Figure 2 is also evident in the posterior expected values from DPmeta with small ; see Table 2 with . With there is good agreement between the posterior expected values for the two methods. For most of the eleven studies there is good agreement in the credible intervals for the two methods with for DPmeta (not shown in Table 2).

The next analysis uses the data from a subset of six of the eleven studies in He et al. These six studies were chosen to illustrate properties of the methodology when there is considerable separation. The results are summarized in Table 3 and Figure 3.

FIGURE 3

Cell () gives the categorized posterior probability that studies and are in the same cluster. (Top) Basic uncertain pooling; (bottom) RJMCMC

From Figure 3 and Table 3 it is apparent that there are two distinct subsets, that is, and . Presumably this separation reflects different characteristics of the two populations and/or different ways that the studies were carried out. As expected, is minuscule, that is, Proceeding as described for the He et al. study, the sum of the posterior probabilities associated with partitions having only a single large cluster (ie, with at least four members) is . Thus, there is no evidence that the true effect sizes from these six studies come from a single source. Moreover, using the discrepancy measure in (13), the posterior predictive ‐value is , showing that the observed data are not concordant with the pool‐all model. Finally, a 95% credible interval for the overall true effect size, , is . This wide interval, together with the clustering, is a strong indication that a single summary value such as the posterior mean of would not be informative. The results from DPmeta are consistent with those given above. For a wide range of values of the posterior expected values for {1, 2, 5} are about 0.14 while those for {6, 7, 11} are about 0.50. With the posterior expected values and intervals for {1, 2, 5} are quite close to those obtained by using the basic uncertain pooling methodology. For {6, 7, 11} almost all of the expected values and interval endpoints are within 0.06 of those from the uncertain pooling methodology.

The final analysis uses the data from the seven study screening subset in Buitrago‐Garcia et al., identified as being of special interest. The results are summarized in Table 4 and Figure 4. In Table 4 the sample SEs play a large role. That is, for the studies {1,2} with the large SEs the posterior expected values are much smaller than the sample effect sizes. For the studies {5,6,7} with the small SEs the posterior expected values and sample effect sizes are approximately equal. Thus, apart from study 4, the posterior expected values suggest greater commonality than was apparent from the corresponding set of sample effect sizes. However, the individual posterior credible intervals are quite different. Using the discrepancy measure in (13), the posterior predictive ‐value is 0.40, indicating that there may be a common source for the true effects. This result is supported by Figure 4 which suggests relatively uniform clustering (except for studies 4 and 6).

FIGURE 4

Cell () gives the categorized posterior probability that studies and are in the same cluster. (Top) Basic uncertain pooling; (bottom) RJMCMC

Since the data in each of Tables 2, 3, 4 are for a subset of a much larger set of studies they are likely to be substantially more homogeneous than the data in the full set of studies. For example, the seven screening studies (Table 4) are a subset of 79 studies with sample proportions ranging from 0.01 to 0.92.

Results from using DPmeta are, for the most part, consistent with these observations. For all of the posterior expected values are approximately 0.31. For the posterior expected values range from 0.25 to 0.35, similar to the posterior means in Table 4. For the intervals corresponding to {5,6,7}, the studies with the smallest SEs, are similar to those from the basic uncertain pooling methodology.

With a few modifications we have implemented the RJMCMC procedure outlined in sections 6.2 and 6.3 of Green, but expand the range for the prior for by taking log (log 100, log 1000) throughout. We summarize by using the posterior means and 95% credible intervals (bottom of Tables 1, 2, 3, 4), and similarity plots (bottom of Figures 1, 2, 3, 4).

Since the sample likelihoods and prior distributions differ between the two approaches, that is, those based on the likelihoods and priors in Sections 3.1 and 3.3, comparisons of the results may not be especially meaningful. However, examining Tables 1, 2, 3, 4 it is notable that the summaries (posterior means and intervals) from the two approaches are consistent, generally close when the SEs are small, less so when the SEs are very large. There are no major differences between the comparable similarity plots corresponding to basic uncertain pooling and RJMCMC.

CONCLUDING REMARKS

The importance of good inferences for the COVID‐19 asymptomatic infection rates is clear, as noted in the quotation from Buitrago‐Garcia et al. in Section 1. Conducting meta‐analyses is a common, often useful, way to summarize information from a collection of studies. However, inference will be misleading if there is pooling of data from studies that are not concordant. For example, Byambasuren et al. note: “A recent review by the center for Evidence Based medicine in Oxford found a range of estimates of asymptomatic COVID‐19 cases which ranged from 5% to 80%. However, many of the identified studies were either poorly executed or poorly documented, making the validity of these estimates questionable.”

In this article, we reanalyze data from three review papers, using three Bayesian methods that have a more general structure than the common meta‐analytic ones. This methodology shows, in a principled manner, the extent and nature of the pooling that can be justified statistically. The more general structure should ensure greater concordance of the data with our model than with a more restricted model.

If the authors of a review have screened the studies so that the ones remaining for analysis have no markedly aberrant characteristics then an analysis showing distinct clusters should prompt a further review, and careful consideration of the inferences to make and the method to use.

In some situations there may be covariates associated with the studies that may help to explain differences in the outcomes. To illustrate, use the basic notation and a linear regression offset. That is, replace (1) with Then inference for can be made using the extension of (6) where it is easily shown that with an matrix of covariates, a vector of regression coefficients, an diagonal matrix with th element , , and

In the reviews we have considered, only He et al. gives more than one covariate for each study, that is, the number of confirmed cases and percent male. Using the covariates, an exploratory analysis of the residuals showed that such an augmented analysis will not improve inferences.

The three methods can be implemented. For DPmeta there is an R package (DPpackage) and the code for DPmeta is included as Supplementary Material. R packages are being developed for the two uncertain pooling methods, that is, basic uncertain pooling and RJMCMC.

Finally, for basic uncertain pooling and DPmeta both the sample proportion and logit of the sample proportion are used in applications. While we have presented results only for the latter, our findings are similar for both choices.

CONFLICT OF INTEREST

The authors declare no potential conflict of interests.

Data S1 Supplementary Material

Click here for additional data file.