Modeling and computation of multistep batch testing for infectious diseases.

We propose a mathematical model based on probability theory to optimize COVID-19 testing by a multistep batch testing approach with variable batch sizes. This model and simulation tool dramatically increase the efficiency and efficacy of the tests in a large population at a low cost, particularly when the infection rate is low. The proposed method combines statistical modeling with numerical methods to solve nonlinear equations and obtain optimal batch sizes at each step of tests, with the flexibility to incorporate geographic and demographic information. In theory, this method substantially improves the false positive rate and positive predictive value as well. We also conducted a Monte Carlo simulation to verify this theory. Our simulation results show that our method significantly reduces the false negative rate. More accurate assessment can be made if the dilution effect or other practical factors are taken into consideration. The proposed method will be particularly useful for the early detection of infectious diseases and prevention of future pandemics. The proposed work will have broader impacts on medical testing for contagious diseases in general.

INTRODUCTION

To fight the COVID‐19 pandemic with limited resources, batch tests were recommended by pooling multiple swab samples. Samples from different individuals are pooled into one batch and then a high‐throughput PCR test is conducted (Cheng, 2020; Lohse et al., 2020; Shani‐Narkiss et al., 2020). If the batch is tested negative, then it can be deduced that all samples were negative. Otherwise, each sample needs to be tested individually. Pooling data was originally proposed by Dorfman (1943) for detecting syphilis in U.S. soldiers during World War II. France et al. (2015) used pooling samples to reduce the number of analyses required for detection of anthrax spores. By grouping individual samples, batch testing significantly reduces the number of tests, providing an efficient method to detect community transmission (Hogan et al., 2020). Batch testing has become more relevant recently, as state and local governments seek to test as many people as possible to transition safely back to normal life. For example, a recent press release by the U.S. Food and Drug Administration (FDA) indicates developers' interest in incorporating pooling samples for domestic testing (Shuren, 2020). Places abroad such as South Korea started using pooling methods to sample in batches of a fixed size in high‐risk communities (Korea Center for Disease Control & Prevention, 2020; Kwak, 2020; Park & Koo, 2020). However, the drawback to batch testing is a much higher false negative rate than individual testing.

In this study, we introduce a batch‐based approach, which simultaneously addresses the problem of limited resources and testing accuracy. We consider a multistep testing procedure with variable batch sizes where each step divides the population into subpopulations based on the previous step's results. These subpopulations are further divided into batches for testing. We introduce a method to estimate the optimal batch sizes given the infection rates of subpopulations to efficiently and accurately test most, if not the entire, population. The proposed method incorporates testing errors and optimizes batch sizes at each step and for each subpopulation. Shani‐Narkiss et al. (2020) also considered a multistep testing procedure with variable batch sizes. However, their approach assumes no testing errors, and their batch sizes are limited to powers of two.

For batch testing, typically a sample from each person is divided into multiple aliquots for separate tests (Mutesa et al., 2020). The population is split into subpopulations with negative test results (batch negative) and positive test results (batch positive). Each subpopulation is given another round of batch tests where the batch size increases for the batch negatives and decreases for the batch positives. We iterate this procedure on each subpopulation, where at each step we can estimate the infection rate. This process is continued until one of the following conditions is satisfied: (i) The process results in three batch negatives or three batch positives, (ii) the infection rate of the subpopulation becomes higher than 30%, and (iii) the optimal batch size is reduced to 2. The samples are randomly assigned to different batches with the size determined by the infection rate in each round. Details of this procedure are given in Section 2.5. To apply our approach most effectively, we can first divide the population based on infection rates, for example, by dividing based on geography, population density, proximity to highly infected regions, etc. Information given by various methods including mobile apps and online mapping (Lee & Lee, 2020) may help track the virus and divide the population into different groups.

We also address the efficacy of the tests. The false negative rate of the COVID‐19 polymerase chain reaction (PCR) test for an individual is known to be near 15%, ranging from 10% to 30% (West et al., 2020; Xiao et al., 2020; Yang and Yan, 2020). By one‐step batch testing, the false negative rate increases (Shuren, 2020). By our multistep batch testing procedure, the false negative rate is substantially reduced. To the best of our knowledge, no other studies have attempted estimating optimal batch sizes by taking testing errors into account. The dilution effect in batch testing (Hwang, 1976; McMahan et al., 2013; Yelin et al., 2020) is not considered in this study. The false negative rate may be increased by the dilution effect.

If three batch negatives occur before getting three batch positives, then we conclude that the individuals in the batch of the final round are not infected. For people whose samples result in three batch positives before three batch negatives, each sample needs to be tested individually to find out which was positive. To reduce the false negative rate, up to three individual tests are performed for the samples from each individual in this group. For a population of size 100,000 with an infection rate of 0.1%, if the false negative rate is 15% and false positive rate is 1% for an individual test, then our method requires around 7000 tests to test the entire population. Results for different infection rates are detailed in Section 3. The simulation results show that our method reduces the false negative rate to approximately 3%, and the false positive rate to near zero.

It is a well‐known fact that the positive predictive value (PPV) is very low when the infection rate is low even if the sensitivity is very high. For a false positive rate of 1% and false negative rate of 15%, the PPV of an individual test is 8% for an infection rate of 0.1%, and 46% for an infection rate of 1%. According to our simulation, our multistep batch testing procedure improves the PPV to 89% and 93%, respectively. This is because individual tests are conducted on the subject in the final positive batches, which have higher infection rates than the entire population (see Section 2.4 and Section 3).

The proposed batch testing procedure is compared with matrix pool testing as well as single batch testing with fixed or variable batch sizes. Two‐dimensional pooling strategy has been studied by researchers including Barillot et al. (1991), Amemiya et al. (1992), Phatarfod & Sudbury (1994), and Hudgens and Kim (2011). The false negative rate decreases to 1% if 12 × 12 matrix pool tests are conducted three times parallel with up to three sequential individual tests to all the positive crossings. However, the number of tests to cover the whole population is much more than that of the proposed multistep batch testing procedure, and it is more than a half the population size even when the infection rate of the population is 0.1% (see Section 3).

The original purpose for batch testing was to prevent the spread of disease in high‐risk communities by testing everyone, symptomatic or not. However, the proposed method can be applied to the general population, due to its flexibility in dealing with various infection rates and its substantially greater sensitivity compared to individual testing and current batch testing approaches. More specifically, our method can accurately test a large population with limited resources by dividing the population based on infection rates, for example, based on geography, population density, proximity to highly infected regions, etc. This method can be applied to effectively combat a future pandemic of new diseases.

MATHEMATICAL MODEL

Optimal batch size assuming no testing errors

We begin by studying a simple case where the accuracy of a virus screening test is 100%. Suppose that the infection rate in a population is p (rate of no infection is ) and the accuracy of a virus screening test is 100%. Let X be a random variable denoting the number of positive cases in a batch of size n. Then, X follows a binomial distribution with n trials and success rate p. The probability of k positive cases in the batch is and the probability that the batch is tested negative is .

Using an initial guess of the batch size n, we want to estimate q. Let . Then after a good number of batch tests, we can estimate q by Suppose we test a population in batches of size n, then test all individuals who were in positively tested batches. For a population of size N, the expected total number of tests to be performed to identify all positive carriers is To find the optimal n, we minimize as We can solve this equation for n numerically.

Optimal batch size given testing errors

We conduct a hypothesis testing with null hypothesis H 0: The subject is uninfected versus alternative hypothesis H 1: The subject is infected. To reduce testing errors, we propose a procedure of conducting multiple rounds of batch tests before carrying out individual tests for a selected group. Denote the probability of a type I error (false positive rate) as α and the probability of a type II error (false negative rate) as β. Let the infection rate be p 1 and in the initial batch tests for the whole population, and X the random variable denoting the number of positive cases in a batch of size n 1. Then, the probability of a batch negative based on (1) can be obtained as Using an initial guess of the batch size n 1, we want to estimate q 1. After a good number of batch tests, we can estimate q 1 by

Suppose all the subjects in the positive batches get individual PCR tests. If the size of the entire population is N 1, the required number of tests can be obtained from (2) by substituting (3) for as To find the optimal batch size x, we minimize as This can be solved for x numerically. In this study, we used the secant method to solve the equation. The optimal batch size n 1 is either floor(x) or ceiling(x), which has the lower value of . For example, when the infection rate is 1% (), , and , the solution of (6) is . We need an integer n 1 minimizing the convex function in (5), which is adjacent to 11.53. Since and , the solution is as shown in Table 1. The initial batch tests can be conducted using n 1 for the whole population.

TABLE 1

Optimal batch size (a) when there are no testing errors, and (b) when the false negative rate is 0.15, false positive rate is 0.01 for individual tests

Infection rate	0.001	0.002	0.003	0.004	0.005	0.006	0.007	0.008	0.009	0.01	0.02	0.03
(a) Without error	32	23	19	16	15	13	12	12	11	11	8	6
(b) With error	35	25	21	18	16	15	14	13	12	12	8	7
Infection rate	0.04	0.05	0.06	0.07	0.08	0.09 to 0.12		0.13 to 0.17		0.18 to 0.25
(a) Without error	6	5	5	4	4	4		3		3
(b) With error	6	6	5	5	5	4		4		3

Continuing this process, after the th round, the subjects have been divided into subpopulations. Each subpopulation was batch tested with batch sizes determined from that round. Fixing one of these subpopulations, we now split it into two smaller subpopulations for the ith round, according to the batch test results from the th round. Let denote the number of subjects in population belonging to test‐negative batches from the th round, and denote the infection rate of this subpopulation. The expected size of the subpopulation in the ith round of batch tests is In this subpopulation, the probability that at least one subject in a batch is infected is and the probability that all the subjects in a batch are not infected is . The estimated number of infected subjects in this subpopulation is Here, is the expected number of infected subjects in a test‐positive batch in the th round, where is a binomial random variable with trials and success rate . Therefore, the infection rate of the subpopulation in the ith round can be used for estimating the optimal batch size and the required number of tests. The optimal number of required tests can be obtained by replacing n 1 with and q 1 with in (3) and (5) to get and the optimal batch size for this round is obtained by minimizing .

For example, suppose the estimated infection rate is and for the population of size 100,000. Then in (5) is minimized when . For the second round, the expected subpopulation size is obtained from (7) as 89,456. From (8), and the estimated infection rate in this subpopulation is obtained from (9) as . For this infection rate, the optimal batch size is obtained by minimizing in (10). These numbers can be found in the second part of Table 5 under Round 2.

TABLE 5

Batch test procedure: population size 100,000, false negative rate 15% and false positive rate 1% for individual tests, infection rates 0.1% and 1%

Infection rate =0.1%. The first round requires 2858 tests with batch size 35
	Round 2			Round 3			Round 4			Round 5			Final
Tests	p 2 a	N 2 b	n 2 c	p 3	N 3	n 3	p 4	N 4	n 4	p 5	N 5	n 5	p	N
−−−	2e‐4	96109	88	2e‐5	94047	224							4e‐6	92684
−−+−	2e‐4	96109	88	2e‐5	94047	224	0.001	1363	30				2e‐4	1302
−+−−	2e‐4	96109	88	0.006	2062	15	0.001	1888	35				2e‐4	1814
+−−−	0.022	3891	8	0.004	3322	18	6e‐4	3102	45				1e‐4	3000
−−++−	2e‐4	96109	88	2e‐5	94047	224	0.001	1363	30	0.027	61	7	0.005	51
−+−+−	2e‐4	96109	88	0.006	2062	15	0.001	1888	35	0.022	74	8	0.004	63
−++−−	2e‐4	96109	88	0.006	2062	15	0.062	175	5	0.012	133	10	0.002	118
+−−+−	0.022	3891	8	0.004	3322	18	6e‐4	3102	45	0.016	102	9	0.003	90
+−+−−	0.022	3891	8	0.004	3322	18	0.049	220	6	0.010	169	12	0.002	152
++−−−	0.022	3891	8	0.13	568	4	0.03	362	7	0.005	300	15	0.001	278
+++	0.022	3891	8	0.13	568	4							0.30	206
++−+	0.022	3891	8	0.13	568	4	0.03	362	7				0.15	62
+−++	0.022	3891	8	0.004	3322	18	0.049	220	6				0.18	51
−+++	2e‐4	96109	88	0.006	2062	15	0.062	175	5				0.22	42
++−−+	0.022	3891	8	0.13	568	4	0.03	362	7	0.005	300	15	0.06	23
+−+−+	0.022	3891	8	0.004	3322	18	0.049	220	6	0.010	169	12	0.08	17
+−−++	0.022	3891	8	0.004	3322	18	6e‐4	3102	45	0.016	102	9	0.11	13
−++−+	2e‐4	96109	88	0.006	2062	15	0.062	175	5	0.012	133	10	0.10	14
−+−++	2e‐4	96109	88	0.006	2062	15	0.001	1888	35	0.022	74	8	0.13	11
−−+++	2e‐4	96109	88	2e‐5	94047	224	0.001	1363	30	0.027	61	7	0.15	9

Infection rate

Subpopulation size

Batch size

Further, we can estimate the optimal batch size for the subpopulation consisting of batch positives. In the th round, the probability that a batch of size is tested positive is The size of this subpopulation can be modified from (7) as In this subpopulation, the probability that at least one subject in a batch is infected is and the probability that all the subjects are not infected is . The estimated infection rate in this subpopulation can be obtained as (9). This infection rate can be used for estimating the optimal batch size and the required number of tests for the subpopulation in the ith round. The accuracy of these formulae developed in this section has been confirmed by matching results with the simulation results given in Section 3.

Figure 1 displays the optimal batch size and the required number of batches for a population of size 100,000 as the infection rate ranges from 0.001 to 0.25 for the first round, and the false negative rate ranges from 0.1 to 0.25. The false positive rate is fixed at 0.01. Table 1 displays the optimal batch sizes for different infection rates (a) when no testing errors are assumed and (b) when the false negative rate is 15% and the false positive rate is 1%.

FIGURE 1

Optimal batch size (left) and required number of batches (right) for the first round of batch tests. False positive rate: 0.01, population size: 100,000

Sensitivity and specificity of a batch test

Let us consider one‐step batch testing. If a batch is tested negative, then we conclude that all samples in the batch are negative. If a batch is tested positive, then each sample in the batch needs to be tested individually. Let the binomial random variable X denote the number of positive cases for a batch of size n with an infection rate of p. We continue to denote α as the probability of a type I error and β as the probability of a type II error for an individual test. Table 2 displays the probabilities for the confusion matrix.

TABLE 2

Probability in each cell of the confusion matrix for single batch testing

		True condition
		No samples are infected	At least one sample is infected
Test	−	(a) (1−α)P(X=0)	(b) βP(X>0)
Result	+	(c) αP(X=0)	(d) (1−β)P(X>0)

Sensitivity

In cells (b) and (d), at least one sample is infected in each batch. It can be deduced that all the samples in cell (b) are negative, and no more tests are given. The probability that an infected individual is incorrectly tested negative from the batch test is . In cell (d), the probability that an infected individual is correctly tested positive from the batch test is . An individual test is given to each sample in this cell, and the probability that a sample is incorrectly tested negative is . Therefore, the false negative rate is obtained as

For example, if , then the false negative rate of batch testing is , and thus the sensitivity is 0.7225. For 0.1, 0.2, and 0.25, the sensitivity of batch testing is 0.81, 0.64, and 0.5625, respectively. Note that the sensitivity of a batch test depends on neither the infection rate nor the batch size. The above result is supported by our simulation given in Section 3. It confirms that the sensitivity is decreased by conventional batch testing. In contrast, our multistep batch testing method has a significantly higher sensitivity than conventional individual tests as well as single batch testing according to our simulation studies.

Specificity

The false positive error occurs in cells (c) and possibly in (d) in Table 2, and all the samples in these cells go through individual tests. In cell (c), the probability that a sample is incorrectly tested positive in the individual tests is In cell (d), the expected infection rate in each batch is . The probability that an uninfected sample is incorrectly tested positive in the individual tests in this cell is The sum of (11) and (12) is Therefore, the false positive rate of batch testing is and the specificity of batch testing is

Unlike the sensitivity of batch testing, the batch size, infection rate, probability of a type I error, and probability of a type II error contribute to (13). For .01 and 0.03, and .1, 0.15, 0.2 and 0.25, the specificity of batch testing (13) using a fixed batch size of 10, and using the optimal batch size is given in Table 3. The specificity is substantially improved by batch testing. These results closely match with our simulation results given in Section 3.

TABLE 3

Specificity for single batch testing with fixed batch size of 10 and optimal batch size for the false positive rate of 0.01, 0.03, and the false negative rate of 0.1, 0.15, 0.2, 0.25 for individual tests

α	β		Infection rate	0.001	0.01	0.03	0.05	0.10
		Batch size 10	Specificity	0.9998	0.9991	0.9978	0.9966	0.9944
	0.1	Optimal	Batch size	34	11	7	5	4
		batch size	Specificity	0.9996	0.9990	0.9984	0.9982	0.9975
		Batch size 10	Specificity	0.9998	0.9992	0.9979	0.9968	0.9948
	0.15	Optimal	Batch size	35	12	7	6	4
0.01		batch size	Specificity	0.9996	0.9990	0.9985	0.9980	0.9976
		Batch size 10	Specificity	0.9998	0.9992	0.9980	0.9970	0.9951
	0.2	Optimal	Batch size	36	12	7	6	4
		batch size	Specificity	0.9996	0.9991	0.9986	0.9981	0.9978
		Batch size 10	Specificity	0.9998	0.9993	0.9981	0.9972	0.9954
	0.25	Optimal	Batch size	37	12	7	6	5
		batch size	Specificity	0.9996	0.9991	0.9987	0.9982	0.9974
		Batch size 10	Specificity	0.9989	0.9968	0.9928	0.9895	0.9831
	0.1	Optimal	Batch size	34	11	7	5	4
		batch size	Specificity	0.9983	0.9966	0.9947	0.9943	0.9920
		Batch size 10	Specificity	0.9989	0.9970	0.9932	0.9900	0.9840
	0.15	Optimal	Batch size	36	12	7	6	4
0.03		batch size	Specificity	0.9983	0.9965	0.9950	0.9935	0.9924
		Batch size 10	Specificity	0.9989	0.9971	0.9936	0.9906	0.9849
	0.2	Optimal	Batch size	37	12	7	6	4
		batch size	Specificity	0.9983	0.9967	0.9952	0.9939	0.9928
		Batch size 10	Specificity	0.9989	0.9972	0.9939	0.9911	0.9859
	0.25	Optimal	Batch size	38	13	8	6	5
		batch size	Specificity	0.9983	0.9966	0.9950	0.9942	0.9917

PPV and NPV

Let us define E as the event that an individual is infected by the virus, and B as the event that an individual got a positive test result. Then by the Bayes' rule, PPV (an individual is infected given a positive test result) is and NPV (negative predictive value: an individual is not infected given a negative test result) is For example, if the infection rate in the population is 1%, sensitivity is 85%, and specificity is 99%, then and . Therefore, the PPV is 0.4620 and the NPV is 0.9985. If the infection rate is 0.1%, then the PPV is 0.0078 and the NPV is 0.9998. For the infection rates , sensitivity within the range of [0.75, 0.90] and the specificity of 0.99, Table 4 illustrates PPV and NPV. Figure 2 displays the PPV (above) and NPV (below). The PPV is substantially improved by our multistep batch testing procedure (see Section 3).

TABLE 4

PPV and NPV

		Sensitivity a
p b		0.75	0.77	0.79	0.81	0.83	0.85	0.87	0.89	0.90
0.001	PPV	0.0070	0.0072	0.0073	0.0075	0.0077	0.0078	0.0080	0.0082	0.0083
	NPV	0.9997	0.9998	0.9998	0.9998	0.9998	0.9998	0.9999	0.9999	0.9999
0.002	PPV	0.1307	0.1337	0.1367	0.1397	0.1426	0.1455	0.1485	0.1514	0.1528
	NPV	0.9995	0.9995	0.9996	0.9996	0.9997	0.9997	0.9997	0.9998	0.9998
0.004	PPV	0.2315	0.2362	0.2409	0.2455	0.2500	0.2545	0.2589	0.2633	0.2655
	NPV	0.9990	0.9991	0.9991	0.9992	0.9993	0.9994	0.9995	0.9996	0.9996
0.006	PPV	0.3116	0.3173	0.3229	0.3284	0.3338	0.3391	0.3443	0.3495	0.3520
	NPV	0.9985	0.9986	0.9987	0.9988	0.9990	0.9991	0.9992	0.9993	0.9994
0.008	PPV	0.3769	0.3831	0.3892	0.3951	0.4010	0.4067	0.4123	0.4178	0.4206
	NPV	0.9980	0.9981	0.9983	0.9985	0.9986	0.9988	0.9989	0.9991	0.9992
0.01	PPV	0.4310	0.4375	0.4438	0.5500	0.4560	0.4620	0.4677	0.4734	0.4762
	NPV	0.9975	0.9977	0.9979	0.9981	0.9983	0.9985	0.9987	0.9989	0.9990
0.02	PPV	0.6048	0.6111	0.6172	0.6231	0.6288	0.6343	0.6397	0.6449	0.6475
	NPV	0.9949	0.9953	0.9957	0.9961	0.9965	0.9969	0.9973	0.9977	0.9979
0.03	PPV	0.6988	0.7043	0.7096	0.7147	0.7197	0.7244	0.7291	0.7335	0.7357
	NPV	0.9923	0.9929	0.9935	0.9941	0.9947	0.9953	0.9960	0.9966	0.9969
0.05	PPV	0.7978	0.8021	0.8061	0.8100	0.8137	0.8173	0.8208	0.8241	0.8257
	NPV	0.9869	0.9879	0.9890	0.9900	0.9910	0.9921	0.9931	0.9942	0.9947
0.08	PPV	0.8671	0.8701	0.8729	0.8757	0.8783	0.8808	0.8832	0.8856	0.8867
	NPV	0.9785	0.9802	0.9819	0.9836	0.9853	0.9870	0.9887	0.9904	0.9913
0.10	PPV	0.8929	0.8953	0.8978	0.9000	0.9022	0.9043	0.9063	0.9082	0.9091
	NPV	0.9727	0.9748	0.9770	0.9791	0.9813	0.9834	0.9856	0.9878	0.9889
0.15	PPV	0.9298	0.9315	0.9331	0.9346	0.9361	0.9375	0.9388	0.9401	0.9408
	NPV	0.9573	0.9606	0.0639	0.9672	0.9706	0.9740	0.9774	0.9808	0.9825
0.20	PPV	0.9494	0.9506	0.9518	0.9529	0.9540	0.9551	0.9560	0.9570	0.9574
	NPV	0.9406	0.9451	0.9496	0.9542	0.9588	0.9635	0.9682	0.9730	0.9754

Specificity is 0.99.

Infection rate.

FIGURE 2

PPV (above) and NPV (below)

Multistep batch testing procedure

More than three batch negatives may not be necessary in the procedure because the infection rate substantially decreases in later rounds. Figure 3 illustrates the proposed batch test procedure. We will further investigate the optimal number of batch tests and the stopping rule in this study. For batch negatives, the batch size increases substantially with most of the subjects remaining in the next round because the infection rate decreases. For batch positives, the subpopulation size substantially decreases in the next round. In this group, not all the samples in the batch are infected, so we can find a sample without infection in the next round of batch tests. This way, the number of required tests to cover the whole population decreases significantly, and it is possible to identify all positive carriers in the population.

FIGURE 3

Proposed multistep batch testing procedure. The oval nodes indicate intermediate nodes and the rectangular nodes indicate terminal nodes

We assume 15% false negative rate and 1% false positive rate of an individual for a population of size 100,000 in this section. Table 5 shows the change in the infection rate and corresponding optimal batch size throughout the process shown in Figure 3 for a population with infection rates of 0.1% and 1%. In the first round, the optimal batch size is 35 resulting in 2858 batch tests when the infection rate is 0.1%, and the optimal batch size is 12 resulting in 8334 batch tests when the infection rate is 1%. Note that the number of batch tests for the first round can be slightly different from this estimation due to the trial for estimating the infection rate with (4) using a small subset of the population with an initial guess of the batch size. The subsequent rounds do not require this trial because the infection rates of subpopulations can be estimated as (9) using the values obtained in the previous rounds. For the ith round, the number of batch tests can be obtained by ceiling. We count only the bold‐faced numbers starting the first round because the rest are duplicates. In each of Round 2 and Round 3, the bold‐faced subpopulation sizes () add up to 100,000 if we ignore the rounding error. In Round 4, the bold‐faced subpopulation sizes plus N in the final column for already finished ( and ) rows add up to 100,000. The calculation is similar for Round 5. The final two columns show the infection rate and resulting subpopulation size in each of the terminal nodes (rectangular frames) in Figure 3. For each infection rate, since the individuals in an upper half of the last column got three batch negatives, they do not need individual tests. A lower half of the last column needs individual tests because they got three batch positives. Thus, the total number of individual tests is the sum of these 10 numbers. The total number of tests can be obtained by adding the number of batch tests in Round 1, the number of batch tests obtained from the bold‐faced numbers in the subsequent rounds, and the number of individual tests.

The expected number of tests to identify all positive carriers is 6144 (5696 batch tests and 448 individual tests) when the infection rate is 0.1%, and 22,436 (19,409 batch tests and 3027 individual tests) when the infection rate is 1%. The sensitivity of this procedure can be obtained from the number of individual tests with the corresponding infection rates given in the last two columns in Table 5. For the infection rate of 0.1%, the expected number of infected cases among the samples who received individual tests at the end of the procedure is 97.8 out of 448. With the 85% sensitivity for individual tests, the sensitivity of this procedure is 83.1% since out of 100. For the infection rate of 1%, the expected number of infected cases among the samples who received individual tests is 975.5 out of 3027. The sensitivity of this procedure is 82.9% since out of 1000. To calculate the specificity, for the infection rate of 0.1%, the expected number of uninfected samples among the samples who received individual tests is 350.2 out of 448. With the 1% type I error rate for individual tests, since , the specificity of this procedure is . For the infection rate of 1%, the expected number of uninfected samples among the samples who received individual tests is 2051.5 out of 3027. Since , the specificity of this procedure is . These results closely match with our simulation results given for method (E) in Table 7 of Section 3.

TABLE 7

Simulation results: 100 repetitions, population size 100,000, sensitivity 85% and specificity 99% for individual tests; mean with standard deviation in parentheses

p (1)		0.001	0.01	0.03	0.05	0.10
(A) a	Acc.(2)	0.9899 (0.0003)	0.9886 (0.0003)	0.9859 (0.0004)	0.9831 (0.0004)	0.9760 (0.0004)
Indiv	Sens.(3)	0.8525 (0.0356)	0.8491 (0.0105)	0.8500 (0.0067)	0.8509 (0.0051)	0.8497 (0.0030)
Tests	Spec.(4)	0.9900 (0.0003)	0.9900 (0.0003)	0.9901 (0.0003)	0.9900 (0.0003)	0.9900 (0.0003)
	PPV	0.0799 (0.0088)	0.4621 (0.0107)	0.7251 (0.0075)	0.8176 (0.0054)	0.9041 (0.0031)
	NPV	0.9998 (0.0000)	0.9985 (0.0001)	0.9954 (0.0002)	0.9921 (0.0003)	0.9834 (0.0004)
	#Tests(5)	100000 (0)	100000 (0)	100000 (0)	100000 (0)	100000 (0)
(B) b	Acc.	0.9995 (0.0001)	0.9964 (0.0002)	0.9897 (0.0003)	0.9831 (0.0005)	0.9675 (0.0006)
Single	Sens.	0.7261 (0.0413)	0.7213 (0.0163)	0.7241 (0.0093)	0.7219 (0.0075)	0.7221 (0.0052)
Batch	Spec.	0.9998 (0.0000)	0.9992 (0.0001)	0.9979 (0.0002)	0.9968 (0.0002)	0.9948 (0.0003)
Tests	PPV	0.8074 (0.0449)	0.8998 (0.0101)	0.9136 (0.0057)	0.9227 (0.0039)	0.9389 (0.0028)
Fixed	NPV	0.9997 (0.0001)	0.9972 (0.0002)	0.9915 (0.0003)	0.9855 (0.0004)	0.9699 (0.0006)
Size 10	#Tests	11841 (133)	18981 (259)	33108 (438)	44693 (500)	65671 (519)
(C) c	Acc.	0.9987 (0.0001)	0.9956 (0.0002)	0.9911 (0.0003)	0.9868 (0.0004)	0.9784 (0.0005)
Single	Sens.	0.8420 (0.0384)	0.8453 (0.0128)	0.8464 (0.0074)	0.8470 (0.0059)	0.8466 (0.0042)
Batch	Spec.	0.9989 (0.0001)	0.9971 (0.0002)	0.9956 (0.0002)	0.9941 (0.0002)	0.9930 (0.0003)
Variable	PPV	0.4308 (0.0268)	0.7460 (0.0124)	0.8548 (0.0067)	0.8835 (0.0040)	0.9303 (0.0027)
Sizes	NPV	0.9998 (0.0000)	0.9984 (0.0001)	0.9952 (0.0002)	0.9920 (0.0003)	0.9832 (0.0005)
Seq Indiv	#Tests	14393 (1128)	37889 (824)	60567 (817)	77952 (793)	98341 (739)
Tests	B+Ind	2858+11535	8334 + 29555	14286 + 46281	16667 + 61285	25000+73341
(D) d	Acc.	0.9999 (0.0000)	0.9994 (0.0001)	0.9970 (0.0002)	0.9939 (0.0003)	0.9863 (0.0005)
Parallel	Sens.	0.9907 (0.0106)	0.9897 (0.0048)	0.9905 (0.0031)	0.9901 (0.0031)	0.9905 (0.0029)
12 × 12	Spec.	0.9999 (0.0000)	0.9995 (0.0001)	0.9972 (0.0002)	0.9941 (0.0003)	0.9858 (0.0005)
Matrix	PPV	0.9402 (0.0332)	0.9511 (0.0077)	0.9168 (0.0053)	0.8980 (0.0041)	0.8857 (0.0032)
Tests	NPV	1.000 (0.0000)	0.9999 (0.0000)	0.9997 (0.0001)	0.9995 (0.0002)	0.9989 (0.0003)
Seq Indiv	#Tests	50727 (342)	56201 (550)	80478 (971)	112177 (1458)	190263 (1951)
Test	B+Ind	49968 + 759	49968 + 6233	49968 + 30510	49968 + 62209	49968+140295
(E) e	Acc.	0.9998 (0.0000)	0.9981 (0.0001)	0.9941 (0.0003)	0.9905 (0.0003)	0.9814 (0.0005)
Multi‐step	Sens.	0.8305 (0.0337)	0.8281 (0.0120)	0.8306 (0.0066)	0.8291 (0.0053)	0.8296 (0.0041)
Batch	Spec.	1.000 (0.0000)	0.9998 (0.0001)	0.9992 (0.0001)	0.9990 (0.0001)	0.9983 (0.0001)
Variable	PPV	0.9503 (0.0241)	0.9722 (0.0058)	0.9693 (0.0037)	0.9776 (0.0021)	0.9815 (0.0014)
Sizes	NPV	0.9998 (0.0000)	0.9983 (0.0001)	0.9948 (0.0002)	0.9911 (0.0003)	0.9814 (0.0005)
1 indiv	#Tests	6221 (159)	22840 (280)	42561 (316)	56089 (377)	84012 (375)
Test	B+Ind	5705 + 516	19439 + 3401	31830 + 10731	41554 + 14535	58622+25390
(F) f Multi‐	Acc.	0.9998 (0.0000)	0.9990 (0.0001)	0.9969 (0.0002)	0.9958 (0.0002)	0.9926 (0.0003)
step Batch	Sens.	0.9689 (0.0165)	0.9700 (0.0057)	0.9730 (0.0033)	0.9723 (0.0025)	0.9725 (0.0017)
Variable	Spec.	0.9999 (0.0000)	0.9993 (0.0001)	0.9976 (0.0002)	0.9970 (0.0002)	0.9949 (0.0002)
Sizes	PPV	0.8852 (0.0290)	0.9307 (0.0085)	0.9259 (0.0044)	0.9446 (0.0028)	0.9547 (0.0018)
Seq Indiv	NPV	1.000 (0.0000)	0.9997 (0.0001)	0.9992 (0.0001)	0.9985 (0.0001)	0.9969 (0.0002)
Tests for	#Tests	7053 (259)	27809 (404)	58428 (609)	75908 (600)	116341 (718)
3 +'s	B+Ind	5704 + 1349	19436 + 8373	31809 + 26619	41530 + 34378	58611+57730

Conventional individual tests.

One‐step batch tests with a fixed batch size of 10, individual tests for positive batches.

One‐step batch tests with variable batch sizes with up to 3 sequential individual tests for positive batches.

Three parallel matrix pool tests; up to 3 sequential individual tests to all positive intersections.

Multistep batch tests with variable optimal batch sizes; an individual test for three batch positives.

Multistep batch tests; up to three sequential individual tests for three batch positives

(1)Infection rate;    (2)Overall accuracy;    (3)Sensitivity;    (4)Specificity;    (5)Number of required tests; (6)Number of batch tests + number of individual tests.

To reduce the false negative rate, we propose to conduct sequential individual tests as follows: We conduct up to three tests for the same person sequentially until a positive test occurs. Let us define E as the event that an individual is infected by the virus, and as the event that an individual got a positive result in the ith individual test. Then, the probability that a sample is tested positive in the first test is the probability that a person is tested negative and then tested positive is and a person is tested negative twice is Thus, the expected number of the individual tests for each person is We can obtain the expected total number of sequential individual tests by substituting the infection rate and the subpopulation size in each of the bottom 10 rows of the last two columns in Table 5. For the infection rate of 0.1%, the expected number of sequential individual tests in (14) is 1155, and the expected number of tests for the whole procedure becomes 6851. For the infection rate of 1%, the expected number of sequential individual tests is 7236, and the expected number of tests for the whole procedure is 26,645. The sensitivity of this method can be obtained by multiplying to the expected number of infected cases in the sequential individual testing. For the infection rate of 0.1%, the sensitivity of this method is 97.5% since out of 100. For the infection rate of 1%, the sensitivity of this method is 97.2% since out of 1000. To calculate the specificity of this method, the probability that an uninfected sample is concluded as positive in the sequential testing is . For the infection rate of 0.1%, since , the specificity of this method is . For the infection rate of 1%, since , the specificity of this method is . These results closely match with our simulation results given for method (F) in Table 7 of Section 3.

To further illustrate how the false negative rate is reduced by this procedure, Table 6 lists all possible cases shown in Table 5 with the false negative rate in each case. Among the 20 possible cases of batch testing, the first 10 cases result in 3 batch negatives, and the second 10 cases result in 3 batch positives. If a sample is in one of the first 10 cases, then it will be deduced as uninfected, and if it is in one of the second 10 cases, then it requires up to 3 sequential individual tests. The false negative rate is for Case (1) because it has three batch negatives, for Cases (2) through (4) because it has three batch negatives and one batch positive, and for Cases (5) through (10) because it has three batch negatives and two batch positives. Since false negative occurs when a sample gets negative in all three individual tests sequentially, β3 is multiplied by the rates given in Table 6 to the second 10 cases. The false negative rate is for Case (11), for Cases (12) through (14), and for Cases (15) through (20). An estimation of the false negative rate of the proposed batch testing procedure can be obtained by multiplying the false negative rate with the subpopulation size given in the last column of Table 5 in each case and divide the sum of these 20 numbers by 100,000. The estimated false negative rate is 3.33% when the infection rate is 0.1%, and 3.25% when the infection rate is 1%. These results yield the sensitivity of 96.67% and 96.75%, respectively. These estimates of the sensitivity are slightly lower and the estimates by combining groups of 10 cases in the previous paragraph (97.5% and 97.2%, respectively) are slightly higher than the simulation results for method (F) in Table 7 given in Section 3, but they are within one standard deviation (1.65% for infection rate 0.1% and 0.57% for infection rate 1%).

TABLE 6

Possible cases of the proposed batch test procedure with false negative rates

	Conclusion to the final batch:							Individual tests
	no infection							for the final batch
		Batch test round					False		Batch test round					False
#tests	Case	1	2	3	4	5	negative rate	Case	1	2	3	4	5	negative rate
3	(1)	−	−	−			β3	(11)	+	+	+			(1−β)^3β^3
4	(2)	−	−	+	−		(1−β)β^3	(12)	+	+	−	+		(1−β)^3β^4
4	(3)	−	+	−	−		(1−β)β^3	(13)	+	−	+	+		(1−β)^3β^4
4	(4)	+	−	−	−		(1−β)β^3	(14)	−	+	+	+		(1−β)^3β^4
5	(5)	−	−	+	+	−	(1−β)^2β^3	(15)	+	+	−	−	+	(1−β)^3β^5
5	(6)	−	+	−	+	−	(1−β)^2β^3	(16)	+	−	+	−	+	(1−β)^3β^5
5	(7)	−	+	+	−	−	(1−β)^2β^3	(17)	+	−	−	+	+	(1−β)^3β^5
5	(8)	+	−	−	+	−	(1−β)^2β^3	(18)	−	+	+	−	+	(1−β)^3β^5
5	(9)	+	−	+	−	−	(1−β)^2β^3	(19)	−	+	−	+	+	(1−β)^3β^5
5	(10)	+	+	−	−	−	(1−β)^2β^3	(20)	−	−	+	+	+	(1−β)^3β^5

SIMULATION STUDIES

We conducted a Monte Carlo simulation study to evaluate the efficiency and efficacy of the proposed batch testing procedure. In this simulation, we assume that the sensitivity and specificity of individual tests are 85% and 99%, respectively. A population of 100,000 people is randomly generated 100 times. The infection rates of 0.1%, 1%, 3%, 5%, and 10% are chosen. Table 7 compares our methods with conventional individual tests, single batch tests, and matrix pool tests using various accuracy measures. We compare the accuracy measures for (A) conventional individual tests, (B) one‐step batch testing with a fixed batch size of 10, (C) one‐step batch testing with optimal batch sizes, with up to three sequential individual tests for each of the positive batches, (D) parallel 12 × 12 matrix pool testing (to cover the whole population, 694 12 × 12 matrices and one 8 × 8 matrix are used); up to three sequential individual tests for all the positive intersections. Since a single matrix testing gives very low sensitivity, we report results from three parallel matrix pool tests. (E) proposed multistep batch tests ending with three batch negatives or three batch positives, with an individual test given to three batch positives, (F) multistep batch tests given in (E), with up to three sequential individual tests for three batch positives. For (C), (E), and (F), the optimal batch sizes estimated in Section 2.5 are used in the subpopulations in each step. For each of the infection rates, the overall accuracy, sensitivity, specificity, PPV, and NPV are calculated, and the required number of tests to cover the whole population is calculated from this simulation, and the values are averaged over the 100 repetitions. The fixed batch size of 10 given in (B) has been used in South Korea for high‐risk facilities. For multistep batch testing, individual tests are conducted to the subpopulation if the infection rate exceeds 30% in any step as mentioned in Section 1. Another set of simulation results for sensitivity of 75% and specificity of 97% is given in the Appendix. The comparison between our methods and the conventional individual tests, single batch testing, or matrix pool tests gives a similar pattern to the one given in this section.

For the matrix pool testing, we also simulated conventional one‐step matrix pool test with an individual test to the positive intersection, but the sensitivity was less than 62% for all the infection rates. For one‐step matrix pool tests with up to three sequential tests to each of the positive intersection, the sensitivity was around 72%. For three parallel matrix pool tests with an individual test to all positive intersections, the sensitivity was around 84%. To achieve high performance for the comparison, we report simulation results for three parallel matrix pool tests with up to three sequential individual tests to all positive intersections in (D). Although not reported in this paper, single‐step batch testing with variable batch sizes also was conducted. The accuracy measures were very close to those of the fixed batch size in (B). However, the batch tests with variable batch sizes require fewer tests.

Note that the number of required tests in the proposed method (F) is close to the estimated number derived in Section 2.5. The sensitivity is reduced in the one‐step batch tests (given in (B)) to approximately 72% from conventional individual tests. This is because if a batch is tested negative, then all the samples in the batch are considered uninfected, and no further tests are given. The results are in line with the sensitivity of 72.25% for one‐step batch testing obtained in Section 2.3.1. Method (C) substantially improved the sensitivity to almost the same level as the sensitivity of conventional individual tests. The sensitivity of method (E) is around 83%, and it is improved by method (F) to approximately 97%. These results show that both the multistep batch testing and sequential individual tests significantly improve the sensitivity. These are in line with the numbers obtained from the model in Section 2.5. This improvement is achieved because sequential individual tests are given to small target subpopulations obtained by our test procedure. Although not included in Table 7, we also conducted a simulation of multistep batch testing with a fixed batch size of 10 followed by up to 3 sequential individual tests. The sensitivity of this method was almost identical to that of method (F) because the sensitivity does not depend on the batch size. However, method (F) requires much fewer tests by using the optimal batch sizes (7053 for method (F) vs. 32,118 for a fixed batch size for infection rate 0.1%, and 27,809 for method (F) vs. 46,416 for a fixed batch size for infection rate 1%). The sensitivity of method (D) is around 99%. However, the number of tests for matrix pool testing is 50,727 for the infection rate of 0.1% and 56,201 for the infection rate of 1%. Although not included in the simulation table, we also simulated three parallel batch testing with a fixed batch size of 10. The sensitivity of the method was near 85%, and the number of tests to cover the whole population was slightly over 30,000 which is larger than that of method (F) for infection rates of 0.1% or 1%, but fewer for the infection rates higher than 1%. The sensitivity of this parallel batch testing was much lower than that of method (F).

As mentioned in Section 2.3.2, the specificity is significantly improved by batch testing from conventional individual testing. The specificity of single batch testing obtained in this simulation closely matches with the expected value given in (13). See also Table 3. Although the specificity of the batch tests decreases as the infection rate increases, the one‐dimensional batch tests (B, C, E, and F) have much higher specificity than the baseline of 99% for all infection rates in Table 7. However, the specificity of the matrix pool tests decreases fast as the infection rate increases and it becomes lower than 99% for the infection rate of 10%.

The improvement of the PPV by our procedure is remarkable. As discussed in Section 2.4 (also shown in (A) of Table 7), the PPV of an individual test is 8% and 46% when the infection rates are 0.1% and 1%, respectively. According to our simulation, single‐step batch testing in (B) improves this to 81% and 90%, respectively, and it is further improved by multistep batch testing given in (E) to 95% and 97%, respectively. The PPV is decreased by sequential individual tests. It is 43% and 75%, respectively, in (C). However, it is improved to 88.5% and 93%, respectively, in (F) by multi‐step batch testing. The PPV is 94% and 95%, respectively, for the matrix pool tests in (D).

The overall number of batch tests from the simulation is close to the estimated number from our model given in Section 2.2. For 0.1% infection rate, the number of multistep batch tests in the simulation (methods (E) and (F)) is around 5700 and the estimated number is 5696. For 1% infection rate, the number is around 19,440 from the simulation and 19,409 from our estimation. However, the simulated number of individual tests for samples with three batch positives is slightly higher than the predicted value. This is likely due to randomness. This discrepancy does not significantly impact the overall number of tests because the size of the subpopulation requiring individual tests is very small (approximately 0.5% for 0.1% infection rate, and around 3% for 1% infection rate) compared to the whole population. For 0.1% infection rate, for example, 516 people received individual tests in the simulation (given in (E)), whereas the model estimates that 448 people need individual tests. This means approximately 99.5% of the population do not require individual tests. The model estimate of this value is 99.552% (100, 000 − 448), and the simulated value is 99.484% (100, 000 − 516).

Note that the overall accuracy is mostly affected by specificity because the infection rates are low. We can incorporate geographic and demographic information for more realistic calculation.

DISCUSSION

The COVID‐19 pandemic changed our lifestyle, seriously impacted the global economy, and took many precious lives. To get back to normalcy, we need a rapid testing of the virus for all the residents of each community. Unlike other coronavirus outbreaks we experienced in the past, the disease rapidly spreads silently by asymptomatic carriers. Since only patients with symptoms have been getting tests, it is a challenging task to identify asymptomatic COVID‐19 carriers. In most countries including the United States, some patients with symptoms could not get tests due to the limited testing capacity. To conduct testing a broader population more efficiently, batch testing methods have been introduced.

The South Korean Center for Disease Control & Prevention used a single‐step batch testing for long‐term care facilities with a fixed batch size of 10 for the entire staff and patients. As seen in our simulation studies, batch testing increases the false negative rate, although this approach can monitor high‐risk groups without symptoms by reducing the number of tests needed to cover the entire community. In this paper, we proposed a multistep batch testing procedure to substantially decrease the false negative rate using a small number of test kits to completely test a large population. The improvement of PPV from individual testing is also remarkable, thus our multistep approach can be trusted for reliable results.

Our approach will be useful for the prevention for early stages of future pandemics. Our method is most effective for diseases with infection rates of up to 3%. We do not recommend this approach for highly contagious large populations with infection rates greater than 5%, as the prescribed amount of tests becomes very large. In contrast, our method is also useful for moderate size, highly contagious communities, which require accurate virus screening for the entire community. Shuren (2020) addresses that conventional batch testing has a higher chance of false negative results because samples are diluted, but it works well when there is a low prevalence of cases. Yelin et al. (2020) found that positive samples can still be well observed in pools in up to 32 samples, and possibly even 64 with additional PCR cycles. However, the detection of positives in a large pool is possible only when viral loads are very high. The viral load presence in COVID‐19 patients can vary over more than nine orders of magnitude (Batson et al., 2020). This can cause a substantial difference in the distributions of Ct values between labs and test populations. Therefore, any lab using batch testing needs to monitor the rate of excess false negatives regularly. In future studies, we will investigate the presence of the dilution effect by focusing on the implementation aspects of the assay such as sampling site, sampling timing, operational consistency of the sampler, and storage of the samples. We will consider implementing a constraint on the batch size as maximum 32 or 64 in our multistep batch procedure. This constraint will increase the number of tests, but it will reduce the dilution effect. We will also investigate optimal stopping rules to further improve the efficiency and efficacy of the multistep batch testing procedure.

CONFLICT OF INTEREST

The authors have declared no conflict of interest.

OPEN RESEARCH BADGES

This article has earned an Open Data badge for making publicly available the digitally‐shareable data necessary to reproduce the reported results. The data is available in the Supporting Information section.

This article has earned an open data badge “Reproducible Research” for making publicly available the code necessary to reproduce the reported results. The results reported in this article could fully be reproduced.

Supporting Information

Click here for additional data file.

TABLE A1

Simulation results: 100 repetitions, population size 100,000, sensitivity 75%, and specificity 97% for individual tests; mean with standard deviation in parentheses

p a		0.001	0.01	0.03	0.05	0.10
(A)	Acc. b	0.9698 (0.0005)	0.9678 (0.0005)	0.9635 (0.0006)	0.9590 (0.0005)	0.9479 (0.0007)
Indiv	Sens. c	0.7528 (0.0427)	0.7505 (0.0127)	0.7498 (0.0077)	0.7502 (0.0062)	0.7497 (0.0040)
Tests	Spec. d	0.9700 (0.0005)	0.9700 (0.0005)	0.9700 (0.0005)	0.9700 (0.0005)	0.9700 (0.0006)
	PPV	0.0249 (0.0029)	0.2019 (0.0060)	0.4354 (0.0065)	0.5681 (0.0059)	0.7347 (0.0050)
	NPV	0.9997 (0.0001)	0.9974 (0.0002)	0.9921 (0.0003)	0.9866 (0.0004)	0.9721 (0.0005)
	#Tests e	100000 (0)	100000 (0)	100000 (0)	100000 (0)	100000 (0)
(B)	Acc.	0.9985 (0.0002)	0.9929 (0.0003)	0.9810 (0.0005)	0.9697 (0.0006)	0.9435 (0.0008)
Single	Sens.	0.5582 (0.0504)	0.5609 (0.0146)	0.5631 (0.0099)	0.5621 (0.0078)	0.5633 (0.0053)
Batch	Spec.	0.9989 (0.0001)	0.9972 (0.0002)	0.9939 (0.0003)	0.9911 (0.0003)	0.9858 (0.0004)
Tests	PPV	0.3444 (0.0359)	0.6717 (0.0158)	0.7413 (0.0079)	0.7690 (0.0067)	0.8154 (0.0042)
Fixed	NPV	0.9995 (0.0001)	0.9956 (0.0002)	0.9866 (0.0004)	0.9773 (0.0005)	0.9531 (0.0008)
Size 10	#Tests	13719 (186)	19825 (279)	31897 (391)	41845 (476)	59885 (494)
(C)	Acc.	0.9948 (0.0004)	0.9877 (0.0005)	0.9778 (0.0005)	0.9710 (0.0007)	0.9521 (0.0008)
Single	Sens.	0.7355 (0.0441)	0.7377 (0.0141)	0.7374 (0.0083)	0.7381 (0.0070)	0.7379 (0.0054)
Batch	Spec.	0.9951 (0.0004)	0.9902 (0.0005)	0.9853 (0.0005)	0.9832 (0.0005)	0.9759 (0.0006)
Variable	PPV	0.1321 (0.0110)	0.4315 (0.0092)	0.6081 (0.0065)	0.6981 (0.0061)	0.7724 (0.0045)
Sizes	NPV	0.9997 (0.0001)	0.9973 (0.0002)	0.9918 (0.0003)	0.9862 (0.0004)	0.9711 (0.0007)
Seq Indiv	#Tests	19141 (1176)	40726 (992)	63015 (984)	74769 (815)	102424 (776)
Tests	B+Ind	2632+16509	7693+33033	12500+50515	16667+58102	20000+82424
(D)	Acc.	0.9991 (0.0003)	0.9967 (0.0003)	0.9887 (0.0005)	0.9791 (0.0007)	0.9966 (0.0011)
Parallel	Sens.	0.9561 (0.0224)	0.9544 (0.0092)	0.9536 (0.0069)	0.9539 (0.0076)	0.9541 (0.0065)
12 × 12	Spec.	0.9991 (0.0003)	0.9971 (0.0003)	0.9897 (0.0005)	0.9804 (0.0007)	0.9568 (0.0010)
Matrix	PPV	0.5389 (0.0878)	0.7672 (0.0202)	0.7422 (0.0083)	0.7189 (0.0061)	0.7108 (0.0048)
Tests	NPV	1.000 (0.0000)	0.9995 (0.0001)	0.9986 (0.0002)	0.9975 (0.0004)	0.9947 (0.0008)
Seq Indiv	#Tests	53039 (986)	60833 (1027)	87098 (1414)	118421 (1809)	192161 (1975)
Test	B+Ind	49968+3071	49968+10865	49968+37130	49968+68453	49968+142193
(E)	Acc.	0.9995 (0.0001)	0.9958 (0.0002)	0.9878 (0.0003)	0.9803 (0.0004)	0.9616 (0.0006)
Multi‐step	Sens.	0.6713 (0.0485)	0.6715 (0.0154)	0.6717 (0.0090)	0.6784 (0.0064)	0.6785 (0.0052)
Batch	Spec.	0.9998 (0.0001)	0.9990 (0.0001)	0.9975 (0.0002)	0.9962 (0.0002)	0.9931 (0.0003)
Variable	PPV	0.7887 (0.0444)	0.8743 (0.0112)	0.8944 (0.0061)	0.9039 (0.0048)	0.9160 (0.0031)
Sizes	NPV	0.9997 (0.0001)	0.9967 (0.0002)	0.9899 (0.0003)	0.9833 (0.0004)	0.9653 (0.0007)
1 indiv	#Tests	7072 (167)	24991 (285)	47494 (355)	60867 (374)	87170 (398)
Test	B+Ind	6375+697	20892+4099	36860+10634	44268+16599	57405+29765
(F) Multi‐ f	Acc.	0.9994 (0.0001)	0.9960 (0.0002)	0.9896 (0.0004)	0.9840 (0.0004)	0.9711 (0.0006)
step Batch	Sens.	0.8887 (0.0289)	0.8832 (0.0104)	0.8841 (0.0061)	0.8904 (0.0045)	0.8914 (0.0034)
Variable	Spec.	0.9995 (0.0001)	0.9971 (0.0002)	0.9929 (0.0003)	0.9889 (0.0003)	0.9799 (0.0005)
Sizes	PPV	0.6342 (0.0346)	0.7557 (0.0115)	0.7930 (0.0063)	0.8083 (0.0044)	0.8314 (0.0036)
Seq Indiv	NPV	0.9999 (0.0000)	0.9988 (0.0001)	0.9964 (0.0002)	0.9942 (0.0002)	0.9878 (0.0004)
Tests for	#Tests	8290 (312)	31393 (467)	63634 (627)	85252 (670)	129406 (746)
3 +'s	B+Ind	6383+1906	20890+10502	36877+26757	44241+41011	57346+72061

Infection rate

Overall accuracy

Sensitivity

Specificity

Number of required tests

Number of batch tests + number of individual tests