The importance of external quality assessment data in evaluating SARS-CoV-2 virus genome detection assays - Authors' reply.

We thank Christoph Buchta and colleagues for their response to our systematic review and meta-analysis. We agree that external quality assessment (EQA) schemes examine the diagnostic test performance of a wide range of nucleic acid assays for SARS-CoV-2 virus genome detection across different laboratories worldwide.

According to WHO, EQA compares test performance within and between laboratories, and such results set a reference for the overall test accuracy and proficiency. However, most EQA done on COVID-19 diagnostics with nucleic acid assays have limited the test performance parameters to cycle threshold (Ct) value, viral concentration, and the number of correct results using synthetic samples which serve as the ground truth in diagnostic evaluation.2, 3

Unlike in the clinical evaluation of diagnostic performance, where sensitivity and specificity are computed from the number of positive and negative patient samples tested in one laboratory in each study, EQA reports the number of correct results as the total number of laboratories correctly identifying a true positive sample. This results in a small sample size per laboratory (a panel of six to seven known samples) which does not resemble real-life practice, where dozens of samples are being tested in one laboratory. Limited sample size might underestimate the false positive rate of the test. Hence, clinical evaluation remains necessary to assess practical diagnostic test performance.

Furthermore, confounding factors during pre-test and post-test procedures might have an impact on the test accuracy because nucleic acid tests are vulnerable to manual errors. As nucleic acid tests target the viral genome, factors that affect the purity of viral genes—such as specimen types and RNA extraction methods—might impact the outcomes of the tests, and thus their diagnostic performance. Also, the actual clinical samples are more complex than synthetic samples in terms of the makeup of the sample matrix and variations in the operational processes such as sample collection and preparation.

Lastly, the use of synthetic samples has been shown to falsely increase the detection limit of diagnostic assays. Hence, our study compares the diagnostic performance of the three nucleic acid tests—ie, digital PCR (dPCR), quantitative PCR (qPCR), and loop-mediated isothermal amplification (LAMP), when different experimental setups are used.

Our findings complement the results from EQA by looking into the experimental factors that are either not evaluated or cannot be assessed using artificial SARS-CoV-2 positive samples. Instead of testing the Ct values of qPCR and LAMP, our analysis uses a different approach in evaluating diagnostic performances as dPCR measures the absolute number of viral copies. The main analytic outcomes are sensitivity, specificity, diagnostic odd ratios, and accuracy presented in the area under the summary receiver operating characteristic curve. The false-positive rate in our findings is also shown to be very low for all three nucleic acid tests evaluated, which agrees with the results from the EQA studies.2, 3 Together, our study substantiates the results of EQA and confirms the test sensitivity, specificity, and accuracy of three different nucleic acid tests using clinical data.

We declare no competing interests.