COVID-19: Black Swan or clumsy use?

"Black Swan" is an unexpected and catastrophic event characterized by three conditions: the anomalous and exceptional value, the extreme impact, and predictability a posteriori. On many sides - by politicians, commentators, scientists - there have been attempts to give the COVID-19 pandemic event the meaning of Black Swan. In reality, one should wonder if the Black Swan was not, instead (or even), the way the epidemic was handled. In Italy, an unreasonable use of a two-step protocol - IgM/IgG anti-COVID followed by RT-PCR in the serologically positive only has caused considerable uncertainty, particularly relevant in measures for maintenance or readmission to work of health personnel. This protocol could be affected by serious uncertainties arising from the analytical limitations of the tests used. To quantify this defect, we evaluated the analytical specifications of serological tests under different prevalence conditions. In conclusion, although laboratory diagnostics represent a useful tool, it can only be used for epidemiological purposes and not to provide healthy pass. ©2021 Pacini Editore SRL, Pisa, Italy.

Introduction

Borrowing from Juvenal, Nassim Nicholas Taleb developed his “Black Swan” theory as an unexpected and abnormal event of great importance, capable of generating significant environmental changes whose meaning, although implicit, is grasped only a posteriori [1].

Politicians and commentators, scientists even, regarded the COVID-19 pandemic as a Black Swan event; actually, we should wonder whether the black swan could be not the pandemic rather than its epidemiological management.

Laboratory test results drove most decisions to manage the pandemics spread; however, their use is often clumsy and inappropriate and deserves to be discussed.

Laboratory tests are useful in distinguishing positive from negative, i.e., sick from healthy. This characteristic is relevant during an epidemic event because it allows the adaptation of both therapeutic and health policy choices to contain the disease’s spread.

The characteristics that allow bettering separate the healthy population from the affected one must be verified for the latter use. Negative predictive values (NPV) and false omission rate (FOR) are the parameters that need consideration. NPV is a test capability to be negative in non-affected people[NPV = (1-prevalence) specificity / (1-prevalence) specificity + prevalence (1-sensitivity)]. FOR is the complementary quantification of false-negatives (FN).

In the early pandemic stages, the lack of knowledge of various diagnostic tests’ analytical performances was a weak point for infection containment measures’ effectiveness.

Later, most serological qualitative tests seemed inadequate in detecting anti-COVID19 antibodies. The window-phase bounding and IgM timing to IgG seroconversion proved to be the most criticalities [2-4].

In Italy, serological tests’ unreasonable use has caused considerable uncertainty, which has proved relevant for the maintenance or readmission to health personnel’s work. These measures include a two-step protocol – IgM/IgG anti-COVID followed by RT-PCR in the serologically positive only – to identify health workers who could continue to be operational or, conversely, to be quarantined [5].

This protocol could be affected by severe uncertainties arising from the analytical limitations of the tests used.

Methods

We evaluated serological tests’ analytical specifications under different prevalence conditions to quantify this defect.

By way of example the serological tests considered are: 1) COVID-19 IgG/IgM (Screen Italia, Perugia, Italy); 2) Liaison SARS-CoV-2 S1/S2 IgG (Diasorin, Saluggia, Italy); 3) Maglumi 2019-nCoV IgG; and 4) Maglumi 2019-nCoV IgM+IgG (Snibe Diagnostic, Shenzhen, China).

Concerning the direct research of the COVID-19 virus with RT-PCR method, we referred to the performances reported for different technologies and [6] on nasopharyngeal swabs (NP) and bronchoalveolar washes (BAL), for which sensitivity of 63% and 93% respectively were reported [7].

The exact prevalence of COVID-19 infection is affected by various factors such as regional variability [8] and an unspecified number of non-swab-tested healthy carriers [9, 10]. We calculated each serological test for FN rate related to their NPV and FOR in different prevalence values. Since FN rate rises as prevalence increases, the resulting number might consider an estimate of health care providers admitted to caring services, despite their infectivity.

Results

As shown in Table I, depending on the NPV and FOR showed by the used test, FNs subjects range between 11 and 134, in a population of 1,000 inhabitants with infection prevalence equal to 0.3 (300 positives). Unfortunately, in the second phase of viral research on serologically positive subjects, further FNs will be are added. For example, in the same population prevalence, an RT-PCR method with a sensitivity of 0.7 will demonstrate VNP and FOR values that will result in 16 FN out of a total of 166 VP [11].

Tab. I.

NPV, FOR and FN rate in different prevalence of infection. NPV is calculated on the basis of sensitivity and specificity values.

	Screen test COVID-19 IgG/IgM			Screen testCOVID-19 IgG/IgM			Liaison SARS-CoV-2 S1/S2 IgG(> 15 days)			Maglumi 2019 - nCoV IgG (CLIA)			Maglumi 2019 - nCoV (CLIA) IgM + IgG
	Se 95.0Sp 98.0(BAL)			Se 65.0Sp 97.0(NF)			Se 97.4Sp 98.5			Se 91.21Sp 97.33			Se 95.80Sp 96.00
Pre	NPV	FOR	FN	NPV	FOR	FN	NPV	FOR	FN	NPV	FOR	FN	NPV	FOR	FN
0.10	0.994	0.006	6	0.961	0.039	39	0.997	0.003	3	0.990	0.010	10	0.995	0.005	5
0.20	0.987	0.013	13	0.917	0.083	83	0.993	0.007	7	0.978	0.022	22	0.989	0.011	11
0.30	0.979	0.021	21	0.866	0.134	134	0.989	0.011	11	0.963	0.037	37	0.982	0.018	18
0.40	0.967	0.033	33	0.806	0.194	194	0.983	0.017	17	0.943	0.057	57	0.972	0.028	28
0.50	0.951	0.049	49	0.735	0.265	265	0.974	0.026	26	0.917	0.083	83	0.958	0.042	42
0.60	0.929	0.071	71	0.649	0.351	351	0.962	0.038	38	0.881	0.119	119	0.938	0.062	62
0.70	0.894	0.106	106	0.543	0.457	457	0.942	0.058	58	0.826	0.174	174	0.907	0.093	93
0.80	0.831	0.169	169	0.409	0.591	591	0.904	0.096	96	0.735	0.265	265	0.851	0.149	149
0.90	0.685	0.315	315	0.235	0.765	765	0.808	0.192	192	0.552	0.448	448	0.717	0.283	283

Pre: prevalence; Se: sensitivity; Sp: pecificity; NPV: negative predictive value; FOR: false omission rate; FN: false negative per thousand. Screen rest Se and Sp are relative – RT-PCR reference gold standard. Real Se and Sp are obtained by considering RT-PCR Broncho Alveolar Lavage (BAL) and RT-PCR nasopharyngeal (NF) as gold standard.

The result of two-step screening is that 152 positive subjects out of 300 could be classified incorrectly as negative. This result comes from 134 IgM/IgG FNs with 18 FNs resulting from RT-PCR under the given conditions. Fewer FNs are achieved using other methodologies and/or different prevalence values; however, the best result of 43 FN remains equally worrying because of the obvious fallout on the possible expansion of contagion.

Discussion

Two step protocol could be affected by serious uncertainties arising from the analytical specifications of the tests under different prevalence conditions.

Our analysis shows that performing RT-PCR research exclusively on positive IgM/IgG subject prevents the discovery of numerous infected operators due to the combined methodological error of used tests in the two steps.

The risk of two-stage screening is that a critical number of positive subjects may be wrongly classified as negative. This risk become more relevant the higher the amount of prevalence of those affected.

Therefore, the consequent unknown presence of false negatives can be a danger and source of outbreaks.

Conclusions

Unreasonable use of serological tests may have caused considerable uncertainty, particularly relevant in maintenance or readmission to work of health personnel, using a two-step protocol - IgM/IgG anti-COVID followed by RT-PCR in the serologically positive only.

The non-segregation of these subjects and their free working activity in the healthcare environment represents an insidious source of new disease outbreaks, making containment null.

This appraisal could be evoked to explain the high incidence of infections (and deaths) among operators and patients in some Italian regions with the highest prevalence, such as Lombardia and Emilia Romagna.

These pieces of evidence could have suggested the combined use (instead of sequential) of both serology and RT-PCR [12]. Such choice would have enhanced the ability to intercept affected subjects and perhaps have averted the Black Swan spread of Covid-19, increasing safety levels.

In conclusion, Laboratory tests can help to distinguish positive from negative, i.e., sick from healthy; but it depends on their characteristics. Although both serological tests and RT-PCR are useful tools, they can only be used for epidemiological purposes and not to provide healthy pass.

At present, the correct behavior would be to consider all subjects present in hospitals as potentially infected, in order to enhance security.