Evaluation of the diagnostic capacities for emerging arboviral diseases in the international network MediLabSecure from 2014 to 2018 - Importance of external quality assessments.

BACKGROUND: Emerging infectious diseases pose an increasing threat to all nations around the world, including to developed countries. By definition, because they are rare or unknown, public health systems are not well prepared against these emerging diseases. To be fully prepared, countries must have implemented surveillance systems to monitor rare or unusual sanitary events.
METHODS: The capacity of diagnostic laboratories is a key component of surveillance systems since they are in charge of identifying the pathogens responsible for outbreaks in a timely manner. The MediLabSecure project aims at implementing a comprehensive surveillance system for vector-borne diseases around the Mediterranean and Black Sea regions. From 2014 to 2018, the human-virology group of MediLabSecure notably supported the implementation of molecular diagnostic capacities for eight arboviruses and one coronavirus in 19 laboratories of its network through sharing of protocols and reagents, and technical training of the scientific staff of beneficiary laboratories.
RESULTS: We report the results of External Quality Assessments for four of these viruses to assess the efficiency of the diagnostic for these threats emerging in the geographic area. The results for these EQA demonstrate the success of the project in the implementation of diagnostic technics for the identification of Dengue, Chikungunya, Zika, and West Niles viruses in laboratories that did not have the capacity before. However, results also show that some work is still to be done to strengthen the newly acquired capacity.
CONCLUSION: The MediLabSecure project deployed an effort to build an efficient capacity in identifying and survey the emergence of arboviruses in the Mediterranean area. Diagnostic technics were successfully implemented in many of the laboratories of the network, but the effort must be maintained over time to strengthen these capacities.

Introduction

Over the early twenty-first century, emerging infectious diseases became a major concern for public health in numerous countries. In 2018 and most recently in 2020, WHO updated the list of priority diseases for research and development [1]. This list now includes mainly emerging diseases caused by arboviruses, such as Rift Valley fever (RVF) or Crimean-Congo haemorrhagic fever (CCHF) viruses; zoonotic haemorrhagic or encephalitic viruses, i.e. Ebola, Lassa, or Nipah viruses; and the zoonotic coronaviruses responsible for lower respiratory infections and acute respiratory distress syndromes, i.e. MERS-CoV (cause of the Middle East respiratory syndrome, MERS), SARS-CoV-1 (associated with severe acute respiratory syndrome, SARS), and SARS-CoV-2 (currently causing the ongoing COVID-19 pandemic).

The capacity of a country to respond quickly to the emergence of a pathogen starts at the laboratory level with the detection of the first cases. This detection is key in controlling the resulting epidemic. In 2013, failure to detect Zika virus (ZIKV) in the South Pacific and in South America resulted in missing the rapid increase in cases and led to a one-year delay in the international response [2]. In 2014, in Guinea, the inability to identify Ebola virus, which was unknown in this region, led to a four-month delay in the response to control the epidemic, partly responsible for the magnitude of the Ebola epidemic [3].

The MediLabSecure project is an international project funded by the European Commission since 2014. Its purpose is to enhance preparedness in preventing and controlling the emergence of zoonotic viruses around the Mediterranean Sea, the Black Sea, and since 2018, the Sahel regions. This project advocates the One Health approach, a concept relying on the intercommunication and interdependence of the human and animal worlds with the environment and the climate [4]. Hence, MediLabSecure includes human-virology laboratories, animal-virology laboratories, medical entomologists, and public health and animal health institutions in each of nineteen non-EU beneficiary countries (Supplemental Fig. 1), and increased to twenty-two countries in 2018 (since 2014 - Albania, Algeria, Armenia, Bosnia and Herzegovina, Egypt, Georgia, Jordan, Kosovo, Lebanon, Libya, Montenegro, Morocco, Palestine, Republic of North Macedonia, Serbia, Tunisia, and Turkey; since 2018 - Burkina-Faso, Mali, Mauritania, Niger, and Senegal; and from 2014 until 2018 - Moldova and Ukraine). It represents a cluster for awareness, risk assessment, surveillance, monitoring, and control of relevant emerging diseases, with a special focus on arboviral infections. MediLabSecure operates at different levels: (i) At the laboratory level, MediLabSecure implements and/or improves pathogen detection [5], with a particular emphasis on biosafety and biosecurity processes; (ii) at the institutional level, the project advocates integrated surveillance, One Health collaborations, and tries to support the establishment of early warning systems in countries [6,7]; (iii) at the international level, MediLabSecure promotes networking to enhance collaborations among neighbouring countries.

From 2014 to 2018, the human-virology group of MediLabSecure worked on developing the capacity of its network of nineteen laboratories for the identification of emerging viral pathogens. The main target pathogens were identified as emerging threats to the region because of their close distribution area or because of the presence of their vectors in the region of MediLabSecure [8]: Dengue virus (DENV), Usutu virus (USUV), West Nile virus (WNV), Chikungunya virus (CHIKV), Zika virus (ZIKV), Crimean-Congo haemorrhagic fever virus (CCHFV), Rift Valley fever virus (RVFV), Yellow fever virus (YFV), and MERS coronavirus (MERS-CoV). The implementation was done through the organization of dedicated workshops, the sharing of protocols and reagents and the evaluation of diagnosis capabilities through External Quality Assessment (EQA) schemes. The work presented here focuses particularly on the assessment of implementation of molecular diagnostic capabilities by Reverse transcription quantitative real-time PCR (RT-qPCR) for four of these viruses (WNV, CHIKV, DENV, ZIKV) over the first four years of the project.

Methods

To evaluate the capabilities and guarantee the quality of laboratory results, the MediLabSecure project sponsored the participation of the beneficiary laboratories to a series of EQA programmes. Two non-profit EQA providers were engaged for the organization of molecular EQAs in 2016 and 2018.

INSTAND [9] EQA schemes were dedicated respectively to the qualitative detection and the optional lineage identification of WNV, and the qualitative detection of CHIKV. Positive samples (WNV and CHIKV) derived from lysates of virus-infected cells (inactivated) and negative samples derived from lysates of non-infected cells. All samples were sent to the participating laboratories in lyophilized form.

Quality Control for Molecular Diagnostics (QCMD) [10], an independent international EQA and Proficiency Testing (PT) organization, proposed four EQA schemes to assess the proficiency of laboratories in the detection of WNV (incl. optional lineage typing), CHIKV, DENV (incl. optional type identification), and ZIKV in reconstituted biological samples. Positive samples (for WNV, CHIKV, DENV, and ZIKV) were obtained from supernatants of virus-infected cell cultures and inactivated afterwards by heat and gamma-irradiation, while samples with other cell culture-derived viruses were used as specificity controls or samples with transport medium only were used as true negative controls.

The EQA providers do not refer to concentrations of samples in qualitative EQA schemes. Only the dilution ratios or the sample relationships by dilution series and order of reducing titer are used in the reports of the respective EQA providers to indicate sensitivities based on low, medium, or high diluted samples. The actual values of titers (high, medium, or low) depend on the virus considered to reflect real biological samples. For this report, we kept the providers’ formal presentations of results.

Results

Laboratory diagnostic capacity situation in 2014 and implementation of molecular diagnostic technique

Nineteen laboratories from nineteen countries participated within the human-virology laboratory network during 2014–2018 (see Supplemental Fig. 1). A questionnaire sent during the selection process allowed to identify the needs and gaps of the newly created network. It also served as basis for the capacity building strategy developed during the following years. Regarding laboratory capabilities for the identification of the viruses targeted within the project, the responses to the questionnaire showed that 79% of the laboratories declared capabilities for the molecular diagnostic for WNV, while 63% implemented DENV diagnostic (see Supplemental Fig. 2). However, for other arboviruses, the declared capabilities were much lower with 37% for CCHFV, 32% for tick-borne encephalitis virus (TBEV), 21% for CHIKV, 16% for RVFV infections, and only 5% (1 laboratory) for YFV. Of note, ZIKV was not initially part of the project and was added following the 2013–2016 epidemic. About 50% of the network laboratories were also involved in the identification of respiratory diseases such as influenza or MERS.

The implementation of diagnostic capacity started in 2015 with a training of the laboratory staffs to technical aspects of molecular and serological diagnostics. Standard Operating Procedures (SOPs), protocols and reagents required for the identification of the targeted arboviruses by quantitative RT-PCR were shared regularly with the laboratories of the network.

Evaluation of molecular diagnostic techniques through External Quality Assessments in 2016

From 2014 to 2018, the MediLabSecure project sponsored the participation of the network laboratories in different EQA programmes. This participation was completely voluntary and some countries were not capable or not willing to participate because of local difficulties. In 2016, the human-virology working group participated in two EQA programmes proposed by INSTAND on molecular diagnostics of WNV and CHIKV. The coordinating laboratory of the human-virology network participated in these two EQAs.

INSTAND EQA schemes “Virus Genome Detection – West Nile virus” 2016

Nineteen laboratories from the network (including the coordination laboratory) participated in the INSTAND EQA scheme on West Nile virus genome detection. Overall, 63.2% (12/19) of the laboratories successfully identified all the core samples of the programme and validated the EQA. Individually, the results were satisfactory with a success rate of 89.5%–94.7% for the high concentrated positive samples. However, the results were lower, at 84.2%, for the low concentrated positive samples 391C and 391F (Table 1 ). Considering the large participation and the fact that 79% of laboratories declared a previous implementation of the technique for WNV molecular diagnostics, the success rate was considered acceptable. Only two laboratories characterized the lineage of WNV (lineage was optional in this EQA, the coordinating laboratory did not participate in the lineage identification). Both laboratories perfectly identified all samples from lineage 1 and 2.

Table 1

Global results of MediLabSecure laboratories to INSTAND EQA scheme on “Virus Genome Detection – West Nile virus” in 2016, incl. lineage identification.

Sample No.	Sample source	Dilution ratios	Participating laboratories (n)	Success ratea	Lineage id (n)	Success rate
391Ab	Lysate of West Nile virus lineage 1 infected cells (inactivated)	1:300b	19	89.5%	2	100%
391B	Lysate of non-infected cell culture	-----	19	89.5%	2	100%
391Cb	Lysate of West Nile virus lineage 1 infected cells (inactivated)	1:30.000b	19	84.2%	2	100%
391Dc	Lysate of West Nile virus lineage 2 infected cells (inactivated)	1:3c	19	94.7%	2	100%
391Ec	Lysate of West Nile virus lineage 2 infected cells (inactivated)	1:30c	19	94.7%	2	100%
391Fd	Lysate of West Nile virus lineage 2 infected cells (inactivated)	1:300.000$	19	84.2%	2	100%
All samples			19	63.2%

Percentage of laboratories successfully identifying each sample or all the panel in the “all samples” line.

The positive samples 391 A and 391 C represent dilution steps of a dilution series of a lysate of West Nile virus lineage 1 infected cells (inactivated).

The positive samples 391 D and 391 E represent dilution steps of a dilution series of a lysate of West Nile virus lineage 2 infected cells (inactivated).

The positive sample 391F derives from the same stock material as the samples 391D and 391E (independent preparation).

INSTAND EQA schemes “Virus Genome Detection - Chikungunya virus” 2016

Eighteen laboratories from the network participated in the INSTAND EQA scheme on detection of the Chikungunya virus. At the beginning of the project, only 21% (4/19) of laboratories declared capabilities to detect CHIKV in biological samples. The global success rate to the EQA was quite satisfactory with 77.8% (14/18). This EQA showed a good capacity (94.4%) of detection of the CHIKV Martinique strain (392G and 392H), but lower success rate (77.8%) for the African strain S27 (392E) used in this panel (Table 2 ).

Table 2

Global results of MediLabSecure laboratories to INSTAND EQA scheme on “Virus Genome Detection – Chikungunya virus” 2016.

Sample No.	Sample source	Dilution ratio	Participating laboratories (n)	Success ratea
392E	Lysate of Chikungunya virus (S 27) infected cells (inactivated)	1:100	18	77.8%
392F	Lysate of non-infected cell culture	-----	18	88.9%
392Gb	Lysate of Chikungunya virus (Martinique) infected cells (inactivated)	1:100b	18	94.4%
392Hb	Lysate of Chikungunya virus (Martinique) infected cells (inactivated)	1:1.000b	18	94.4%
All samples			18	77.8%

Percentage of laboratories successfully identifying each sample or all the panel in the “all samples” line.

The positive samples 392G and 392H represent dilution steps of a dilution series of a lysate of Chikungunya virus infected cells (inactivated).

Evaluation of molecular diagnostic techniques through External Quality Assessments in 2018

Despite globally satisfactory results for the identification of both WNV and CHIKV in 2016, the network still presented with room for improvement. In 2018, MediLabSecure proposed to all human-virology laboratories to participate in EQA programmes from QCMD regarding some of the viral targets of the project: WNV, CHIKV, ZIKV, and DENV. Others as MERS-CoV EQA were performed but not reported here. Beneficiary laboratories were free to register for some or all programmes. The nineteen laboratories responded, and all of them registered to at least one of the EQAs, often to the four. Unfortunately, some laboratories were not capable to answer in a timely manner. Each responding laboratory sent one dataset to QCMD. Altogether, an average of 74.3% of laboratories that registered participated in the EQAs and responded in time (75% for WNV (15/20), 76% for CHIKV (13/17), 82% for ZIKV (14/17), 72% for DENV (13/18)).

QCMD EQA “MediLabSecure 2018 West Nile Virus Study”

In 2018, the human-virology laboratories registered to a second EQA programme on WNV conducted by QCMD (Table 3 ). The coordinating laboratory of the human-virology network participated in this EQA. Overall, 60% (9/15) of the laboratories successfully identified all the core samples of the programme and validated the EQA. This rate was equivalent to the one measured two years before, suggesting stability but no improvement. However, individually, at least 93.3% (14/15) of the laboratories correctly identified each sample with the notable exception of the two lower-concentrated WNV lineage 1 (WNV NY99) samples, MEDWNV18-02 (80% success (12/15)) and MEDWNV18-07 (66.7% success (10/15)). This result suggests a lineage specific sensitivity issue for the techniques employed since the lineage 2 (WNV Heja) was correctly identified by 93.3% of laboratories at its lower concentration. The MEDWNV18-07 was considered an educational sample and its correct identification was not necessary to validate the quality programme. The specificity of identification was quite good with at least 93.3% of correct identification for the three negative samples, including two specificity controls with other flaviviruses. Thus, the lower global success rate of 60% was due to six laboratories doing one misidentification rather than a couple of laboratories with an inadequate technique.

Table 3

Global results of MediLabSecure laboratories in the QCMD EQA “MediLabSecure 2018 West Nile Virus Study”.

Sample No.	Sample source	Sample relation-ships [1]	Detection frequency [2]/Sample status [3]	Datasets (n)	Success rate [4]
MEDWNV18-01	WNV Heja Lineage 2	DS1_3	Detected/CORE	15	93.3%
MEDWNV18-02	WNV NY99 Lineage 1	DS2_2	Detected/CORE	15	80.0%
MEDWNV18-03	Non-WNV flaviviruses (DENV 1/2/4 & JEV)		Negative/CORE	15	93.3%
MEDWNV18-04	WNV NY99 Lineage 1	DS2_1	Frequently Detected/CORE	15	100%
MEDWNV18-05	WNV Heja Lineage 2	DS1_1	Frequently Detected/CORE	15	100%
MEDWNV18-06	WNV Heja Lineage 2	D1, DS1_2	Frequently Detected/CORE	15	100%
MEDWNV18-07	WNV NY99 Lineage 1	DS2_3	Detected/EDUCATIONAL	15	66.7%
MEDWNV18-08	Non-WNV flaviviruses (DENV 3, TBEV, YFV & ZIKV)		Negative/CORE	15	100%
MEDWNV18-09	Negative		Negative/CORE	15	93.3%
MEDWNV18-10	WNV Heja Lineage 2	D1, DS1_2	Frequently Detected/CORE	15	100%
All CORE samples				15 (9/15 laboratoriesa)	60.0%a

[1] Sample Relationships: Indicates the relationships of the samples within this challenge. The highest titer member of dilution series DS1 is indicated by DS1_1 and further members of the series as DS1_2, DS1_3 etc. in order of reducing titer. Additional dilution series are indicated by DS2 (e.g., DS2_1, DS2_2 etc.). One duplicate pair is present and is indicated by ‘D1’.

[2] Detection Frequency: To aid qualitative analysis each panel member is assigned a frequency of detection. This is based on the peer group consensus of all qualitative results returned from participants within the EQA challenge/distribution.

[3] Sample Status: EQA samples are defined as “CORE” or “EDUCATIONAL”. Core proficiency samples are reviewed by the QCMD Scientific Expert(s). This is on the basis of scientific information, clinical relevance, current literature and, where appropriate, professional clinical guidelines. Participating laboratories are expected to report core proficiency samples correctly within the EQA challenge/distribution.

[4] Percentage Correct (All): Percentage of datasets (%) reporting the correct qualitative result.

The overall success rate for all core samples refer to the number of participating laboratories. Laboratories having reported results obtained by several methods are recorded only once in the calculation.

QCMD EQA “MediLabSecure 2018 Chikungunya Virus Study”

The results of the second EQA on CHIKV identification organized by QCMD are displayed in Table 4 . Around 77% (10/13) of the group succeeded to correctly identify all the core samples and validate the EQA programme, showing stability over the years. Regarding the individual samples, they were correctly identified in at least 92.3% of the analyses, with the same limitation for some laboratories concerning low concentration samples. In this case, difficulties were met with MEDCHIK18-08 (84.6% success, 11/13), with a dilution of the 1st WHO International Standard for CHIKV RNA, and MEDCHIK18-09 (15.4% success, 2/13), an educational samplewith a very low concentration of CHIKV ECSA (East/Central/south African genotype) strain. The specificity however reached 100% correctly identified for the three negative samples that included also WNV and DENV positive samples as specificity controls.

Table 4

Global results of MediLabSecure laboratories in the QCMD EQA “MediLabSecure 2018 Chikungunya Virus Study”.

Sample No.	Sample source	Sample relation-ships [1]	Detection frequency [2]/Sample status [3]	Datasets (n)	Success rate [4]
MEDCHIK18-01	CHIKV WHO ISb	DS2_1	Detected/CORE	13	92.3%
MEDCHIK18-02	CHIKV ECSA strainc	DS1_1	Detected/CORE	13	92.3%
MEDCHIK18-03	WNV NY99		Negative/CORE	13	100%
MEDCHIK18-04	CHIKV ECSA strainc	DS1_3	Frequently Detected/CORE	13	100%
MEDCHIK18-05	DENV-4		Negative/CORE	13	100%
MEDCHIK18-06	Negative		Negative/CORE	13	100%
MEDCHIK18-07	CHIKV ECSA strainc	DS1_2	Frequently Detected/CORE	13	100%
MEDCHIK18-08	CHIKV WHO ISb	DS2_2	Detected/CORE	13	84.6%
MEDCHIK18-09	CHIKV ECSA strainc	DS1_5	Infrequently Detected/EDUCATIONAL	13	15.4%
MEDCHIK18-10	CHIKV ECSA strainc	DS1_4	Detected/CORE	13	92.3%
All CORE samples				13 (10/13 laboratoriesa)	76.9%a

[1] Sample Relationships: Indicates the relationships of the samples within this challenge. The highest titer member of dilution series DS1 is indicated by DS1_1 and further members of the series as DS1_2, DS1_3 etc. in order of reducing titer. Additional dilution series are indicated by DS2 (e.g., DS2_1, DS2_2 etc.).

[2] Detection Frequency: To aid qualitative analysis each panel member is assigned a frequency of detection. This is based on the peer group consensus of all qualitative results returned from participants within the EQA challenge/distribution.

[3] Sample Status: EQA samples are defined as “CORE” or “EDUCATIONAL”. Core proficiency samples are reviewed by the QCMD Scientific Expert(s). This is on the basis of scientific information, clinical relevance, current literature and, where appropriate, professional clinical guidelines. Participating laboratories are expected to report core proficiency samples correctly within the EQA challenge/distribution.

[4] Percentage Correct (All): Percentage of datasets (%) reporting the correct qualitative result.

The overall success rate for all core samples refer to the number of participating laboratories. Laboratories having reported results obtained by several methods are recorded only once in the calculation.

World Health Organization (WHO) International Standard, PEI code 11785/16; generated from isolate R91064 which represents the East/Central/south African genotype (ECSA).

ECSA strain (from the epidemic in the Indian Ocean area 2004–2007).

QCMD EQA “MediLabSecure 2018 Zika Virus Study”

After the global epidemic of 2013–2016, the molecular detection of this virus was implemented in the region partly thanks to the MediLabSecure project [5]. The QCMD EQA programme [11] was also the opportunity to evaluate the implementation of the techniques in the participating laboratories. The results were quite satisfactory (Table 5 ) and around 64% (9/14) of the laboratories participating in the EQA programme correctly identified all core samples. Individually, five samples were identified with a success rate of 100%, and two samples by 92.9% (13/14) of the participants. As for the other EQAs, the difficulty was met for the lowest concentrated ZIKV African lineage sample ZIKA18S-06 with only 71.4% (10/14) successfully identifying this sample. The specificity was very good since 100% of the laboratories identified the negative samples (including also a specificity control with other flaviviruses).

Table 5

Global results of MediLabSecure laboratories in the QCMD EQA “MediLabSecure 2018 Zika Virus Study”.

Sample No.	Sample source	Sample relation-ships [1]	Detection frequency [2]/Sample status [3]	Datasets (n)	Success rate [4]
ZIKA18S-01	ZIKV French Polynesian (IS candidate (11474/16)) b		Frequently Detected/CORE	14	100%
ZIKA18S-02	ZIKV French Polynesian (IS candidate (11468/16)) c	DS2_2	Detected/CORE	14	92.9%
ZIKA18S-03	ZIKV French Polynesian (IS candidate (11468/16)) c	DS2_1	Frequently Detected/CORE	14	100%
ZIKA18S-04	ZIKV African straind	DS1_2	Frequently Detected/CORE	14	100%
ZIKA18S-05	ZIKV African straind	DS1_1	Detected/CORE	14	92.9%
ZIKA18S-06	ZIKV African straind	DS1_3	Detected/CORE	14	71.4%
ZIKA18S-07	Non-ZIKV flaviviruses (DENV 2, WNV NY99 & YFV)		Negative/CORE	14	100%
ZIKA18S-08	Negative		Negative/CORE	14	100%
All CORE samples				14 (9/14 laboratoriesa)	64.3%a

[1] Sample Relationships: Indicates the relationships of the samples within this challenge. The highest titer member of dilution series DS1 is indicated by DS1_1 and further members of the series as DS1_2, DS1_3 etc. in order of reducing titer. Additional dilution series are indicated by DS2 (e.g., DS2_1, DS2_2 etc.).

[2] Detection Frequency: To aid qualitative analysis each panel member is assigned a frequency of detection. This is based on the peer group consensus of all qualitative results returned from participants within the EQA challenge/distribution.

[3] Sample Status: EQA samples are defined as “CORE” or “EDUCATIONAL”. Core proficiency samples are reviewed by the QCMD Scientific Expert(s). This is on the basis of scientific information, clinical relevance, current literature and, where appropriate, professional clinical guidelines. Participating laboratories are expected to report core proficiency samples correctly within the EQA challenge/distribution.

[4] Percentage Correct (All): Percentage of datasets (%) reporting the correct qualitative result.

The overall success rate for all core samples refer to the number of participating laboratories. Laboratories having reported results obtained by several methods are recorded only once in the calculation.

ZIKV reference material 11474/16 prepared by the Paul-Ehrlich-Institut (PEI) (from French Polynesian ZIKV strain PF13/251013-18, representing the Asian lineage).

WHO IS preparation 11468/16 developed by PEI (from French Polynesian ZIKV strain PF13/251013-18, representing the Asian lineage).

Zika virus strain Uganda MR766 provided by the Robert Koch-Institut (RKI) (representing the African lineage).

QCMD EQA “MediLabSecure 2018 Dengue Virus Study”

The results for the QCMD EQA among the participating countries were acceptable despite a moderate global success rate of 61.5% (8/13) in the programme. Individually, five samples were correctly identified by 92.3% (12/13) of the participants and four samples were correctly identified by all participants (Table 6 ). The educational sample DENVRNA18S-03, with the lowest concentration of DENV-3, was correctly identified by 84.6% (11/13) of the laboratories. The specificity was excellent with 100% identification of the negative samples (including also other flaviviruses as specificity control). The moderate global success rate was due to five different laboratories reporting one mistake each with the core samples.

Table 6

Global results of MediLabSecure laboratories in the QCMD EQA “MediLabSecure 2018 Dengue Virus Study”.

Sample No.	Sample source	Sample relation-ships [1]	Detection frequency [2]/Sample status [3]	Datasets (n)	Success rate [4]
DENVRNA 18S-01	DENV-3	DS1_1	Frequently Detected/CORE	13	100%
DENVRNA 18S-02	DENV-2		Detected/CORE	13	92.3%
DENVRNA 18S-03	DENV-3	DS1_3	Detected/EDUCATIONAL	13	84.6%
DENVRNA 18S-04	DENV-1	D1	Detected/CORE	13	92.3%
DENVRNA 18S-05	DENV-3	DS1_2	Detected/CORE	13	92.3%
DENVRNA 18S-06	DENV-4	DS2_1	Detected/CORE	13	92.3%
DENVRNA 18S-07	Negative		Negative/CORE	13	100%
DENVRNA 18S-08	DENV-1	D1	Detected/CORE	13	92.3%
DENVRNA 18S-09	Non-DENV flaviviruses (JEV, WNV NY99, TBEV, YFV & ZIKA)		Negative/CORE	13	100%
DENVRNA 18S-10	DENV-4	DS2_2	Frequently Detected/CORE	13	100%
All CORE samples				13 (8/13 laboratoriesa)	61.5%a

[1] Sample Relationships: Indicates the relationships of the samples within this challenge. The highest titer member of dilution series DS1 is indicated by DS1_1 and further members of the series as DS1_2, DS1_3 etc. in order of reducing titer. Additional dilution series are indicated by DS2 (e.g., DS2_1, DS2_2 etc.). One duplicate pair is present and is indicated by 'D1'.

[2] Detection Frequency: To aid qualitative analysis each panel member is assigned a frequency of detection. This is based on the peer group consensus of all qualitative results returned from participants within the EQA challenge/distribution.

[3] Sample Status: EQA samples are defined as “CORE” or “EDUCATIONAL”. Core proficiency samples are reviewed by the QCMD Scientific Expert(s). This is on the basis of scientific information, clinical relevance, current literature and, where appropriate, professional clinical guidelines. Participating laboratories are expected to report core proficiency samples correctly within the EQA challenge/distribution.

[4] Percentage Correct (All): Percentage of datasets (%) reporting the correct qualitative result.

The overall success rate for all core samples refer to the number of participating laboratories. Laboratories having reported results obtained by several methods are recorded only once in the calculation.

Discussion

The MediLabSecure project aims to prevent emerging infectious diseases through multiple and complementary actions in its geographical area. At the laboratory level, the project notably promotes the implementation of detection techniques for a panel of emerging arboviruses and respiratory viruses of zoonotic origin. Indeed, the detection of the pathogens is the first key step to raise awareness and alert authorities about the risk of a particular epidemic.

The first action of the MediLabSecure project was to create a network of legitimate sentinel laboratories. The selection of the laboratories for each specialty was key in the process. The national and international visibility of the laboratories selected regarding the targeted viruses had to be well established. For the human-virology network, the project took advantage of the former European projects Episouth and Episouth Plus [12], whose objectives were to create a framework of collaboration on epidemiological issues to improve communicable disease surveillance, communication, and training across the countries of the Mediterranean and Balkans regions. The Episouth projects notably worked on the implementation of identification technique for WNV and DENV. Some of the laboratories involved in the Episouth projects were included in the MediLabSecure project, thus explaining the better level of implementation of diagnostics for WNV and DENV compared to other arboviruses at the beginning of the MediLabSecure project (see Supplemental Fig. 2).

The MediLabSecure project invested on the hands-on training of the scientists and laboratory staffs of the network. This report focuses on molecular diagnostics, nonetheless, training on serological diagnostics was also dispensed. Workshops were a central activity of the project. From 2014 to 2018, four workshops took place for the human-virology network. Next to the first training on molecular and serological diagnostics of arboviruses, three complementary workshops were organised: one on biosafety and bio-risk assessments in laboratories and shipment of hazardous substances; and two workshops on sequencing, data analysis, and phylogenetics (basic and expert levels). Additionally, multiple international meetings reunited specialists from the different laboratories and institutions. During these meetings, technical presentations on pathogens of regional concern were organized. Workshops and meetings, along with EQAs, were the occasion to share SOPs on molecular diagnostics with the beneficiary laboratories. From 2014 to 2018, eight viruses were targeted: CCHFV, CHIKV, YFV, ZIKV, DENV, WNV, USUV, and MERS-CoV. It is important to note that the goal of the project was not to impose any technique nor a particular protocol for the detection of the targeted viruses. SOPs and reagents were sent to volunteer laboratories, but they were never imposed nor were supposed to replace an already implemented technique. Several laboratories kept the commercial or in-house protocols for diagnostics when a sensitive technique was already in place. The project aimed at giving the capacity for the detection of these viruses to all the participating laboratories. A critical example of the positive impact of MediLabSecure was the implementation of ZIKV diagnostic techniques in all the laboratories of the network during the pandemic of 2014–2016.

EQAs are a common tool to control and guarantee the efficiency and quality of the techniques implemented in diagnostic laboratories and are mandatory for some quality certifications and accreditation. From 2014 to 2018, the MediLabSecure project sponsored the participation of the network laboratories from veterinarian and human-virology sectors to different commercial [11] or house-made EQA programmes [13,14] to test the diagnostic capabilities on WNV, USUV, ZIKV, CHIKV, DENV, YFV, RVF, and MERS-CoV. Specific EQA for the identification of mosquito vectors was created for medical entomology laboratories from the network [15]. The participation to these EQAs was on voluntary basis and some countries could not, or would not, participate because of certain situations (virus absent from the region, customs or shipment difficulties, local epidemic overwhelming the lab capacities, etc.). In this report, the results that the MediLabSecure human-virology laboratories obtained in six EQA programs were presented to evaluate the implementation of molecular diagnostic technics within the network. However, the actual aim of the EQAs was for each beneficiary laboratory to make its own auto-evaluation and serve as quality control for its laboratory qualifications. The coordinator laboratory, which was the sponsor for these programmes, only received the results as a global anonymous group result.

The global results on the six EQA programmes were quite satisfactory. Overall, above 60% of the laboratories validated the EQAs by correctly identifying all core samples of the programmes. Looking at individual samples, often above 90% of the laboratories successfully identified the virus proposed. For WNV and CHIKV, two successive EQA programmes showed a comparable global success rate over time. Since the EQA programmes were conducted by different EQA providers, a direct comparison between results would not be adequate. However, individual identification rates improved between 2016 and 2018, showing that these EQAs were also the opportunity for the laboratories that failed at a particular programme to correct their procedures and improve the techniques. The main issues were the sensitivity of the techniques, with less efficiency, as a group, to identify the limit samples (low concentrated samples). Over the six EQA programmes, it was never possible to associate the sensitivity issues to a specific technic, in-house protocol, or commercial kit. The misidentifications were equally distributed amongst the declared technics, suggesting that the limit of detection for different viruses depends on other factors, such as the type of apparatus used, the training of staff, or the local conditions. This sensitivity issue was sometimes strain-specific (or lineage-specific). The results for the WNV EQA from QCMD (2018) suggested a better sensitivity of the techniques for the lineage 2 (WNV Heja) compared to lineage 1 (NY99), which was not expected since lineage 2 is quite recent in the Mediterranean area compared to the lineage 1 and laboratories were more used to identify the latest. Sensitivity limitation is well known for molecular diagnostic and might be an issue to detect arboviruses in blood samples since the viremia is often low and last only a few days. This also shows that there is still room for improvement in the network and that MediLabSecure will benefit from an additional four years. The specificity of the implemented techniques was very good for the arboviruses. Above 90% of the group, often 100%, were capable of correctly identify the negative samples, even when another close virus was present in the samples.

In this work, we used the results obtained from EQA programmes to evaluate the efficiency of the MediLabSecure project to transfer competencies. Overall, the EQA programmes showed that the transfer was done and effective for most of the laboratories considering the diagnostics of WNV, CHIKV, ZIKV, and DENV. For WNV and DENV diagnostics, most of the laboratories had already implemented the techniques by 2014 (15 and 12 laboratories, respectively). However additional laboratories implemented and validated the techniques (18 laboratories participated for WNV in 2016, 15 in 2018; 13 laboratories participated to the DENV EQA in 2018). The results for the CHIKV and ZIKV diagnostics are more remarkable since in 2014 only four laboratories could diagnose Chikungunya and probably none had diagnostic capacities for Zika. In 2016, 17 laboratories participated in the EQA programme for CHIKV identification, and 13 in 2018. Fourteen laboratories participated to the ZIKV EQA in 2018. Thanks to the MediLabSecure project, all these key sentinel laboratories of the Mediterranean area are now capable to identify these emerging viruses, next to a few additional others, in case suspected cases are found.

These valuable data must not be understood as the final goal for the project, neither a success nor a failure, but rather as a temporary picture of the progress made, and work left to be done in the capacity building strategy of MediLabSecure. The repetition of EQAs on WNV and CHIKV in 2016 and 2018 showed a slight progression in the group. An important conclusion that these results highlighted is the fact that a correct and sustainable implementation of a diagnostic technique requires more than sharing protocols and reagents, but also necessitates the training of the staff and regular external quality controls over time.

Conflict of interest

None declared.

Funding statement

MediLabSecure Project is funded by the through the Instrument contributing to Stability and Peace and the EU CBRN CoE initiative (DG DEVCO IFS/2013/330 961). The contents of this article are the sole responsibility of the authors and do not necessarily reflect the views of the European Union.