Mapping the coverage, availability and uptake of External Quality Assessment programmes across One Health sectors in Asia.

INTRODUCTION: Establishing effective external quality assessment (EQA) programmes is an important element in ensuring the quality of, and building capacity for, antimicrobial resistance (AMR) laboratory surveillance.
OBJECTIVES: To understand the current coverage of, and challenges to participation in, EQAs in National Reference Laboratories (NRLs) across One Health (OH) sectors in Asia.
METHODS: Current EQA coverage was evaluated through desktop review, online surveys and interviews of both EQA participants and providers. EQA coverage was mapped and summarized by laboratory type and 'readiness' level and identified challenges evaluated qualitatively.
RESULTS: Of the 31 identified NRLs [16 Human Health (HH) and 15 Animal/Food Safety laboratories (A/FS)], 14 HH and 7 A/FS laboratories currently participated in international EQA schemes and several participated in two or more different schemes. Seven laboratories were currently not participating in any EQA scheme and two of these (one HH and one A/FS) do not currently perform microbiology; six HH NRLs provided national EQAs. Of the eight surveyed international EQA providers, three were based in Asia and all offered varying programmes in terms of pathogens, frequency and support mechanisms for reporting and follow-up. Only one provider currently served laboratories across all OH sectors.
CONCLUSIONS: The current coverage of EQA programmes for AMR in Asia was heterogeneous across countries but especially across OH sectors. This updated overview of the coverage and challenges associated with participation in, and provision of, EQAs for AMR suggest the benefit and relevance of introducing one comprehensive and high-quality EQA programme across OH sectors in Asia.

Introduction

Antimicrobial resistance (AMR) remains a global challenge and burden despite increased focus and efforts to control and prevent it in the past decade., It is estimated that by 2050, mortality attributed to AMR could reach 10 million annually, with the majority of these occurring in low- and middle-income countries (LMICs). This AMR disease burden will be associated with an estimated economic cost of up to 100 trillion USD if no action is taken to effectively prevent the continued emergence of AMR. These estimates are, however, based on modelling using incomplete data of questionable scientific accuracy, particularly from LMICs. LMICs exhibit a relatively larger burden of infectious diseases combined with the lack of AMR data of consistently good quality for surveillance, in part due to inadequate resources in laboratory diagnostics. Therefore, it is recommended to focus on improving laboratory surveillance capacity to ensure that robust data inform decision-making. An important part of building capacity for AMR laboratory surveillance is the establishment of effective external quality assessment (EQA) schemes.

Effective EQA programmes are important to ensure correct identification of pathogens, antimicrobial susceptibility testing (AST) and to verify that laboratory procedures conform to acceptable international standards., Since AMR is a cross-sectoral problem, the benefit of applying a One Health (OH) approach in the context of quality assurance for AMR activities is evident and has been increasingly gaining momentum. The Fleming Fund (FF) was established by the UK Government to support activities that improve AMR surveillance in LMICs in Asia and sub-Saharan Africa. Part of their early scoping to inform funding priorities was a comprehensive review of current surveillance networks, including specific analysis of existing initiatives relevant to quality management. Despite identifying a large number of initiatives within quality assurance, there was no clear or common framework in terms of programme content or governance, and very few identified coordinated efforts outside the human (public) health sector.

Furthermore, although previous global and regional EQA programmes have been targeting most of the WHO’s Global Antimicrobial Resistance and Use Surveillance System (GLASS) pathogens, with some specific programmes also targeting specific foodborne pathogens, they have had disproportionate participation from higher-income countries and highlighted significant limitations in both content, geographical coverage and larger deviations in results among laboratories in LMICs.,, As such, despite significant progress achieved in AMR surveillance efforts in Asia in recent years, gaps in available EQA schemes, as well as variation in capacity between countries and across OH sectors, still remain. Therefore, improving the quality of bacteriology diagnostics through implementing EQA provision across all OH sectors was identified as one of the specific priorities to be addressed within the FF portfolio of regional grants. Starting in January 2020, the ‘Strengthening External Quality Assessment’ in Asia (EQASIA) project was awarded one of these grants to implement EQA schemes in national reference laboratories (NRLs) in the Asian region.

To inform the appropriate approach to implementing a comprehensive regional EQA programme for AMR, EQASIA conducted an initial mapping exercise to understand the current EQA coverage across OH sectors in Asia. Here, EQASIA describes the findings from this exercise, including current uptake and provision of EQA programmes for AMR, as well as the identified challenges associated with implementation of these programmes.

Methods

Existing information sources

A desktop review was conducted to identify existing reference laboratories and current EQA providers in the 12 FF-designated priority countries in Asia: Bangladesh, Bhutan, India, Indonesia, Laos, Myanmar, Nepal, Pakistan, Papua New Guinea, Sri Lanka, Timor-Leste and Vietnam. However, active implementation of the project was only in 11 of the 12 FF-designated countries, which are the focus of this analysis. As an initial step, laboratories designated as AMR reference laboratories by the national AMR coordinating committees, as part of the National Action Plans (NAPs) in the respective countries, as well as other relevant regional/global reference centres and/or EQA providers across the OH sectors, were identified where possible. The desktop review was complemented with data from other FF activities, including initial scoping visits conducted by Mott MacDonald between August 2018 and February 2020, input from communications with other ongoing FF country grants in Indonesia, Pakistan and Vietnam, and FF regional Round 1 grants. The WHO Global AMR team and the Food and Agriculture Organization Regional Office for Asia and the Pacific (FAO-RAP) also provided relevant information from their recent, respective survey activities on AMR surveillance capacities in the region. Finally, additional, and some newly appointed, reference laboratories were identified through communication and engagement with country stakeholders during three project symposia held in Quarter 3 (Q3) of 2020 (Figure 1).

Figure 1.

Data collection and consultation process.

Survey tools and interviews

Two online surveys addressed to ‘EQA participants’ and ‘EQA providers’ were developed using survey software and distributed by sharing the URLs. All survey questions and answers were written/captured in English. The ‘EQA participant survey’ (Survey S1, available as Supplementary data at JAC Online), was sent to 18 Human Health (HH) and Animal/Food safety (A/FS) NRLs identified in eight countries. Supporting information was also collected through communication with FF country grantees. Using the ‘EQA participant survey’, the identified NRLs were queried on their general microbiology capacities, participation in EQA schemes and challenges associated with participation, as well as whether they already provided any EQAs nationally. For laboratories that were not yet participating in any EQA schemes, their perceived challenges for future participation were assessed.

The ‘EQA provider survey’ (Survey S2) was administered to the identified global/regional EQA providers. Additionally, if an NRL responded in the ‘EQA participant survey’ that they offered EQA within the country, they were asked to respond to the ‘EQA provider survey’. Each EQA provider was expected to respond to the survey once. EQA providers were queried on the details and the quality/comprehensiveness of their programmes based on: (1) coverage of GLASS pathogens; (2) frequency of EQAs; (3) ISO/IEC 17043 accreditation; (4) use of IT modules for reporting (and feedback); (5) content and quality of follow-up exercises; and (6) funding structure of the EQAs, including funding sources and capacity for expansion of current programmes. Data from the remaining 3 of the 11 countries were obtained via different sources (as described above) or through direct communication at a later stage. Figure 1 illustrates the combined processes of consultations and data collection comprising the final, complete data sources.

Data analysis

The survey information received from both the participant and provider laboratories were compared for initial cross-validation of the replies. Where details of EQA participation and/or provision were inconsistent or unclear, additional follow-up queries and/or interviews were attempted to provide more granular assessments.

A map to provide overviews and characterize the current EQA provision geographically and across sectors was created for an initial display of how well the designated priority recipients (NRLs) were covered through currently running programmes. Maps were created using R software (https://rstudio.com/).

In addition to mapping out the coverage of programmes, challenges to participation and general questions on microbiological diagnostic capacity and laboratory quality management procedures were also summarized. To identify the most commonly cited challenges by participants, qualitative data analysis within four prespecified domains (communication with the EQA provider, capacity, resources and logistics) was conducted using Microsoft Word 2016 version 16.0 and Displayr: Analysis and Reporting Software for Survey data (https://www.displayr.com/).

Ethics

The collection of data in this project did not require human subject involvement. As such, the project was granted exemption from ethical review by the Institutional Review Board of the International Vaccine Institute (IVI). The purpose of collecting and storing information was explained, and consent to do so obtained from the respective respondents at the beginning of the survey.

Results

EQA participation and provision

Of the 31 laboratories identified, 14 HH and 7 A/FS laboratories stated that they currently participated in international EQA schemes. Three of the HH laboratories and one A/FS laboratory mentioned that they participated in two different schemes and two HH laboratories listed participation in three different EQA schemes. Seven NRLs (one HH and six A/FS) were currently not participating in any EQA scheme and two of these (one HH and one A/FS) do not currently perform microbiology. Three laboratories remained unknown because they did not provide details of their EQA participation. Six of the HH NRLs but none of the A/FS NRLs provided national EQAs. Figure 2 summarizes the current EQA participation and national provision across all surveyed OH reference laboratories. Nine international EQA providers were identified through desktop review and consultations (eight of these replied to the survey) and six HH reference laboratories indicated that they provided national EQAs on the ‘EQA participant survey’ (four of these replied to the ‘EQA provider survey’). None of the surveyed A/FS NRLs provided EQAs. Of the international providers who responded to the survey, three were based in Asia and only one global provider currently served laboratories across all OH sectors, whereas the remaining seven were sector-focused (Figures 3 and 4).

Figure 2.

EQA participation of NRLs across OH sectors in 11 countries in Asia. This figure appears in colour in the online version of JAC and in black and white in the print version of JAC.

Figure 3.

Overview of HH EQA provision in Asia. This figure appears in colour in the online version of JAC and in black and white in the print version of JAC.

Figure 4.

Overview of A/FS EQA provision in Asia. This figure appears in colour in the online version of JAC and in black and white in the print version of JAC.

Challenges of NRLs participating in EQA programmes

When trying to summarize the challenges identified by laboratories of participating in EQAs, laboratories were grouped into three ‘participation categories’ in order to best characterize the challenges identified (‘no EQA participation’, ‘EQA participation only’ and ‘EQA participation and provision’). Information on challenges was collected under four thematic areas: communication with EQA providers; capacity; resources; and logistics (Table 1). For the laboratories that did not participate in EQA, in terms of resources, challenges included ‘cost of participating in EQA’ and ‘lack of qualified staff’. Logistically, the laboratories mentioned ‘difficulty with customs clearance’ and ‘lack of internet access’. In terms of capacity, ‘lack of training for staff’ was mentioned, whereas for communication with the provider, ‘lack of knowledge’ and ‘access to EQA schemes’ were mentioned. In the category of ‘EQA participant only’, the A/FS laboratories did not highlight any challenges. The HH laboratories cited ‘limited knowledge’ and ‘limited access to EQA schemes’ as challenges within the area of communication with the EQA provider. ‘Lack of training’, ‘limited human resource’ and ‘high workload’ were mentioned within the resource and capacity domains. In the ‘EQA participation and provision’ category, ‘financial constraints’ was a common resource challenge, with one laboratory being specific on the ‘annual fee’ and ‘lack of access to current CLSI guidelines’. Laboratories experienced logistical challenges relating to ‘difficulty in importing samples’ and also mentioned ‘difficulty to access the internet’. When it came to communication with the EQA provider, ‘limited knowledge of available EQA schemes in the region and internationally’ was mentioned. In terms of capacity, ‘lack of support to implement corrective actions’ was commonly mentioned.

Table 1.

Challenges of participating in EQA programmes by NRLs across OH sectors in 11 countries in Asia

Participation	Communication with the EQA provider	Capacity	Logistics	Resources
EQA participation and provision	Lack of support like staff training, supplies including reagents and equipment Limited knowledge about EQA schemes	Frequent reagent stockout Lack of back-up for equipment Lack of automated system for blood culture, shortage of high-quality antibiotic discs Lack of support to implement corrective actions	Customs delay Lack of internet access	Financial constraints Lack of qualified staff Frequent staff turnover
EQA participation only	Limited knowledge about EQA schemes Lack of access to EQA schemes Lack of support like training, supplies and equipment	Lack of training for staff Frequent reagent stockout Lack of back-up for equipment	Difficulty in importing samples	Financial constraints High workload Lack of qualified staff Frequent staff turnover
No EQA participation	Lack of support like training, supplies and equipment Lack of knowledge about EQA schemes Lack of access to EQA schemes	Lack of training for staff Frequent reagent stockout Lack of back-up for equipment	Difficulty in importing samples e.g. customs delay, lengthy process to import Lack of internet access	Lack of funds to organize EQA and financial constraints High workload Lack of qualified staff Frequent staff turnover

Quality of programmes and identified challenges for EQA providers

The general impression of current EQA schemes was rather heterogeneous and while most of the providers conducted EQAs across the spectrum of WHO GLASS priority pathogens, none currently provided comprehensive schemes covering all pathogens or the full range of antimicrobials. While almost all provided EQAs for pathogen identification, some only offered AST for selected isolates. Only five of eight international EQA providers were ISO/IEC 17043 accredited. Four of eight international and one of four national EQA providers indicated using information technology (IT) systems for reporting and analysis. One of the national providers only provided an EQA scheme for pathogen identification and not for AST. The frequency of the EQA distributions varied across providers, with 4 of 12 sending them three times per year. Some (3 of 12) also stated that they provided EQA distributions monthly, but whether this was a misconception of the nature of the programmes rather than being based on re-testing of isolates was not fully ascertained. Three of eight international providers and three of four national providers had a dedicated EQA budget. On the other hand, five of eight international EQA providers and one of four national EQA providers asked the EQA participants to pay an annual subscription fee. When highlighting challenges, five of eight international and one of four national EQA providers commented on ‘lack of resources to support comprehensive follow-up activities for participating laboratories’. Three of the national EQA providers described the nature of their EQA programmes as ‘facilitation of another international EQA programme’.

Discussion

The current coverage of EQA programmes for AMR in Asia appears rather heterogeneous across countries, but especially across OH sectors, likely due to a wide range of current capacity levels and hence a wide range in readiness to participate in EQA programmes. Among current programmes, the coverage is variable in both content and frequency and there are redundancies due to participation in multiple programmes seen in several laboratories. The main shortcomings and challenges identified by current EQA providers are lack of IT solutions to support programme reports and evaluations, limited financial resources for sustaining participation, and follow-up exercises for underperforming laboratories. In 2017, a comprehensive review of existing programmes related to quality management in AMR surveillance identified 27 different initiatives in LMICs functioning at a regional or global level. Eleven of these were coordinated by a supranational body and the remaining ones by a mix of academic groups, commercial entities and governmental or non-governmental institutions. The content of the programmes also varied, with about half offering EQA programmes only and the other half offering different combinations of EQA programmes with training, assessments, standard/policy setting and accreditation or mainly functioning as a reference material provider. Such heterogeneity in other aspects of country efforts within AMR surveillance has also been evident in previous reviews from across the Asian region., Since these publications, multiple efforts have been initiated in the region, including establishment of NAPs to enable commitment from countries to support and share data with the WHO GLASS. In combination with the substantial efforts invested through the FF country grant portfolio, as well as other important initiatives, the laboratory capacity across bacteriology reference laboratories in the region has also been strengthened. However, many of the generated data are still not sufficiently quality assured and therefore still of limited value in surveillance efforts in LMICs. To appropriately guide the implementation of an EQA programme across the Asian region, the EQASIA project has provided an updated overview of the current coverage and challenges faced in participating in these programmes. This scoping activity has served to illustrate not only the coverage of existing EQA activities in 11 countries in Asia but also highlighted some of the challenges that laboratories face, either as EQA participants or providers. While most of the HH laboratories participated in EQA programmes, only half of the A/FS laboratories that provided feedback reported participation in an EQA programme. This shows a critical gap that needs to be addressed, particularly in this age of emerging and re-emerging zoonotic communicable diseases, and in the face of limited resources.,

Only one of the surveyed EQA providers already served both HH and A/FS laboratories in the region. Combining this EQA provider’s expertise with the relevant and already-present EQA provider capacities working with WHO within the HH sector and with FAO within the A/FS sector in Asia represents a unique opportunity to bring together local expertise and presence while leveraging an existing global EQA platform to introduce a One Health EQA in line with both WHO and FAO priorities to strengthen existing regional programmes.

At the national level, several of the HH NRLs reported that they also provided EQA, while none of the A/FS laboratories did. This demonstrates recent achievements and development within the HH sector but also highlights the need for similarly building the capacities of A/FS NRLs to become national EQA providers while ensuring that all national EQA providers are trained and supported in assuring quality in their further national implementation of EQAs in line with ISO/IEC 17043. It is important to ensure EQA providers are ISO/IEC 17043 certified to ensure the schemes they run are competent; its application applies to planning and design, management, personnel, equipment, quality assurance and confidentiality of the EQA.

No global EQA scheme is currently comprehensively supporting the WHO GLASS. There are multiple programmes that offer EQAs worldwide but the associated costs are a barrier to participation for laboratories in LMICs. This obstacle was also highlighted by laboratories in the surveys. To maintain commitment to the EQAs, and compliant and equitable participation from all countries in the region, EQASIA deems it important to offer EQAs free of charge. Limited financial resources also affect the ability to access the needed support materials, as described previously by other EQA programmes. This supports the idea that not only should participation in EQAs be offered free of charge, but programmes should also provide support for procurement of guidelines (e.g. CLSI documents) and financial support for associated training activities wherever feasible.

In addition to financial constraints, laboratories also face diverse challenges that can hinder participation in EQA, such as limited knowledge of providers, limited human resources and inadequate training. For any EQA programme to be truly effective, associated training activities and follow-up of underperforming laboratories is crucial. There is still limited use of IT for electronic microbiology data capture and collation from laboratories in the region, which also represents an opportunity that needs to be considered in the design and implementation of a regional EQA programme.

Finally, all of the abovementioned challenges jointly underline the importance of sufficient funding and political support to sustain comprehensive EQA programmes in Asia in the future. As such, the success and sustainability of a regional EQA programme will depend on the adoption and/or support by supranational organizations, such as WHO, FAO and the World Organisation for Animal Health (OIE), as well as commitment from global funding bodies.

Conclusions

The current coverage of EQA programmes for AMR in Asia is rather heterogeneous across countries but especially across OH sectors, with a wide range of current capacity levels and readiness to participate in EQA schemes. Variable content and identified redundancies between programmes, as well as both technical and financial limitations, further challenges the successful implementation and optimal impact of EQA programmes in the region.

These findings suggest the benefit and relevance of introducing one comprehensive and high-quality EQA programme for AMR, free of charge to all NRLs across the OH sectors in the Asian region. Such a programme should include an IT solution for efficient reporting, a follow-up regimen for underperforming laboratories and also a comprehensive capacity-building programme to support participants’ continuous development, allowing them to benefit from participation in the EQA programme. To ensure inclusion of all laboratories, regardless of current capacity levels, the training programme should be broad but flexible. Further studies to determine the optimal number of reference laboratories required by sector and by country is needed to sufficiently address EQA needs. A sustainability plan informed by realistic costing and forecasting analyses for the continued provision of the programme in the Asian region is also needed. Finally, the adoption and support of the programme by relevant supranational tripartite organizations, as well as committed funding bodies, are imperative for its future success and sustainability.

Click here for additional data file.