External quality assessment of the molecular diagnostics and genotyping of meticillin-resistant Staphylococcus aureus.

Two multicentre external quality assessments (EQA) for the molecular detection and genotyping of meticillin-resistant Staphylococcus aureus (MRSA) were arranged. Firstly, 11 samples containing various amounts of inactivated MRSA strains, meticillin-susceptible S. aureus (MSSA), meticillin-resistant coagulase-negative staphylococci (MRCoNS) or Escherichia coli were distributed to 82 laboratories. Samples containing 102 or 103 MRSA cells were correctly scored in only 16 and 46% of the datasets returned, respectively. Two of the used MSSA strains contained an SCCmec cassette lacking the mecA gene. There was a marked difference in the percentage of correct results for these two MSSA strains (37 and 39%) compared to the MSSA strain lacking the SCCmec cassette (88%). Secondly, a panel for MRSA genotyping, consisting of ten samples (two identical, three genetically related and five unique strains) was distributed to 19 laboratories. Seventy-three percent of the datasets recorded all samples correctly. Most pulsed-field gel electrophoresis (PFGE) protocols proved to be suboptimal, resulting in inferior resolution in the higher or lower fragment regions. The performance of molecular diagnostics for MRSA shows no significant changes since our first EQA in 2006. The first molecular typing results are encouraging. Both assessments indicate that programme expansion is required and that major performance discrepancies continue to exist.

Introduction

Adequate infection control of meticillin-resistant Staphylococcus aureus (MRSA) strongly depends on the speed and quality of (molecular) identification and characterisation strategies used by the clinical microbiological laboratory [1, 2].

Over the past 4–5 decades, cultivation assays have been primarily used for the detection and subsequent identification of MRSA. However, cultivation requires prolonged incubation periods and, in general, clinically relevant meticillin resistance still needs to be confirmed by the detection of the mecA gene or its product. Nucleic acid amplification techniques (NAATs) offer benefits over traditional culture-based assays, in particular, a reduced time to identification and an improved specificity and sensitivity. Over the past decade, a range of commercial and in-house developed NAATs has been introduced. The sensitivity and specificity of these assays may be compromised, as a result of inhibition or false-positivity due to the presence of meticillin-resistant coagulase negative staphylococci (MRCoNS) or variability within the mec-resistance cassette. This may lead to false-negative results (new staphylococcal cassette chromosome mec [SCCmec] variants) or false-positive results (deletion of the mecA gene) [3-6].

After MRSA detection, genetic typing may be necessary in order to assess whether local cross-infection occurs and whether preventive measurements are mandatory. Currently, many different genotyping methods are in use in the diagnostic laboratory, but pulsed-field gel electrophoresis (PFGE) of SmaI digested genomic DNA still remains the most frequently used method [7]. Only when outbreaks are properly defined, adequate infection control measurements can be implemented.

The current multicentre external quality assessment (EQA) study determined the performance of molecular assays to detect MRSA and genotyping techniques to differentiate MRSA strains. The studies were coordinated by Quality Control for Molecular Diagnostics (QCMD) in Glasgow, Scotland.

Material and methods

EQA for molecular MRSA detection and identification

In August 2009, the EQA MRSA panel for MRSA detection and identification was distributed to 80 participating laboratories in 15 countries, along with detailed sample processing instructions. Participants were given 6 weeks to examine the samples and to report their results to the QCMD by using an online data collection system. The QCMD MRSA panel consisted of six samples containing 106, 105 (n = 2), 104, 103 and 102 CFU/ml MRSA bacterial cells, one meticillin-susceptible S. aureus (MSSA) sample, one sample containing MRCoNS, two samples containing MSSA harbouring an SCCmec cassette lacking the mecA gene and one sample containing Escherichia coli (Table 1). The production laboratory quantified the contents of the samples on the basis of colony counting, optical density measurements and real-time molecular amplification results. All bacterial samples were heat-inactivated for 10 min at 100°C.

Table 1

Composition of the QCMD 2009 panel for MRSA detection and identification

Sample	Sample content	Target value (CFU/ml)	Sample status
MRSA09-01	MRCoNS c	1 × 107	Negative
MRSA09-02	MSSA d	5 × 106	Negative
MRSA09-03	MSSA b	5 × 106	Negative
MRSA09-04	MRSA a	5 × 105	Positive
MRSA09-05	MRSA a	5 × 106	Positive
MRSA09-06	MRSA a	2.5 × 102	Positive
MRSA09-07	MRSA a	2.5 × 104	Positive
MRSA09-08	MRSA a	5 × 105	Positive
MRSA09-09	MSSA e	1 × 107	Negative
MRSA09-10	MRSA a	2.5 × 103	Positive
MRSA09-11	E. coli f	5 × 106	Negative

MRSA a: meticillin-resistant Staphylococcus aureus N315

MSSA b: meticillin-sensitive Staphylococcus aureus ATCC 29213

MRCoNS c: meticillin-resistant coagulase-negative staphylococci 634

MSSA d: meticillin-sensitive Staphylococcus aureus (MecA negative 92-1866 [SSCmec III])

MSSA e: meticillin-sensitive Staphylococcus aureus (MecA negative 93-2886 [SCCmec I])

E. coli f: Escherichia coli ATCC 35218

EQA for MRSA genotyping

The EQA panel for MRSA genotyping was distributed to 19 participants in eight countries in August 2009. The panel consisted of ten samples of viable MRSA strains in Müller Hinton broth. Genetic relatedness of the MRSA panel was originally determined with PFGE [8]. The current panel consisted of two identical strains, three genetically related strains and five unique strains (Table 2). Genotype and subtype were reported by the production laboratory. A different letter signifies the detection of a different genotype, whereas a different number signifies the detection of a different subtype. All data were reported in relation to the reference strain in panel sample MRSATP09-01.

Table 2

Genotyping results per panel member and technology type of the QCMD 2009 panel for MRSA genotyping

Sample	Expected genotype	Total datasets		PFGE a		AFLP b		spa c
		n = 15		n = 11		n = 2		n = 2
		n	%	n	%	n	%	n	%
MRSATP09-01	A	15	100	11	100	2	100	2	100
MRSATP09-02	B	11	73.3	8	72.7	1	50.0	2	100
MRSATP09-03	C	14	93.3	11	100	1	50.0	2	100
MRSATP09-04	D	14	93.3	11	100	1	50.0	2	100
MRSATP09-05	A	13	86.7	10	90.9	1	50.0	2	100
MRSATP09-06	E	13	86.7	11	100	2	100	0	0.0
MRSATP09-07	F	12	80.0	9	81.8	1	50.0	2	100
MRSATP09-08	G	14	93.3	11	100	1	50.0	2	100
MRSATP09-09	G1	11	73.3	11	100	0	0.0	0	0.0
MRSATP09-10	G2	11	73.3	11	100	0	0.0	0	0.0

PFGE a: BioRad GenePath group 1 reagent kit (n = 2), in-house PFGE (n = 8) and double-locus sequence typing (n = 1)

AFLP b: in-house AFLP (n = 2)

spa c: in-house spa typing (n = 2)

The QCMD Neutral Office analysed the data, which was anonymously released to all participants in a detailed EQA final report.

Results

Out of the 80 participants, 68 (85%) responded. Twelve participants did not return results. Five of these withdrew officially, indicating ‘assay not offered’ (n = 2), ‘internal issues’ (n = 1) and ‘other’ (n = 2) as the reason for withdrawal.

The following commercial amplification assays were used for MRSA detection: BAG HealthCare Hyplex StaphyloResist (n = 1) (BAG Healthcare, Lich, Germany), BD Diagnostic GeneOhm MRSA Assay (n = 13) (BD Diagnostics, GeneOhm, San Diego, California), BD Diagnostics GeneOhm Staph SR Assay (n = 5) (BD Diagnostics , GeneOhm), Cepheid IDI MRSA (n = 1) (Cepheid, Sunnyvale, California), Cepheid Xpert MRSA Test (n = 11) (Cepheid), Cepheid Xpert MRSA/SA Test (n = 2) (Cepheid), Roche LightCycler MRSA Advanced Test (n = 4) (Roche Diagnostics, Basel, Switzerland), TIB MOLBIOL LightMix Kit MRSA (n = 2) (TIB MolBiol, Berlin, Germany) and Hain Lifescience GenoQuick MRSA (n = 2) (Hain Lifescience, Nehren, Germany). This diversity overlaps with the spectrum of currently available commercial tests.

All results are summarised in Table 3. Results for the panel samples with 106 MRSA cells (MRSA09-05), 105 MRSA cells (MRSA09-04 and MRSA09-08) and 104 MRSA cells (MRSA09-07) were reported correctly in 100, 100, 99 and 98% of the datasets, respectively. The samples containing lower amounts, MRSA09-10 (103 CFU/ml) and MRSA09-06 (102 CFU/ml), were reported correctly in only 46 and 16% of the datasets, respectively. No statistically significant differences in sensitivity or specificity could be seen between the different tests or between commercial and in-house testing. MRCoNS sample MRSA09-01 was correctly reported as MRSA-negative by 87% (40 out of 46) of the commercial polymerase chain reaction (PCR) tests and in 70% (23 out of 33) of the in-house PCR assays. MSSA sample MRSA09-03 was correctly reported as MRSA-negative by 89% (41 out of 46) of commercial PCR tests and in 85% (28 out of 33) of datasets generated by in-house PCR assays, respectively. The MSSA samples containing the SCCmec cassette but lacking the mecA gene (MRSA09-02 and MRSA09-09) were both incorrectly reported as positive by commercial PCR tests in 87% (40 out of 46). For in-house assays, these samples were reported incorrectly in 24% (8 out of 33) and 30% (10 out of 33), respectively. These percentages of incorrect results underline the need for improved specificity of these MRSA tests and, therefore, positive results should always be confirmed by a culture method or a second molecular test. For laboratories with high false-positivity rates or in regions with a low prevalence of MRSA, confirmation is essential [9]. For the E. coli sample, commercial PCR results were reported correctly in 87% (39 out of 45), whereas in-house PCR tests recorded correct results in 97% (29 out of 30).

Table 3

Number of correct qualitative results per panel member and technology type

Sample	Sample content	Total datasets		PCR								NASBA e		Other f
				Conventional				Real-time
				Commercial a		In-house b		Commercial c		In-house d
		n = 82		n = 1		n = 3		n = 45		n = 30		n = 1		n = 2
		n	%	n	%	n	%	n	%	n	%	n	%	n	%
MRSA09-05	MRSA	82	100	1	100	3	100	45	100	30	100	1	100	2	100
MRSA09-04	MRSA	82	100	1	100	3	100	45	100	30	100	1	100	2	100
MRSA09-08	MRSA	81	98.8	1	100	3	100	44	97.8	30	100	1	100	2	100
MRSA09-07	MRSA	80	97.6	1	100	3	100	45	100	28	93.3	1	100	2	100
MRSA09-10	MRSA	38	46.3	0	0.0	2	66.7	22	48.9	13	43.3	0	0.0	1	50.0
MRSA09-06	MRSA	13	15.9	0	0.0	0	0.0	6	13.3	6	20.0	0	0.0	1	50.0
MRSA09-03	MSSA	72	87.8	1	100	3	100	40	88.9	25	83.3	1	100	2	100
MRSA09-09	MSSA	30	36.6	1	100	3	100	5	11.1	20	66.7	1	100	0	0.0
MRSA09-02	MSSA	32	39.0	1	100	3	100	5	11.1	22	73.3	1	100	0	0.0
MRSA09-01	MRCoNS	66	80.5	0	0.0	2	66.7	40	88.9	21	70.0	1	100	2	100
MRSA09-11	E. coli	73	89.0	1	100	1	33.3	39	86.7	29	96.7	1	100	2	100

Commercial a: BAG HealthCare Hyplex Staphyloresist (n = 1)

In-house b: details not presented

Commercial c: BD Diagnostics, details not provided (n = 7), BD Diagnostics GeneOhm MRSA Assay (n = 13), BD Diagnostics GeneOhm Staph SR Assay (n = 5),Cepheid IDI MRSA (n = 1), Cepheid Xpert MRSA Test (n = 11), Cepheid Xpert MRSA/SA Test (n = 2), Roche LightCycler MRSA Advanced Test (n = 4),TIB MolBiol LightMix Kit MRSA (n = 2)

In-house d: details not presented

NASBA e: details not presented

Other f: Hain Lifescience GenoQuick MRSA (n = 2)

Out of the 19 potential participants, 14 (74%) responded. Four of the non-responders withdrew officially, indicating ‘panel used for research’ (n = 1) and ‘assay not offered’ (n = 3). The majority of datasets were generated by PFGE (n = 11), with the remainder generated by AFLP (n = 2) and spa typing (n = 2). Only eight participants (73%) scored all samples correctly, all with PFGE (Table 2).

Discussion

To maintain high-quality clinical care, quality control of molecular diagnostics is very important. The primary aim of our EQA programmes was to assess the proficiency of laboratories in the molecular detection and characterisation of MRSA strains. Here, we conclude that the molecular detection of MRSA in samples with high CFU counts is reliable, which can, and has been, implemented in various laboratory settings with confidence. All tests performed equally well. However, for direct molecular diagnostics, we have to conclude that current commercial and in-house tests do not meet the required quality criteria. The sensitivity of many tests is relatively low and the specificity needs to be improved. The FDA submission in 2004 of the IDI-MRSA test had a detection limit of 325 CFU/swab [10]. This is much more sensitive than documented in this study. Pre-enrichment of clinical samples leads to concentrations of MRSA exceeding 109 CFU/ml, which is higher than the concentrations of MRSA likely to be found in a patient sample and also those in this EQA panel. However, pre-enrichment reduces one of the major advantages offered by NAATs, namely, rapidity. In this year’s panel, only inactivated cells were present. As a consequence, pre-enrichment was not possible. This may have influenced the results of some laboratories. For the years to come, viable cells will be distributed, which is more similar to the real clinical situation.

This is the fourth year that the QCMD has offered the MRSA DNA EQA Programme and the number of participants has increased from 51 in 2006, 61 in 2007 and 74 in 2008 to 80 in 2009 [11]. Over the years, we observe a statistically non-significant decrease in the overall test sensitivity. The most pronounced discrepancies were observed in the low-concentration panel samples (103 and 102 CFU/ml). Conversely, the percentage of correct results showed an overall improvement for the ‘specificity’ samples (containing MSSA and MRCoNS or E. coli) and the ‘true-negative’ samples (Table 4). Again, this was not statistically significant. Still, these incorrect results underline the need for improved specificity of molecular MRSA tests and, therefore, positive results should always be confirmed by a culture method or a second molecular test. For laboratories with high false-positivity rates or in regions with a low prevalence of MRSA, confirmation is necessary.

Table 4

Comparison of performance on like samples in the QCMD 2006, 2007, 2008 and 2009 MRSA EQA programmes

Sample content	Sample concentration (CFU/ml)	Percentage of correct results
		2006	2007	2008	2009
MRSA a	1 × 109	96.6	96.3	93.2	NIP
MRSA a	5 × 106	82.8	96.3	91.9	100
MRSA a	5 × 105	NIP	NIP	NIP	100
MRSA a	5 × 105	NIP	NIP	NIP	98.8
MRSA a	2.5 × 104	NIP	NIP	NIP	97.6
MRSA a	2.5 × 103	51.7	72.2	54.1	46.3
MRSA a	1 × 103	51.7	66.7	55.4	NIP
MRSA a	2.5 × 102	12.1	37.0	20.3	15.9
MSSA b	1 × 109	87.9	92.6	89.2	NIP
MSSA b	5 × 106	NIP	NIP	NIP	87.8
MSSA b	1 × 103	NIP	94.4	95.9	NIP
MRCoNS c	1 × 107	82.8	88.9	94.6	80.5
MSSA b + MRCoNS c	1 × 103 + 1 × 105	94.8	83.3	86.5	NIP
MSSA b + MRCoNS c	1 × 103 + 1 × 104	96.6	77.8	83.8	NIP
MSSA d	5 × 106	NIP	NIP	NIP	36.6
MSSA e	1 × 107	NIP	NIP	NIP	39.0
E. coli f	5 × 106	93.1	92.6	97.3	89.0
S. aureus negative medium	–	NIP	92.6	98.6	NIP

NIP: not in panel

MRSA a: meticillin-resistant Staphylococcus aureus N315

MSSA b: meticillin-sensitive Staphylococcus aureus ATCC 29213

MRCoNS c: meticillin-resistant coagulase-negative staphylococci 634

MSSA d: meticillin-sensitive Staphylococcus aureus (MecA negative 92-1866 [SSCmec III])

MSSA e: meticillin-sensitive Staphylococcus aureus (MecA negative 93-2886 [SCCmec I])

E. coli f: Escherichia coli ATCC 35218

S. aureus negative medium: Müller Hinton broth.

In 2009, two MSSA samples harbouring an SCCmec cassette, but lacking the mecA gene, were included. There was a marked difference in the percentage of correct qualitative results for the MSSA strain containing the mecA gene compared to the two strains lacking it. These data show that confirmation on results generated using assays that only target the SCCmec cassette is mandatory.

In conclusion, the quality of molecular diagnostic tests still needs improvement, and proper and regular quality control and international standardisation for MRSA diagnostics should be mandatory for the years to come.

We present the first QCMD EQA programme for the genetic characterisation of MRSA strains. Clear differences in resolution were observed between the datasets. Some PFGE protocols, which were implemented by most of the participating laboratories, proved to be suboptimal, as a low level of discrimination in the high and/or low molecular weight fragments was observed in the majority of the results reported. This suggests the need for optimisation of the PFGE programme. This lack of resolution was most evident within the group of closely related MRSA strains in the panel (MRSATP09-08, MRSATP09-09 and MRSA09-10). These strains were incorrectly reported in 27% of datasets. Participants reported using a range of criteria for determining genotype and subtype. The guidance according to Tenover et al. was the most prominent method [12].

To improve the performance and quality of molecular diagnostics, both laboratories and manufacturers should be encouraged to participate in EQAs. The availability of EQA panels for detection and typing should also be developed for other important (nosocomial) infectious agents, including vancomycin-resistant enterococci (VRE) and extended-spectrum β-lactamase (ESBL)-producing enterobacteriaceae.