Performance evaluation of three rapid antigen tests for the diagnosis of group A Streptococci.

OBJECTIVE: To compare the diagnostic performance of three rapid antigen detection tests (RADTs) for group A Streptococcus (GAS).
DESIGN: A hospital-based, cross-sectional, retrospective study.
SETTING: A comparative study of rapid diagnostic tests for GAS using clinical specimens in a single institute. PARTICIPANTS: 225 children in the outpatient clinics ofKorea University Guro Hospitalwith suspicious symptoms were subjected to throat swab sampling. A dual-swab applicator was used. Samples were stored at below -70°C in a 10 mL transport tube containing 1 mL liquid Stuart's transport medium. OUTCOME MEASURES: All tests were performed in the laboratory by trained clinical laboratory scientists. Sensitivity, specificity, accuracy and kappa index of three RADTs were compared with the reference PCR test and culture results.
RESULTS: Of the 225 patients suspected of having GAS, 67 and 90 were positive for GAS in the culture and PCR tests, respectively. Compared with the reference culture, the sensitivity for GAS was 92.5% (CI 83.4 to 97.5), 71.6% (CI 59.3 to 81.9) and 74.63% (CI 62.5 to 84.4) for careUS Strep A Plus, SD Bioline and BD Veritor, respectively, and the specificity was 97.0% (CI 93.1 to 99.0), 94.6% (CI 90.1 to 97.5) and 92.9% (CI 87.8 to 96.2) for careUS Strep A Plus, SD Bioline and BD Veritor, respectively. Compared with the reference GAS real-time PCR, the sensitivity was 73.3% (CI 62.9 to 82.1), 63.3% (CI 52.5 to 73.2) and 67.8% (CI 57.1 to 77.2) for careUS Strep A Plus, SD Bioline and BD Veritor, respectively, and the specificity was 99.3% (CI 95.9 to 99.9), 100.0% (CI 97.3 to 100.0) and 99.3% (CI 95.9 to 99.9) for careUS Strep A Plus, SD Bioline and BD Veritor, respectively.
CONCLUSIONS: The careUS Strep A Plus is a useful test that showed highly comparable results with those of the culture test and superior performances among the three RADTs. The use of RADTs should be encouraged to provide acceptable and fast results using simple equipment. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study compared the diagnostic performance of three rapid antigen detection tests (RADTs) with two references, the group A Streptococcus streptococcal pyrogenic exotoxin B (speB) gene PCR and culture methods.

The careUS Strep A Plus is a useful test that is highly comparable with the ‘reference standard’ test.

We used samples stored below −70°C in a 10 mL transport tube containing 1 mL liquid Stuart’s transport medium to evaluate RADTs.

Introduction

Group A Streptococcus (GAS) is the most common bacterial aetiology of pharyngitis, with a prevalence of 5%–15% in adults, while approximately a quarter of cases in children are due to GAS.1 Suppurative and non-suppurative complications such as retropharyngeal abscess, acute rheumatic fever, rheumatic heart disease and poststreptococcal glomerular nephritis2–5 can occur in patients who do not receive timely treatment with antimicrobial agents. The burden of invasive GAS diseases is unexpectedly high, with at least 663 000 new cases and 163 000 deaths yearly.6 Therefore, early antimicrobial treatment of bacterial pharyngitis can be beneficial in preventing the sequelae and diminishing medical costs. However, it is difficult to distinguish bacterial and viral pharyngitis since no symptoms or signs have been shown to have a sufficiently high likelihood ratio to permit an accurate diagnosis of GAS pharyngitis.7 Using antimicrobials to treat viral pharyngitis blindly is ineffective and contributes to the growing problem of antimicrobial resistance.8–10 Thus, correct identification of GAS enables detection of GAS-positive cases that could lead to complications, and the correct exclusion of GAS prevents unnecessary use of antibiotics.11

The Infectious Diseases Society of America recommends rapid antigen detection test (RADT), throat swab culture or both in patients with pharyngitis, except for those with overt viral symptoms. Negative RADT results should be confirmed by a throat culture in children, and positive RADTs do not require backup culture tests due to its high specificity.12 Commercially available RADTs have sensitivity and specificity of 77%–98.9% and 62%–100%, respectively, compared with the throat swab culture method.13 The differences between various RADTs could be attributed to the skill level of individuals performing the tests, relatively subjective interpretation of RADT end point, different commercial kits and the quality of culture. Given the importance of RADT in diagnosing acute pharyngitis, more accurate performance evaluation of RADTs is required.

Therefore, the objective of this study was to evaluate the performance of three RADTs—careUS Strep A Plus (Wells Bio, Korea), SD Bioline Strep A (Yongin, Korea) and BD Veritor system (Becton, Dickinson and Company, Sparks, Maryland)—to provide more accurate information about RADTs in the detection of GAS in suspected cases of bacterial pharyngitis.

Methods

Study design

From September to November 2015, 225 children aged 4–17 years suspected of having streptococcal pharyngitis (defined as the presence of a painful throat and evidence of inflammation of the throat or tonsils on physical examination) were recruited for potential enrolment. Those who presented with symptom onset more than 7 days previously or who presented signs of viral respiratory infection were subsequently excluded from this group. The throat swabs were collected by two physicians using a dual-swab applicator (Copan Diagnostics, California), submitted for routine testing in the outpatient clinics of Korea University Guro Hospital and tested in a culture study.

After rubbing the pharyngeal mucosa with the dual-swab applicator, the sample was immediately transported to the microbiology department. Each applicator was stored in a 10 mL transport tube containing 1 mL liquid Stuart’s transport medium. The bacterial culture was performed immediately. Within 7 days after storage below −70°C, aliquoted specimens were tested using the three RADTs (careUS Strep A Plus, SD Bioline Strep A and BD Veritor system) according to package insert instructions and PCR by trained clinical laboratory scientists who were blinded to the culture result. In addition, another aliquot from a second applicator was stored as a backup at below −70°C (figure 1).

Figure 1

Flow diagram of specimen collection and group A Streptococcus (GAS) rapid antigen detection test (RADT).

All RADT results were compared with the culture test and streptococcal pyrogenic exotoxin B (speB) real-time PCR assay as the reference assay.

Throat swab culture

Dual throat swabs were taken simultaneously from all patients with clinical pharyngitis using rayon-tipped swabs. The first swab was taken to the laboratory, streaked onto a 5% sheep blood agar dish and incubated in an atmosphere of 5% CO2 at 35°C for 24–48 hours. After overnight incubation, the plate was examined to detect the presence of beta-haemolytic colonies. GAS was identified using the bioMérieux Vitek MS V.2.0 system (bioMérieux, France) according to the manufacturer’s instructions.

speB gene PCR assay

To confirm the presence of GAS, we performed speB gene real-time PCR using forward (f, 5ʹ-CTAAACCCTTCAGCTCTTGGTACTG-3ʹ) and reverse (r, 5ʹ-TTGATGCCTACAACAGCACTTTG-3ʹ) primers and a probe (Cy3-CGGCGCAGGCGGCTTCAAC-BHQ2), which has shown excellent sensitivity and specificity against GAS in a previous report.14 Primer and dual-labelled probe sequences for the speB were synthesised by Macrogen (Seoul, Korea). DNA was extracted from 200 µL clinical samples using proteinase K enzymatic digestion and DNA isolation using the QIAamp DNA Mini Kit (Qiagen, France). Briefly, 5 µL DNA extract was mixed with 12.5 µL iQ Multiplex Powermix (Bio-Rad, USA); 1 µL each of 10 µM f-primer, r-primer and probe; and 4.5 µL distilled water. The speB real-time PCR was performed using a CFX96 real-time PCR detection system (Bio-Rad) with an initial activation of 95°C for 3 min followed by 35 cycles at 95°C for 20 s and 60°C for 20 s.

careUS Strep A Plus

Strep A Plus is an in vitro rapid chromatographic immunoassay for the qualitative detection of GAS antigens directly extracted from throat swab specimens of symptomatic patients. This test involves the chemical extraction of Strep A antigens followed by solid-phase immunoassay for the detection of extracted antigens. In this test, anti-Strep A antibody is printed on the test line region of the test strip. After the throat swab specimen is collected, Strep A antigens are extracted for 1 min from the specimen using extraction reagents. The sample is then dispensed directly onto the sample well of the cassette. During testing, the antigens extracted from the throat swab specimen react with anti-Strep A antibodies conjugated with coloured nanobeads. The complex migrates through the membrane to bind with the anti-Strep A antibodies on the membrane and produce a red line in the test region. The presence of two coloured lines, one in the control region and the other in the test region, indicates a positive result. The absence of the test line indicates a negative result. If the control line does not appear, the test result is not valid. All tests were repeated by careUS Strep A Plus analyser, Lite-G (complementary metal-oxide semiconductor (CMOS) camera).

SD Bioline Strep A

SD Bioline Strep A strip test is also a chromatographic, solid-phase immunoassay for the qualitative detection of GAS antigens. The test can be performed directly using throat swabs. Using goat and rabbit anti-Strep A antibodies, the reactions can be observed as a purple-coloured line of the antibody–antigen–antibody complex formation. The results can be read 5 min after the start of the test, and the negative reaction takes a minimum of 5–10 min to complete by visual reading only.

BD Veritor system

BD Veritor system is a qualitative lateral flow chromatographic immunoassay for qualitative detection of GAS antigens. In this test, antibodies specific for the Strep A antigen are coated on the testing line. During the test, the treated throat swab specimens react with antibodies against Strep A antigen bound to the particles in the detector. The mixture moves to the membrane and is captured by the antibodies lined on the membrane. Positive results are determined using an automated reader when the antigen–antibody complex is precipitated at the test (T) line and control (C) line. The test takes approximately 10 min to set up and run. The results can be interpreted by visual reading, but can also be interpreted using an optical reader in ambiguous cases.

Statistical analysis

The number of subjects was calculated using the sample size calculation formula provided in the reference based on throat swab culture.15 Sensitivity, specificity, positive predictive value and negative predictive value were analysed using the GAS Reverse Transcription Polymerase Chain Reaction (RT-PCR) and throat swab culture results as standards. The kappa index was used to determine the level of agreement between the RADTs and GAS RT-PCR or culture results. The CIs for sensitivity, specificity, accuracy and kappa index were calculated using the Clopper-Pearson method. The CIs for the kappa index and the p value were calculated using QuickCalcs (GraphPad Software). Fisher’s exact test with p<0.05 was considered statistically significant.

Patient and public involvement

No patients or members of the public were involved in the design of this study.

Results

A total of 225 throat swab specimens were collected and tested using speB gene real-time PCR assay as reference methods. The median age of patients was 9.6 (range, 4–17) years, and the percentage of male and female patients was 54.2% (122/225) and 45.8% (103/225), respectively. The most common symptoms were fever (95.2%), hyperaemia (92.7%), oedema (66.5%), pain and enlargement of the gland (38.4%), and exudate (34.2%) in GAS-positive cases. These five symptoms rated higher in RADT positive cases than in negative cases.

Of the 225 tested specimens, 90 and 135 were positive and negative, respectively, for GAS based on the reference PCR test. In the test of throat swab cultures, 67 and 168 were positive and negative, respectively. All specimens were simultaneously tested using the three RADTs, and their characteristics and the results are summarised in tables 1–3.

Compared with the culture tests, careUS Strep A Plus showed the highest sensitivity at 92.5% (CI 83.4 to 97.5). In contrast, SD Bioline and BD Veritor revealed lower sensitivities at 71.6% (CI 59.3 to 81.9) and 74.6% (CI 62.5 to 84.4), respectively. careUS Strep A Plus showed the best specificity at 97.0% (CI 93.1 to 99.0), and the other two RADTs also showed good specificity (SD, 94.6% (CI 90.1 to 97.5); BD, 92.9% (CI 87.8 to 96.2)). When overall accuracy was calculated, careUS showed the highest accuracy at 95.7% (CI 92.3 to 97.9). The kappa index was used to determine the level of agreement between the RADTs and the GAS culture results.

careUS showed high degree of agreement with a kappa index of 0.896. The kappa indices of SD Bioline and BD Veritor were 0.694 and 0.690, respectively. All Fisher’s exact test results were statistically significant (p<0.0001).

Among the three RADTs compared with speB gene real-time PCR assay, careUS Strep A Plus showed the highest sensitivity at 73.3% (CI 62.9 to 82.1). On the other hand, SD Bioline and BD Veritor revealed much lower sensitivities at 63.3% (CI 52.5 to 73.2) and 67.8% (CI 57.1 to 77.2), respectively. The SD Bioline test showed the highest specificity at 100% (CI 97.3 to 100.0), while the other tests also showed good specificity (careUS Strep A Plus, 99.3% (CI 95.9 to 99.9); BD, 99.3% (CI 95.9 to 99.9)). When overall accuracy was calculated, careUS showed the highest accuracy at 88.9% (CI 84.1 to 92.6). careUS showed a high degree of agreement with a kappa index of 0.758, while those of the SD Bioline and BD Veritor were 0.675 and 0.707, respectively, which are considered to be at a ‘good’ level. All results of the Fisher’s exact test were statistically significant between careUS and SD Bioline, careUS and BD Veritor, and SD Bioline and BD Veritor (p<0.001).

Overall, careUS showed good performance compared with the reference GAS culture and PCR tests.

Discussion

It is difficult to distinguish the causative pathogens of viral and GAS-induced pharyngitis solely based on clinical symptoms, and therefore it is necessary to perform an RADT or throat culture. Throat culture is considered to be the standard reference method, but culture tests require microbiological facilities with expertise, as well as a long wait time of approximately 1–2 days. It is not practical to perform culture tests in a private clinic, and it is also difficult for a patient to revisit the hospital to obtain culture results. Since acute pharyngitis is mainly due to viral infections, prescribing antibiotics without proper testing leads to unnecessary medical costs and abuse of antibiotics. Furthermore, in contrast to the USA, in Korea antibiotics can be prescribed empirically without the results of RADT or culture.

To reduce the complications associated with untreated GAS pharyngitis, there is a tendency to prescribe antibiotics to almost all patients with acute pharyngitis.16 As a result, abuse of antibiotics for upper respiratory tract infections in Korea is serious, and a high frequency of resistance to erythromycin in GAS has been reported.17 Consequently, the use of RADT could be beneficial in reducing antibiotic resistance and properly managing patients. Facilitated by simple devices or analysers, RADTs allow easy interpretation of results in approximately 10–15 min in small hospitals or private clinics. The diagnostic accuracy of GAS RADTs has also been widely studied and has been provided for attending doctors.18–22 The performance of SD Bioline Strep A and BD Veritor system was also reported to be comparable with that of GAS culture.16 21–23

To the best of our knowledge, this is the first study of careUS Strep A Plus as a method to detect Streptococcus from throat swabs in children with acute pharyngitis. Unfortunately, our study showed low sensitivities for SD Bioline Strep A compared with that obtained in a previous Korean study reported in 2009, which showed a sensitivity of 95.9%.16 However, the result was higher than that reported by a research group in India, which showed sensitivity and specificity of 55% and 100%, respectively.21 The sensitivity of BD Veritor compared with the GAS culture was similar to that reported in previous studies.23 Among the three RADTs, careUS Strep A Plus showed the highest sensitivity, specificity, accuracy and kappa index compared with culture and PCR test.

careUS does not need a specific analyser to read the results according to the manufacturer’s instruction, which is advantageous compared with BD Veritor. Additionally, the repeated results of careUS using the Lite-G CMOS camera analyser were similar to those of the visual readings. This suggests that the result is accurate and without ambiguity. Compared with SD Bioline Strep A, which can be visually read, careUS revealed excellent performance characteristics in terms of sensitivity, accuracy, negative predictive value and kappa index. It was originally expected that the results of BD with nanoparticle technology would be better than those using existing conventional methods. However, the results of the study differed from this expectation, and the careUS Strep A Plus test showed good sensitivity and specificity among the three RADTs. The nanoparticle used in careUS is NanoAct, comprising coloured cellulose nanoparticles which have a larger surface than those of other labels such as colloidal gold, coloured latex and fluorescent latex. NanoAct has higher visibility than the other nanoparticles and can be easily detected under visible light conditions and multicolour labelling. In addition, NanoAct has a 10-fold higher sensitivity than that of the other particles and there are no requirements for specific instruments.24 This difference in reagent performance may be the reason for our unexpected test results, including the complete agreement between the visual and analyser reader. Since negative RADT results do not preclude GAS infection as shown in our studies, confirmation using culture is recommended due to its high specificity. On the other hand, positive RADTs do not require backup culture,12 which may be advantageous for the use of Strep A antigen RADTs with high specificity.

The limitations of our study were that clinical data, including antibiotic utilisation, were not available since patient information was not provided. Samples from patients who administered antibiotics prior to the culture test may produce false negative culture results (or similar). When symptoms such as sore throat or cough caused by GAS occur, most Korean patients visit private clinics or small hospitals rather than tertiary or university hospitals. It is logistically difficult for small clinics or hospitals to have all the equipment or trained personnel for appropriate culture or PCR testing. Currently, most patients with pharyngitis are prescribed antibiotics without appropriate culture, PCR or susceptibility results. This practice leads to increased rates of antibiotic resistance and the emergence of multidrug-resistant pathogens. Both physicians and patients should be aware of the serious problems associated with the unnecessary use of antibiotics, and efforts should be made to minimise this practice. GAS RADT, which was in considerable agreement with the culture test, would be useful in hospital laboratories and point-of-care testing in clinics.

In conclusion, careUS Strep A Plus is a useful test that is highly comparable with the ‘reference standard’ test. Among the three RADTs evaluated in this study, careUS Strep A Plus showed good performance in terms of sensitivity, specificity, accuracy and agreement with the culture and PCR test results. It would be expedient to encourage the increased use of RADTs to obtain acceptable and fast results using simple equipment. Increasing the use of RADTs could redefine current strategies in GAS pharyngitis treatment in the medical field and would be beneficial to patients with pharyngitis.

Table 1

Characteristics of each rapid diagnostic test for group A Streptococcus detection

Rapid test kits	Approximate assay time (min)	Recommended specimen	Interpretation
careUS Strep A Plus	10	Throat swab	Eye (±CMOS camera)
SD Bioline Strep A	10	Throat swab	Eye
BD Veritor system	10	Throat swab (posterior pharynx)	Optical reader

CMOS, complementary metal-oxide semiconductor.

Table 2

Positive and negative results (n) compared with reference PCR test and culture results of three rapid antigen detection tests

		Culture results		PCR results
		GAS+ (67)	GAS− (168)	GAS+ (90)	GAS− (135)
careUS + visual reading	Positive	62	5	66	1
	Negative	5	163	24	134
careUS + CMOS camera	Positive	62	5	66	1
	Negative	5	163	24	134
SD Bioline	Positive	48	9	57	0
	Negative	19	159	33	135
BD	Positive	50	12	61	1
	Negative	17	156	29	134

CMOS, complementary metal-oxide semiconductor; GAS, group A Streptococcus.

Table 3

Sensitivity, specificity, accuracy and kappa index analysis of three rapid antigen detection tests compared with reference PCR test and culture results

	careUS		SD Bioline		BD Veritor
	Culture reference	PCR reference	Culture reference	PCR reference	Culture reference	PCR reference
Sensitivity, % (95% CI)	92.5 (83.4 to 97.5)	73.3 (62.9 to 82.1)	71.6 (59.3 to 81.9)	63.3 (52.5 to 73.2)	74.6 (62.5 to 84.4)	67.8 (57.1 to 77.2)
Specificity, % (95% CI)	97.0 (93.1 to 99.0)	99.3 (95.9 to 99.9)	94.6 (90.1 to 97.5)	100 (97.3 to 100.0)	92.9 (87.8 to 96.2)	99.3 (95.9 to 99.9)
Accuracy, % (95% CI)	95.7 (92.3 to 97.9)	88.9 (84.1 to 92.6)	88.1 (83.2 to 91.9)	85.3 (80.0 to 89.6)	87.7 (82.7 to 1.5)	86.7 (81.5 to 90.8)
PPV (95% CI)	92.5 (83.9 to 96.7)	98.5 (90.3 to 99.8)	84.2 (73.5 to 91.1)	100	80.7 (70.36 to 87.9)	98.4 (89.5 to 99.7)
NPV (95% CI)	97.0 (93.3 to 98.7)	84.8 (79.8 to 88.7)	89.3 (85.1 to 92.4)	80.4 (75.7 to 84.2)	90.2 (85.8 to 93.2)	82.2 (77.3 to 86.1)
Kappa index (95% CI)	0.896 (0.83 to 0.95)	0.758 (0.67 to 0.84)	0.694 (0.58 to 0.79)	0.675 (0.57 to 0.77)	0.690 (0.58 to 0.79)	0.707 (0.61 to 0.80)
Kappa index p value	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001

NPV, negative predictive value; PPV, positive predictive value.