Field accuracy of fourth-generation rapid diagnostic tests for acute HIV-1: a systematic review.

INTRODUCTION: Fourth-generation HIV-1 rapid diagnostic tests (RDTs) detect HIV-1 p24 antigen to screen for acute HIV-1. However, diagnostic accuracy during clinical use may be suboptimal.
METHODS: Clinical sensitivity and specificity of fourth-generation RDTs for acute HIV-1 were collated from field evaluation studies in adults identified by a systematic literature search.
RESULTS: Four studies with 17 381 participants from Australia, Swaziland, the United Kingdom and Malawi were identified. All reported 0% sensitivity of the HIV-1 p24 component for acute HIV-1 diagnosis; 26 acute infections were missed. Specificity ranged from 98.3 to 99.9%.
CONCLUSION: Fourth-generation RDTs are currently unsuitable for the detection of acute HIV-1.

Introduction

An estimated 5–20% of HIV infections are due to transmission from recently infected individuals [1], although this depends on epidemic characteristics [2,3]. Diagnosing HIV in the weeks after an individual has acquired infection is likely to be critical to epidemic control as prompt diagnosis allows early entry into care and treatment, reducing the risk of onward transmission. Laboratory diagnosis relies on the appearance of HIV antigen (e.g. HIV-1 p24 antigen) or nucleic acid in the bloodstream prior to detectable antibodies to HIV-1/2, but testing procedures can be burdensome. Easy-to-use and accurate in-vitro diagnostics for detection of acute HIV infection that can be used at point of care are required to realize the potential public health benefits of early diagnosis, especially in resource-limited settings wherein the large majority of HIV transmissions occur.

New fourth-generation point-of-care rapid diagnostic tests (RDTs) for HIV-1 detect HIV-1 p24 antigenaemia as well as antibodies to HIV-1/2. They have the potential to meet these requirements, and have been approved for their sale in many settings. However, the sensitivity of the p24 antigen component has been variable (2–92%) in laboratory evaluations on banked panels of specimens [4-9] leading to concerns that this may translate into poor diagnostic accuracy in the field for the diagnosis of acute HIV-1 [10]. We therefore performed a systematic review to assess the field accuracy of fourth-generation RDTs for the diagnosis of acute HIV-1 in clinical practice.

Methods

In accordance with our published protocol [11], we systematically searched Medline (using PubMed) and ISI Web of Science for studies published between 1 January 2005 and 22 April 2015 that evaluated the diagnostic accuracy of fourth-generation RDTs for acute HIV-1 (Supplementary Table 1: Search Strategy). Studies were eligible if they evaluated the field performance of at least one fourth-generation RDT for acute HIV-1 in adults (aged >15 years). We defined field performance as the RDT being used on an unselected population, at the point of care (defined as being conducted in the presence of the patient [12]), in the setting of intended use and carried out by the intended user. Cross-sectional studies, cohort studies and randomized control trials were included. Studies that evaluated fourth-generation RDTs within laboratories or retrospectively against specimen panels were excluded. We placed no restrictions on assay manufacturer, language or country.

Two reviewers (P.M., E.A.) independently reviewed the titles and abstracts of all studies against inclusion and exclusion criteria, with discrepancies resolved by group decision with a third reviewer (M.T.). Two reviewers (J.L., P.M.) independently reviewed the full text of all selected studies using a piloted data extraction form to determine final selection status. Discrepancies between reviewers were resolved by group decision with a third reviewer (M.T.). Participants receiving the index test were defined as true positive, false positive, false negative and true negative for the presence of a positive HIV-1 p24 antigen reaction on the fourth-generation HIV RDT compared with the study-defined reference standard for acute HIV-1 infection. We recognized that there is no current consensus reference standard for the diagnosis of acute HIV infection and specified that an acceptable reference standard for the diagnosis of acute HIV-1 infection would include at least one nonreactive antibody test result using a third-generation HIV immunoassay, and at least one reactive antigen test result on a fourth-generation HIV immunoassay, and/or at least one reactive result using an HIV-1 p24 antigen immunoassay. However, where this was not used, we used study-reported reference standards. Details of studies excluded after full-text review are shown in the supplementary appendix (Supplementary Table 2) [5-9,13-37].

Risk of bias in the included studies was independently assessed by two reviewers (J.L. and P.M.) using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS 2) tool [38] with signalling questions used across four domains (selection, index test conduct, reference standard conduct and participant flow and timing) to assess overall risk of bias.

Study flow and results are reported in accordance with the Standards for Reporting of Diagnostic Accuracy (STARD) checklist [39]. Given the high degree of heterogeneity in study populations and conduct, we did not perform meta-analysis, but would have done so if results had indicated this to be appropriate.

Results

Of 2056 unique studies identified, four met inclusion criteria and contributed data relating to 17 381 participants to this analysis (Fig. 1). Studies came from Swaziland [40], Malawi [41], Australia [42] and the United Kingdom [43] (Table 1). Three studies included community members [41], MSM [42] or IDUs [43] attending specialist sexually transmitted diseases or HIV testing clinics, whereas in the fourth, participants were recruited from a national HIV incidence survey [40]. All had a cross-sectional study design.

Fig. 1

Flowchart of included and excluded studies.

Table 1

Characteristics of included studies investigating diagnostic accuracy of fourth-generation HIV rapid diagnostic tests.

Study	Countries	Study population	Proportion men	Reference standard for diagnosis	True positive	False positive	False negative	True negative	Sensitivity (95% CI)	Specificity (95% CI)
Conway et al., 2014 [42]	Australia	Adult (≥18 years old) MSM attending STD clinics in four suburbs	100%	Algorithm comprising of:	0	6	3	3181	0.00b	0.998b
				Fourth-generation immunoassay (Architect HIV Ag/Ab Combo or Elecsys HIV Combi PT) – All participants
				Supplementary antibody immunoassay (Serodia HIV-1 Antibody particle agglutination assay or Genscreen HIV 1&2 Antibody EIA) – If index test antigen or antibody reactive
				HIV p24 antigen immunoassay (Genscreen HIV-1 p24 Antigen) – If index test antigen or antibody reactive
				HIV Western blot (HIV Western Blot 2.2, MP Diagnostics) – If index test antigen or antibody reactive
Duong et al., 2014 [40]	Swaziland	Adult (18–49 years old) community members participating in a national HIV incidence survey	39%	Algorithm comprising of:	0	12	13	12 345	0.00 (0–0.23)	0.999 (0.998–0.999)
				Third-generation RDT (Uni-Gold Recombigen) – Index test antibody reactive
				Fourth-generation RDT (Determine HIV-1/2 Ag/Ab Combo) – Index test antigen reactive
				HIV viral load quantification (COBAS AmpliPrep/COBAS TaqMan) – Index test antigen reactive
				Pooled (with deconstruction and individual retesting if detected) HIV viral load quantification (COBAS AmpliPrep/COBAS TaqMan) – Index test antigen and antibody nonreactive
Jones et al., 2012 [43]	United Kingdom	Adult (≥18 years old) MSM, IDUs, individuals from high-prevalence settings and at-risk partners attending STD clinic in London	87%	Algorithm comprising of:	0	3	2	980	0.00b	0.997b
				Third-generation RDT (Determine HIV-1/2) – All participants
				Fourth-generation immunoassay (Architect HIV Ag/Ab Combo) – All participants
				Fourth-generation immunoassay (VIDAS platform, assay not specified) – Index test reactive
Rosenberg et al., 2012 [41]	Malawi	Adults (18–49 years old) attending hospital STDa and HIV testing clinics in Lilongwe	STD: 65%	Third-generation RDT (Uni-Gold Recombigen, SD Bioline HIV 1/2 3.0) – All participants	0	14	8	824	0.00 (0.00–0.37)	0.983 (0.972–0.991)
			HTC: 52%	HIV viral load quantification (Roche Monitor) – Participants with nonreactive or discrepant HIV antibody RDT results

CI, confidence interval; HTS, HIV testing services; NR, not reported; PWID, people who inject drugs; RDT, rapid diagnostic test; STIs, sexually transmitted infections. True positive/false positive/true negative/false negative all refer to p24 antigen strip of index test.

aOnly patients with negative or discrepant HIV RDTs results at high risk for acute HIV infection attending STD clinic were invited to participate.

b95% CIs not reported.

All four studies evaluated the diagnostic accuracy of Alere Determine HIV-1/2 Ag/Ab Combo (Alere Medical Co. Ltd., Chiba, Japan), hereafter described as the index test. This is a fourth-generation RDT that can detect antibody to HIV-1/2 and HIV-1 p24 antigen. Only two studies used reference standards as defined in our study protocol [42,43], with the other two studies variously using laboratory-performed testing algorithms for characterization of acute HIV comprising of a third-generation RDT, a fourth-generation RDT and HIV quantitative nucleic acid testing (NAT) technology (HIV viral load) for participants with a reactive HIV-1 p24 antigen index test result [40], and a third-generation RDT (all participants) and HIV viral load quantification (participants with nonreactive or discrepant antibody index test results) [41].

In all four studies, sensitivity of the HIV-1 p24 antigen component of the index test for reference standard-classified acute HIV infection was 0%, with a total of 26 acute HIV infections missed (Table 1). Specificity of HIV-1 p24 antigen component of the index test for acute HIV infection was similar across all studies (98.3–99.9%); a total of 35 false-positive results were observed. It was possible to extract data on the accuracy of the antibody component of the index test from two of the four studies; in the first, the antibody component detected zero of three (0%) acute HIV infections [42]; in the second, two of eight (25%) [41].

The risk of bias in these studies as assessed across the four assessed domains was largely low or unclear. Only in one domain of one study was there a high risk of bias: in patient selection; in this study, a nonconsecutive, nonrandom cohort of patients were enrolled – only patients who were assessed as high risk for acute HIV infection [41]. The full assessment of risk of bias is shown in the supplementary appendix (Supplementary Table 3).

Discussion

This systematic review reveals unacceptably low-field sensitivity in the diagnosis of acute HIV-1 of the p24 antigen component of Determine HIV 1/2 Ag/Ab Combo (Alere Medical Co. Ltd, Chiba, Japan) – the only product evaluated in the included studies – with not a single true positive for acute infection detected in over 17 000 individuals from at-risk groups in sub-Saharan Africa, Australia and the United Kingdom. Across these studies, 26 reference-standard-defined acute HIV infections were missed, and 35 false positives were observed. On the basis of these findings, we recommend that Determine HIV 1/2 Ag/Ab Combo (Alere Medical Co. Ltd) is not currently used for screening for acute HIV-1 infection at the point of care.

There are a number of likely reasons for the poor test performance. The assay detects free HIV-1 p24 antigen; this has a short window of detection, before developing antibodies from immune complexes with antigen in the bloodstream and preclude binding of antigen to the test strip. This technical difficulty can be overcome with an antigen dissociation step [44], but this may be hard to implement at the point of care. Differential affinity of binding to different HIV subtypes could also contribute to poor detection of HIV-1 p24 antigen.

We excluded a number of studies as they did not meet our inclusion criteria. The majority were not field evaluations but rather evaluations in reference laboratory settings on specimen panels. Of those studies that did perform a field evaluation, but were excluded from this review, the reference and/or index test(s) were often inconsistently applied across all patients making evaluation of true sensitivity and specificity impossible. This likely represents the inherent difficulty in designing a study to assess an in-vitro diagnostic for acute HIV infection, which is rare even in areas of high HIV prevalence and has a short timeframe for detection.

The lack of a consistent application of a clinical definition of acute HIV-1 infection makes it challenging to compare results across studies and compare diagnostic accuracy. The included studies defined acute HIV-1 infection using a variety of biomarkers and clinical parameters. In laboratory terms, acute HIV-1 infection has been clearly defined in terms of the Fiebig stages of acute HIV-1 infection, characterized by sequential appearance of HIV-1 nucleic acid, HIV-1 p24 antigen followed by immunoglobulin M and then immunoglobulin G antibodies to HIV-1 [45]. These stages were originally defined using second and third-generation immunoassays and attempts have been made to redefine these stages in terms of more modern third and fourth-generation assays [46]. Regardless of the existence of these clear laboratory definitions, we found that clinical definitions of acute HIV-1 are inconsistently applied across studies. Internationally agreed and applied consensus definitions of acute HIV infection for use in clinical studies would go some considerable way to clarifying a confusing field.

Despite a comprehensive literature search and rigorous analysis, there are some limitations to our study: data may be reported as supplementary material, and have been missed by our search; only adults were included. Owing to the small number of studies identified, and high degree of heterogeneity in study settings and populations, it was inappropriate to undertake meta-analysis, meaning that overall summary estimates for diagnostic accuracy cannot be made. All studies assessed the accuracy of only one fourth-generation RDT; others are available [e.g. SD Bioline HIV Ag/Ab (Standard Diagnostics, Inc., Giheung-gu, Republic of Korea) and ImmunoComb II HIV 1&2 TriSpot Ag-Ab (Orgenics Ltd, Yavne, Israel)]. The generalizability of our results are unclear but it seems likely that the same technical difficulties will affect other fourth-generation RDTs [47], and clinicians should interpret the results of HIV-1 RDT antigen testing with caution.

In conclusion, the fourth-generation RDT for HIV that was the subject of this systematic review has low sensitivity for detection of acute HIV-1, and is unsuitable for this purpose. Point-of-care options for diagnosis of acute HIV-1 are therefore currently limited to third-generation RDTs alongside careful assessment of timing of exposure and serial testing, with a continuing need for laboratory support for antigen testing or NAT wherein the suspicion of acute HIV infection is high. New approaches to diagnosis of acute HIV such as point-of-care testing with third-generation RDT exhibiting best possible seroconversion sensitivity, or NAT – including the prospect of expansion of qualitative NAT platforms for use at point of care that have been developed for early infant diagnosis – need evaluating in the field against agreed definitions of acute HIV in order to realize the public health benefits of making this important diagnosis.

Acknowledgements

The authors gratefully acknowledge the assistance of Indo Sachiko for translation of one manuscript from Japanese to English. Joseph Lewis is financially supported by the Wellcome Trust as a Wellcome Trust funded clinical PhD fellow (grant number 109105/Z/15/Z).

Author contributions: P.M., E.A., E.O., A.S. and M.T. conceived the idea and developed the protocol. J.L., P.M., E.A. and M.T. performed the systematic review and data extraction. J.L. wrote the manuscript. All authors redrafted and approved the final manuscript.

Sources of funding: none.

Conflicts of interest

There are no conflicts of interest.