Performance of the Bio-Rad Geenius HIV1/2 Supplemental Assay in Detecting "Recent" HIV Infection and Calculating Population Incidence.

OBJECTIVE: HIV seroconversion biomarkers are being used in cross-sectional studies for HIV incidence estimation. Bio-Rad Geenius HIV-1/2 Supplemental Assay is an immunochromatographic single-use assay that measures antibodies (Ab) against multiple HIV-1/2 antigens. The objective of this study was to determine whether the Geenius assay could additionally be used for recency estimation.
DESIGN: This assay was developed for HIV-1/2 confirmation; however, quantitative data acquired give information on increasing concentration and diversity of antibody responses over time during seroconversion. A quantitative threshold of recent HIV infection was proposed to determine "recent" or "nonrecent" HIV infection; performance using this cutoff was evaluated.
METHODS: We tested 2500 highly characterized specimens from research subjects in the United States, Brazil, and Africa with well-defined durations of HIV infection. Regression and frequency estimation were used to estimate assay properties relevant to HIV incidence measurement: mean duration of recent infection (MDRI), false-recent rate, and assay reproducibility and robustness.
RESULTS: Using the manufacturer's proposed cutoff index of 1.5 to identify "recent" infection, the assay has an estimated false-recent rate of 4.1% (95% CI: 2.2 to 7.0) and MDRI of 179 days (155 to 201) in specimens from treatment-naive subjects, presenting performance challenges similar to other incidence assays. Lower index cutoffs associated with lower MDRI gave a lower rate of false-recent results.
CONCLUSIONS: These data suggest that with additional interpretive analysis of the band intensities using an algorithm and cutoff, the Geenius HIV-1/2 Supplemental Assay can be used to identify recent HIV infection in addition to confirming the presence of HIV-1 and HIV-2 antibodies.

INTRODUCTION

Guidelines for HIV testing in most countries, including the United States,[1] recommend using an algorithm where specimens that are reactive on a sensitive screening test [preferably a fourth generation antibody (Ab)/antigen (Ag) test] are retested to confirm the presence of HIV antibodies. This supplemental testing has historically been performed using a Western blot or immunofluorescence assay; however, newer US and European guidelines require using an HIV-1/HIV-2 discriminatory assay for this confirmatory testing.[2,3] Previously, only the Bio-Rad Multispot rapid test had been approved for both confirmation and HIV-1/2 differentiation. Subsequently, another confirmatory assay that can differentiate between HIV-1 and HIV-2 infection has been introduced, the Bio-Rad Geenius HIV-1/2 Supplemental Assay test (Geenius).[4] Its performance in confirmatory testing has been compared to Multispot and Innolia assays,[5,6] and it is approved for use in both the United States (FDA) and Europe (European Community CE marked).

The Geenius is a dual path lateral flow–based unitary assay test that measures IgG antibody responses by individual bands to 4 HIV-1 antigens (gp41, gp160, p31, and p24) and 2 HIV-2 antigens (gp36 and gp140), as previously described.[7,8] Antibodies specific for the bound antigens remain after eluting unbound antibodies and pink/purple bands occur where bound antibodies are identified by colloidal gold-linked protein A reagents. The assay is read by the Geenius reader, an automated optical reader that can produce quantitative measurements of antigen band intensities. For confirmation of HIV infection status, results are read in the Geenius reader: the software compiles and analyzes information on the intensity of each band and produces a final diagnostic test result. Typically, the band intensity data are not released to the user; however, the software contains information on both antibody specificity and band intensity. It has been hypothesized that data on band intensities obtained from standard Geenius supplemental testing could also be interpreted to distinguish between recent and long-standing HIV infections.

Tests for recent HIV infection are very important for HIV surveillance programs. They are used widely in cross-sectional population surveys to calculate population incidence rates and to monitor the efficacy of HIV prevention programs[9-13] and interventions.[14] In the United States and Europe,[15,16] extensive re-testing of samples from newly diagnosed patients is performed as part of enhanced case-based surveillance.[17] If a routinely used HIV diagnostic assay could perform the dual function of testing for recent HIV infection, this could have broad implications for HIV care and for surveillance of the HIV epidemic.

The Consortium for the Evaluation and Performance of HIV Incidence Assays (CEPHIA) developed a 2500-member specimen panel to facilitate comparative performance evaluations of tests for recent HIV infection. This panel has been used to independently evaluate the performances of several existing candidate assays for recent HIV infection, using a general framework for the application of these assays to infer incidence from cross-sectional surveys.[18] In this study, we report the first such analysis of the Geenius assay as a test for recent HIV infection using the CEPHIA “Evaluation Panel.” Using this panel, the properties of the assay most relevant for incidence inference are estimated for a number of chosen subpopulations and at various assay recency index cutoffs. In the present analysis, we also evaluated additional panels of CEPHIA samples to investigate the reproducibility of measurements as well the sensitivity of measurements to variations in testing procedures.

METHODS

CEPHIA Specimen Repository and the Evaluation Panel

CEPHIA has retrospectively collected specimens to facilitate comparative evaluation of tests to identify recent HIV infection, intended for use in incidence surveillance.[18] Specimens and data have been contributed by participating clinical research cohorts including the UCSF Options and SCOPE cohort studies, San Francisco Men's Health Study, the UCSD Acute HIV Infection Study, AMPLIAR cohort, and IAVI Protocol C, according to the Declaration of Helsinki. The UCSF Committee on Human Subjects Research (CHR #10-02365) approved study procedures. A 2500-plasma specimen “Evaluation Panel” was designed for the full assessment of promising tests in identifying recent HIV infection. As previously described, longitudinal specimens were collected from 928 subjects (2–13 specimens per subject, median of 3 specimens per subject). The time of follow-up after the estimated date of HIV infection (discussed below) ranged from 1 week to more than 10 years, with a median follow-up of 3 years (described in Table 1).

TABLE 1.

Estimated Test Properties (and 95% CIs) for Various Specimen Sets and Different Thresholds Cutoff

Laboratory Procedures

Testing was performed independently in a CEPHIA laboratory (Blood Systems Research Institute) by technicians trained by the test developer and blinded to specimen background information. After calibrating the reader, Geenius quality positive and negative controls were tested at the beginning of each testing day. As previously described, the assay was performed using a calibrated pipettor and disposable pipette tips to add 5 μL of plasma to the sample well in the Geenius cassette. Next, 3 drops of assay buffer were added to the sample well, and after 5 minutes, 5 drops of the assay buffer were added to the buffer well in the Geenius cassette. After an incubation of 15–20 minutes, all test cassettes were read by the automated reader. Photographs of the Geenius cassette and HIV band interpretation results were recorded using the Geenius reader; and these results were transferred to Bio-Rad where information on band intensities was extracted. The product as used here is still investigational, and the assay is not currently approved for use as described.

Interpretation of the Assay Results

The Geenius reader measures the intensity of bands specific for antibodies to 4 HIV-1 antigens (gp41, gp160, p31, and p24), 2 HIV-2 antigens (gp36 and gp140), and a protein A total IgG-binding control band. A “Geenius Index” was developed by Bio-Rad based on results from testing a smaller 250-member CEPHIA “Qualification Panel” of specimens. The index incorporates information from 3 of the HIV-1 bands that appeared to evolve most consistently during seroconversion and is defined as the sum of the band intensities of gp41, gp160, and p31 bands, divided by the intensity of the control band. A test result below the index cutoff of 1.5 proposed by the test developer is interpreted as indicating “recent” HIV infection.

Estimation of Test Properties for Discrimination of Recent Infections and Estimation of Incidence

The software used by CEPHIA to store and analyze the data, the stratification of the data into specimen sets, and the methods used for estimating test properties were all previously described.[18] Test properties were evaluated in each of a number of specimen sets, created by stratifying on treatment history, viral load, CD4 cell count, time from infection to specimen draw, and HIV subtype (based on country when unknown).[18] The Evaluation Panel was purposefully enriched with specimens from subjects with risk factors for “false-recent” misclassification—ie, specimens from individuals under antiretroviral treatment (ART) and specimens from elite controllers (who suppress viremia in the absence of treatment) that were specifically sought from the SCOPE cohort. To avoid biasing results, these specimens were analyzed separately in main analyses.

The following 2 test properties are of relevance for incidence estimation[18]:

The mean duration of recent infection (MDRI), which is the average time that a subject is classified as “recently” infected, while infected for less than some time cutoff T; and

The false-recent rate (FRR), which is the probability that a subject, who is infected for longer T, will produce a “recent” result.

The consistent and general definition of the MDRI and FRR rely on the use of the postinfection time cutoff, T, which is set at T = 2 years for this analysis.[18]

In practice, the notion of “infection” depends on the particular HIV diagnostic test used in the incidence study and refers to “detectable infection.” In this analysis, “infection” was defined as infection that is detectable using an HIV viral lysate–based Western blot assay. The methodology used to estimate subjects' infection times (time of seroconversion on Western blot) from their testing histories has been described.[18] Estimated infection dates were calculated for subjects with a documented history of a negative HIV diagnostic test within 120 days of their first positive HIV test, using average durations of Fiebig stages[19,20] to estimate times at which patients seroconverted on a Western blot.[19,20] Since publishing earlier CEPHIA analyses, more complete testing history data have been retrieved, leading to some refinements in estimated infection times for particular subjects. Subjects without complete testing histories were not included in this analysis unless they were known to be “long-standing” because of the specimen draw date being more than 2 years from a documented HIV-positive test result or entry into a research cohort as a person known to have HIV.

The MDRI was estimated using linear binomial regression for the probability of testing “recent.” Bootstrapping (by resampling subjects) was used to obtain 95% confidence intervals (CIs). Three different parametric forms and 2 rules for including data were implemented.[18] In the results presented below, the regression model used a logit link function and a cubic polynomial of (estimated) time since infection as the predictor, and all data points within time 1.1 × T of infection were used in the model fitting. The sensitivity of results to changes in parametric form and when including more data (up to 2 × T) was investigated.

The FRR was approximated by the proportion of “recent” subjects among those subjects infected for longer than T, using the most frequent classification per subject and 95% CIs are provided (exact Clopper–Pearson intervals). The cutoff used to distinguish between “recent” and “nonrecent” results was varied from the proposed Geenius Index cutoff of 1.5, and results are presented for index cutoff values ranging from 0.5 to 1.75.

Evaluation of Assay Reproducibility

Within the Evaluation Panel, each of 3 blinded controls appeared as 25 uniquely labeled specimens. These were included to determine reproducibility of test results at high, medium, and low levels of antibody response (Specimens A, B, and C, respectively). In addition, repeat testing of 4 labeled control specimens (Specimens D–G) was regularly performed (6–10 repeats). Reproducibility of test results were measured including calculating the coefficient of variation (ratio of standard deviation to mean) of test results per specimen.

Sensitivity of Quantitative Assay Results to Variation in Procedures (The “Guard Band” Study)

The “Guard Band” (or robustness) study was designed to determine the sensitivity of results to variations in sample and sample buffer volumes. First, the sample volumes were varied from 2.5 and 10 μL from the recommended 5 μL, to explore the impact of sample volume on band intensities. Second, the volume of assay buffer added to well 1 was varied from 2 or 4 drops, from the recommended 3 drops of buffer to explore the impact on antibody binding and therefore band intensities. Each of 4 control specimens used in the study, chosen based on different band intensities on the 4 HIV antigens, were tested 3 times under each condition. Test results were analyzed using multiple linear regression: the mean Geenius Index was determined for each sample, sample volume, and buffer volume (no interactive conditions were tested).

RESULTS

Assay Dynamics

Figures 1 and 2 describe the Geenius assay characteristics when excluding treated subjects and SCOPE elite controllers, for individual band intensities and the overall Geenius Index, respectively. The evolution of individual band intensities over time since infection (Fig. 1A) demonstrate that although the individual HIV Ag band intensities are in different ranges of values, all band intensities increase rapidly after infection. When comparing the results of different bands on the same group of specimens (Fig. 1B) the measurements for gp41 and gp160 bands are strongly correlated, although gp41 band shows a relatively faster progression of band intensity after infection. The p31 band results may be negative, and when positive may range over a larger range of band intensities.

FIGURE 1.

Antigen-specific band intensities. A, Box-and-whisker plots of band intensities, as a function of time since infection (in 6-month intervals), for p31, gp160, and gp41. B, Pairwise scatter plots of band intensities for p31, gp160, and gp41. Excludes treated subjects and SCOPE elite controllers.

FIGURE 2.

Incidence assay results over time since infection. A, Box-and-whisker plots of the Geenius result, as a function of time since infection. Results are summarized for each 6-month interval after infection, and results for all specimens drawn more than 2 years after infection are captured in the rightmost box plot. B, Proportion of “recent” results (using a cutoff of 1.5) as a function of time since infection (with a 95% CI), stratified by HIV subtype (B, C, A1, and D). Excludes treated subjects and SCOPE elite controllers.

When evaluating the Geenius Index, an increase in band intensities over time to above the index cutoff of 1.5 is apparent (Fig. 2A), although about half of the specimens drawn within the first 6 months of infection provided “nonrecent” results at this cutoff, and there were some “recent” results obtained for specimens drawn more than T = 2 years after infection. The evolution of band intensities over time potentially varies by HIV subtype (Fig. 2B), and subtype D specimens may return “recent” results for longer periods.

Test Properties

Table 1 shows the estimated Geenius Index properties for all incident HIV-positive subjects originally screened by HIV viral lysate–Western blot. Using the proposed HIV recency index cutoff of 1.5 to discriminate between “recent” and “nonrecent” results, the estimated MDRI (excluding specimens from treated subjects and SCOPE elite controllers) is 179 days (95% CI: 155 to 204). When the cutoff is lowered to 1.25, the MDRI decreases considerably to 109 days (95% CI: 88 to 131), and when the cutoff is increased to 1.75, the MDRI increases considerably to 325 days (95% CI: 297 to 353). In sensitivity analyses, these MDRI estimates change by up to 8% when changing the parametric form of the model or the data inclusion rules. At the lower cutoffs, there are little data to inform the model fitting and results rely heavily on estimated infection times. When stratifying by subtype, MDRI point estimates changed by up to 15% when varying the model or data inclusion rules.

The overall FRR (excluding specimens from treated subjects and SCOPE elite controllers) was 4.1% (95% CI: 2.2 to 7.0) for a cutoff of 1.5. This decreases to 2.5% (95% CI: 1.1 to 5.0) when the cutoff is lowered to 1.25 and increases to 14.6% (95% CI: 10.9% to 19.1%) when the cutoff is raised to 1.75. Including only specimens with detectable viral loads (≥75 copies per milliliter), the FRR remains high at 5.9% (95% CI: 3.0 to 10.1) at the 1.5 cutoff. Higher FRRs of larger than 30% were observed in elite controllers and treated subjects, and even higher FRRs of above 50% in subjects with undetectable viral loads and who received early ART.

Reproducibility

Figure 3 presents the variability of repeat measurements for each of the blinded controls (A, B, and C) and the labeled controls (D, E, F, and G) for each band used to calculate the Geenius Index. The coefficient of variation of the Geenius Index ranges from 5% to 13%.

FIGURE 3.

Reproducibility of assay results. Box-and-whisker plots of the 25 repeat measurements for the 3 blinded controls (A–C) and 6–10 repeat measurements for the 4 labeled controls (D–G), for each band used as well as the Geenius index result.

Guard Band or Robustness Study

Figure 4 presents the mean Geenius Index result for the various sample and sample buffer volumes tested separately with each of the 4 specimens. The regression analysis results indicate an impact of sample and buffer volume changes on band intensities. When the sample volume is decreased from 5 to 2.5 μL, the mean index value decreased by 0.26 (95%CI: −0.52 to 0.00; P-value: 0.05), and if increased to 10 μL, the mean index value increases by 0.22 (95% CI: −0.04 to 0.48; P-value: 0.10). When the number of drops of sample buffer is lowered from 3 to 2, there is no change in the mean index (95% CI: −0.26 to 0.26; P-value: 1.00), but if raised to 4 drops, the mean index value increases by 0.77 (95% CI: 0.51 to 1.03; P-value: <0.001).

FIGURE 4.

Sensitivity of assay results to variations in testing procedures. For each of the 4 specimens (distinguished by marker shape), the individual measurements are shown by markers, and mean measurements connected by dashed lines. The input plasma volume is varied (left plot) or number of drops of buffer is varied (right plot).

DISCUSSION

Tests for recent HIV infection are widely used by national-level surveillance programs and for epidemiological research. The relevance of such testing to more local public health and surveillance practice has been limited by the complexity of assays used (requiring batch testing of large numbers of samples in well-resourced, central laboratories). In this study, we determined that the Bio-Rad Geenius, a new single-use assay with fast turn-around time approved in the United States and Europe for HIV antibody confirmation and for HIV-1/HIV-2 differentiation can also function as a test for recent HIV infection. Using the measured “band intensities” that quantify antibody responses to specific HIV antigens, a composite biomarker for “recent” HIV infection can be constructed, which increases over time after seroconversion. Using the proposed index cutoff of 1.5 to distinguish “recent” from “nonrecent” infection, subjects remain “recent” for about half a year. The corresponding FRR is large at about 4% (excluding treated subjects and identified elite controllers). This FRR will likely preclude the use of the Geenius as a stand-alone assay for cross-sectional incidence estimations. Using a lower index cutoff of 1.25 will reduce the FRR to 2.5% but also decreases the MDRI to 109 days.

The application of a point of care test that can simultaneously act as a confirmatory HIV diagnostic assay and a test for recent infection is highly novel.[21] Other diagnostic tests detect early infection but do not help distinguish those with ongoing seroconversion from those with established infection.[22] A diagnostic test providing this information could improve clinical decision making, help target public health interventions, and enhance case-based surveillance. Clinically, immediate and rapid initiation of ART (eg, before genotype testing) in the first 3 months of HIV infection can lead to better treatment outcomes including lower HIV reservoirs,[23] better immune control of the virus,[24] and less systemic inflammation.[25] Early ART can also quickly reduce viral load and secondary transmission[26] when infectivity is greatest.[27-29] Public health interventions that can be effectively targeted to acute/recent infection cases[30] include contact tracing and partner services such as HIV testing and antiretroviral PrEP for partners at high risk of acquiring or continuing to transmit HIV.[31,32] Finally, having information on recent infection status from routine diagnostic test results would provide a new metric of early case detection for HIV testing programs and case-based surveillance.[33] Because all of these potential use, cases may apply to a different period of acute or recent infection (eg, 1, 3, or 6 months after seroconversion); additional studies will be needed to determine best practices for using recent infection tests for individual disease staging.

For purposes of incidence estimation in population surveys, we recommend that the Geenius assay might be used in combination with other HIV recency tests (eg, to provide one result within a multiassay algorithm). Rapid testing with minimal training requirements is important in testing facilities that are not located in a clinical laboratory. This would allow testing at disseminated testing sites and easy transfer to field sites in settings where clinical laboratories or other research infrastructure are unavailable. However, the results of our guard band studies also introduce an important additional caution. We recommend calibrated precision pipettes or validated plastic microtube pipettes when using results of a Geenius assay for early infection staging.

During our evaluation, it was possible to run between 90 and 120 samples on the Geenius system in an 8-hour period, so throughput is reasonable but test kit costs limits the application of the assay to additional interpretation of recency status when the test is already being run as an HIV confirmatory assay. It is especially important to have the option of using these assays in determining whether the individual has been infected recently in clinical settings close to patients, to avoid loss to follow-up. Although the Geenius assay does not meet the criteria for an optimal incidence assay as identified by the Target Product Profile outlined in our previous publications (FRR <1% with an MDRI of 1 year),[34] the fast turn-around time of the Geenius assay, its easy transferability with minimal laboratory requirements, and its use as a confirmatory test for HIV and surveillance provides much more than currently used incidence assays, and with access to the antibody band intensity data, it could be an important new tool in identifying recent infections at the point of care. The results of this study suggest that, if routine use of the Geenius or similar confirmatory tests expands, it is feasible to incorporate testing for recent HIV infection into clinical and public health practice on an unprecedented scale. This raises important opportunities and new challenges for public health and clinical implementation science.