Comparison of Rapid Point-of-Care Tests for Detection of Antibodies to Hepatitis C Virus.

Background.  Hepatitis C is one of the most prevalent blood-borne diseases in the United States. Despite the benefits of early screening, among 3.2 million Americans who are infected with hepatitis C virus (HCV), 50%-70% are unaware of their infection status. Methods.  Data were collected between 2011 and 2014, from 1048 clients who were in the following groups: (1) injection drug users, (2) women at sexual risk, (3) gay and bisexual men, and (4) transgender individuals. The sensitivity and specificity of point-of-care tests included (1) the MedMira rapid human immunodeficiency virus (HIV)/HCV antibody test, (2) MedMira hepatitis B (HBV)/HIV/HCV antibody test, (3) Chembio HCV Screen Assay used with both whole blood and (4) oral specimens, (5) Chembio HIV-HCV Assay also used with both whole blood and (6) oral specimens, (7) Chembio HIV-HCV-Syphilis Assay, and (8) OraSure HCV Rapid Antibody Test used with whole blood. The gold standard for the HCV tests were HCV enzyme immunoassay (EIA) 2.0. Results.  OraSure had the highest sensitivity at 92.7% (95% confidence interval [CI] = 88.8%-96.5%) followed closely by Chembio's 3 blood tests at 92.1% (95% CI = 87.7%-96.4%), 91.5% (95% CI = 87.2%-95.7%), and 92.3% (95% CI = 88.4%-96.2%). The sensitivities of MedMira HIV/HCV and MedMira HIV/HCV/HBV tests were the lowest, at 79.1% (95% CI = 72.6%-85.5%), and 81.5% (95% CI = 75.2%-87.8%), respectively. Specificity for the OraSure was 99.8% (95% CI = 99.4%-100%); specificity for the Chembio blood tests was 99.2% (95% CI = 98.6%-99.9%), 99.4% (95% CI = 98.8%-99.9%), and 99.3% (95% CI = 98.8%-99.9%); and specificity for the MedMira was100% and 100%. False-negative results were associated with HIV and hepatitis B core antibody serostatus. Conclusions.  The OraSure and Chembio blood tests (including those multiplexed with HIV and syphilis) appear to good performance characteristics. This study has identified potential limitations of rapid testing in those testing positive for HIV and HBcAb. There should be discussion of updates to the 2013 CDC guidance.

Hepatitis C is one of the most prevalent and deadly blood-borne diseases in the United States [1]. Prompt treatment, adequate follow-up, birth cohort, and risk factor-guided screening programs recommended by the US preventative task force in 2012 were reported to be cost-effective interventions [2, 3]. However, despite the benefits of early screening, among 3.2 million Americans who are chronically infected with hepatitis C virus (HCV), 50%–70% are unaware of their infection status [4-6]. A high prevalence of HCV was reported among individuals with identifiable risk such as injecting drug use (IDU) [7-9], incarceration history [10, 11], and individuals with human immunodeficiency virus (HIV) [12-14] or blood transfusion-related risk [15, 16]. Screening and follow up of HCV treatment were reported to be low among IDUs, with only 53% of HCV-positive patients following up for their results, and only 20% of those received antiviral treatment [17]. A study of IDUs in Long Beach, California found that only 42% of them followed up for further diagnosis or treatment [18].

Current serological testing includes enzyme immunoassays (EIAs), RNA detection assays, as well as rapid diagnostic testing and point-of-care (POC) testing. In the context of the current study, “rapid test” will be used synonymously with “POC test.” Point-of-care testing can provide an important contribution to current clinical practice if there is satisfactory performance of the POC testing compared with the “gold standard” conventional laboratory testing [19]. If it can be demonstrated that sensitivity and specificity are acceptable, then most clinicians may prefer the advantages of POC testing.

One advantage of POC testing is to minimize turnaround time, which allows for making rapid clinical decisions [20]. The use of POC HIV testing has shown that patients are more likely to return on time for hepatitis and sexually transmitted infection test results [21]. In addition, the POC tests may alleviate the problem of underdiagnosing HCV by increasing the availability of screening [22]. There are many times when patients, who may be homeless or indigent, are lost to follow up because the patient did not have a method of communicating with the healthcare provider, which then further exacerbates the risk for delayed treatment [23-25].

Currently, there are several rapid tests being manufactured, and they all use immunochromatography to show the presence of antihepatitis C antibodies in the test fluid. A recent meta-analysis found that POC testing of blood (serum, plasma, or whole blood) has the highest accuracy, followed by Rapid Diagnostic Tests of serum or plasma, with POC tests of oral fluids being lower [19]. The role of POC testing using the OraQuick HCV Rapid Antibody Test (OraSure Technologies, Inc.) and linkage to care has been demonstrated in 1 pilot study in Colorado, which found that patients preferred to receive their test results along with counseling in the same visit, although some barriers to treatment were loss to follow up and access to care [26].

Point-of-care HCV testing may have advantages over gold standard testing if it can reach more of the at-risk populations and identify more cases [27]. The OraQuick HCV Rapid Antibody Test is currently the only rapid HCV test approved by the US Food and Drug Administration for patients at risk or with symptoms [28]. In a comparative evaluation of rapid test devices used for prescreening blood donation, the OraQuick had the highest sensitivity and exceeded all other tests being compared [29]. However, a European study found that the OraQuick had “limited specificity” at 88%, leaving some questions to be answered about clinical application [30].

The sensitivity of rapid tests have been variable, with decreased sensitivity demonstrated in HIV-seropositive individuals and oral rapid tests [4, 31–33]. One study reported a significant association of MedMira false results with gender (MedMira Laboratories, Inc.) [34]. Other studies found that multiplex testing successfully confirmed hepatitis B virus (HBV) DNA detection without compromising HIV or HCV RNA detection [35].

In 2009, a Federal Register Notice called for an “Opportunity to collaborate in the evaluation of rapid diagnostic tests for HIV and HCV” to which 3 manufacturers (Chembio Diagnostic Systems, Inc., MedMira Laboratories, Inc., and OraSure Technologies, Inc.) responded [4] by providing the Centers for Disease Control and Prevention (CDC) with rapid screening assays. The CDC tested 1100 specimens with the assays in a laboratory setting [4]. In addition, the National HIV Behavioral Surveillance System tested 1592 specimens (490 from New York, 389 from Denver, 265 from Seattle, and 448 from Dallas) [32]. The Study to Assess Hepatitis C Risk tested 409 specimens with assays provided by Chembio and MedMira with all specimens being from San Diego, California [34]. The purpose of the current study was to conduct a head-to-head evaluation of the performance of several experimental POC tests for HCV from Chembio, MedMira, and Orasure. This study included not only the singleplex HCV tests, but also in combination with other tests on the same test platform, in a large (N = 1028) high-risk population in Long Beach, California. In addition, the study investigated whether the false-positive/ false-negative results were associated with factors such HIV and HBV positivity status, gender, ethnicity, and age.

METHODS

Data for this study were collected from 26 May 2011 to 28 April 2014. The participants were recruited at the Center for Behavioral Research and Services (CBRS), which is an off-campus research center of the California State University, Long Beach (CSULB). The CBRS provides free HIV and sexually transmitted disease testing to the community as well as conducts research. Eligible clients were 15 years of age and older, had not participated previously, and reported being in a behavioral risk group. Behavioral risk groups were defined as follows: (1) injection drug users (IDUs) with verified track marks (ie, visible signs of injection) [36]; (2) women who reported at least 2 male partners in the last 2 years or engaging in anal intercourse, sex trading, or sex with a man who has sex with men (MSM), an IDU, or an HIV-positive man; (3) MSM and men who have sex with men and women (MSMW); and (4) transgender individuals. These definitions of the risk groups were based on guidelines from the Los Angeles County Department of Public Health (LACDPH). Clients were not excluded based on prior infection history. When an eligible client agreed to participate, they gave written informed consent under a protocol approved by the CSULB, Institutional Review Board (IRB), and a California State licensed phlebotomist drew a venous blood sample by standard laboratory practices for the POC tests, as well as the gold-standard confirmatory tests. The IRB had approved having the client receive the results of the POC tests at the initial visit, and the IRB also approved allowing 15-year-old participants in the study, which is consistent with California State Law. Every test that had been provided by the manufacturers was completed on the whole blood specimen of each participant. However, there was variation in sample size by test because not all experimental test kits were available at all times. During the study session, the participant also completed the Risk Behavior Assessment [37, 38] to gather demographic data, behavioral data, and a questionnaire to obtain information about tattoo experience. Two weeks after the initial visit, the participant returned for the gold-standard results. This meant that the phlebotomists who were reading the POC results were blind to the gold-standard results. The POC tests included the following: (1) the MedMira rapid HIV/HCV antibody test; (2) MedMira HBV/HIV/HCV antibody test; (3) Chembio HCV Screen Assay, which is used with both whole blood and (4) oral specimens; (5) Chembio HIV-HCV Assay, which is also used with both whole blood and (6) oral specimens; (7) Chembio HIV-HCV-Syphilis Assay; and (8) OraSure HCV Rapid Antibody Test used with whole blood. The test kits were stored in a temperature-controlled setting, with the temperature being both monitored and recorded. The test procedures were based on the manufacturer's venipuncture whole-blood specimen instructions, and the phlebotomists were trained in person onsite in Long Beach by Chembio staff for the Chembio tests. The phlebotomists were trained (1) via videoconference by MedMira staff on the MedMira tests and (2) by LACDPH on the Orasure procedure.

The sensitivities and specificities of those 8 new experimental diagnostic tests were evaluated in comparison with the gold-standard, conventional laboratory testing. The gold standard for the HCV tests were HCV EIA 2.0 (Abbott Laboratories, Abbott Park, IL). The Clopper-Pearson method was used for the confidence intervals of the sensitivities and specificities [39]. Marginal regression models using Generalized Estimating Equations were conducted to determine whether false-negative results were associated with sample characteristics such as race (White, Black, Hispanic), HIV positivity status, HBV positivity status, and sex [40]. All analyses were performed with SAS software version 9.3 (Cary, NC).

RESULTS

Among the 1028 tested specimens with the gold standard, there were 197 (19%) that had a positive result for HCV. Only 12% of the sample had a college degree and 23% had a paid job. Most of the sample (66%) had been incarcerated and 42% were homeless. Almost one tenth of the sample had shared needles and 42% had a tattoo done unprofessionally. Table 1 shows that the sample was mostly male and of Black race/ethnicity. Most of the sample injected stimulants, with cocaine injected by 48% and amphetamines being injected by 40%. Six percent of the sample was HIV positive.

Table 1.

Selected Demographic Characteristics (N = 1048)

Characteristic	Proportion of Sample, %
Sex
Male	58
Female	42
Age (M SD)	39.3 (11.79)
Race/Ethnicity
Hispanic	22
Black, not Hispanic	50
White, not Hispanic	28
Sexual Preference
Heterosexual	43
Gay	22
Bisexual	31
Drugs injected (may have injected more than 1)
Cocaine	48
Amphetamines	40
Heroin	32
Other opiates	20
Speedball (cocaine and heroin)	18
Illicit methadone	9
Education
8th Grade or Less	3
Less than High School	25
GED (High School Equivalent)	8
High School Graduation	25
Trade/Technical	4
Some College	24
College Graduation	12
Homeless
No, Not Homeless	58
Yes, Homeless	42
Paid job, salary, or business
No	77
Yes	23
Needle Sharing
No times	91
Used needles	9
Unprofessional Tattoo
No tattoo	58
Yes tattoo	42
Ever Incarcerated
No	34
Yes	66
Hepatitis C Antibody
Negative	81
Positive	19
HIV test result
Negative	94
Positive	6
Hepatitis B Core Antibody
Negative	79
Positive	21

Abbreviations: HIV, human immunodeficiency virus; M, mean; SD, standard deviation.

Table 2 shows that of these 8 diagnostic tests, OraSure had the highest sensitivity at 92.7% followed closely by Chembio's 3 blood tests. The sensitivities of MedMira HIV/HCV and MedMira HIV/HCV/HBV multiplex rapid POC testing (HCV only being reported), compared with the gold standard, were the lowest, at 79.1% and 81.5%, respectively. However, the specificities of both of these diagnostic tests compared with the gold standard were 100%.

Table 2.

Performance Characteristics of Anti-HCV Point-of-Care Tests by Assay with Blood and Oral Specimens

	Sensitivity (95% CI)	Specificity (95% CI)	TP	FP	FN	TN
Blood
MedMira HIV/HCV	79.1% (72.6%–85.5%)	100%	121	0	32	718
MedMira HIV/HCV/HBV	81.5% (75.2%–87.8%)	100%	119	0	27	680
Chembio HCV	92.1% (87.7%–96.4%)	99.2% (98.6%–99.9%)	139	5	12	646
Chembio HIV/HCV	91.5% (87.2%–95.7%)	99.4% (98.8%–99.9%)	150	4	14	686
Chembio HIV/HCV/Syphilis	92.3% (88.4%–96.2%)	99.3% (98.8%–99.9%)	167	5	14	756
OraSure HCV	92.7% (88.8%–96.5%)	99.8% (99.4%–100%)	164	2	13	792
Oral
Chembio HCV	84.9% (78.9%–90.8%)	99.3% (98.6%–99.9%)	118	4	21	580
Chembio HIV/HCV	84.2% (78.4%–90.0%)	99.5% (98.9%–100%)	128	3	24	616

Abbreviations: CI, confidence interval; FN, false negative; FP, false positive; HBV, hepatitis B virus; HCV, hepatitis C virus; HIV, human immunodeficiency virus; TN, true negative; TP, true positive.

Table 3 shows that the false-negative results were significantly associated with HIV seropositivity for all test kits except the MedMira HIV/HCV/HBV. False-negative results on the MedMira HIV/HCV/HBV, the Chembio HIV/HCV Blood, and the OraSure HCV Blood were associated with hepatitis B core antibody (HBcAb). Results for the models with race and gender are not shown because none were significant. We do not present the associations with the false-positive results because there were so few false positives that the associations were either not significant or the models failed to converge.

Table 3.

Bivariate Associations with False-Negative Results*

Variable	Relative Risk	95% Confidence Interval	Wald χ2	P Value
MedMira HIV/HCV
HIV	3.59	2.04–6.34	19.61	.0001
HBcAb	0.55	0.28–1.05	3.29	.0696
MedMira HIV/HCV/HBV
HIV	2.12	0.89–5.02	2.92	.0875
HBcAb	0.40	0.18–0.85	5.65	.0175
Chembio HCV Blood
HIV	3.95	1.27–12.30	5.63	.0176
HBcAb	0.33	0.09–1.15	3.05	.0809
Chembio HCV Oral
HIV	3.16	1.36–7.32	7.18	.0074
HBcAb	0.57	0.26–1.26	1.91	.1671
Chembio HIV/HCV Blood
HIV	4.76	1.78–12.71	9.68	.0019
HBcAb	0.26	0.07–0.87	4.72	.0297
Chembio HIV/HCV Oral
HIV	2.65	1.13–6.22	5.05	.0246
HBcAb	0.62	0.29–1.27	1.70	.1922
Chembio HIV/HCV/Syphilis
HIV	4.47	1.63–12.22	8.5	.0036
HBcAb	0.54	0.19–1.52	1.37	.2415
OraSure HCV Blood
HIV	4.45	1.59–12.50	8.04	.0046
HBcAb	0.18	0.04–0.76	5.37	.0204

Abbreviations: HBcAB, hepatitits B core antibody; HCV, hepatitis C virus; HIV, human immunodeficiency virus.

* Findings significant at P < .05 are in bold.

DISCUSSION

This comparison of 8 POC rapid assays found that sensitivity ranged from 79.1% to 92.7% and specificity ranged from 99.2% to 100%. The Chembio HIV/HCV/Syphilis and the Orasure tests had substantially better sensitivity than both of the MedMira tests. The sensitivities of the oral tests were lower than the other Chembio tests done on blood and the Orasure done on blood, but the confidence intervals were wide. This finding is consistent with meta-analysis results [19]. The only tests to achieve 100% specificity were the 2 MedMira tests, but they did so at the cost of lower sensitivity.

Our HIV prevalence of 6% compares to similar results in other studies [32, 41]. False-negative results for all tests except the MedMira HIV/HCV/HBV were strongly associated with HIV seropositivity. This is consistent with other reports [4, 32].

A unique finding of this study is the inverse association we found between the HBcAb positivity and having a false result on the MedMira HIV/HCV/HBV, Chembio HIV/HCV Blood, and OraSure HCV Blood test kits. We were unable to find any other reports noting this association.

Several general observations can be made from these findings. Almost all of the false results were false negatives. The fact that the sensitivities of the POC tests were lower than the specificities may be explained by the limit of detection of the gold standard test being lower than the limits for the POC tests, which are too high. We would speculate that the reason for the associations with HIV is that the HCV titers were lowered just enough to be undetectable by some of the POC tests, but not low enough to be undetectable by the gold-standard test. Three mechanisms that could exacerbate this difference are as follows: (1) immunodeficiency, which is seen as the most common reason for false negatives [42]; (2) some of the participants may have been recently infected and still in a “window period” where the titers may not have peaked yet [43-45]; and (3) some of those samples may have had titers lowered because the individuals were in the process of resolving the infection [46].

The finding that HBcAb is inversely associated with false-negative results, that is, people who were HBcAb positive were less likely to have a false-negative result, may be counterintuitive. We speculate that, in this population, HBcAb may be a marker for having a competent immune system that is capable of producing antibody titers for HCV that are detectable by the POC tests. Not finding this effect for HBsAg complicates this speculation. This effect was found for only 3 of the test kits, so it needs to be replicated by other studies.

Limitations

There are several major limitations of this study. The first is that we were not able to obtain antibody titer levels, nor viral load values. These data would have been helpful in interpreting our results. The second major limitation is that our gold standard was only the HCV EIA 2.0. There was no reflex testing with a nuclei acid test as recommended by the CDC [47]. Had we done the reflex testing as recommended, it may have changed the results. This has been termed a “composite reference standard” [48]. This was considered and rejected as being cost-prohibitive. We were not able to assess feasibility, which includes the impact of POC tests on patient care, such as linkage to care, or placement on therapy.

CONCLUSIONS

Rapid testing has become an important tool for many clinicians in determining HCV status. Of the 8 diagnostic tests evaluated, OraQuick HCV had the highest sensitivity, which is consistent with previous studies. However, the Chembio blood tests were not substantially different from the Orasure. This opens up the possibility that the Chembio tests, if approved by the US Food and Drug Administration, could be used not only for initial screening, but also to differentiate resolved HCV infection from biologic false positivity as recommended in the 2013 guidance [49]. The guidance states “If testing is desired to distinguish between true positivity and biologic false positivity for HCV antibody, then, testing may be done with a second HCV antibody assay approved by FDA for diagnosis of HCV infection that is different from the assay used for initial antibody testing. HCV antibody assays vary according to their antigens, test platforms, and performance characteristics, so biologic false positivity is unlikely to be exhibited by more than one test when multiple tests are used on a single specimen” [49]. This research study has an important contribution to clinical medicine because it has identified potential limitations of rapid testing in those testing positive for HIV. Potential explanations for these findings are many, but they may suggest a reduction in antibody production that is manifested in this study by false negatives on the POC tests. The HBcAb finding may take further study to fully comprehend. Further research and discussion should take place to both replicate the current findings and to consider how this may impact recommendations for HCV testing in these populations.