The evaluation of low cut-off index values of Elecsys® HIV combi PT assay in predicting false-positive results.

OBJECTIVE: To analyze the results of different cut-off index (COI) values of Elecsys® HIV combi PT assay and to assess the role of COI in reducing the frequency of false-positive results.
METHODS: We conducted a retrospective study of samples analyzed by Elecsys® HIV combi PT assay, a 4th-generation ECLIA, between 2016 and 2017. A total amount of 379 122 samples were collected for HIV (Human Immunodeficiency Virus) screening.
RESULTS: A total of 379 122 samples were analyzed. 2528 (0.67%) were positive by Elecsys® HIV combi PT. Of these, 468 were false-positive results, and most of them (94.87%) were in samples with 1 < COI < 15. The false-positive rate was 0.12%. Patients with false-positive samples were more distributed in elder (P < .001) and female (P < .001) than true-positive specimens. The median COI in true-positive specimens was (385.20), which is significantly higher than false-positive specimens (2.08). The consistency between Elecsys® HIV combi PT assay and 3rd-generation and positive predictive value (PPV) increased with higher COI values. Cancer, infection, and neurological diseases were considered the potential confounding factors of HIV false-positive results (19.44%, 11.11%, and 6.62%, respectively).
CONCLUSION: Samples with low COI values, especially those contain confounding factors, need to be further scrutinized to determine whether the confounding factors may cause false-positive problem. In addition, the hypothesis that low COI values may predict false-positive results is valid.

INTRODUCTION

HIV/AIDS continues to pose a serious burden of morbidity and mortality globally. As reported by WHO, till 2018, there were more than 37.9 million people living with HIV and only 75% of them were aware of their infection status. The spread of HIV/AIDS is alarming to many countries, including China, where the HIV prevalence is <0.1%. In recent years, China has strengthened advocating to improve public awareness of HIV/AIDS. The implementation has gone all out to achieve the UNAIDS/WHO 90‐90‐90 target, namely 90% of infected people should know their status, 90% of diagnosed cases should be on antiretroviral medication, and 90% of those being treated should have fully suppressed viral load by 2020.

Novel 4th‐generation screening and confirmatory assays are now commercially available and have been incorporated into new diagnostic algorithms. , The 4th‐generation immunoassay can identify HIV‐1 p24 antigen, anti‐HIV IgM, and IgG antibodies simultaneously with sufficient sensitivity and specificity, thus narrowing the “window period” into approximately two weeks, and enabling the detection of acute and early HIV infection. , Due to the extremely high sensitivity of these techniques, the positive predictive value (PPV) in settings with a low prevalence of HIV may not be optimal, thus potentially leading to false‐positive results, arousing unnecessary concerns and interfering the progress of clinical diagnosis. , This study was carried out to analyze results for different COI values and the use of COI to distinguish between false‐positive and true‐positive results in a low‐prevalence setting through the presented hypothesis, we suggest that low COI values could predict false‐positive HIV results.

MATERIALS AND METHODS

Settings

We conducted a retrospective study in a large general teaching hospital with 4300 beds and a catchment population of approximately 16.33 million inhabitants in Sichuan, China. We included serum samples delivered to our laboratory, from inpatients and outpatients, for HIV testing between 2016 and 2017.

The screening tests

Samples were first analyzed using Elecsys® HIV combi PT assay, performed on the MODULAR ANALYTICS E170 or Cobas e 601 platform (Roche Diagnostics), following the manufacturer's instructions. Samples were considered as reactive (COI ≥ 1.0), borderline (0.9 ≤ COI < 1.0), or non‐reactive (COI < 0.9). All initially reactive or borderline samples were redetermined in duplicate with 3rd‐generation assay—colloidal gold method—Anti‐HIV (Livzon Diagnostics Inc) or 3rd‐enzyme‐linked immunosorbent assay (ELISA) which was run on the TECAN (freedom evolyzer, Switzerland) according to the national guide of China. Samples were considered repeatedly reactive (control line and test line both appear/ COI ≥ 0.9 in either of the determinations) and non‐reactive (only control line appear or COI < 0.9) by 3rd‐generation assay. These assays were all conducted following the manufactures’ instructions.

The supplemental tests

Repeatedly, reactive samples must be confirmed according to recommended confirmatory algorithms. Western Blot HIV Blot 2.2 (MP Diagnostic) detects IgG antibodies specific to viral antigens. On the basis of manufacturer's criteria, the results of individual specimens were reported as positive (the presence of at least two bands, including two env bands, or one env band plus p24 band), indeterminate (reactivity to any of the bands but not compatible with the criteria for a positive interpretation), or negative (the absence of any of the specific bands). As a negative result only means that no IgG antibody detected, patients with indeterminate or negative results required further tests or follow‐up. HIV p24 antigen and HIV‐1 viral load could both detect early infection and narrow the gap of window period. HIV p24 antigen quantitative test was performed by Elecsys HIV Ag assay (Roche Diagnostics) on MODULAR ANALYTICS E170 or Cobas e 601 platform. Samples were considered as reactive (COI ≥ 1.0), borderline (0.9 ≤ COI < 1.0), or non‐reactive (COI < 0.9). Cobas AmpliPrep/Cobas TaqMan HIV‐1 test (Roche Diagnostics) was applied for HIV‐1 RNA detection, with lower detection limit of 15 CPs/mL. Samples were classified into three groups (>5000 CPs/mL, ≤5000 CPs/mL, and below detection limits).

Samples were considered as confirmed positive if they were reactive by immunoblot, HIV‐RNA, or follow‐up. HIV p24 Ag was used to help determine the infection status.

Blocking test

With sufficient sample size, a total amount of 193 samples with initially and repeatedly reactive, uncertain, or negative in antibody confirmation tests stepped into blocking test. HIV Ag/Ab was rerun, before and after blocking the samples with 40 μL Heterophilic Blocking Reagent (HBR, 20 μg/mL) added to each sample and reacted for 1 hour under room temperature.

Statistical analysis

SPSS 23.0 software was used for statistical analysis, and OriginPro 2016 was used for plotting. Quantitative data were expressed as median and interquartile range (IQR). Comparisons between continuous variables were made using the t test or variance analysis, depending on the normality of the distribution. If not, the non‐parametric test was adopted for analysis. The data were counted by chi‐square test or Fisher's exact probability test. A P‐value less than or equal to .05 was considered significant.

RESULTS

The distribution of true‐positive and false‐positive specimens

A total of 379 122 samples were analyzed by Elecsys® HIV combi PT according to the recommended algorithm by Chinese Centers for Disease Control and Prevention (CDC), and the distribution of samples by results is shown in Figure 1. Briefly, 2528 samples (0.67%) were initially reactive by Elecsys®, and 1908 (75.47%) of the 2528 samples were confirmed reactive. After ruling out 893 samples that have been lost in follow‐up, samples selected for WB/RNA/follow‐up according to the algorithm. At last, 1167 samples with true‐positive results (including 153 diagnosed before) and 468 samples with false‐positive results were reported from 2528 initially reactive samples. It is worth noting that the false‐positive rate accounted for 0.12%.

FIGURE 1

The distributions of the results of the Elecsys® HIV combi PT assay

Compared features of true‐positive group, false‐positive group was prone to be older [52 (37‐65) vs 45 (31‐57), P < .001] [median (IQR)]. Notably, the proportion of male in false‐positive group was lower than that in true‐positive group [54.06% (253/468) vs 81.36% (825/1014), P < .001]. There was no significant difference in terms of ethnicity, educational level, marital status, and occupation.

COI analysis

The median COI value in true‐positive group was 385.20 (IQR:197.60‐769.60), while value in false‐positive group was 2.08 (IQR:1.36‐4.17), with statistically significant difference (P < .0001) (Figure 2).

FIGURE 2

The COI values of the true‐positive specimens and the false‐positive specimens

Nearly, all (94.87%) of the false‐positive results had a COI between 1 and 15, and 434 out of 468 false‐positive specimens were non‐reactive by 3rd‐generation assay. COI ranged from 15 to 50 in 22 false‐positive samples, and only two samples fluctuated at 50‐100 COI, with value of 57.84 as the peak in all false‐positive samples. The consistency and PPV showed different trends with varied COI values. When COI value climbed, the consistency and PPV increased simultaneously, while the false‐positive specimens decreased (Table 1).

TABLE 1

Consistency and PPV of different COI values

Screening tests (Elecsys® HIV Combi PT assay)		Retesting by 3rd‐generation			PPV (%)
COI Values	No.	Reactive (FP)	Non‐reactive (FP)	Consistency (%)
1~	425	5 (4)	420 (420)	1.18	0.24
15~	46	23 (14)	23 (12)	50.00	43.48
50~	91	86 (16)	5 (2)	94.51	80.2
100~	357	356 (0)	1 (0)	99.72	100
300~	432	431 (0)	1 (0)	99.77	100
800~	284	283 (0)	1 (0)	99.65	100
Total	1635	1184 (34)	451 (434)	72.42	71.38

Abbreviation: FP, false‐positive.

The sensitivity and specificity for Elecsys® HIV Combi PT assay were 100% (95% CI: 100‐100) and 99.88% (95% CI: 99.78‐99.98), respectively. ROC analysis showed the COI valued at 52.74 with 0.998 AUC (P < .0001). Correspondingly, the false‐positive rate declined to 0.01% and PPV rose to 99.92% (95% CI: 99.75‐100).

The clinical characteristics of false‐positive samples and blocking test

The three leading clinic diagnoses among false‐positive patients were cancer, infection, and neurological diseases, with the proportions of 19.44%, 11.11%, and 6.62%, respectively (Table 2). The 91 cancer samples consisted of adenocarcinoma, invasive cancer, tumor metastasis, and advanced tumor.

TABLE 2

Clinical characters of false‐positive specimens

Clinic diagnosis	No.	Proportion（%）
Cancer	91	19.44
Infection	52	11.11
Neurological diseases	31	6.62
Cardiovascular diseases	29	6.20
Just examination	26	5.56
physical examination	22	4.70
Orthopedic diseases	21	4.49
Ophthalmic diseases	20	4.27
Mental diseases	17	3.63
Preoperative examination	13	2.78
Lithiasis	12	2.56
Autoimmune diseases	11	2.35
Liver cirrhosis	11	2.35
Chronic obstructive pulmonary disease	10	2.14
Disease of digestive tract	9	1.92
Renal insufficiency	7	1.50
Others	86	18.38
Total	468	100

Ruling out 60 of 193 samples showed negative results already by retesting HIV Ag/Ab, so we blocked the remaining 133 samples then retested HIV Ag/Ab. The results indicated 16 out of 133 samples turned to be negative after blocking. The median COI of 16 samples changed from 1.46 (IQR:1.67‐5.97) to 0.82 (IQR:0.39‐0.89) with no significant difference (P > .05). Moreover, the clinical diagnosis of the 16 turning negative patients has no characteristic.

DISCUSSION

The 4th‐generation assay is capable for detecting both HIV‐1 p24 antigen and antibody simultaneously, which can narrow the “window period” into about two weeks, making it widely used as screening assay. However, Western blot used as supplemental assay can only detect the HIV‐1 antibody. West China Hospital uses the Elecsys® HIV combi PT assay for screening, and WB, HIV‐RNA, and follow‐up visit as confirmation. The HIV infection rate of the included participants in this study was 0.31% (1167/379 122), which indicated a relatively low HIV prevalence in Chengdu region.

The Elecsys® HIV combi PT assay, a 4th‐generation assay, is currently recognized and recommended as HIV screening test internationally. It has a special pre‐treatment of samples to release antigen and can evaluate the diagnostic performance. , According to our study, the specificity and sensitivity of the Elecsys® HIV combi PT were 99.88% and 100%. The false‐positive rate and positive predictive value were 0.12% and 71.38%. The results were similar to previous studies that the Elecsys® HIV combi PT performed well in specificity (>99.50%) and sensitivity (nearly 100%) (Table 3). Considering all these merits of the 4th‐generation assay, it is more appropriate for routine clinical screening and HIV early infection screening in high‐risk groups.

TABLE 3

Diagnose performance of Elecsys® HIV Combi PT assay of this study and different studies before

Authors	Sensitivity（%）	Specificity（%）	FP（%）	PPV（%）
This study	100 (100‐100)	99.88 (99.78‐99.98)	0.12	71.38 (70.68‐75.00)
Annette Blaich 8	NP	99.70（99.60‐99.90）	NP	71.80（57.70‐85.90）
Denise L.Uettwiller‐Geiger 14	100（99.75‐100）	99.94（99.85‐99.98）	NP	NP
Eun Young Song 15	100（98.20‐100）	99.50（99.00‐99.90）	0.49	NP
Yongming Liu 16	100 (100‐100)	99.93 (99.91‐99.95)	0.07	82.21 (78.54‐85.88)

Abbreviations: FP, false‐positive; NP, not provided; PPV, positive predictive value.

In this study, false‐positive specimens showed some similarities. The COI value was significantly lower in the false‐positive group than that in the true‐positive group. Nearly, all false‐positive samples were distributed at a low level of COI (1‐15), and a higher COI value accompanied by fewer false‐positive samples. Results also showed that most patients with false‐positive results were elders and females, which was consistent with previous studies. , Recent studies revealed that many factors, such as cancers, rheumatoid factors, autoimmune diseases, and pregnancy, might lead to poor specificity in HIV screening by the 4th‐generation assay and cause an indeterminate result for the WB test. , , , , , More elderly patients in the false‐positive group are more likely to suffer from underlying diseases. These diseases and some potential confounding substances in patients may blamed for some false‐positive results. Although follow‐up visit and use of HBR for blocking can eliminate interference to certain extent, proceeding HIV‐RNA test as soon as possible, the history of epidemiology and confounding factors should also be taken into consideration. Besides, it is of vital importance to enhance the communication between the laboratory and clinic.

Our study is subject to limitations. First, it is a retrospective study and some data were missing. But this will be perfected by our follow‐up study. Second, the data came from one hospital and it cannot be generalizable to the entire local population. Therefore, multi‐center clinical experiments should be conducted to further confirm the results.

In conclusion, samples with low COI values, especially those with confounding factors, need to be further censored to determine whether confounding factors could have an effect on the results or not. In addition, the hypothesis that low COI values may predict false‐positive results is valid.

ETHICAL CONSIDERATIONS

The study was conducted in full compliance with the principles of the Helsinki Declaration and local regulations. The study protocol was approved by ethics committee of the West China Hospital of Sichuan University. Exemption for obtaining informed consents from subjects was granted as a retrospective analysis of routinely collected programmatic data, and there was no direct contact with patients.