Sensitivity and specificity of rapid hepatitis C antibody assays in freshly collected whole blood, plasma and serum samples: A multicentre prospective study.

BACKGROUND: This study evaluated performance of two hepatitis C virus (HCV) rapid diagnostic tests (RDTs) performed by intended users in resource-limited settings.
METHODS: Testing was conducted at three facilities in two countries (Georgia, Cambodia) using matched fingerstick whole blood, plasma and serum samples. Investigational RDTs were compared with a composite reference standard (CRS) comprised of three laboratory tests, and a reference RDT.
RESULTS: In matched samples from 489 HCV positive and 967 HCV negative participants, specificity with both investigational RDTs was high using either reference method (≥98.4% in all sample types). Sensitivity was lower in whole blood versus plasma and serum for both RDTs compared with the CRS (86.5-91.4% vs 97.5-98.0% and 97.3-97.1%) and reference RDT (93.6-97.8% vs 100% and 99.4%). Sensitivity improved when considering only samples with detectable HCV viral load.
CONCLUSION: Sensitivity was highest in serum and plasma versus whole blood. The World Health Organization prequalification criterion (≥98%) was narrowly missed by both RDTs in serum, and one in plasma, possibly due to the intended user factor. Performance in whole blood was considered adequate, given potential roles of HCV infection history, improved sensitivity with detectable viral load and performance similarities to the reference RDT.

Introduction

World Health Organization (WHO) member states have committed to the elimination of viral hepatitis as a public health threat by 2030 [1]. Screening for hepatitis C virus (HCV), a pathogen that affects approximately 71 million people worldwide (2015 estimate), is critical to the success of these targets, especially as only an estimated 20% of infected people are aware of their HCV status [1]. According to WHO recommendations, screening should be performed through the detection of HCV-specific antibodies using a single quality-assured serological in vitro diagnostic test, which can be either a laboratory-based immunoassay or a rapid diagnostic test (RDT) [2]. A positive RDT test is followed by confirmatory testing for viraemic infection via detection of HCV viral load (VL) or core antigen [2].

Low- and middle-income countries (LMICs) have the highest burden of HCV, representing over 70% of the global total [3]. However, access to laboratory-based testing services in these settings is often limited by the absence of suitable equipment, stringent training requirements and sample or patient transportation challenges. RDTs, which can be used outside of the laboratory, are an attractive alternative due to their affordability, ease of use and feasibility of utilizing various sample types, including plasma, serum, fingerstick whole blood or oral fluid [2]. WHO prequalification status intents to indicate that an RDT is likely to have reliable performance in LMICs, as it requires the generation of performance data in LMICs in intended use settings by intended users, with at least a portion of these data generated using freshly collected samples [4]. However, of the many commercially available HCV RDTs, only four have obtained WHO prequalification status to date [5]. The scarcity of quality-assured RDTs is an important barrier to HCV screening in LMICs on a large scale [6].

A previous retrospective study evaluated the performance of 13 HCV RDTs in archived plasma samples [7]. In this study, the majority of RDTs exhibited performance in line with WHO criteria for selection of HCV diagnostics in samples from patients without human immunodeficiency virus [HIV] co-infection (sensitivity ≥98% and specificity of ≥97% in serum or plasma samples [8, 9]). Sensitivity was lower in samples from HIV infected participants compared with samples from HIV uninfected participants; interestingly, the majority of false negative HIV infected samples did not have detectable HCV VL/core antigen. However, the retrospective study was performed on archived samples by highly trained staff in evaluation laboratories, a setting that does not fully reflect the reality in which HCV RDTs are intended or likely to be used. In the field, HCV RDTs are most likely to be performed in primary care or screening facilities by staff with limited training, using whole blood by finger prick as the most common sample type. Data on RDT performance in whole blood is often limited or absent, particularly in comparison with matched samples of other types.

The objective of the current study was to evaluate the sensitivity and specificity of HCV RDTs in a real-world setting. Performance was assessed in fresh, matched whole blood, plasma and serum samples that were collected and tested in resource-limited settings by intended users, i.e. nurses and primary healthcare personnel.

Methods

Study design

This prospective, multicentre study (NCT04139941) assessed the performance of two HCV RDTs: the HCV-Ab Rapid test (Beijing Wantai Biological Pharmacy Enterprise Co., Ltd, Beijing, China) and the First Response HCV card test (Premier Medical Corporation Ltd., Mumbai, India). Operational characteristics of these tests are shown in S1 Table. These RDTs were selected as they met WHO prequalification criteria in archived plasma samples in the previous study [7], and the manufacturers had demonstrated a commitment to seeking WHO prequalification status.

Testing was conducted at three primary healthcare facilities in two countries. These were: a general outpatient clinic at the Sihanouk Hospital Center of Hope (SHCH), a non-governmental hospital providing low-cost medical care in Phnom Penh, Cambodia; an HCV screening facility at the National Center for Disease Control and Public Health (NCDC) in Tbilisi, Georgia; and an opioid substitution treatment facility at the Centre for Mental Health and Prevention of Addiction (CMHPA), also in Tbilisi, Georgia.

RDTs were tested on three sample types: fingerstick whole blood, ethylenediaminetetraacetic acid (EDTA) plasma, and serum (matched samples), all collected and tested on the same day. Performance was compared with three WHO prequalified laboratory reference tests, of which two were enzyme immunoassays (EIAs; Murex Anti-HCV version 4.0, Fujirebio INNOTEST HCV Ab IV) and one was a line immunoassay (LIA; Fujirebio INNO-LIA HCV Score), using a previously described composite reference standard (CRS) that incorporated the results of all three reference tests [7]. The algorithm was based on WHO prequalification evaluation protocols, with the final decision being based on the LIA test result. A signal-to-cut-off ratio of ≥1 (based on the measured optical density) was used for the EIAs; interpretation of LIA results was performed according to manufacturer instructions. Performance of the two investigational RDTs was also compared with a reference RDT, the WHO prequalified SD Bioline HCV test (Abbott Laboratories, Lake Bluff, USA; operational characteristics shown in S1 Table).

Reference testing was conducted at diagnostic reference laboratories (R. Lugar Center for Public Health Research, Tbilisi, Georgia and Biobykhin Medical Analysis Laboratory, Phnom Penh, Cambodia) using plasma samples, collected and tested on fresh or non-frozen samples (stored at 4°C) within seven days of sample collection, in accordance with manufacturer instructions for use. Confirmatory testing to obtain HCV VL and genotyping information was performed on fresh plasma samples. Tests used for determination of HCV VL were the RealTime HCV viral load assay (Abbott Laboratories, Lake Bluff, USA; limit of detection [LOD] 12 IU/mL) in Georgia and the AccuPid HCV Real-time PCR Quantification Kit (Khoa Thuong Biotechnology, Ho Chi Minh City, Vietnam; LOD 21 IU/mL) in Cambodia. Testing was performed between July 2019 and December 2019. Ethics approval for this study was obtained from the Cambodian National Ethics Committee for Health Research and the Georgian National Center for Disease Control and Public Health Institutional Review Board. Written informed consent was obtained from all study participants.

Participant recruitment

Participants providing samples were required to be aged ≥18 years, have no history of HCV treatment (past or present), and be willing to perform an HIV test. At SHCH in Cambodia, all individuals visiting the facility as outpatients were invited to participate in the study until the daily recruitment target (~10 participants/day) was met. At CMHPA and NCDC in Georgia, all individuals visiting the facility were invited to participate. Additionally, known HCV positive individuals from the site databases were contacted and invited to participate. Participant demographic and medical history information was collected, including age, HIV status, other medications and infections, and recent vaccinations. Counselling related to HCV test results was offered, and all participants received HCV confirmatory testing. Participants were assigned to the HCV positive and HCV negative group based on the result of the composite reference standard. If positive, Cambodian participants were given free treatment; Georgian participants received treatment via the national HCV treatment programme. The HCV status of participants was not known to RDT testers.

RDT performance assessments

Every sample type was tested and interpreted once per RDT. Invalid results were repeated once, and plasma and serum samples were repeated in duplicate if the initial result was different to the reference RDT SD Bioline. Two lots of each RDT were used; the complete sample population was tested to approximately 50% with lot 1 and 50% with lot 2. Testers were nurses and primary healthcare personnel who are intended to perform RDT screening as per each countries’ healthcare system. A number of different testers performed the tests at each site. The number of different testers for whole blood samples was 6, 2 and 4 at SHCH, CMHPA and NCDC, respectively. The corresponding numbers of testers for plasma and serum were 3, 2 and 3.

Data capture

Participant demographic, medical history, RDT and LIA results were initially captured on paper case report forms. Viral load and genotype results, as well as EIA results were captured in electronic format on the respective analyses platforms. All data were subsequently entered into the electronic databased Open Clinica v4.0.

Outcome measures

The primary outcome was the estimates of sensitivity and specificity of the two RDTs in each of the three sample types, compared with the CRS. Sensitivity and specificity compared with the reference RDT SD Bioline was a secondary outcome. For both outcomes, sensitivity and specificity were calculated for the overall sample set, by country and in the subset of samples with detectable HCV VL. Furthermore, statistical difference in performance between the sample types was assessed for both outcomes. Additionally, a multivariate analysis was performed to evaluate the impact of different demographic factors on RDT sensitivity in whole blood.

Statistical analyses

For sample size calculations, sensitivity and specificity was assumed to be 90% for whole blood and 95% for plasma and serum samples. However, using these assumptions, the minimum sample sizes to achieve 80% power with a 95% CI of ±5% were lower than WHO Technical Specification Series-7 (TSS-7) requirements of 400 HCV positive and 1000 HCV antibody and RNA negative samples for diagnostic assessments of HCV RDTs [4]. Therefore, the TSS-7 values were used, with a 10% increase to account for sample exclusion due to indeterminate HCV status with the CRS (based on experience from the previous study [7]). Final sample size targets were 440 HCV antibody positive (HCV positive) and 1,100 HCV antibody negative (HCV negative) samples.

Point estimates with 95% confidence intervals based on Wilson’s score method, were calculated for sensitivity and specificity. A performance comparison was performed using Pearson’s chi-square test with Bonferroni adjustment to estimate statistical differences in RDT performance between sample types and by sample type between the two countries. Statistical analysis was performed using R (version 3.6).

Covariates included in the multivariate logistic regression were age, gender, presence of detectable viral load, HCV genotype and country. The model was applied separately for each of the two investigational RDTs and the two reference methods (CRS and reference RDT SD Bioline). Estimates of coefficients and p-values were calculated using glm function with binomial logit specification in R.

Results

Population and sample characteristics

Of 1,540 individuals recruited, 11 were excluded, thus 1,529 samples of each type were provided in total. Characteristics of the individuals who provided samples are shown in Table 1. Mean age ranged from 40.3 years at CMHPA to 51.8 years at SHCH. Of the 1,529 samples, 489 were HCV positive, 966 were HCV negative, and 74 were excluded due to indeterminate results on the CRS (Fig 1). The number of HCV positive individuals encountered at NCDC in Georgia was higher than expected, thus more HCV positive participants were recruited than was anticipated in the predefined site enrolment targets.

Table 1

Study population characteristics.

	SHCH Cambodia (N = 770)	CMHPA Georgia (N = 439)	NCDC Georgia (N = 320)
Male, n (%)	269 (34.9)	360 (82.0)	153 (47.8)
Mean age, years (range)	51.8 (±13.7)	40.3 (±10.5)	42.6 (±13.6)
HCV positive on CRS, n (%)	214 (27.8)	209 (47.6)	66 (20.6)
HIV positive, n (%)	4 (0.5)	0	0
On ARV, n (%)	2 (0.3)	0	0
On other medicationa, n (%)	375 (48.7)	259 (59.0)	49 (15.3)
Other infectionsa,b, n (%)	43 (5.6)	13 (3.0)	8 (2.5)
Recent vaccinationa,c, n (%)	46 (6.0)	18 (4.1)	23 (7.2)

aAll self-reported

bHepatitis B virus, syphilis, hepatitis A virus, hepatitis D virus, influenza, measles, tuberculosis

cIn the past 12 months; includes vaccination against hepatitis B virus, influenza, tetanus, rabies, human papillomavirus, measles-mumps-rubella, yellow fever. ARV, antiretroviral therapy; CMHPA, Centre for Mental Health and Prevention of Addiction; CRS, composite reference standard; HIV, human immunodeficiency virus; NCDC, National Center for Disease Control and Public Health; SHCH, Sihanouk Hospital Center of Hope.

Fig 1

Number of samples by HCV status.

HCV VL was detectable in 63% of HCV positive samples. HCV genotype 1, 1a and 1b were the most common, followed by genotype 3 and genotype 6. However, there were no genotype 3 samples from Cambodia, and no genotype 6 samples from Georgia (Table 2).

Table 2

HCV VL and genotype status of HCV positive samples.

	SHCH Cambodia	CMHPA Georgia	NCDC Georgia
	(N = 214)	(N = 209)	(N = 66)
HCV VL status, n (%)
HCV VL undetectable	79 (36.9)	81 (38.8)	23 (34.8)
HCV VL detectable	135 (63.1)	128 (61.2)	43 (65.2)
Samples per HCV genotype, n (%)
1, 1a, 1b	63 (46.7)	33 (25.8)	22 (51.2)
2	11 (8.1)	11 (8.6)	6 (14.0)
3	--	60 (46.9)	9 (20.9)
4	--	--	--
5	--	--	--
6	59 (43.7)	--	--
Mixed	--	20 (15.6)	6 (14.0)
Not determinable	2 (1.5)	4 (3.1)	--

SHCH, Sihanouk Hospital Center of Hope; CMHPA, Centre for Mental Health and Prevention of Addiction; NCDC, National Center for Disease Control and Public Health; HCV, hepatitis C virus; VL, viral load.

Sensitivity and specificity versus composite reference standard

When compared with the CRS, specificity in the overall sample set was high (≥98.4% for both RDTs in all three sample types), with no differences observed across sample types (adjusted p = 1.0) (Table 3). Sensitivity was lower in whole blood for the HCV-Ab Rapid test (86.5%) and the First Response HCV card test (91.4%), versus plasma (97.5% and 98.0%, respectively, adjusted p<0.001) and serum (97.3% and 97.1%, adjusted p<0.001 for the HCV-Ab Rapid test and adjusted p = 0.005 for the First Response HCV card test). Sensitivity was higher in the subset of samples with detectable HCV VL (>95.4% for both RDTs) for all sample types compared with the overall sample set.

Table 3

Investigational RDT performance versus composite reference standard in the overall sample set.

	TN, n		TP, n		FN, n	FP, n	Sensitivity, % (95% CI)			Specificity, % (95% CI)
Point estimates
Whole blood (all samples)
HCV-Ab Rapid	958		423		66	8	86.5 (83.2, 89.2)			99.2 (98.4, 99.6)
First Response HCV	964		447		42	2	91.4 (88.6, 93.6)			99.8 (99.2, 99.9)
Whole blood (samples with detectable VL)
HCV-Ab Rapid	—		292		14	—	95.4 (92.5, 97.3)			—
First Response HCV	—		301		5	—	98.4 (96.2, 99.3)			—
Plasma (all samples)
HCV-Ab Rapid	951		477		12	15	97.5 (95.8, 98.6)			98.4 (97.5, 99.1)
First Response HCV	963		479		10	3	98.0 (96.3, 98.9)			99.7 (99.1, 99.9)
Plasma (samples with detectable VL)
HCV-Ab Rapid	—		304		2	—	99.3 (97.6, 99.8)			—
First Response HCV	—		304		2	—	99.3 (97.6, 99.8)			—
Serum (all samples)
HCV-Ab Rapid	955		476		13	11	97.3 (95.5, 98.4)			98.9 (98.0, 99.4)
First Response HCV	964		475		14	2	97.1 (95.3, 98.3)			99.8 (99.2, 99.9)
Serum (samples with detectable VL)
HCV-Ab Rapid	—		303		3	—	99.0 (97.2, 99.7)			—
First Response HCV	—		303		3	—	99.0 (97.2, 99.7)			—
Performance comparison (all samples), p-values
		Sensitivity						Specificity
Sample type		HCV-Ab Rapid		First Response HCV				HCV-Ab Rapid	First Response HCV
Whole blood vs plasma		<0.001		<0.001				1.0	1.0
Whole blood vs serum		<0.001		0.005				1.0	1.0
Plasma vs serum		1.0		1.0				1.0	1.0

CI, confidence interval; FN, false negative; FP, false positive; TN, true negative; TP, true positive; VL, viral load

Sensitivity in whole blood was considerably lower in Cambodia than Georgia for both RDTs (76.6% vs 94.2% for the HCV-Ab Rapid test and 85.0% vs 96.4% for the First Response HCV card test; adjusted p<0.001; Table 4). The majority of whole blood false negative samples with detectable VL from Cambodia were of genotype 1b, while those from Georgia were found across all genotypes (S2 Table). No significant differences in sensitivity between the two countries were observed for plasma or serum, and no significant differences in specificity were observed between countries for any sample type (adjusted p>0.215).

Table 4

Investigational RDT performance versus composite reference standard by country.

	TN, n		TP, n		FN, n	FP, n	Sensitivity, % (95% CI)			Specificity, % (95% CI)
Point estimates
Cambodia: whole blood
HCV-Ab Rapid	510		164		50	5	76.6 (70.5, 81.8)			99.0 (97.7, 99.6)
First Response HCV	515		182		32	0	85.0 (79.7, 89.2)			100 (99.3, 100)
Georgia: whole blood
HCV-Ab Rapid	448		259		16	3	94.2 (90.8, 96.4)			99.3 (98.1, 99.8)
First Response HCV	449		265		10	2	96.4 (93.4, 98.0)			99.6 (98.4, 99.9)
Cambodia: plasma
HCV-Ab Rapid	503		210		4	12	98.1 (95.3, 99.3)			97.7 (96.0, 98.7)
First Response HCV	512		211		3	3	98.6 (96.0, 99.5)			99.4 (98.3, 99.8)
Georgia: plasma
HCV-Ab Rapid	448		267		8	3	97.1 (94.4, 98.5)			99.3 (98.1, 99.8)
First Response HCV	451		268		7	0	97.5 (94.8, 98.8)			100 (99.2, 100)
Cambodia: serum
HCV-Ab Rapid	505		208		6	10	97.2 (94.0, 98.7)			98.1 (96.5, 98.9)
First Response HCV	513		209		5	2	97.7 (94.6, 99.0)			99.6 (98.6, 99.9)
Georgia: serum
HCV-Ab Rapid	450		268		7	1	97.5 (94.8, 98.8)			99.8 (98.8, 100)
First Response HCV	451		266		9	0	96.7 (93.9, 98.3)			100 (99.2, 100)
Performance comparison, p-values
		Sensitivity						Specificity
Cambodia vs Georgia		HCV-Ab Rapid		First Response HCV				HCV-Ab Rapid	First Response HCV
Whole blood		<0.001		<0.001				1.0	1.0
Plasma		1.0		1.0				0.541	1.0
Serum		1.0		1.0				0.217	1.0

CI, confidence interval; FN, false negative; FP, false positive; TN, true negative; TP, true positive

Sensitivity and specificity of the reference RDT SD Bioline compared with the CRS are shown in S3 Table. Performance of this test was similar to the investigational RDTs in plasma and serum (sensitivities of 95.1% and 93.9%, respectively), and even slightly lower in whole blood (90.4%). However, contrary to the other tests, no differences between sensitivity in whole blood samples and plasma or serum were observed for the reference RDT (adjusted p-value >0.160 for sensitivity and specificity across different sample types).

Sensitivity and specificity versus RDT reference SD Bioline

When the RDT SD Bioline was used as a reference for comparison, specificity in the overall sample set was high for both investigational RDTs in all three sample types (≥96.8%), with no differences observed across sample types (adjusted p>0.099) (Table 5). For both investigational RDTs, sensitivity in whole blood increased when using the SD Bioline RDT as a reference (93.6% for the HCV-Ab Rapid test and 97.8% for the First Response HCV card test) and further increased in samples with detectable HCV VL (97.3% and 99.3%, respectively). Sensitivity in plasma and serum was also slightly increased when the RDT SD Bioline was used as a reference to evaluate performance (>99.4% for both sample types and RDTs). Sensitivity was considerably lower for both RDTs in whole blood compared with plasma (adjusted p<0.001 for the HCV-Ab Rapid test and adjusted p = 0.060 for the First Response HCV card test), and for the HCV-Ab Rapid test in whole blood compared with serum (adjusted p<0.001).

Table 5

Investigational RDT performance versus reference RDT in the overall sample set.

	TN, n		TP, n		FN, n	FP, n	Sensitivity, % (95% CI)			Specificity, % (95% CI)
Point estimates
Whole blood (all samples)
HCV-Ab Rapid	1063		422		29	15	93.6 (90.9, 95.5)			98.6 (97.7, 99.2)
First Response HCV	1064		441		10	14	97.8 (96.0, 98.8)			98.7 (97.8, 99.2)
Whole blood (samples with detectable VL)
HCV-Ab Rapid	—		293		8	—	97.3 (94.8, 98.6)			—
First Response HCV	—		299		2	—	99.3 (97.6, 99.8)			—
Plasma (all samples)
HCV-Ab Rapid	1023		472		0	34	100 (99.2, 100)			96.8 (95.5, 97.7)
First Response HCV	1036		472		0	21	100 (99.2, 100)			98.0 (97.0, 98.7)
Plasma (samples with detectable VL)
HCV-Ab Rapid	—		304		0	—	100 (98.8, 100)			—
First Response HCV	—		304		0	—	100 (98.8, 100)			—
Serum (all samples)
HCV-Ab Rapid	1026		465		3	35	99.4 (98.1, 99.8)			96.7 (95.4, 97.6)
First Response HCV	1038		465		3	23	99.4 (98.1, 99.8)			97.8 (96.8, 98.6)
Serum (samples with detectable VL)
HCV-Ab Rapid	—		302		1	—	99.7 (98.2, 100)			—
First Response HCV	—		302		1	—	99.7 (98.2, 100)			—
Performance comparison (all samples), p-values
		Sensitivity						Specificity
Sample type		HCV-Ab Rapid		First Response HCV				HCV-Ab Rapid	First Response HCV
Whole blood vs plasma		<0.001		0.060				0.136	1.0
Whole blood vs serum		<0.001		1.0				0.099	1.0
Plasma vs serum		1.0		1.0				1.0	1.0

CI, confidence interval; FN, false negative; FP, false positive; TN, true negative; TP, true positive; VL, viral load

RDT sensitivity in whole blood was lower in Cambodia than in Georgia for both tests (87.4% vs 97.8%, adjusted p<0.001 for the HCV-Ab Rapid test and 95.1% vs 99.6%, adjusted p = 0.022 for the First Response HCV card test; Table 6). For both RDTs, specificity was lower in Cambodia compared with Georgia in plasma (94.5% vs 99.4%, adjusted p<0.001 for the HCV-Ab Rapid test and 96.6% vs 99.6%, adjusted p = 0.006 for the First Response HCV card test) and serum (94.2% vs 99.6%, adjusted p<0.001 for the HCV-Ab Rapid test and 96.2% vs 99.8%, adjusted p<0.001 for the First Response HCV card test). There were no significant differences between study countries in specificity for whole blood for either test. The multivariable logistic regression analysis showed that country was the most significant covariate associated with sensitivity (S4 Table). Besides the country, only gender was associated with sensitivity (slightly higher in males). However, gender only passed the threshold of statistical significance in one case (HCV Ab Rapid compared with the CRS).

Table 6

Investigational RDT performance versus reference RDT by country.

	TN, n	TP, n	FN, n	FP, n	Sensitivity, % (95% CI)	Specificity, % (95% CI)
Point estimates
Cambodia: whole blood
HCV-Ab Rapid	575	159	23	13	87.4 (8.18, 91.4)	97.8 (96.3, 98.7)
First Response HCV	576	173	9	12	95.1 (90.9, 97.4)	98.0 (96.5, 98.8)
Georgia: whole blood
HCV-Ab Rapid	488	263	6	2	97.8 (95.2, 99.0)	99.6 (98.5, 99.9)
First Response HCV	488	268	1	2	99.6 (97.9, 100)	99.6 (98.5, 99.9)
Cambodia: plasma
HCV-Ab Rapid	536	203	0	31	100 (98.1, 100)	94.5 (92.3, 96.1)
First Response HCV	548	203	0	19	100 (98.1, 100)	96.6 (94.8, 97.8)
Georgia: plasma
HCV-Ab Rapid	487	269	0	3	100 (98.6, 100)	99.4 (98.2, 99.8)
First Response HCV	488	269	0	2	100 (98.6, 100)	99.6 (98.5, 99.9)
Cambodia: serum
HCV-Ab Rapid	539	196	2	33	99.0 (96.4, 99.7)	94.2 (92.0, 95.9)
First Response HCV	550	197	1	22	99.5 (97.2, 100)	96.2 (94.2, 97.4)
Georgia: serum
HCV-Ab Rapid	487	269	1	2	99.6 (97.9, 100)	99.6 (98.5, 99.9)
First Response HCV	488	268	2	1	99.3 (97.3, 99.8)	99.8 (98.9, 100)
Performance comparison, p-values
	Sensitivity				Specificity
Cambodia vs Georgia	HCV-Ab Rapid		First Response HCV		HCV-Ab Rapid	First Response HCV
Whole blood	<0.001		0.022		0.145	0.221
Plasma	N/A		N/A		<0.001	0.006
Serum	1.0		1.0		<0.001	<0.001

CI, confidence interval; FN, false negative; FP, false positive; TN, true negative; TP, true positive

Discussion

In this prospective study of RDT performance in freshly collected whole blood, plasma and serum samples, sensitivity of both the HCV-Ab Rapid test and the First Response HCV card test was high in plasma and serum, but lower in whole blood. The concentration of antibodies is likely to be lower in whole blood compared with plasma and serum, which could explain the lower sensitivity seen in this study. However, although variability in sensitivity of HCV RDTs in whole blood has been previously reported in some studies [10-13], those that directly compared performance to plasma and serum have reported similar sensitivities across sample types [14, 15]. Other aspects that may have affected sensitivity include the possibility that some patients participating in the study had cleared their HCV infections, as evidenced by the absence of detectable VL in around one third of samples, and the improved sensitivity in the subset of samples with detectable viral load. Other studies have noted declines in HCV antibody levels following treatment-induced or spontaneous HCV clearance [16, 17], and a recent study observed reduced sensitivity of an HCV RDT in subjects with treatment-induced clearance [18]. While this would have affected all three sample types, it may have had a larger impact on sensitivity in whole blood as antibody concentrations would have been closer to the lower LOD compared with plasma and serum. Notably, WHO prequalification criteria are specifically designed for evaluation of plasma samples; no guidance is provided on expected performance in whole blood [8]. Given the variability in sensitivity in whole blood with HCV RDTs seen in earlier studies, achieving lower but acceptable sensitivity in whole blood may be considered adequate performance for the two investigational RDTs evaluated here. Nevertheless, HCV screening programmes using these RDTs must take into account the potential for lower performance in whole blood in real-world versus laboratory settings, particularly given that testing of fingerstick blood in non-laboratory settings is likely to be a common usage of these tests.

In our previous study using archived plasma samples [7], sensitivity of the investigational RDTs met the WHO prequalification sensitivity criterion of ≥98% [8], when compared with the laboratory-based CRS. In the current study, this criterion was narrowly missed by both RDTs in serum, and one of two in plasma. Unlike the previous study, in this evaluation the RDTs were performed by nurses and primary healthcare personnel, to represent a real-world setting. As such, variability in conditions, such as low lighting when reading RDTs, and user factors such as differences in line interpretation for low positive samples where lines can be more difficult to identify, could have impacted test performance. Similar factors, as well as the added technical challenge of fingerstick blood collection, may also have been a contributing factor to the lower sensitivity in whole blood. The fact that specificity was high in all sample types and sensitivity was close to WHO prequalification criteria in plasma and serum samples, suggests that the RDTs perform well in real-world settings and are likely to be beneficial to HCV screening programmes.

Consistent with our previous study in archived plasma samples [7], in this analysis, false negatives mostly occurred in samples with undetectable HCV VL. However, in our previous study this effect was more apparent in HCV and HIV coinfected samples [7]. As only four participants in the current study were HIV positive, the effect of HCV VL on test performance in this study was not linked to HIV. Other studies have reported similar observations of improved HCV RDT performance in samples with detectable VL [19]. HCV VL testing is used to confirm viraemic infection in people who test positive for HCV antibodies [2], thus these samples represent participants who had active HCV infections. Because the sensitivity of the investigational RDTs was higher in samples with detectable VL compared with the overall sample set for all sample types, this provides some reassurance in the feasibility of using these RDTs to detect HCV in the people in need of treatment.

RDT test performance in Cambodia was considerably lower than in Georgia, in terms of sensitivity in whole blood compared with either reference test (CRS or reference RDT) and in terms of specificity in serum and plasma compared with the RDT reference. Differences in specificity when compared to the RDT reference might be explained by the lower sensitivity of the SD Bioline RDT in serum and plasma samples from Cambodia, resulting in a higher number of apparent false positives for the investigational RDTs.

The reason for the lower sensitivity of the investigational RDTs in Cambodia is not clear.

Although the majority of false negative samples with detectable VL from Cambodia were of genotype 1b, while those from Georgia were found across all genotypes, different methodologies were used at the different sites to determine HCV genotype, so it is difficult to determine whether this represents a meaningful difference. A prozone effect, whereby the ability of antibodies to form immune complexes is impaired at high concentrations, may also have resulted in false negatives, as has been shown with other RDTs [20]. Alternatively, it is possible that HCV positive participants from Cambodia with undetectable HCV VL had lower antibody titres, as suggested by the fact that proportionally, there were more true positives in samples with undetectable VL from Georgia compared with Cambodia (87.5% versus 50.6% for the HCV-Ab Rapid test and 92.3% vs 63.3% for the First Response HCV card test). Historically, the HCV epidemic in Cambodia has been largely driven through past unsafe medical practices [21, 22], whereas Georgia has an ongoing HCV epidemic in injection drug users [23]. Additionally, one of the two centres in Georgia was an opioid substitution treatment facility, thus a high proportion of Georgian participants would have been injection drug users. This suggests a possibility that the between-country differences in sensitivity may be due to Cambodian participants having generally cleared infections longer ago, while more Georgian participants had ongoing infections. Previous studies have shown that HCV screening tests can provide discrepant results in people with waning antibodies [24]. However, it was not possible to test this hypothesis in this study, as it was not designed to recruit participants to represent the proportionate occurrence of ongoing and past infections. Further research is needed to better understand sensitivity differences across different population groups or HCV endemic areas.

It is interesting to note that the WHO prequalified RDT SD Bioline, used as a reference RDT in this study, also had lower than expected sensitivity in whole blood in the overall sample set (including samples with and without detectable VL) when compared with the laboratory-based CRS. The quality of SD Bioline is well established [25, 26], thus this further highlights how regional and demographic differences in population can impact on RDT performance, even with established RDTs, and demonstrates the generally lower sensitivity of RDTs compared with laboratory-based immunoassays as antibody screening tests.

Specificity was high in all sample types for both investigational RDTs when compared with the CRS, meeting the WHO prequalification specificity criterion of ≥97% for HCV serology RDTs in plasma or serum specimens [8]. Specificity also met this criterion when compared with a WHO prequalified reference RDT test, except for one of two tests in plasma and serum samples, for which specificity dropped just below the threshold.

A limitation of this study is the stringent CRS used, which led to 73 samples being excluded from the study. While this provides confidence in the accuracy of the characterisation of the samples used in the study, it is possible that inclusion of the excluded samples would have affected sensitivity and specificity estimates. Additionally, the number of testers was higher for whole blood than for plasma and serum at two out of the three study sites, which may have contributed to differences in performance across sample types. However, previous studies have suggested that provision of training substantially reduces user errors with RDTs [27]. Training was provided to all testers involved in this study, thus the impact of user variability is likely to have been minimal.

In summary, both investigational RDTs performed well in fresh plasma and serum samples. Although sensitivity in whole blood performance was lower, particularly in Cambodia, given the potential impact of variability in HCV infection history, population drivers, conditions and user factors, data from other studies evidencing variable performance in whole blood with quality assured tests, and the fact that performance was similar to that of the reference RDT, test performance can be considered adequate. Additionally, overall performance in whole blood for samples with detectable VL was high. Comparative studies in different sample types should be taken into consideration when selecting HCV RDTs for screening programmes, bearing in mind that whole blood performance in real-world settings may be different from expectations based on data generated in laboratory evaluations.

TREND statement checklist.

(PDF)

Click here for additional data file.

Investigational and reference RDT operational characteristics.

(DOCX)

Click here for additional data file.

HCV genotype of false negative whole blood samples by country.

(DOCX)

Click here for additional data file.

Performance of the reference RDT test SD Bioline compared with the composite reference standard.

(DOCX)

Click here for additional data file.

Multivariable logistic regression analysis for RDT performance in whole blood (p-values).

(DOCX)

Click here for additional data file.

Protocol deviations.

(DOCX)

Click here for additional data file.

Clinical study protocol.

(PDF)

Click here for additional data file.

23 Sep 2020

PONE-D-20-21354

Sensitivity and specificity of rapid hepatitis C antibody assays in freshly collected whole blood, plasma and serum samples: a multicentre prospective study

PLOS ONE

Dear Dr. Vetter,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The reviewers' comments can be found at the end of this email.

Please submit your revised manuscript by Nov 07 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Mohamed Fouda Salama, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a well conducted study evaluating the accuracy of two hepatitis C virus (HCV) rapid diagnostic tests (RDTs) in matched whole blood, plasma and serum samples compared to a WHO composite reference standard in three hospitals in Cambodia and Georgia. Statistical analysis was generally well conducted using the even too conservative approach of Bonferroni correction for p-values. I have a few questions, suggestions and comments:

Main Points

1. Known HCV positive individuals from the site databases were contacted and invited to participate in the study. How many of these were confirmed HCV+ by CSR? It would be interesting to do a sensitivity analysis restricting to only people with unknown HCV-status as this is the typical target population for screening.

2. The number of testers for full blood was larger than for the other type of samples. Could this explain some of the discrepancy?

3. The whole analysis focusses on sensitivity and specificity of the RDTs. Besides random variation and variability due to the testers, these are expected to be fixed features of the tests regardless of the setting in which the test was used. On the other hand, because DAA provide cure in 99% of cases and are well tolerated, the most important thing for screening and for individuals is a high positive predictive value (PPV). Because prevalence of HCV was much lower in Cambodia as opposed to Georgia, I expect the PPV also to be much lower. It would be good to quantify this and give the probability of being HCV+ and viremic in people who are tested positive with these RDTs.

4. A number of potential reasons for the lower sensitivity of the tests in full blood and in Cambodia have been provided but are rather speculative (undetectable HCV-RNA, older infections, HCV subtype, number and expertise of the testers, etc.). Average age was much older in the Cambodia site and could be used as a proxy of duration of infection. Is it not possible to identify which of these factors are more likely to explain the discrepancy by formally modelling the false positive rate (e.g. by multivariable logistic regression analysis)?

5. Eleven participants have been excluded. It would be good to describe the reasons for exclusion. It is also unclear why 73 samples were excluded. If each participant provided 3 samples, why 73 and not 33?

6. How were the Bonferroni corrected p-values actually obtained? Bonferroni suggested to use a change of the threshold for significance from the original α to α/k where k is the number of tests performed. To my knowledge, a Bonferroni-corrected p-value as such does not exist. Maybe the authors used the Benjamini–Hochberg false discovery rate correction? It would be a better approach, as Bonferroni is known to be too conservative.

7. Prequalification WHO sensitivity criterion was narrowly missed by both tested RDTs in serum, and one of two in plasma. This should be mentioned in the Conclusions of the abstract and the tone of the whole paragraph consequently lowered.

Other Points

1. Table 1. I would flip row and columns like in Table 2 (country as column headers). I would also include number and proportion of HCV+ by CRS and add percentages for all variables (HIV, on ARV, etc.) and p-values for difference between countries.

2. I would add a decimal figure to whole p-values even for those fully compatible with the null hypothesis (p=1.0).

3. Lines 259-261. Given the variability in sensitivity in whole blood with HCV RDTs seen in earlier studies, achieving lower but acceptable sensitivity in whole blood may be considered adequate performance for the two investigational RDTs evaluated here. Sentence seems a little too bald? Are not RDTs most useful as tests to be done outside of the laboratory on fingerstick whole blood samples?

Reviewer #2: This is a well written manuscript addressing one of the important issues in hepatitis C diagnostics - the use of DBS for detection of anti-HCV. I would have much preferred for authors to include additional WHO-prequalified RDTs.

Reviewer #3: This work shows the results of the comparison of a comprehensive number of two rapid diagnostic tests (RDT) for hepatitis C antibodies in the real world setting on a relatively high number of samples, and compared to a reference RDT and to a composite reference standard cconsisting in gold-stadard serology tests performed in the same samples.

The paper is well-writen and the main conclusion is that the two evaluated RTDs perform well in the filed, perhaps with decreased sensitivity for whole-blood samples and for negative viremia samples.

SPECIFIC COMMENTS:

-In the abstract include the numbers for sensitivity and specificiy; and mention the performance with respect to the composite reference.

- P5, L104-106. I would recommend movind this sentences ti the first paragraph in the "study design"

- P5, L112: What were the methods used for VL determination? What was the LOD? Was VL performed in freshly frozen samples? Pelase, specify.

- P5, L117. Plese detail inclusion and exclusion criteria.

- P6, L167. Specify why samples were excluded, based in the inclusion/exclusion criteria.

- P11, L204-205. The majority of FN in whole blood in Cambodia were HCV-1b..This is intringuing, do you have any potential explanation? What was the viral load for these samples?

- P14, L245 ...mention test name / brand here.

- P15, L254 other studies have also shown decreased sensitivity of Ab-RTDs after treatment-induced clearance in the clinical setting (see i.e. Carvalho-Gomes, Plos One, 2020)

MINOR COMMENTS:

p3, L58 ...ease of use and feasibility to...

p4 L69-70 ...lower in samples from HCV infected patientes as compared samples from HI uninfected...//...majority of FN anti-HCV in samples from HIV infected patients did not...

P16, L281...Because the sensitivity...

P16, L283...feasibility of the RDTs...

P16, L293 HCV genotypes...

P17, L328 ...particularly...

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

4 Nov 2020

Reviewer #1

This is a well conducted study evaluating the accuracy of two hepatitis C virus (HCV) rapid diagnostic tests (RDTs) in matched whole blood, plasma and serum samples compared to a WHO composite reference standard in three hospitals in Cambodia and Georgia. Statistical analysis was generally well conducted using the even too conservative approach of Bonferroni correction for p-values. I have a few questions, suggestions and comments:

1. Known HCV positive individuals from the site databases were contacted and invited to participate in the study. How many of these were confirmed HCV+ by CSR? It would be interesting to do a sensitivity analysis restricting to only people with unknown HCV-status as this is the typical target population for screening.

Response: Although we included known HCV positive people in our study, their HCV status was unknown to the testers performing the RDTs. It is therefore unlikely that HCV status could have impacted our results. Additionally, none of the participants in this study had received or were receiving HCV treatment, and so our population may be considered comparable to those with unknown HCV status in this respect.

2. The number of testers for full blood was larger than for the other type of samples. Could this explain some of the discrepancy?

Response: We cannot exclude the possibility that user variability may have led to differences across sample types. Studies with RDTs for other diseases have noted that accuracy is highly-user dependent, however, user errors with RDTs have generally been related to issues relating to interpretation of manufacturer instructions, and are improved with the provision of training (Rennie et al. Trans R Soc Trop Med Hyg 2007;01:9–18). In this study, training was provided to all testers, so we believe the impact of user variability is likely to have been minimal. We have now mentioned this as a limitation on page 19 of the manuscript.

3. The whole analysis focusses on sensitivity and specificity of the RDTs. Besides random variation and variability due to the testers, these are expected to be fixed features of the tests regardless of the setting in which the test was used. On the other hand, because DAA provide cure in 99% of cases and are well tolerated, the most important thing for screening and for individuals is a high positive predictive value (PPV). Because prevalence of HCV was much lower in Cambodia as opposed to Georgia, I expect the PPV also to be much lower. It would be good to quantify this and give the probability of being HCV+ and viraemic in people who are tested positive with these RDTs.

Response: Unfortunately, the design of this study makes it unsuitable for PPV calculation. Because a proportion of known positive participants were contacted for study participation, the study population was therefore not representative of the true prevalence in either country, which would have needed “random” recruitment. Also, in Cambodia, word of the study spread, which may have artificially increased participation of positive participants. Both of these factors may artificially increase the PPV of a test.

4. A number of potential reasons for the lower sensitivity of the tests in full blood and in Cambodia have been provided but are rather speculative (undetectable HCV-RNA, older infections, HCV subtype, number and expertise of the testers, etc.). Average age was much older in the Cambodia site and could be used as a proxy of duration of infection. Is it not possible to identify which of these factors are more likely to explain the discrepancy by formally modelling the false positive rate (e.g. by multivariable logistic regression analysis)?

Response: As per the reviewers suggestion, we performed a multivariable logistic regression analysis of RDT performance in whole blood, taking into account country, gender, age, detectable viral load and the different genotypes. In all cases, the country was the most significant covariate associated with sensitivity. Besides the country, only gender was associated with sensitivity (sensitivity was slightly higher in males). However, gender only passed the p-value threshold of 0.05 in one case (HCV Ab Rapid compared with the composite reference standard). Therefore, the difference in sensitivity between Cambodia and Georgia cannot be explained by the difference in characteristics of patients involved in these trials (at least, using these covariates). A table with results of the multivariable logistic regression analysis has been added as a supplement and a description of the methods added to the main text.

5. Eleven participants have been excluded. It would be good to describe the reasons for exclusion.

Response: Brief reasons for exclusion of these 11 samples are shown in Figure 1 (10 sample mix-ups and one participant <18 years of age). Additional details are as follows: The participant that was younger than 18 years of age was only 16 years old, but this was only noted after enrolment, upon review of the participant’s demographics and the date of birth. The remaining 10 samples were excluded as there was a mix up of the serum and plasma samples collected from these participants. These 10 participants were all recruited on the same day at a single site and serum/plasma aliquots were prepared on the same day. The mix up of samples occurred during the procedure of aliquoting into pre-labelled tubes. We have added these additional details to the footnote of Figure 1.

6. It is also unclear why 73 samples were excluded. If each participant provided 3 samples, why 73 and not 33?

Response: This refers to the samples that had indeterminate results on the CRS, not to the 11 participants who were excluded. As described on page 6, the CRS was composed of two enzyme immunoassays and one line immunoassay. Samples that had conflicting results between the two CRS assay types, or that had indeterminate results on the line immunoassay, were excluded. Please see Figure 1 for further details. Note that the value of 73 stated in the text at line 170 is in error; the correct number is 74, as shown in Figure 1. The text has been amended accordingly.

7. How were the Bonferroni corrected p-values actually obtained? Bonferroni suggested to use a change of the threshold for significance from the original α to α/k where k is the number of tests performed. To my knowledge, a Bonferroni-corrected p-value as such does not exist. Maybe the authors used the Benjamini–Hochberg false discovery rate correction? It would be a better approach, as Bonferroni is known to be too conservative.

Response: Bonferroni adjusted p-values were calculated with a "p.adjust" function in R. Benjamini-Hochberg FDR correction is another valid method of correction for multiple hypotheses, which is, indeed, less conservative than Bonferroni. The idea behind our choice of Bonferroni was related to the conservative behaviour of this approach as we aimed to identify only the most reliable statistically significant results and filter out as many false positives as possible. In this case the probability of having at least 1 false positive is equal to 5%, while in case of Benjamini-Hochberg there are, on average, 5% false positives among all significant results. To check whether our general conclusions are affected by the method of correction, we calculated adjusted p-values using Benjamini-Hochberg approach. This method resulted in similar p-values and did not alter our conclusions.

8. Prequalification WHO sensitivity criterion was narrowly missed by both tested RDTs in serum, and one of two in plasma. This should be mentioned in the Conclusions of the abstract and the tone of the whole paragraph consequently lowered.

Response: We have amended the conclusions of the abstract accordingly.

8. Table 1. I would flip row and columns like in Table 2 (country as column headers). I would also include number and proportion of HCV+ by CRS and add percentages for all variables (HIV, on ARV, etc.) and p-values for difference between countries.

Response: We have amended Table 1 as suggested.

9. I would add a decimal figure to whole p-values even for those fully compatible with the null hypothesis (p=1.0).

Response: We have made this change as suggested.

10. Lines 259-261. Given the variability in sensitivity in whole blood with HCV RDTs seen in earlier studies, achieving lower but acceptable sensitivity in whole blood may be considered adequate performance for the two investigational RDTs evaluated here. Sentence seems a little too bald? Are not RDTs most useful as tests to be done outside of the laboratory on fingerstick whole blood samples?

Response: We agree and have added the following sentence to page 16 of our manuscript: “Nevertheless, HCV screening programmes using these RDTs must take into account the potential for lower performance in whole blood in real-world versus laboratory settings, particularly given that testing of fingerstick blood in non-laboratory settings is likely to be a common usage of these tests.”

Reviewer #2

This is a well written manuscript addressing one of the important issues in hepatitis C diagnostics - the use of DBS for detection of anti-HCV. I would have much preferred for authors to include additional WHO-prequalified RDTs.

Response: The aim of this study was to assess the performance of RDTs that do not currently hold WHO prequalification status, but for which the manufacturers had demonstrated a commitment to seeking WHO prequalification. Although there are already four HCV RDTs with prequalification status, as we mention on page 4 of our manuscript, there is a need for additional prequalified RDTs to facilitate HCV screening on a large scale in low- and middle-income countries. We included the WHO prequalified RDT SD Bioline (Abbott Laboratories) in our study as a reference RDT, to allow for comparison of the investigational RDTs with an already prequalified test.

The reviewer may be interested in another article that we have recently published, which assessed the performance of 13 different HCV RDTs in archived plasma samples, including the four WHO prequalified tests (Vetter et al. J Infect Dis 2020; doi: 10.1093/infdis/jiaa389).

Reviewer #3:

This work shows the results of the comparison of a comprehensive number of two rapid diagnostic tests (RDT) for hepatitis C antibodies in the real world setting on a relatively high number of samples, and compared to a reference RDT and to a composite reference standard consisting in gold-standard serology tests performed in the same samples. The paper is well-written and the main conclusion is that the two evaluated RTDs perform well in the field, perhaps with decreased sensitivity for whole-blood samples and for negative viremia samples.

1. In the abstract include the numbers for sensitivity and specificity; and mention the performance with respect to the composite reference.

Response: We have added this information to the abstract as requested.

2. P5, L104-106. I would recommend moving this sentences to the first paragraph in the "study design"

Response: We believe the reviewer is referring to the description of the reference RDT, SD Bioline, suggesting that this be placed above the description of the composite reference test. However, as the comparison with the composite reference test was the primary outcome of the study (as mentioned in the Outcome Measures section on page 8), we believe it is more appropriate to detail this first. Sensitivity and specificity compared with the reference RDT was a secondary outcome, as defined in the study protocol.

3. P5, L112: What were the methods used for VL determination? What was the LOD? Was VL performed in freshly frozen samples? Please, specify.

Response: The tests used for determination of viral load were the RealTime HCV viral load assay (Abbott Laboratories) in Georgia, and the AccuPid HCV Real-time PCR Quantification Kit (Khoa Thuong Biotechnology) in Cambodia. Limits of detection for these two tests are 12 and 21 IU/mL, respectively. Viral load assessments were performed on fresh samples, as per protocol. This information has now been added to page 6 of the manuscript.

4. P5, L117. Please detail inclusion and exclusion criteria.

Response: Key inclusion criteria are detailed on page 7 of the manuscript, where it is stated that participants providing samples were required to be aged ≥18 years and have no history of HCV treatment (past or present). We have added the additional criteria that participants must have been willing to perform an HIV test. Other inclusion criteria are standard for clinical trials (i.e. provision of consent and willingness to undergo study procedures) and are therefore not described. Participants could have had known or unknown HCV serology, and could have been already registered at the local site or could register for the first time when enrolling in the study. There were no exclusion criteria other than inability to provide consent.

5. P6, L167. Specify why samples were excluded, based in the inclusion/exclusion criteria.

Response: Brief reasons for exclusion of these 11 samples are shown in Figure 1 (10 sample mix-ups and one participant <18 years of age). Additional details are as follows: The participant that was younger than 18 years of age was only 16 years old, but this was only noted after enrolment, upon review of the participant’s demographics and the date of birth. The remaining 10 samples were excluded as there was a mix up of the serum and plasma samples collected from these participants. These 10 participants were all recruited on the same day at a single site and serum/plasma aliquots were prepared on the same day. The mix up of samples occurred during the procedure of aliquoting into pre-labelled tubes. As the study protocol did not contain instructions on how to manage sample mix ups, it was decided to exclude these samples from the study. We have added these additional details to the footnote of Figure 1.

6. P11, L204-205. The majority of FN in whole blood in Cambodia were HCV-1b. This is intriguing, do you have any potential explanation? What was the viral load for these samples?

Response: For the reviewer’s information, the viral loads (copies/mL) of the false negative samples with genotype information are shown in the response document. The samples from Cambodia have high viral loads, however, this is not surprising as samples with low viral load would have had insufficient genetic material to determine the genotype. While it is possible that genotype 1b may be more difficult to detect using these RDTs compared with other genotypes, we do not have supporting evidence for this given that the vast majority of genotyped samples from Cambodia were genotype 1b. The genotyping method used in Georgia was different to that used in Cambodia, thus results across countries cannot be compared.

7. P14, L245 ...mention test name / brand here.

Response: Test names have been added as suggested (now page 15, line 259).

8. P15, L254 other studies have also shown decreased sensitivity of Ab-RTDs after treatment-induced clearance in the clinical setting (see i.e. Carvalho-Gomes, Plos One, 2020)

Response: We thank the reviewer for drawing our attention to this important paper. We have now mentioned this study on page 16 of our manuscript.

9. p3, L58 ...ease of use and feasibility to...

Response: We have made the suggested edit (see page 4, line 62).

10. p4 L69-70 ...lower in samples from HCV infected patients as compared samples from HI uninfected...//...majority of FN anti-HCV in samples from HIV infected patients did not...

Response: We have made the suggested edit (see page 5, line 74).

11. P16, L281...Because the sensitivity...

Response: We have made the suggested edit (see page 17, line 299).

12. P16, L283...feasibility of the RDTs...

Response: We have made the suggested edit (see page 17, line 301).

13. P16, L293 HCV genotypes...

Response: We have made the suggested edit (see page 17, line 311).

14. P17, L328 ...particularly...

Response: We have made the suggested edit (see page 19, line 350).

Submitted filename: HCV fresh sample reviewer response_D1_04Nov.docx

Click here for additional data file.

10 Nov 2020

PONE-D-20-21354R1

Sensitivity and specificity of rapid hepatitis C antibody assays in freshly collected whole blood, plasma and serum samples: a multicentre prospective study

PLOS ONE

Dear Dr. Vetter,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we invite you to submit a revised version of the manuscript that addresses a final point raised by the reviewer regarding the data in one of the supplemental tables.

Please submit your revised manuscript by Dec 25 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Mohamed Fouda Salama, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

You need to address the reviewer's comment regarding data in S. table 4

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors addressed adequately all reviewers' query. My only last comment regards Supplemental Table 4. It would be useful to add the estimates of the OR and 95% CI and report the exact p-values rather than the cryptic '<0.05'

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

13 Nov 2020

Reviewer #1

The authors addressed adequately all reviewers' query. My only last comment regards Supplemental Table 4. It would be useful to add the estimates of the OR and 95% CI and report the exact p-values rather than the cryptic '<0.05'

Response: Thank you for this comment, we have now added all p-values down to the third decimal point in Supplemental Table 4. While reviewing the table, we noted that the p-values in the second column for HCV-Ab Rapid compared to the reference RDT were incorrect. We have now corrected this, apologise for the error and note that all statements as made in the manuscript remain correct (lines 245-248).

Submitted filename: HCV fresh sample reviewer response_D1_12Nov.docx

Click here for additional data file.

16 Nov 2020

Sensitivity and specificity of rapid hepatitis C antibody assays in freshly collected whole blood, plasma and serum samples: a multicentre prospective study

PONE-D-20-21354R2

Dear Dr. Vetter,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Mohamed Fouda Salama, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

26 Nov 2020

PONE-D-20-21354R2

Sensitivity and specificity of rapid hepatitis C antibody assays in freshly collected whole blood, plasma and serum samples: a multicentre prospective study

Dear Dr. Vetter:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Mohamed Fouda Salama

Academic Editor

PLOS ONE