Consistency evaluation of Liferiver, Yaneng, Darui, and the Cobas 4800 test for high-risk human papillomavirus screening.

BACKGROUND: In recent years, several high-risk human papillomavirus (HR-HPV) tests have been developed. The assay capabilities need to be systematically reviewed. Here, we compared the clinical sample performance of three novel HR-HPV assays (Liferiver, Yaneng, and Darui) based on different platforms with the widely adopted cobas4800 test.
METHODS: A total of 346 cervical swabs from women who were screened for cervical cancer were analyzed for the presence of 14 HR-HPV genotypes. The distinction between the four assays was investigated by the genotyping and direct sequencing.
RESULTS: The positive rates of the four assays ranged from 61.56% to 64.16%. The overall concordance was 88.15%. The Yaneng assays displayed the best sensitivity (100%) and specificity (98.43%). The sensitivity (98.17%) and specificity (98.43%) of the Darui assay were superior to those of the cobas4800 test (97.72% and 93.70%, respectively). The Liferiver assay displayed comparable sensitivity with the cobas4800 test (95.89% and 97.72%, respectively). The specificity of the cobas4800 was lower than that of the Liferiver assay (93.70% vs. 97.64%).
CONCLUSIONS: The three novel HR-HPV assays displayed good agreement with the cobas4800 test. The analytical performance of all four fulfilled the requirements of sensitivity and specificity for HR-HPV detection.

INTRODUCTION

Cervical cancer is one of the most common malignant cancers in women. In 2018, approximately 311 000 women died from cervical cancer. More than 85% were from low‐ or middle‐income countries. , It is widely accepted that human papillomavirus (HPV) infection is an essential factor of cervical intraepithelial neoplasia and invasive cervical cancer. Almost all cervical cancer cases (99%) are linked with high‐risk HPV (HR‐HPV) infection. HR‐HPVs are classified into several genotypes, including HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, and HPV59. Of these, HPV16 and HPV18 are the most common genotypes. They cause more than two‐thirds of cervical cancer cases. , , Early detection of HR‐HPV infection is critical to prevent and decrease the morbidity of cervical cancer.

There are various assays for detecting and genotyping HPV. These assays differ in targets, technologies, genotyping abilities, capacity for automation, and high‐throughput analysis. , Among these assays, real‐time polymerase chain reaction (real‐time PCR) assays applied with multiplex amplifications are considered the best approach for the detection and genotyping of HPV. The Cobas 4800 assay (Roche Diagnostics) was the first product based on the real‐time PCR technique. It was approved by the United States Food and Drug Administration (USFDA) in April 2011. The Cobas 4800 assay can detect 14 types of HR‐HPV, and can differentiate HPV16 and HPV18 from the other pooled HR‐HPV groups.

Genotyping is critical role in determining the prevalence and relative risk degree of each type of HPV. Genotyping can also be beneficial for monitoring the recurrence of cancer after treatment and in evaluating the efficacy of prophylactic vaccines. , Several novel assays have been devised to improve the capacity of HPV genotyping. The Liferiver HR‐HPV assay (Liferiver, Shanghai, China) can identify and differentiate 15 HR‐HPVs based on a real‐time PCR platform. The assay was approved by the China Food and Drug Administration (CFDA) in 2015. The Yaneng 23 HPV assay (Yaneng Biology, Shenzhen, China) is a PCR‐based reverse blot‐hybridization assay (PCR‐RBD) that can simultaneously detect and genotype 18 HR‐HPVs and six LR‐HPVs. The assay was approved by CFDA in 2014.

Aside from the PCR approaches, the matrix‐assisted laser desorption/ionization‐time‐of‐flight mass spectrometry (MALDI‐TOF MS) is another capable technique for the detection and genotyping of HPVs. This technique is the basis of the novel 18 HR‐HPV detection assay (Darui Diagnostic). However, HPV testing based on the MALDI‐TOF MS platform has not been scrutinized by any expert committee.

Although the aforementioned assays have already been applied for clinical testing, their capability in the detection and genotyping of HR‐HPVs has not been systematically compared. We report the evaluation of the analytical performance of three novel HPV detection tests by comparing them to the cobas4800 HPV test in the detection of HPV16, HPV18, and a pool of 12 other HR‐HPV genotypes.

MATERIALS AND METHODS

Clinical specimens' collection

We selected a total of 346 clinical cervical specimens from women who had visited the Shenzhen Luohu Hospital Group for routine HR‐HPV screening. The histopathology or cytomorphology findings of these patients were not taken into account. Cervical specimens were collected in Cobas and Liferiver PCR cell collection medium (n = 179 and 167, respectively). After collection from September to November 2019, the samples were stored at −80°C before processing.

Comparison of the two PCR cell collection medium

The chemical composition of the two PCR cell collection media used in this study was not available from the manufacturers. Thus, elementary analysis and nuclear magnetic spectroscopy were performed to examine the media ingredients. To further confirm the accuracy of the analysis, a single HPV16 liquid standard (GWB(E)090671; BDS) was diluted to a concentration approximating the limit of detection (LoD) of the two assays (10 000 copies/mL for Liferiver and 300 copies/mL for Cobas). To reduce interference and simulate the practical situation, HPV‐negative clinical specimens stored in the Cobas or Liferiver cell media were used as the diluent to prepare two separate sets of samples. One set consisted of two samples containing ~10 000 copies/mL diluted in Cobas or Liferiver cell medium. The other set comprised two samples containing ~300 copies/mL diluted in either medium. The results of these dilutions detected by the two assays were compared.

Clinical specimens HPV detection and genetyping by Real‐Time PCR HPV test

Analytical comparison was performed to investigate the performance of the Cobas 4800 HPV assay and the Liferiver Real‐Time HR‐HPV assay in HPV detection and genotyping. One milliliter of liquid cytology sample was separated into two aliquots for these two tests (Table 1). The Cobas assay can detect 14 HPV genotypes (HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66, and HPV68) and can differentiate HPV16 and HPV18 from the other pooled high‐risk HPV groups. In addition to these 14 HPV types, the Liferiver assay also capable detects HPV82. DNA extraction and PCR amplification were performed by different fully automated sample preparation processes. The Roche HPV‐DNA extraction kit (Roche Diagnostics), the cobas × 480 PCR amplification instrument (Roche Diagnostics), and the Cobas z480 analyzer (Roche Diagnostics) were used for the Cobas 4800 assay. The Liferiver HPV‐DNA kit (Liferiver), Autrax automatic nucleic acid extraction workstation (Liferiver), and Slan‐96S PCR analyzer (Hongshi) were used for the Liferiver assay. The experiment and result interpretation were carried out according to the manufacturer's protocols.

Table 1

Comparison of the basic characteristics for each of the HPV‐DNA tests

Assay characteristics	Liferiver HPV (15 genetypes) assay	Darui HPV (18 genetypes) assay	Yaneng HPV (23 genetypes) assay	CobasX 4800 HPV
Manufacturer	Liferiver (Shanghai)	DARUI dignostic (Guangzhou)	Yaneng biology (Shenzhen)	Roche (The USA)
Principle of test	Multiplex real‐time PCR	MALDI‐TOF MS	PCR‐RDB	Multiplex real‐time PCR
Analyzed gene (size of PCR product)	L1 (200 bp)	L1 (80‐120 bp)	L1 (150‐200 bp)	L1 (200 bp)
Internal control	MNBH	β‐globin	β‐globin	β‐globin
Detected genotypes	15 hrHPV (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59,66, 68, 82)	18 hrHPV (16, 18, 26, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 68, 73, 82)	23 HPV (18HR HPV:16, 18, 26, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 68, 73, 82; 6 LR HPV:6, 11, 42, 43, 81, 83)	14 hrHPV (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68)
Form of results	Full genotyping	Full genotyping	Full genotyping	Partial genotyping (HPV16, HPV18, other HPV)
Limit of detection a (HPV16; HPV18)	1 × 104 copies/mL	100 copies/mL	1 × 104 copies/mL	300‐600 copy/mL; 600 copy/mL
Sample processing	Automated sample extraction	Semiautomatic sample extraction	Semiautomatic sample extraction	Automated sample extraction
Turnaround time/technician time	1.5‐2/0.5 h	10 h/1.5 h	6 h/1 h	4‐5/0.5 h
Throughput	42	192	96	96
Special equipment required	SLAN‐96P, 96S Autrax automatic nucleic acid extraction workstation (instrument for automatized sample preparation)	Smart32 nucleic acid extraction instrument Applied Biosystems Veriti 384 (Thermal cycler) DR MassARRY	Auto‐Pure96 automatic nucleic acid extraction instrument Applied Biosystems Veriti 384 YN‐H96 Automatic nucleic acid molecular hybridizer	The cobas x 480 (instrument for automatized sample preparation); cobas z 480
Regulatory status	CFDA approved	/	CFDA approved	FDA approved

HPV genotypes detectable by all tested methods are shown in bold.

Abbreviations: CFDA, China Food and Drug Administration; FDA, Food and Drug Administration; HR‐HPV, high‐risk human papillomavirus.

Limit of detection was determined by the manufacturer of each assay.

HPV detection and genotyping by the HPV MALDI‐TOF MS assay

The Darui 18 HR‐HPV test kit was recently developed for HR‐HPV detection based on multiplex PCR together with the MALDI‐TOF MS platform (Sequenom). The target DNAs were extracted from the clinical DNA specimens by the MagPure Tissue & Blood DNA LQ Kit (Magen) on the Smart32 nucleic acid extraction instrument (Daan). PCR amplification was performed on an Applied Biosystems Veriti 384 apparatus (Thermo Fisher Scientific). Samples were detected and genotyped using the 18 HR‐HPV MALDI‐TOF MS assay to detect HPV16, HPV18, HPV26, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV53, HPV56, HPV58, HPV59, HPV66, HPV68, HPV73, and HPV82 on the Darui MassARRY MALDI‐TOF MS platform. The genotyping results were automatically generated using Typer 4.0.22 Software (Sequenom). The samples were clustered into positive and negative groups according to the software algorithm, which is based on the ratios of unextended/extended primer and on the relative intensity of each expected mass‐to‐charge ratio. These criteria were used to discriminate positive samples for each genotype. The human β‐globin gene was included as an internal control to ensure specimen adequacy (Figure [Link], [Link]).

HPV detection and genotyping of the 23 types by RBD‐PCR

PCR‐RDB HPV genotyping for the 23 HPV types (Yaneng Biotech) identifies 18 HR‐HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 68, 73, 82, and 83) and five LR‐HPV types (6, 11, 42, 43, and 81). DNA was extracted from liquid cytology samples by the auto‐Pure96 automatic nucleic acid extraction instrument. The extracts and the positive and negative controls were amplified in a thermal cycler under the conditions set by the manufacturer. After amplification, HPV genotyping was conducted by hybridization and RDB on strips fixed with 23 specific probes. Blue spots could be identified as positive with the naked eye (Figure [Link], [Link]).

Genotyping and direct sequencing

For the samples showing discrepant results among the four HPV detection assays were further examined for HPV genotypes by PCR and direct sequencing. Nested PCR was applied using the general primer pair MY09/11 for primary PCR and the GP6+/MY09 primer pairs. The amplified products were further purified by electrophoresis, and the sequencing PCR was performed using the general primers GP6+. All positive bands of type‐specific nested PCR amplicons were purified by gel electrophoresis and sequenced using one of the genotype‐specific primers as the sequencing primer. HPV type identification was performed by the Alignment Search Tool (BLAST) database on the NCBI website.

The experimental conditions for all tests mentioned above followed the relevant guidelines and regulations as well as the protocol provided by the manufacturer. In each assay, both positive and negative controls were employed to ensure all cells were detected and to avoid carry‐over contamination. Although genotyping results differed in each detection assay, only 14 HR‐HPV types were selected for further analysis (HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66, and HPV68). All 14 were detected by all comparison assays. Among these genotypes, the HPV16 and HPV18 results were analyzed individually. The remaining 12 high‐risk HPV types were pooled together as the high‐risk HPV group.

Clinical performance of HPV detection assays

The sensitivities and specificities of the four HPV‐DNA detection assays were calculated based on the diagnostic accuracy criteria, which were determined based on the results of the genotyping and direct sequencing. If the HPV‐DNA detection results of the four assays were all concordant, they were regarded as true positive or negative. Otherwise, the diagnostic accuracy criteria were identified using PCR and direct sequencing.

Statistical analyses

All statistical analyses were performed using SPSS23.0 (IBM). Concordance rates and kappa coefficients (k) with 95% confidence intervals were calculated to estimate the concordance between the results from different assays. McNemar's test Chi‐square test was used to compare HPV‐positive rates, sensitivity, and specificity of each test. A P value < .05 was considered statistically significant.

RESULTS

Effect of two PCR cell media on detection capability of HPV16

The major components of both PCR cell collection media are methanol‐water buffer containing common elements (Figure [Link], [Link], Table S1). There was little difference in the contents and concentrations of the media components. Both PCR cell collection media used for samples at a LoD level were measured and positive results were identified. The threshold cycle (Ct) value of the same concentration of sample was higher in the Liferiver PCR cell media than that in the Cobas PCR cell media when the analyses were performed on both HPV detection systems. The difference was more obvious when the test was performed on the Cobas 4800 system (Figure [Link], [Link]).

HPV‐positive rates in collected cases

The distribution data of the HPV types among 346 samples detected by the four HPV‐DNA assays are presented in Table 2. The total positive rate of the four assays ranged from 61.56% to 64.16%. The results from the different classifications revealed no significant differences (P > .05 in all cases). Notably, the only case co‐infected with HPV16 and HPV18 was successfully identified by the Liferiver, Yaneng, and Darui HPV assays. The Cobas test yielded a negative result. The cases co‐infected with HPV16, HPV18, and other HPVs were detected most often in the Cobas HPV assay (1.73%) than in other three assays (Table 2).

Table 2

Prevalence and genotype of HPV for clinical specimens by the three HPV assays

	Liferiver	%	Darui	%	Yaneng	%	Cobas4800	%
Total number	346	100.00	346	100.00	346	100.00	346	100.00
HPV negative	133	38.44	129	37.28	125	36.13	124	35.84
HPV positive	213	61.56	215	62.14	221	63.87	222	64.16
HPV16	47	13.58	46	13.29	44	12.72	44	12.72
HPV18	25	7.23	21	6.07	21	6.07	22	6.36
Other HPV	104	30.06	110	31.79	117	33.82	112	32.37
HPV16 and HPV18	1	0.29	1	0.29	1	0.29	0	0.00
HPV16 and other HPV	19	5.49	22	6.36	21	6.07	22	6.36
HPV18 and other HPV	13	3.76	14	4.05	14	4.05	16	4.62
HPV16, HPV18, and other HPV	4	1.16	3	0.87	3	0.87	6	1.73

Abbreviation: HPV, human papillomavirus.

Consistency evaluation of the assays

Comparison data of the three HR‐HPV assays with the Cobas 4800 assay are presented in Table 3. The coincident rate of the four HPV assays was 92.49% (320 cases/346 cases), regardless of HPV genotype. The coincident rate was 88.15% (305 cases/346 cases) considering the different genotypes. Compared with the Cobas 4800 test, agreement was high in the Yaneng assay 96.82%, Kappa, 0.931), Liferiver assay (92.77%, Kappa, 0.845), and the Darui assay (95.09%, Kappa, 0.894). In cases of infection with HPV16 alone, the agreement of the results of the three assays with those of the Cobas test exceeded 98.27% (Kappa, 0.947). For single HPV18 infection, the consistency reached 97.98% (Kappa, 0.904). Except for HPV16 and HPV18, the other genotypes in the three assays were also consistently detected (91.33%, Kappa, 0.823) compared with the detection by the Cobas test (Table 3).

Table 3

Concordance between the results of the four HPV assays

	Liferiver		Darui		Yaneng		CobasX 4800
	HPV+	HPV−	HPV+	HPV−	HPV+	HPV−	HPV+	HPV−
Liferiver
HPV+			210	3	210	3	205	8
HPV−			7	126	11	122	17	116
HPV16+			70	2	68	3	69	2
HPV16−			1	273	1	274	3	272
HPV18+			38	5	38	5	40	3
HPV18−			1	302	1	302	4	299
HPV others+			139	1	190	1	133	7
HPV others−			10	196	16	139	23	183
Darui
HPV+	Kappa = 0.939 (0.902‐0.976) P = .344				216	5	211	6
HPV−					1	124	11	118
HPV16+	Kappa = 0.974 (0.945‐1.00) P = 1				68	4	69	3
HPV16−					1	273	3	271
HPV18+	Kappa = 0.917 (0.852‐0.982) P = .219				38	1	38	6
HPV18−					1	306	1	301
HPV others+	Kappa = 0.935 (0.898‐0.972) P = .012				149	0	143	6
HPV others−					6	191	13	184
Yaneng
HPV+	Kappa = 0.914 (0.869‐0.959) P = .057		Kappa = 0.963 (0.934‐0.992) P = .219				216	6
HPV−							5	119
HPV16+	Kappa = 0.964 (0.929‐1.00) P = .625		Kappa = 0.955 (0.916‐0.994) P = .375				68	1
HPV16−							4	273
HPV18+	Kappa = 0.917 (0.852‐0.982) P = .033		Kappa = 0.971 (0.93‐1.00) P = 1				38	6
HPV18−							1	301
HPV others+	Kappa = 0.900 (0.853‐0.947) P < .001		Kappa = 0.965 (0.885‐1.045) P = .031				147	8
HPV others−							9	182
Roche cobas
HPV+	Kappa = 0.845 (0.786‐0.904) P = .108		Kappa = 0.894 (0.845‐0.943) P = .332		Kappa = 0.931 (0.892‐0.970) P = 1
HPV−
HPV16+	Kappa = 0.956 (0.917‐0.995) P = 1		Kappa = 0.947 (0.906‐0.988) P = 1		Kappa = 0.955 (0.916‐0.994) P = .375
HPV16−
HPV18+	Kappa = 0.908 (0.841‐0.975) P = 1		Kappa = 0.904 (0.833‐0.975) P = .125		Kappa = 0.904 (0.833‐0.975) P = .125
HPV18−
HPV others+	Kappa = 0.823 (0.762‐0.884) P = .005		Kappa = 0.889 (0.84‐0.938) P = .167		Kappa = 0.901 (0.856‐0.946) P = 1
HPV others−

Every intersection of method row and method column corresponds to a 2 *2 contingency table for those two methods.

kappa (95% CI) concordance metrics.

P value was calculated using the McNemar test.

Abbreviation: HPV, human papillomavirus.

In summary, the highest concordance was found between the Cobas and Yaneng HPV test with a kappa value > 0.9, while the lowest concordance was observed between the Cobas and Liferiver assay with a kappa value of 0.845. The inconsistent infection cases with corresponding Ct values between the Liferiver and Cobas 4800 HPV tests are presented in Table 5. A total of 21 cases with single HPV type infection were successfully identified by the Cobas assay and failed to be detected by the Liferiver assay. The positive samples had a Ct value approximate to each cutoff setting in the Liferiver (cutoff at 38) and Cobas 4800 (cutoff at 40) HPV assays. Interestingly, there were seven positive cases in the Liferiver assay that displayed a negative result in the Cobas 4800 test, with a Ct value of 35 (Table 5).

Table 5

Ct values for discordant single HPV infection results between the Liferiver tests and Cobas4800 HPV test

	N	Ct (all) median	Ct (all) average
Cobas+/Liferiver−	21	38.24	38.9
Cobas−/Liferiver+	7	35	35

Liferiver cutoff, 38 and cobas cutoff, 40.5.

Abbreviations: Ct, cycle threshold; HR‐HPV, high‐risk HPV.

Analytical performance of the four HPV assays

The sensitivity and specificity data of the four assays are summarized in Table 4. All tests displayed a comparably high specificity for both HPV16 (98.92% to 99.64%) and HPV18 (98.69% to 100%). A high sensitivity (98.55% to 100%) was obtained for HPV16, while the value remained relatively low for HPV18 (95.12% to 97.56%). As for other 12 HR‐HPV genotypes, the Liferiver assay showed the lowest sensitivity (92.05%) and the highest specificity (99.49%). The Cobas 4800 tests produced the lowest specificity (93.85%) for other HPV types. The Yaneng assay produced a relatively high sensitivity and specificity for HPV genotypes or all HPV infections compared to the values for the other three assays.

Table 4

Sensitivity, specificity, and Cohen's k coefficient of the different methods calculated using golden standard (PGMY09/11 PCR and sequencing) as a reference

HPV type	Method	All samples
		N	SE (%)	SP (%)	kappa (95% CI)	P value
HPV	Liferiver	356	95.89	97.64	0.926 (0.88‐0.97)	.146
	Darui	356	98.17	98.43	0.963 (0.93‐0.99)	.687
	Yaneng	356	100	98.43	0.988 (0.97‐1.00)	.500
	CobasX 4800	356	97.72	93.70	0.919 (0.88‐0.96)	.581
HPV16	Liferiver	356	98.55	98.92	0.964 (0.93‐1.00)	.625
	Darui	356	100	98.56	0.965 (0.93‐1.00)	.125
	Yaneng	356	100	99.64	0.991 (0.97‐1.00)	1.0
	CobasX 4800	356	100	98.92	0.973 (0.94‐1.00)	.250
HPV18	Liferiver	356	97.56	99.02	0.946 (0.81‐1.0)	.625
	Darui	356	95.12	100	0.972 (0.93‐1.0)	.500
	Yaneng	356	95.12	100	0.972 (0.93‐1.0)	.500
	CobasX 4800	356	97.56	98.69	0.933 (0.87‐0.99)	.375
HPV others	Liferiver	356	92.05	99.49	0.923 (0.88‐0.96)	.003
	Darui	356	97.35	98.97	0.965 (0.94‐0.99)	.687
	Yaneng	356	100	97.95	0.977 (0.95‐1.00)	.125
	CobasX 4800	356	95.36	93.85	0.889 (0.84‐0.94)	.359

DISCUSSION

Persistent infection of HR‐HPV are the main factors for cervical cancer. It is important to identify in advance the type‐specific HPV with a satisfactory performance for clinical diagnosis and cancer prevention. Although various HPV‐DNA tests have been commercialized to address the growing demand for cervical cancer screening in China, the unsatisfactory consistency in the genotyping results among HPV kits has been described. , , , Herein, the identified consistency was evaluated with clinical cases among two commonly used domestic HPV assays (Liferiver and Yaneng), a newly developed HR‐HPV test based on MALDI‐TOF MS (Darui), and the internationally recognized Cobas 4800 test. ,

The HPV detection results revealed a high level of agreement of the three HR‐HPV tests with the results of the Cobas HPV test. These three tests performed similarly with positive rates ranging from 61.56% to 63.87%. The high HPV‐positive rates were not surprising because samples were taken from registered HPV infected patients who were undergoing treatment. The kappa value for all HR‐HPV tests compared with cobas4800 ranged from 0.845 to 0.931. In cases infected with HPV16, HPV18, and other HR‐HPVs, the results from the three HR‐HPV tests also agreed well with those from cobas4800 test. This finding demonstrated that the detection efficacy of HPV infection of the three newly developed HPV detection assays is comparable to that of the cobas4800 test. All the HPV‐DNA tests had good agreement for HPV16 and HPV18 with a kappa coefficient exceeding 0.9 between each assay. Results of other HR‐HPV genotypes agreed with each other with a kappa coefficient of 0.823 to 0.965, which is similar to the overall concordance rate among the four assays, regardless of genotype.

Of note, the MassARRY HPV assay displayed outstanding performance. This assay allowed the detection and genotyping of 18 HR‐HPVs and could also be used for a fully automated middle‐throughput assay with a process capacity of 10 × 384‐well format within 2 working days. The capability of MALDI‐TOF MS platform in HPV‐DNA testing has been demonstrated. , Cai et al compared an 18 HR‐HPV detection assay based on the MALDI‐TOF MS platform with the Cobas4800 and described that this assay was superior to the Cobas 4800 HPV test in sensitivity and specificity. Consistent with this finding, we found that the Darui 18 HR‐HPV detection assay based on MALDI‐TOF MS platform had high levels of agreement with the other three HPV tests approved by the USFDA and CFDA, and was more sensitive and specific than the Cobas 4800 test.

In this study, the analytical sensitivity and specificity of four detection methods were evaluated. Among these assays, the Yaneng assay was the most sensitive and specific in detecting 14 HR‐HPV genotypes. The Cobas 4800 and Yaneng assays yielded concordant results, with consensus of up to 97% of all samples. The 99.42% consensus of the Yaneng assay indicated that the assays was superior. The sensitivity and specificity of the Liferiver assay may have been influenced by the lower LoD compared with the other three methods. Improving the detection of the cutoff value for these genotypes should be readily solved by the manufacturer. The sensitivity and specificity of the Cobas 4800 and Liferiver assays were calculated from 179 samples in Cobas PCR cell collection media and 167 samples in Liferiver PCR cell media. This difference in media may explain the missed detection of genotypes by the Cobas 4800 test compared with the Liferiver assay.

There are several limitations in the study. The sample size (n = 346) was a relatively small. Studies involving more samples are required to verify our findings. Due to the limited sample volume and message, only analytic sensitivity and specificity were examined. Systematically investigation of the performance of these assays is warranted. Thirdly, for the 14 HR‐HPV genotypes, with the exception of HPV16 and HPV18, the other HPV types were all investigated during the concordance test. This means that the total HR‐HPV concordance rates could have been overestimated. Moreover, the inconsistent results among the four assays were likely to have originated from the different types of PCR cell collection media used. Therefore, we suggest using a general PCR cell collection media.

In conclusion, the findings suggest the three novel HR‐HPV tests commonly used in China have a comparable analytic performance with the Cobas HPV test. The Yaneng HPV assay displayed the best HPV detection and genotyping capabilities. All four tests displayed exceptional sensitivity and specificity using the golden standard PGMY09/11 PCR and sequencing as a reference. Efforts should be made to confirm the clinical performance of the four tests using pathological diagnosis. Studies with more samples are required to further compare an increased number HPV tests as well.

Fig S1

Click here for additional data file.

Fig S2

Click here for additional data file.

Fig S3

Click here for additional data file.

Fig S4

Click here for additional data file.

Tab S1

Click here for additional data file.

App S1

Click here for additional data file.