Unsatisfactory rates vary between cervical cytology samples prepared using ThinPrep and SurePath platforms: a review and meta-analysis.

OBJECTIVE: To compare unsatisfactory rates between the two major liquid-based cytology (LBC) platforms, namely ThinPrep (Hologic) and SurePath (Becton Dickinson).
DESIGN: The authors performed both a systematic review and a meta-analysis. Inclusion criteria were English language, data presented on unsatisfactory rates for either ThinPrep or SurePath, utilising actual patient samples (ie, not laboratory manipulated samples) and no manipulation using acetic acid to increase the satisfactory rate. The authors searched PubMed for articles using the keywords 'SurePath' or 'ThinPrep' and 'unsatisfactory'. References of retrieved studies were searched for additional articles. Key researchers in the field were also contacted. PARTICIPANTS AND
INTERVENTIONS: Eligible studies were reviewed for rates of unsatisfactory cervical cytology smears processed on either the ThinPrep or SurePath platforms (compared with a general linear model) or data on unsatisfactory rates for both platforms for the same laboratory and the same patient population (compared with a meta-analysis using a random effects model and pooled RR). PRIMARY OUTCOME MEASURE: Unsatisfactory rate of cervical cytology smears.
RESULTS: A total of 1 120 418 cervical cytology smears were reported in 14 different studies using the SurePath platform for an overall unsatisfactory rate (weighted average) of 0.3%. 28 studies reported on 1 148 755 smears prepared using the ThinPrep platform for an overall unsatisfactory rate (weighted average) of 1.3%. The general linear model did not show a difference between LBC platforms when other variables were controlled for; however, the power to detect a difference (0.087) was very low. The meta-analysis performed on four studies where both ThinPrep and SurePath results were reported from the same laboratory showed fewer unsatisfactory tests from the SurePath platform (RR 0.44, 95% CI 0.25 to 0.77, p=0.004).
CONCLUSIONS: Multiple factors affect LBC unsatisfactory rates. In a meta-analysis, cervical cytology samples prepared on the SurePath platform show significantly fewer unsatisfactory smears than those prepared on the ThinPrep platform.

Introduction

Since first introduced in the late 1990s, liquid-based cytology (LBC) was quickly adopted by many laboratories as a great improvement in the performance of gynaecologic cytology. The method permits laboratories to create slides rather than having prepared slides sent to them in various degrees of fixation and preparation. The LBC methodology also provides a cleaner smear as the proprietary methods remove obscuring elements such as blood and inflammation. This ‘cleaning’ of samples is often cited as the reason there are lower unsatisfactory rates in LBC.1 The liquid platforms also allow for reflex testing for human papilloma virus for borderline cases. However, large trials and a meta-analysis have recently shown that the performance of LBC may be equivalent to conventional cytology, thus making it difficult for laboratories to justify the added expense of LBC.2–4

When considering LBC, laboratories are faced with a decision between two main platforms produced by ThinPrep (Hologic) and SurePath (Becton Dickinson). Both technologies are Food and Drug Administration approved but employ different methodologies in sample preparation. In light of these differences in preparation, we set out to evaluate the existing English literature comparing the performance of each platform with respect to unsatisfactory rates.

Methods

Information sources and search strategy

Our meta-analysis adhered to the PRISMA statement for reporting on meta-analyses.5 The initial literature search for articles was conducted using PubMed. Articles published in English between 1 January 1990 and 1 August 2011 were retrieved using the search words ‘unsatisfactory’ and ‘ThinPrep or SurePath’. These date range were chosen to coincide with the period following the introduction of The Bethesda System (TBS) in 1989.6 The references of the retrieved papers were examined for further possible studies not detected in the initial literature search. Additionally, we searched major clinical trial registries (http://clinicaltrials.gov, http://isrctn.org, http://www.umin.ac.jp/ctr/index/htm/, http://www.anzctr.org.au/Default.aspx, http://www.trialregister.nl/trialreg/index.asp, http://apps.who.int/trialsearch) using the search words ‘liquid-based cytology’, contacted both manufacturers and contacted key researchers in the field to check for ongoing or unpublished studies.

Study selection and eligibility criteria

Eligible studies for the review presented rates of technically unsatisfactory cervical cytology smears performed on either ThinPrep or SurePath platform. For inclusion in the meta-analysis, studies must have presented data on unsatisfactory cervical cytology smears performed on both ThinPrep and SurePath platforms by the same laboratory on the same patient population. Only studies presented in English were considered. Both published and unpublished studies were considered to be eligible for inclusion.

The initial search retrieved 76 papers. The abstracts of these papers were reviewed by CN and DF and 46 were chosen for full review. An additional six published studies were subsequently identified as well as one unpublished study. Of these articles, 42 met the further inclusion criteria of presenting data on unsatisfactory rates for either ThinPrep or SurePath and utilising actual patient samples (ie, not laboratory manipulated samples) and not employing glacial acetic acid to increase the satisfactory rate. Four of these studies presented data on both ThinPrep and SurePath use in the same population by the same laboratory and were therefore included in the meta-analysis. Our contacts with industry and other researchers did not reveal additional usable studies. The selection process is summarised in figure 1.

Figure 1

Selection of studies of unsatisfactory rates of cervical cytology smears prepared with ThinPrep and SurePath platforms.

Data extraction

Two of the authors (CN and DF) extracted data independently from the 42 studies meeting the inclusion criteria. These results were then compared and any discrepancies resolved. The following data were extracted: (1) number of patients included, (2) type of liquid-based system used (ThinPrep or SurePath), (3) per cent of unsatisfactory cases, (4) country of study, (5) year of study, (6) whether image analysis software was used to screen the slides and (7) the number of women in the study (sample size). The lack of ambiguity of the outcome measure (only two end points are possible: unsatisfactory or satisfactory), combined with the irrelevance of a follow-up (latent) period, obviated the need for quality scoring of the included studies.

Statistical analysis

Unsatisfactory rates were first compared using a univariate general linear model in SPSS V.19. The dependent variable was unsatisfactory rate, and the dependent variables were year of study (divided into 2002 and before to capture studies published before TBS revision in 2001 and 2003 and later), platform (ThinPrep vs SurePath), country where study was performed and whether image analysis software was used. Studies were weighted by the number of women participating in each one. In addition to the significance level, observed power was calculated for each variable. Differences were considered significant at an α level of 0.05. Meta-analysis was then performed using Review Manager V.4.2.7 A χ2 test for heterogeneity among studies included in the meta-analysis revealed an I2 of 97% with a p value of <0.00001, indicating significant heterogeneity among these studies (although the heterogeneity was quantitative rather than qualitative as all the trials demonstrated the same trend). Therefore, a random effects model was employed to calculate the pooled OR.

Results

A total of 1 120 418 cervical cytology smears were reported in 14 different studies using the SurePath platform (table 1). The pooled unsatisfactory rate (weighted average) was 0.3%. For cervical cytology smears processed on the ThinPrep platform, 28 studies reported on 1 148 755 smears for a pooled unsatisfactory rate (weighted average) of 1.3% (table 2). A general linear model as previously described was employed to test for a difference in unsatisfactory rate ThinPrep and SurePath. Year of publication (p=0.021) and country where research was performed (p=0.004) were independent predictors of unsatisfactory rate. The use of image analysis software (p=0.091) and the LBP platform used (p=0.558) were not independent predictors of unsatisfactory rate. However, the observed power for LBP platform was very low at 0.087, making the risk of a type II error high. To attempt to gain statistical power, we performed a meta-analysis where we considered only studies where both platforms were evaluated on the same patient population by the same laboratory. We found three such studies in the published literature and added data from one additional unpublished study (table 3). All four of these studies reported similar populations but different patients who were screened with either ThinPrep or SurePath platform. The pooled RR from the meta-analysis was 0.44 (95% CI 0.25 to 0.77) in favour of SurePath over ThinPrep (figure 2). A Z-test for overall effect was statistically significant at p=0.004.

Table 1

Unsatisfactory cervical cytology smear rates in populations using the SurePath platform

Lead author	Year	Total number of cervical cytology smears	Number of unsatisfactory smears	Percentage of unsatisfactory smears	Location of study
Colgan et al8	2004	352 680	915	0.3	Canada
Zhao et al9	2011	972	2	0.2	China
Kirscher et al10	2006	84 414	292	0.3	Denmark
Beerman et al11	2008	35 315	46	0.1	the Netherlands
Sykes et al12	2008	451	12	2.7	New Zealand
Sykes et al12	2008	457	12	2.6	New Zealand
Kitchener et al13	2011	22 145	354	1.6	UK
Narine and Young14	2007	53 982	208	0.4	UK
Narine and Young14	Unpublished	27 738	327	1.2	UK
Alsharif et al15	2009	232 022	360	0.2	USA
Stark16	2007	137 703	302	0.2	USA
Nance17	2007	92 875	158	0.2	USA
Fremont-Smith et al18	2004	58 580	130	0.2	USA
Sass19	2003	8771	14	0.2	USA
Wilbur et al20	2009	12 313	27	0.2	USA
Total		1 120 418	3159	0.3

Table 2

Unsatisfactory cervical cytology smear rates in populations using the ThinPrep platform

Lead author	Year	Total number of cervical cytology smears	Number of unsatisfactory smears	Percentage of unsatisfactory smears	Location of study
Davey et al21	2007	52 665	795	1.5	Australia
Roberts et al22	1997	35 560	235	0.7	Australia
Shield et al23	1999	300	19	6.3	Australia
Halford et al24	2009	87 284	758	0.9	Australia
Duggan et al25	2006	2288	10	0.4	Canada
Belison et al26	2002	1450	158	10.9	China
Zhao et al9	2011	1033	15	1.5	China
Yeoh et al27	1999	16 541	93	0.6	Hong Kong
Cheung et al28	2003	190 667	608	0.3	Hong Kong
Grace et al29	2002	1000	13	1.3	Ireland
Treacy et al1	2009	41 312	1342	3.2	Ireland
Ronco et al30	2007	22 708	583	2.6	Italy
Rahimi et al31	2008	467	9	1.9	Italy
Luthra et al32	2001	1024	36	3.5	Kuwait
Siebers et al33	2008	46 064	153	0.3	the Netherlands
Williams34	2006	78 064	1493	1.9	Scotland
Taylor et al35	2006	3184	70	2.2	South Africa
Tuncer et al36	2005	4322	72	1.7	Turkey
Kitchener et al13	2011	23 436	456	1.9	UK
Narine and Young14	Unpublished	58 973	1250	2.1	UK
Alsharif et al15	2009	5419	124	2.3	USA
Nance17	2007	88 575	567	0.6	USA
Bentz37	2002	23 790	209	0.9	USA
Bentz37	2002	15 154	197	1.3	USA
Bolick and Hellman38	1998	10 694	31	0.3	USA
Wachtel et al39	2009	2890	77	2.7	USA
Harkness et al40	2003	3000	80	2.7	USA
Duby and DiFurio41	2009	55 438	1027	1.8	USA
Duby and DiFurio41	2009	53 209	783	1.5	USA
Zhao and Austin42	2009	222 242	3230	1.4	USA
Total		1 148 755	15 593	1.3

Table 3

Studies included in the meta-analysis

Lead author	Year	Platform	Total number of cervical cytology smears	Number of unsatisfactory smears	Percentage of unsatisfactory smears	Location of study
Zhao et al9	2011	TP	972	2	1.5	China
Zhao et al9	2011	SP	1033	15	0.2	China
Kitchener et al13	2011	TP	23 436	456	2.0	UK
Kitchener et al13	2011	SP	22 145	354	1.6	UK
Narine and Young14	Unpublished	TP	58 973	1250	2.1	UK
Narine and Young14	Unpublished	SP	27 738	327	1.2	UK
Nance17	2007	TP	88 575	567	0.6	USA
Nance17	2007	SP	92 875	158	0.2	USA

SP, SurePath; TP, ThinPrep.

Figure 2

Meta-analysis of the association between rate of unsatisfactory cervical smears and processing platform (ThinPrep and SurePath).

Because of the small number of studies included in the meta-analysis, a funnel plot was not informative. However, the four included showed similar unsatisfactory rates to other studies and therefore we felt were representative of the other studies retrieved.

Discussion

Despite being one of the most successful screening tests for cancer, cervical cytology smears have been criticised for low sensitivity. False-negative cervical cytology smears are responsible for this decrease in sensitivity and may result from collection errors, screening errors or interpretation errors.

We set out to examine unsatisfactory rates in LBC as these have been demonstrated to be largely reproducible within laboratories but not across laboratories.43 Most laboratories use TBS or some modification thereof for interpretation of cervical cytology smears. We could not, however, establish the criteria used to determine adequacy for each platform; some laboratories use the methods described in the manufacturer's guide (TBS based), while others have adopted their own criteria and many authors do not elaborate on this in their methods. Furthermore, our results showed that year of publication and to a greater extent country were significant confounding factors in the comparison of unsatisfactory rates between ThinPrep and SurePath platforms. These confounding factors resulted in very low power to detect a statistically significant difference in unsatisfactory rates, despite a wide difference in mean unsatisfactory rates between the two platforms. In contrast, a meta-analysis comparing four head-to-head studies did reveal a lower unsatisfactory rate for the SurePath platform. These differences underscore the fact that multiple factors are associated with overall unsatisfactory rates with the platform used representing only one of these.

It is also acknowledged that there are other possible biases in the source literature that our analyses could not control for. These include the relative experience of the cytologists in reading slides prepared using one or the other platform and any possible undisclosed biases in the allocation of individuals to treatment groups in the four head-to-head studies. The differences in unsatisfactory rates due to LBC platform we report here are most likely a function of differences in the proprietary methodology for each platform. ThinPrep employs a filter-based technology, while SurePath uses a density/sedimentation process. Differences are also encountered when collecting the sample; ThinPrep requires the collector to rinse the collection device in the liquid media that is followed by disposal of the collection device. In contrast, the SurePath collection device is placed in the media and sent to the laboratory for processing. This difference has been demonstrated to account for up to 38% loss of the ThinPrep sample in a study by Bigras et al.44 These preparatory differences represent a significant technical difference, which laboratories must consider prior to adopting LBC. However, counterbalancing this loss of sample is the fact that ThinPrep samples are significantly less labour intensive than SurePath samples for the processing laboratory.

As tables 1 and 2 show, there was considerable variation in unsatisfactory rates among jurisdictions, with countries in the European Union in particular tending to show higher unsatisfactory rates in both studies of SurePath and ThinPrep. This is likely attributable to variations in adequacy criteria compared with TBS.

Both the SurePath (FocalPoint GS) and ThinPrep (ThinPrep Imager) systems include optional computer-assisted digital image analysis software. If we examine only studies using this technology, three studies using the ThinPrep Imager all gave unsatisfactory rates of 1.5%21 41 42—similar to other ThinPrep studies included in this review. The single study evaluating the FocalPoint GS system showed an unsatisfactory rate of 0.2%,19 again similar to the other SurePath studies reviewed.

Reprocessing of unsatisfactory ThinPrep slides using glacial acetic acid can decrease the unsatisfactory rates by 30%–40%.45 However, use of this technique still would not result in equivalency with SurePath unsatisfactory rates and introduces additional issues including a higher false-positive rate46 and interferes with the ability to perform hybrid capture human papilloma virus testing.47

The data presented here show a significant difference between the different platforms with SurePath having a significantly lower rate of unsatisfactory samples. This pattern was observed when considering different countries and methods used to determine thresholds for unsatisfactory designation. Consistent with this observation are data from a College of American Pathologist survey reporting differences in unsatisfactory rates between laboratories using SurePath (median unsatisfactory rate 0.3%) and ThinPrep (median unsatisfactory rate 1.1%) platforms.48 These numbers are similar to the pooled unsatisfactory rates we report in this study (0.3% for SurePath and 1.3% for ThinPrep).

A limitation of our study is that it considers only one aspect of LBP platform selection (unsatisfactory rate), whereas other factors such as specificity and sensitivity must also be considered. A discussion of this is outside the scope of this paper but interested readers are referred to the meta-analysis of Arbyn et al4 for further information. The implication of this unsatisfactory rate difference is also a consideration for specimens, which could be biobanked for further studies to evaluate new biomarkers and how they may perform in unsatisfactory LBC samples.49

Unsatisfactory samples represent a missed opportunity for screening50 and are more often associated with a cervical abnormality.51 It has also been well described that patients with an invasive cervical cancer have a much greater rate of unsatisfactory Pap smears, often as a result of scant cellularity and obscuring elements such as blood and inflammation.52–54 In addition to the technical requirements of sample preparation, laboratories may wish to consider the variation in unsatisfactory sample rates when choosing an LBC platform.