Test execution variation in peritoneal lavage cytology could be related to poor diagnostic accuracy and stage migration in patients with gastric cancer.

PURPOSE: Peritoneal lavage cytology is part of the routine staging workup for patients with advanced gastric cancer. However, no quality assurance study has been conducted to show variations or biases in peritoneal lavage cytology results. The aim of this study was to demonstrate a test execution variation in peritoneal lavage cytology between investigating surgeons.
MATERIALS AND METHODS: A prospective cohort study was designed for determination of the positive rate of peritoneal lavage cytology using a liquid-based preparation method in patients with potentially curable advanced gastric cancer (cT2~4/N0~2/M0). One hundred thirty patients were enrolled and underwent laparotomy, peritoneal lavage cytology, and standard gastrectomy, which were performed by 3 investigating surgeons. Data were analyzed using the chi-square test and a logistic regression model.
RESULTS: The overall positive peritoneal cytology rate was 10.0%. Subgroup positive rates were 5.3% in pT1 cancer, 2.0% in pT2/3 cancer, 11.1% in pT4a cancer, and 71.4% in pT4b cancer. In univariate analysis, positive peritoneal cytology showed significant correlation with pT stage, lymphatic invasion, vascular invasion, ascites, and the investigating surgeon. We found the positive rate to be 2.1% for surgeon A, 10.2% for surgeon B, and 20.6% for surgeon C (P=0.024). Multivariate analysis identified pT stage, ascites, and the investigating surgeon to be significant risk factors for positive peritoneal cytology.
CONCLUSIONS: The peritoneal lavage cytology results were significantly affected by the investigating surgeon, providing strong evidence of test execution variation that could be related to poor diagnostic accuracy and stage migration in patients with advanced gastric cancer.

Introduction

The presence of free cancer cells in the peritoneal cavity of patients with gastric cancer, demonstrated by positive peritoneal cytology (CY1), indicates an incurable condition and is the strongest risk factor for peritoneal gastric cancer recurrence.1-3 Peritoneal lavage cytology (PLC) during laparotomy was introduced by Japanese surgeons in the early 1970s as a part of the routine staging workup for patients with gastric cancer, and its clinical significance has been studied by several Japanese researchers.1-6 According to the Japanese Gastric Cancer Association staging system established in 1998, CY1 was considered to indicate metastatic disease. This diagnostic tool was also introduced in the American Joint Committee on Cancer (AJCC) staging system in 2010.7,8 PLC is now a routine staging procedure used worldwide for the treatment of patients with gastric cancer, and also a decisive basis for development of a treatment plan.9-16 Typically, prior to its general application, a new test should be validated by comparing with a gold standard;17-20 however, no reliable validation study of PLC has been conducted, probably because there was no gold standard test for comparison.21,22 Furthermore, to the best of our knowledge, no quality assurance or control study has been conducted to confirm the reproducibility or variability of PLC results thus far.

The first and most important step of PLC is sample collection. Inappropriate or inconsistent sampling procedures may provide faulty specimens that could lead to misinterpretations.22-27 The aim of this study was to demonstrate test execution variation between investigating surgeons during sampling for PLC in patients with gastric cancer.

Materials and Methods

1. Patients

A prospective cohort study investigating the positive rate of PLC using a liquid-based cytology preparation method was conducted in patients with advanced gastric cancer (AGC) in a single institute in Korea. Patients who met all of the following criteria were included: presence of biopsy-proven primary gastric adenocarcinoma, clinical T2~4/N0~2/M0 in the preoperative TNM evaluation and adequate organ function for major upper abdominal surgery as indicated by an American Society of Anesthesiologists score of <3. Exclusion criteria were as follows: synchronous malignancy in the abdomen, peritoneal dissemination (P1), hepatic metastasis (H1), extensive lymph node metastasis (N3), bulky N2 lymph node metastasis, para-aortic lymph node metastasis, or other distant metastasis on preoperative or intraoperative evaluation, and unresectable gastric cancer invading adjacent organs. A total of 132 patients were enrolled between October 2008 and September 2009 at Korea Cancer Center Hospital; however, 2 patients were excluded because of peritoneal dissemination on intraoperative evaluation. All patients enrolled in the study voluntarily agreed to participate and provided written informed consent.

2. Peritoneal lavage and surgical procedure

After laparotomy, the peritoneal cavity was thoroughly examined for synchronous malignancy, peritoneal dissemination, hepaticmetastasis, para-aortic/bulky N2 lymph node metastasis, other distant metastasis, and tumor resectability. In addition, gross serosal invasion of the primary tumor and ascites was evaluated. In cases involving a suspicious metastatic lesion, a biopsy sample was obtained and a frozen section biopsy was performed. If any exclusion criteria were observed during intraoperative evaluation, the cases were excluded from the study.

The peritoneal lavage procedure for sample collection was as follows: (i) 100 ml of sterile saline was poured around the tumor of the stomach; (ii) 100 ml of sterile saline was poured into the Douglas pouch; (iii) the bowels and peritoneal fluid were gently stirred for approximately 30 seconds; (iv) lavage was aspirated from the dependent area around the tumor, including the left subphrenic space and from the Douglas pouch, in that order; (v) peritoneal lavage samples were labeled and sent immediately to the pathology department. All patients subsequently underwent standard gastrectomy with D2 lymph node dissection, which was performed by 1 of the 3 investigating gastric surgeons.

3. Liquid-based cytology

All peritoneal lavage specimens were prepared by a single cytotechnologist using the ThinPrep (Cytic Corporation, Boxborough, MA, USA) liquid-based cytology preparation system at the pathology laboratory.28 The sample was centrifuged at 600 g for 10 minutes, and the supernatant was poured off carefully. The cell pellet was re-suspended and washed with 30 ml CytoLyt solution. The specimen was added to a PreservCyt (Cytic Corporation, Malborough, MA, USA) Solution Vial and allowed to stand for 15 minutes. The vial was then placed in a Cytic ThinPrep 2000 processor utilizing a computerized process and patented membrane technology for dispersion control, collection, and transfer of diagnostic cells from the sample to a 20-mm circular area on a glass slide. The slide was fixed in 95% ethanol and stained by Papanicolaou and diastase-periodic acid-Schiff staining methods.29-31

All ThinPrep slides were reviewed and diagnosed by an experienced pathologist with a specialization in gastrointestinal oncology. Cytological diagnosis was graded according to the Papanicolaou classification. Patients whose cytology specimens were strongly suggestive of malignancy (class IV) or consistent with malignancy (class V) were included in the positive cytology group.2,3,32,33

4. Data and statistical analysis

We anticipated CY1 rates of 10% for T2/3 cancer and 30% for T4a/4b cancer on the basis of the results from previous retrospective studies.34,35 Patients with T2/3 and T4a/4b cancer were expected to be evenly enrolled in the study; therefore, a total sample size of 126 was calculated.36 To compare clinicopathological variables according to the PLC result and investigating surgeon, the chi-square test and Fisher exact test were used for categorical variables, and the Student t-test was used for continuous variables. A binary logistic regression model with the enter method was used to determine risk factors for CY1. P-values less than 0.05 were considered statistically significant. Statistical analysis was performed using SPSS statistical software ver. 14.0 for Windows (SPSS Inc., Chicago, IL, USA).

Results

The clinicopathological characteristics of the 130 patients are summarized in Table 1. The distribution of stage pT1, pT2/3, pT4a, and pT4b was 14.6%, 38.5%, 41.5%, and 5.4%, respectively. Further, the proportion of patients with pT1 and pT2/3 cancer was 53.1%, accounting for approximately half of the total sample size.

Table 1

Characteristics of the gastric cancer patients (n=130)

Values are presented as median (range), number (%), or mean±standard deviation. *T and N stage were based on the 7th edition of the American Joint Committee on Cancer cancer staging system. †Cytological diagnosis was graded according to the Papanicolaou classification, and class IV and V were regarded as positive cytology.

Overall CY1 rate using the liquid-based cytology preparation method was 10.0% for the 130 patients with potentially curable AGC (cT2~4/N0~2/M0), which is lower than our expected rate. The CY1 rates of pT1/2/3 and pT4a/4b cancer were 2.9% and 18.0% (P=0.004). There was a close correlation between CY1 and T stage (P<0.001). Univariate analysis also identified lymphatic invasion, vascular invasion, ascites, and the investigating surgeon to be correlated with CY1 (Table 2). By logistic regression analysis, pT stage, ascites, and the investigating surgeon were significantly associated with CY1 (Table 3). The remarkably high odds ratios for ascites and investigating surgeon appear to be overestimated, because the logistic regression model tends to systematically overestimate odds ratios in small samples.36 Both pT stage and malignant ascites are well-known risk factors for CY1; however, investigating surgeon as a risk factor is an unexpected result.

Table 2

Characteristics of gastric cancer that correlated with cytological findings

Values are presented as number (%) or number only. CY0: negative peritoneal cytology, CY1: positive peritoneal cytology. *T and N stage were based on the 7th edition of the American Joint Committee on Cancer cancer staging system.

Table 3

Logistic regression analysis of risk factors for positive cytology in gastric cancer

Values are presented as odds ratio (95% confidence interval). *T and N stage were based on the 7th edition of the American Joint Committee on Cancer cancer staging system.

As shown in Table 4, no significant differences in characteristics were observed between the 3 groups of patients with regards to the investigating surgeon. However, CY1 rates were 2.1% for surgeon A, 10.2% for surgeon B, and 20.6% for surgeon C (P=0.024). The difference in CY1 rates between surgeons A and C is quite remarkable (odds ratio, 21.37; P=0.026). Fig. 1 also shows a considerable difference in the CY1 rates of T stage subgroups between investigating surgeons. These differences are possibly attributable to inappropriate or inconsistent sampling procedures during peritoneal lavage carried out by the investigating surgeons, particularly in the surgeon A group. These results provide strong evidence of test execution variation in PLC.

Table 4

Characteristics of gastric cancer that correlated with the investigating surgeon

Values are presented as number (%) or number only. CY0: negative peritoneal cytology, CY1: positive peritoneal cytology. *T and N stage were based on the 7th edition of the American Joint Committee on Cancer cancer staging system.

Fig. 1

A remarkable difference in positive cytology rates according to T stage in patients with advanced gastric cancer was noted between surgeon groups (P=0.024). The difference suggests a variation in sample collection between investigating surgeons and it can be related to poor diagnostic accuracy and stage migration in patients with advanced gastric cancer.

All cases with CY1 are summarized in Table 5. According to the 6th edition of the AJCC cancer staging manual37 for gastric cancer staging, there were 2 cases of stage I, 2 cases of stage II, 4 cases of stage III, and 5 cases of stage IV among the patients with CY1, whereas according to the 7th edition of the AJCC cancer staging manual,8 all patients had stage IV cancer because CY1 was defined as distant metastasis. As a result, the proportion of patients with stage IV cancer in the 3 groups, with respect to the investigating surgeon, showed similar differences with the positive rates of PLC as follows: 2.1% for surgeon A, 10.2% for surgeon B, and 20.6% for surgeon C (P=0.024). In other words, a considerable number of patients with AGC may be incorrectly staged. Therefore, the test execution variation of PLC is directly related to stage migration in patients with AGC, especially owing to the revised 7th edition of the AJCC staging manual.38-40

Table 5

Case summary regarding the staging migration effect of PLC among 13 gastric cancer patients with positive cytology

PLC = peritoneal lavage cytology; PLN = positive lymph nodes; AJCC = American Joint Committee on Cancer; SM = invades submucosa; MP = invades muscularis propria; SE = serosa exposure or invades visceral peritoneum; SI = serosal invasion or invades adjacent structures. *TNM staging excluding the PLC result. †TNM staging including the PLC result. ‡Immunohistochemical staining of the cytologic specimen was positive for pancytokeratin.

Discussion

To the best of our knowledge, this is the first prospective cohort study that used a liquid-based cytology method for PLC in patients with resectable AGC of clinical stage T2~4/N0~2/M0 to demonstrate a test execution variation in PLC between investigating surgeons. We found an overall CY1 rate of 10%, which is much lower than that reported by previous retrospective studies.34,35 This value was also lower than that anticipated in our study: 10% for T2/3 and 30% for T4a/4b. One possible explanation for the low CY1 rate is the difference in the study population. This study included 19 cases (14.6%) of pT1 stage and excluded P1, M1, or unresectable cases, whereas previous PLC studies only retrospectively analyzed pT2-4 stages and included a few P1, M1, or unresectable cases. However, even after exclusion of pT1 cases from the study population, the overall CY1 rate showed only a slight increase (10.8%).

Therefore, the investigators reanalyzed risk factors of CY1 by including the investigating surgeon variable, which was newly documented as an independent risk factor of CY1 in addition to T stage and malignant ascites.2,3,33,41 The differences in CY1 rates between investigating surgeons was quite remarkable. The CY1 rate of the surgeon C group was almost the same as that of the estimated CY1 rate in this study, whereas the CY1 rate of the surgeon A group was very low and was significantly lower than that of the surgeon C group. There were no differences in other components of the study method (study period, inclusion criteria, exclusion criteria, PLC protocol, cytology preparation method, cytotechnologist, and pathologist), except for the investigating surgeon; therefore, we concluded that the difference resulted from improper or inconsistent technique in PLC sampling by the investigating surgeons,26,27 and that the results showed evidence of test execution variation in PLC between investigating surgeons.22

Further, this study is the first to document significant variations in test execution during PLC sample collection between investigating surgeons; however, this study did not use a typical statistical method, such as kappa statistics, to measure the agreement between observers because the results of repeated PLC samples from the same patient with gastric cancer cannot guarantee an identical result.23,24,42 It also suggests other sources of bias and variation, which can be related to poor diagnostic accuracy in patients with AGC.

In principle, prior to its clinical application, new diagnostic tests should be validated in comparison with a gold or reference standard and evaluated in terms of bias or variation.17 Consequently, the apparent performance of a poor test may increase while obscuring the performance of a good test.27,43 On the other hand, variability indicates the scope or amplitude of probability that an index test may not consistently yield the same result when repeated, and it mostly arises from differences in population, setting, test protocol, observer, or definition of the target disease among individual diagnostic accuracy studies. Accordingly, diagnostic tests with high variability commonly show a correlation with imprecision, poor reproducibility, and low reliability.17 22,23

PLC is now a routine diagnostic test in staging workup and is helpful in therapeutic decision making for patients with AGC.7,8 It was introduced in the early 1970s; however, only a few studies have aimed at validating the performance of PLC, and no study has focused on bias or variation of PLC. In a review of 22 patients with T3M0 among 127 patients with gastric cancer undergoing laparoscopic PLC, Burke et al.44 reported on the performance measurements of PLC for the first time: 40% sensitivity, 93% specificity, and 68% accuracy. Bando et al.2 reported that PLC performance was 91%, with 56% sensitivity and 97% specificity in patients with AGC. These 2 studies established the reference standard for PLC as peritoneal recurrence, whereas Kodera et al.45 defined the reference standard for PLC as either synchronous peritoneal dissemination or peritoneal recurrence within 2 years after curative resection, and the performance measurements were 56% sensitivity and 91% specificity. Therefore, even if either of the 2 above assumptions is chosen as the reference standard for PLC, the performance of PLC appears to be too low for use in clinical practice.

On the other hand, the purpose of a PLC test is not to identify minute peritoneal disseminations or to predict peritoneal recurrence of gastric cancer; therefore, in our opinion the above reference standards of PLC, such as synchronous peritoneal dissemination or peritoneal recurrence, are inappropriate, and have 'imperfect or inappropriate gold standard' bias. First, it is well known that most free cancer cells attached to peritoneal mesothelial cells cannot survive owing to the existence of a 'peritoneal-blood barrier', preventing submesothelial invasion.46,47 Second, the CY1 rate of PLC is only 43% to 78% in P1 groups.1,3,44 Third, the overall survival rates of CY1P0 (P0, no peritoneal metastasis) groups showed remarkable improvement after systemic or intraperitoneal chemotherapy,6,10,12,16,48 and gastric cancer recurrence occurred at a number of different sites besides the peritoneal cavity in CY1P0 patients.12,35,49 Finally, the paradoxical evidence supporting our opinion is that the overall survival of the CY1P1 group is significantly worse than that of the CY0P1 (CY0, negative peritoneal cytology) group,2,35 which indicates that CY1 itself had an independent prognostic influence apart from the influence of gross peritoneal dissemination. Therefore, according to this evidence, the assumption by previous investigators, that free cancer cells in the peritoneal cavity show exclusive progression to gross peritoneal dissemination, is difficult to support.2,44,45 Thus, it is more reasonable to conclude that we have no gold standard test for comparison with PLC.

On the contrary, PLC itself should be established as a gold standard test. To achieve this goal, well-designed and unbiased quality assurance studies as well as efforts to minimize variability between investigators are required to improve the accuracy and precision of PLC.24 Therefore, we selected and summarized 10 large-scale studies of PLC so as to re-evaluate them in terms of bias and variation (Table 6, 7).1-5,33,44,50-52 At a glance, we observed major differences in overall CY1 rates, with a range of 7% to 39% between PLC studies. This large difference may mainly be caused by the different proportions of study subpopulations between PLC studies and the different study periods owing to increasing prevalence of early gastric cancer, particularly in East Asia,53,54 wherein the differences represent variations in disease severity and disease prevalence between PLC studies; in other words, spectrum bias.55 Therefore, the results of PLC were stratified by T or P staging to reduce the potential effect of spectrum bias, as shown in Table 7. Consequently, the difference in CY1 rates for the P1, T1~3, or T4 groups was relatively small. Another explanation for spectrum bias is that PLC studies, which began in the 1970s, show higher overall CY1 rates than recent PLC studies, whereas the difference of CY1 rates of P1 groups is not remarkable.

Table 6

Summary and comparison of large-scale peritoneal lavage cytology studies

CY1: positive peritoneal cytology, Class IV/V: strongly suggestive of malignancy or consistent with malignancy according to the Papanicolaou classification, P0: no peritoneal metastasis, P1: peritoneal metastasis. U = upper abdomen; D = Douglas pouch; P = Papanicolaou; G = Giemsa; PAS = periodic acid-Schiff; AB = alcian blue; DPAS = diastase-periodic acid-Schiff; CR = curative resection; NCR = non-curative resection.

Table 7

The subgroup outcomes of peritoneal lavage cytology stratified by T or P stage*

*TNM stage was based on the 7th edition of the American Joint Committee on Cancer Cancer staging system in 2010. P0: no peritoneal metastasis, P1: peritoneal metastasis, CY1: positive peritoneal cytology. N/A = not available; CR = curative resection.

Considerable differences were observed in approach route, lavage site, preparation method, and staining method used in individual PLC studies, which may be correlated with test execution variation. Furthermore, the classification system and category for CY1 were different between the 10 PLC studies. In half of these studies, cytological findings were classified as positive or negative. Papanicolaou classification was used in 3 studies, and only the clustered form of malignant cells was regarded to indicate CY1 in 2 studies. Therefore, this difference implies that the arbitrary choice of threshold value, a kind of variation, may have been introduced in PLC studies in order to maximize the sensitivity and specificity of the test. Thus, several sources of variation and bias can be found in these PLC studies.22 However, no quality assurance or control study to minimize variation and bias in PLC results has been conducted until now.

Since the revised 7th edition of the AJCC staging manual has considered CY1 to indicate metastatic disease, i.e. stage IV, the diagnostic accuracy of PLC has become more important than before, because it is not only directly related to stage migration, but is also a decisive basis for the treatment plan for patients with gastric cancer. The diagnostic accuracy of PLC and its stage migration effect are the most problematic for T4a/b staging in a population with no other metastatic lesions. For example, in false CY1 cases, patients may lose the opportunity to undergo curative resection, whereas, in false CY0 cases, patients may lose an opportunity for appropriate treatment options.12,15,16,33 Therefore, considering potential variation and bias in PLC, special attention should be paid to the development of a treatment strategy for patients with gastric cancer. Significant efforts to improve the accuracy and precision of PLC are necessary. In addition, new techniques, such as a liquid-based preparation method, should be evaluated in future studies. The liquid-based cytology preparation method for examination of specimen slides under the microscope is quick and easy, and it provides fewer unsatisfactory specimens and residual samples, which can be used for further confirmatory tests or other purposes. However, it is not known to demonstrate better performance than conventional Papanicolaou tests in cervical cytology screening.56-58

In conclusion, PLC is a valuable diagnostic test for detecting free cancer cells in the peritoneal cavity of patients with gastric cancer; however, the accuracy and precision of PLC has not been established owing to the lack of a reference standard and quality assurance or control studies. Findings from the present study indicate a variation in test execution during PLC sample collection. Further, several sources of bias and variation in PLC studies have been recognized by review of the methods and results of representative PLC studies. Consequently, until now, the generalization of the results of individual PLC studies to clinical practice worldwide has been difficult. Therefore, development and establishment of a consensus PLC protocol, including a sampling method and well-designed quality assurance studies are required to support the reproducibility and reliability of PLC.