DNA hypermethylation analysis in sputum for the diagnosis of lung cancer: training validation set approach.

BACKGROUND: Lung cancer has the highest mortality of all cancers. The aim of this study was to examine DNA hypermethylation in sputum and validate its diagnostic accuracy for lung cancer.
METHODS: DNA hypermethylation of RASSF1A, APC, cytoglobin, 3OST2, PRDM14, FAM19A4 and PHACTR3 was analysed in sputum samples from symptomatic lung cancer patients and controls (learning set: 73 cases, 86 controls; validation set: 159 cases, 154 controls) by quantitative methylation-specific PCR. Three statistical models were used: (i) cutoff based on Youden's J index, (ii) cutoff based on fixed specificity per marker of 96% and (iii) risk classification of post-test probabilities.
RESULTS: In the learning set, approach (i) showed that RASSF1A was best able to distinguish cases from controls (sensitivity 42.5%, specificity 96.5%). RASSF1A, 3OST2 and PRDM14 combined demonstrated a sensitivity of 82.2% with a specificity of 66.3%. Approach (ii) yielded a combination rule of RASSF1A, 3OST2 and PHACTR3 (sensitivity 67.1%, specificity 89.5%). The risk model (approach iii) distributed the cases over all risk categories. All methods displayed similar and consistent results in the validation set.
CONCLUSIONS: Our findings underscore the impact of DNA methylation markers in symptomatic lung cancer diagnosis. RASSF1A is validated as diagnostic marker in lung cancer.

Lung cancer has the highest mortality rate of all cancers, because of the presence of metastases at time of presentation (Siegel ). Since the 1970s, the average overall five-year survival rate hovers at 15%, despite new insights in therapeutic strategies (Siegel ). For late-stage disease, treatment options remain limited and of palliative intent. However, prognosis improves considerably when lung cancer is detected at stage I or II, where patients are treated with curative intent (Patz ).

Currently, lung cancer is detected and staged by imaging techniques. Ideally, the diagnosis is pathologically confirmed. Therefore, tumour tissue needs to be obtained through invasive methods, such as bronchoscopy or transthoracic needle aspiration. In daily practice, this is not always possible, due to for instance, localisation of the tumour or physical stress for the patient.

Thus, there is need for a novel diagnostic method. The use of sputum is of interest, as procurement is non-invasive, inexpensive and simple. Tumour cells and tumour DNA are shed in the respiratory epithelial lining fluid and usually make up <1% of sputum composition. Sputum cytology has a low sensitivity of 66% (range 42–97%) for lung cancer diagnosis (Rivera ). More promising is the application of molecular techniques that are able to detect minimal amounts of aberrant tumour DNA in sputum (Honorio ; Shivapurkar ).

DNA promoter hypermethylation of tumour-suppressor genes leads to transcriptional silencing (Esteller, 2011). Previous research has shown that various genes are hypermethylated in lung cancer patients as opposed to controls and can be detected in sputum (Belinsky ; Cirincione ; Shivapurkar ). In a preliminary study, we investigated DNA hypermethylation of biomarkers RASSF1A, APC and cytoglobin (CYGB) in sputum of lung cancer patients (Hubers ). A novel classification system for lung cancer prediction was introduced, which proved to be reproducible in two independent sets of subjects. In particular, hypermethylated RASSF1A demonstrated to have potential as a diagnostic marker.

Here, we report on an independent validation of these and additional novel discovered biomarkers (Shivapurkar ; Steenbergen ) in an external cohort of prospectively collected sputum obtained from lung cancer patients and cancer-free controls. In addition, the diagnostic value of the molecular sputum analysis was compared with sputum cytology.

Methods

Subjects were included between June 2009 and February 2013 by pulmonologists in the regions of Amsterdam and Nieuwegein, the Netherlands (Figure 1). Cases were patients diagnosed with lung cancer. Their sputum was collected before lung cancer treatment, or when patients showed lung cancer progression while on treatment. Staging was performed according to the 7th edition of UICC TNM system (Sobin and Gospodarowicz, 2009). Controls were cancer-free subjects, mainly diagnosed with chronic obstructive disease (COPD), classified according to the GOLD criteria (Gold, 2009; Table 1). Patients who were cancer-free for a period of at least 3 years after curative treatment for lung cancer were also considered as controls. From the initially included controls (without symptoms at time of sputum collection), six patients developed lung cancer within a period of 6 months and were placed in the lung cancer group at time of analyses. Controls who developed lung cancer >6 months after sputum collection (n=14) were excluded from the main analyses and analysed separately.

Figure 1

Enrolment and follow-up of study subjects.

Table 1

Sociodemographic characteristics of subjects in learning and validation set

	Learning set					Validation set
	Lung cancer (n=73)		Control (n=86)			Lung cancer (n=159)		Control (n=154)
	n	%	n	%	P-value	n	%	n	%	P-value	P-valuea
Sex
Male	49	67.1%	55	64.0%	0.68	107	67.3%	103	66.9%	0.94	0.98
Female	24	32.9%	31	36.0%		52	32.7%	51	33.1%
Age
Mean (±s.d.)	66.0 (±9.6)	68.7 (±10.0)	0.07	64.6 (±10.5)	67.4 (±9.8)	0.011	0.29
COPD statusb
I	10	17.2%	18	22.0%	0.009	30	22.9%	34	23.0%	<0.001	0.23
II	20	34.5%	22	26.8%		46	35.1%	48	32.4%
III	11	19.0%	25	30.5%		10	7.6%	42	28.4%
IV	2	3.4%	11	13.4%		2	1.5%	10	6.8%
COPD, GOLD unknown	5	8.6%	0	0.0%		11	8.4%	4	2.7%
No COPD	10	17.2%	6	7.3%		32	24.4%	10	6.8%
Smoking status
Current	23	31.9%	24	27.9%	0.57	53	33.8%	57	37.5%	0.77	0.94
Former	44	61.1%	52	60.5%		92	58.6%	83	54.6%
Never	5	6.9%	10	11.6%		12	7.6%	12	7.8%
Pack years
Median (IQR)	40.5 (24.8–52.8)		36.5 (21.5–50.0)		0.32	40.0 (22.8–53.0)		31.0 (20.0–47.8)		0.014	0.81
Stagec
IA	9	12.3%				24	15.1%				0.33
IB	5	6.8%				5	3.1%
IIA	6	8.2%				7	4.4%
IIB	3	4.1%				10	6.3%
IIIA	13	17.8%				34	21.4%
IIIB	11	15.1%				13	8.2%
IV	25	34.2%				66	41.5%
Unknown	1	1.4%				0	0.0%
Diagnosis
SCC	31	42.5%				50	31.4%				0.50
AC	26	35.6%				66	41.5%
NSCLC NOS	7	11.0%				21	13.2%
SCLC	1	1.4%				6	3.8%
Other	8	9.6%				16	10.1%

Abbreviations: COPD=chronic obstructive pulmonary diseases; GOLD=Global Initiative for Chronic Obstructive Lung Disease; IQR=interquartile range; SCC=squamous cell carcinoma; AC=adenocarcinoma; NSCL=non-small-cell lung cancer; SCLC=small-cell lung cancer; C NOS=cancer not other specified.

Comparison between lung cancer patients of both sets.

According to GOLD classification (Gold, 2009).

Staging was conform 7th edition of TNM criteria (Sobin and Gospodarowicz, 2009).

The study was approved by the institutional review boards of the participating hospitals. All subjects provided signed informed consent. Oral and written information was provided to all subjects. Sociodemographic details and smoking habits were assessed by a questionnaire, completed by the subjects (Wood ; Field ). Clinical data were retrieved from medical records, blinded to outcome of methylation analysis.

Collection, recoding and processing of sputum with dithiothreitol, DNA isolation and hypermethylation analysis were performed as described before (Hubers ). Based upon previous research of our group, DNA hypermethylation of the promoter regions of the following biomarkers were tested using multiplex quantitative methylation-specific PCRs: RASSF1A, CYGB, APC (Shivapurkar ; Hubers ) and recently discovered PRDM14, FAM19A4 and PHACTR3 (Snellenberg ; Steenbergen ). 3OST2 was tested in a singleplex quantitative methylation-specific PCR assay (Shivapurkar ). Samples were tested in a blinded manner.

Cytological analysis

Following dithiothreitol processing, 0.2 ml of each sample was used for cytological analysis. Single layer slides were prepared using the Hettich Cyto-System. Cytological analysis was blinded for molecular analysis results and case–control status. Sputum cytology was scored for the following parameters: cell abundance; amount of neutrophilic granulocytes; cellular debris; squamous and/or cylinder cells; squamous metaplasia; and (suspicious) cancer cells. Sputum samples were considered representative for the respiratory tract if alveolar macrophages or respiratory epithelial cells were present. Cytology was defined ‘positive' when cancer cells or cells suspicious for cancer (atypia) were identified.

Data and statistical analysis

A sputum bank was composed from the prospectively collected sputum samples. Only the first sputum canister (days 1–3) of subjects on which all biomarkers and cytology were assessed was included for analyses. An independent learning and validation set were randomly assigned from the sputum bank using a 1:2 ratio of both cases and controls, respectively (Figure 1).

To evaluate the diagnostic value of methylation for lung cancer in sputum, three different approaches were used, as described previously (Hubers ). Receiver operating characteristic curves were composed with the ratio values of each marker. The second statistical model is based on a recent review (Hubers ) that assesses the ‘diagnostic' value of a biomarker (that is, minimal number of false-positive test results). In this review, we developed a rationale to determine the true diagnostic capacity of the methylation markers, substantiating that undiagnosed lung cancer is present in maximally 4% of the control population (based on combination of prevalence, risk and time). This resulted in a threshold setting for all markers at a fixed specificity per marker of 96%. Positive and negative predictive values (PPV and NPV, respectively) and diagnostic odds ratios (DORs) of markers were calculated for both the first and second approach with 95% confidence intervals (95% CI). In addition, multivariate logistic regression (markers as categorical variable) with a forward selection procedure was performed with the biomarkers in the learning set, leading to a combination rule with the highest sensitivity. This combination of markers with same thresholds was subsequently tested in the validation set. Biomarkers with a P-value ≤0.05 entered the logistic regression model. Results between the two sets were compared with the χ2 test or Fisher's exact test. Chi-square tests were used to examine differences in DNA hypermethylation frequency between COPD patients without lung cancer and lung cancer patients without COPD. Moreover, to investigate whether COPD could be a possible confounder in the association between methylation and lung cancer, stratified DNA hypermethylation analysis and COPD-corrected logistic regression analysis were performed.

Furthermore, the complementary effect of DNA hypermethylation to cytology for lung cancer diagnosis was evaluated for the whole set with the McNemar test. To assess the additive value of sampling sputum during a prolonged time of 4–9 days, cumulative hypermethylation analysis was performed as described before using the cutoff obtained by Youden's J index (Hubers ). To examine the learning effect in time over canisters I to III, generalised estimating equations were used for each biomarker. Repeated measures for each subject were defined as the outcome of biomarker (positive or negative; using the cutoff obtained via the first approach) of one to three different canisters. An exchangeable structure was chosen for the correlation matrix, the logit-function was used as link function between the true status (case or control) of the subject and the outcome of biomarker, the number of the canister and their two-way interaction.

All statistical tests were two-sided with a significance level at 0.05 (P≤0.05). SPSS version 20.0 was used (IBM Corp., Armonk, NY, USA).

Results

Characteristics of subjects

Figure 1 shows the enrolment and follow-up of study subjects. Of 472 subjects, information on DNA hypermethylation analysis and cytology was available from the first sputum canister. Samples were randomised in a learning set (n=73 cases, n=86 controls) and validation set (n=159 cases, n=154 controls). Median duration of follow-up was 23 months in controls (range 0–43) and 8 months in cases (range 0–43).

Sociodemographic and clinical characteristics of cases, with sputum collection <6 months before diagnosis and controls in learning and validation sets are described in Table 1. In the validation set, mean age of controls was higher than of cases, and controls had smoked less pack years (P=0.011 and P=0.014, respectively). COPD was more prevalent in controls in both learning and validation sets (P=0.009 and P<0.001, respectively). Similar distributions were observed between the sets for the other variables.

Fifty per cent of lung cancer patients were diagnosed with stage IIIB and IV lung cancer. Adenocarcinoma (40%) and squamous cell carcinoma (35%) were the most prevalent histological types.

Cytology

Cytological analysis was performed for all subjects of learning and validation set combined, showing a positive result in 13.8% (95% CI: 9.6%–18.9%) of lung cancer cases with a specificity of 99.6% (95% CI: 97.7%–99.99%). Sensitivity marginally improved when cases had collected sputum during 9 days; 10 of 169 lung cancer patients, who had a negative cytology result in canister I, were detected in either canister II or III (5.9% 95% CI: 2.9%–10.6%).

DNA hypermethylation analysis

Approach (i): discrimination capability of biomarkers between lung cancer patients and controls

DNA hypermethylation analysis of RASSF1A, APC, CYGB, 3OST2, PHACTR3, FAM19A4 and PRDM14 was performed for all samples in learning and validation sets. Receiver operating characteristic curves for each marker are shown in Figure 2, for learning and validation set, respectively. Cutoff values were calculated based on Youden's J index. Univariate analyses with 95% CIs of all biomarkers in both learning and validation sets are shown in Table 2. Regarding high specificity, RASSF1A showed the best diagnostic performance in both learning and validation sets (sensitivity and specificity were 42.5% and 96.5%, 36.5% and 88.3%, respectively). PPV was 91.2% (95% CI: 76.3%–98.1% Supplementary Table 1a), NPV was 66.4% (95% CI: 57.4%–74.6%) and DOR was 20.4 (95% CI: 5.9–70.7). The combination rule of biomarkers RASSF1A, 3OST2 and PRDM14 was selected by multivariate logistic regression from the learning set for independent evaluation in the validation set. Both 3OST2 and PRDM14 showed individual high AUC scores (Table 2) with comparable results in the validation set. Positive DNA hypermethylation in one or more of these three markers demonstrated a sensitivity for lung cancer diagnosis of 82.2% (95% CI: 71.5%–90.2%) with a specificity of 66.3% (95% CI: 55.3%–76.1%) in the learning set. Similar results were observed for this panel in the validation set: sensitivity of 79.2% (95% CI: 72.1%–85.3% P=0.60) and specificity of 64.3% (95% CI: 56.2%–71.8% P=0.76). Diagnostic efficiency of the biomarker panel remained more or less similar with the addition of cytology (sensitivity of 83.6% in learning set). Molecular sputum analysis is superior over sputum cytology (P<0.001).

Figure 2

Receiver operator characteristic (ROC) curves were composed with the ratio values of markers RASSF1A, APC, CYGB, FAM19A4, 3OST2, PHACTR3 and PRDM14 for (A) learning set and (B) validation set. The true positive rate (sensitivity) is plotted against the false-positive rate (1-specificity) for the different possible cutoff values.

Table 2

DNA hypermethylation markers evaluated as binary marker (positive or negative) based on two statistical approaches (Youden's J index and fixed specificity) with different threshold setting on learning set (A) and subsequent evaluation on validation set (B)

	AUC	95% CI	Sensitivity (%)	95% CI (%)	Specificity (%)	95% CI	P-values
A. Learning set
Cutoff based on Youden's J index
RASSF1A	0.69	0.60–0.77	42.5	31.0–54.6	96.5	90.1–99.3	<0.001
APC	0.64	0.55–0.73	52.1	40.0–63.9	70.9	60.1–80.2	0.003
CYGB	0.65	0.56–0.74	56.2	44.1–67.8	74.4	63.9–83.2	<0.001
3OST2	0.72	0.64–0.80	50.7	38.7–62.6	86.0	76.9–92.6	<0.001
PRDM14	0.71	0.63–0.79	60.3	48.1–71.5	76.7	66.4–85.2	<0.001
FAM19A4	0.59	0.50–0.68	86.3	76.2–93.2	29.1	19.8–39.9	0.02
PHACTR3	0.69	0.61–0.77	57.5	45.4–69.0	77.9	67.7–86.1	<0.001
RASSF1A, 3OST2 and PRDM14			82.2	71.5–90.2	66.3	55.3–76.1	<0.001
Cutoff based on fixed specificity of >96% in learning set
RASSF1A			42.5	31.0–54.6	96.5	90.1–99.3	<0.001
APC			16.4	8.8–27.0	96.5	90.1–99.3	0.005
CYGB			19.2	10.9–30.1	96.5	90.1–99.3	0.001
3OST2			31.5	21.1–43.4	96.5	90.1–99.3	<0.001
PRDM14			17.8	9.8–28.5	96.5	90.1–99.3	0.003
FAM19A4			15.1	7.8–25.4	96.5	90.1–99.3	0.01
PHACTR3			28.8	18.8–40.6	96.5	90.1–99.3	<0.001
RASSF1A, 3OST2 and PHACTR3			67.1	55.1–77.7	89.5	90.1–99.3	<0.001
B. Validation set
Cutoff based on Youden's J index
RASSF1A	0.67	0.61–0.73	36.5	29.0–44.5	88.3	82.2–92.9	<0.001
APC	0.63	0.57–0.69	52.2	44.1–60.2	69.5	61.676.6	<0.001
CYGB	0.64	0.58–0.70	49.7	41.7–57.7	68.2	60.2–75.4	0.001
3OST2	0.71	0.65–0.77	49.7	41.7–57.7	85.1	78.4–90.3	<0.001
PRDM14	0.75	0.69–0.80	64.8	56.8–72.2	74.0	66.4–80.8	<0.001
FAM19A4	0.66	0.59–0.72	77.4	70.1–83.6	22.1	15.8–29.5	0.90
PHACTR3	0.67	0.61–0.73	60.4	52.3–68.0	62.3	54.2–70.0	<0.001
RASSF1A, 3OST2 and PRDM14			79.2	72.1–85.3	64.3	56.2–71.8	<0.001
Cutoff based on fixed specificity of >96% in learning set
RASSF1A			36.5	29.0–44.5	88.3	82.2–92.9	<0.001
APC			22.0	15.8–29.3	96.8	92.6–98.9	<0.001
CYGB			19.5	13.6–26.5	98.1	94.4–99.6	<0.001
3OST2			34.0	26.6–41.9	96.8	92.6–98.9	<0.001
PRDM14			27.0	20.3–34.7	96.8	92.6–98.9	<0.001
FAM19A4			26.4	19.7–34.0	97.4	93.5–99.3	<0.001
PHACTR3			25.2	18.6–32.6	91.6	86.0–95.4	<0.001
RASSF1A, 3OST2 and PHACTR3			64.8	56.8–72.2	80.5	73.4–86.5	<0.001

Abbreviations: AUC=area under the curve; 95% CI=95% confidence intervals.

AUC and 95% CI were calculated for the learning set. Combination rules were defined using multivariate logistic regression. P-values are given for the statistical difference between cases and controls.

No relation was observed between early (stage I–II) and advanced (stage III–IV) lung cancer and DNA hypermethylation (P-values >0.10). Regarding histology of the tumours (adenocarcinoma versus squamous cell carcinoma), PHACTR3 showed to be more hypermethylated in adenocarcinomas when compared with squamous cell carcinomas (P=0.001; Table 3). Although not significant, RASSF1A hypermethylation was more observed in squamous cell carcinomas.

Table 3

DNA hypermethylation analysis in relation to tumour histology

	Diagnosis
	SCC (n=81), %	AC (n=92), %	P-value
RASSF1A
Negative	41	59	0.056
Positive	56	44
APC
Negative	52	48	0.23
Positive	43	57
CYGB
Negative	52	48	0.15
Positive	41	59
3OST2
Negative	52	48	0.20
Positive	42	58
PRDM14
Negative	55	45	0.11
Positive	42	58
FAM19A4
Negative	45	55	0.84
Positive	47	53
PHACTR3
Negative	62	38	0.001
Positive	36	64

Abbreviations: SCC=squamous cell carcinoma; AC=adenocarcinoma.

Cutoff for positive hypermethylation is based on Youden's J index of biomarkers.

In the group of never-smokers (22 cases and 17 controls), hypermethylation of most biomarkers was comparable for sensitivity and specificity in smokers with >15 pack years (P>0.04; data not shown). RASSF1A and 3OST2 demonstrated high specificity (95% and 91%, respectively) with a sensitivity of 47% and 53%, respectively. When smokers <15 pack years were combined with never smokers similar results were obtained.

For clinical parameters, such as age and smoking status, no association was observed with DNA hypermethylation. In comparing COPD patients without lung cancer with lung cancer patients without COPD, all tested methylation markers have a (significantly) higher fraction of positive cases in lung cancer (Supplementary Table 2). To examine whether COPD is a confounding factor, cases of learning and validation sets were combined and logistic regression analysis revealed after correcting for COPD that the regression coefficient changed less than 10% for all tested methylation markers (for example from b=1.798 to 1.793 for RASSF1A), excluding COPD as confounding factor. Furthermore, analyses of the association between methylation markers and lung cancer stratified by COPD status did not reveal relevant differences (Supplementary Table 3) neither did analyses of the association between methylation markers and COPD status stratified by group (lung cancer or control; Supplementary Table 4).

Fourteen subjects presented with lung cancer more than 6 months after sputum collection. Of these, four were positive for RASSF1A hypermethylation.

Approach (ii): diagnostic value of biomarkers

The diagnostic value of the methylation markers was examined starting with a fixed 96% specificity for each marker in the learning set (Table 2). Multivariate logistic regression analysis was performed and resulted in the combination of RASSF1A, 3OST2 and PHACTR3, yielding a sensitivity of 67.1% (95% CI: 55.1%–77.7%) and specificity of 89.5% (95% CI: 90.1%–99.3%) in the learning set, versus 64.8% (95% CI: 56.8%–72.2%) and 80.5% (95% CI: 73.4%–86.5%) in the validation set, respectively. No differences were observed between both sets for sensitivity and specificity (P=0.73 and 0.07, respectively). PPV, NPV and DOR for all methylation markers are shown in Supplementary Table 1b.

Approach (iii): risk classification model

The risk classification model was composed with samples of the learning set and subsequently evaluated on the validation set. Logistic regression analysis first included RASSF1A for identification of high-risk individuals and next 3OST2 and PRDM14 for lower-risk categories in the model (Table 4).

Table 4

Risk classification model based on post-test probabilities for the presence of lung cancer

		Learning set				Validation set				P-value
		Lung cancer		Control		Lung cancer		Control		Lung cancer	Control
RASSF1A	3OST2 and PRDM14	n/73	%	n/86	%	n/159	%	n/154	%
≥60%		29	39.7	2	2.3	54	34.0	15	9.7	0.56	0.04
	40–60%	11	15.1	2	2.3	29	18.2	0	0.0
	20–40%	11	15.1	12	14.0	34	21.4	19	12.3
	0–20%	22	30.1	70	81.4	42	26.4	120	77.9

RASSF1A was included as diagnostic marker to identify high-risk individuals (≥60% chance on lung cancer), 3OST2 and PRDM14 as risk marker for lower risk groups (40–60%, 20–40% and 0–20%, respectively).

In the learning set, RASSF1A classified 39.7% of lung cancer patients in the high-risk group (≥60% chance on lung cancer) with few false-positive controls (2.3%). The risk factors 3OST2 and PRDM14 assigned half of the remaining lung cancer cases in the moderate lung cancer risk groups and 30% in the lowest risk group, whereas the majority of controls (81.4%) were allocated to the lowest risk group. Consistent results were demonstrated in the validation set with slightly more lung cancer patients in the moderate risk groups (18.2% and 21.4%, respectively), and with a marginally lower specificity for RASSF1A.

Prolonged sputum sampling

From 195 cases and 228 controls, a complete set of the three canisters (i.e., I, II and III) with sufficient DNA for hypermethylation analysis was available (Figure 1). Using cutoff values based on Youden's J index, McNemar tests and Cochran Q tests did not show statistically significant differences in frequency of hypermethylation among the three canisters, except for CYGB, which demonstrated significantly more hypermethylation in the third canister, when compared with canisters I and II (50%, 51% and 58%, respectively; P=0.03). Therefore, sputum quality is comparable among the canisters.

The number of lung cancer patients who tested negative in canister I and with positive hypermethylation in canisters II and III is shown in Table 5. Individual marker analysis showed that proportion of additional positive cases is larger for the risk markers (mean 41%), compared with diagnostic marker RASSF1A (11%). Seventeen of thirty-seven cases were detected in addition when either canister II or III was tested for hypermethylation of biomarker panel RASSF1A, 3OST2 and PRDM14 (45.9% 95% CI: 29.5%–63.1%). Generalised estimating equations did not show an interaction between canister number and the outcome of the biomarker, indicating that no learning effect for sputum sampling occurred over time.

Table 5

Additive hypermethylation analysis of biomarkers in canisters II and III from lung cancer patients who tested negative in canister I

Negative in I, but positive in sample II or III				P-value
	n/N	%	95% CI (%)	Canister	Outcome of marker	Canister * outcome
RASSF1A	14/125	11.2	6.3–18.1	0.977	<0.001	0.97
APC	32/99	32.3	23.3–42.5	0.425	<0.001	0.541
CYGB	38/100	38.0	28.5–48.3	0.132	<0.001	0.419
3OST2	26/96	27.1	18.5–37.1	0.74	<0.001	0.515
PRDM14	27/71	38.0	26.8–50.3	0.02	<0.001	0.198
FAM19A4	28/39	71.8	55.1–85.0	0.183	0.191	0.963
PHACTR3	31/81	38.3	27.7–49.7	0.504	<0.001	0.441
RASSF1A, 3OST2 and PRDM14	17/37	45.9	29.5–63.1

Abbreviation: 95% CI=95% confidence interval.

Cutoff for positive hypermethylation was based on Youden's J index of canister I samples in the learning set. 95% CI are provided. The P-values represent the results of the generalised estimating equations to investigate the learning effect for all biomarkers.

Discussion

This study reports on a training-validation approach of DNA hypermethylation analysis of biomarkers in sputum for the diagnosis of lung cancer. All tested biomarkers were able to discriminate between lung cancer patients and controls. RASSF1A showed best performance with high positive predictive value and high DORs; not only in two different and independent sets, but also using three different statistical approaches. This confirms the value of RASSF1A hypermethylation as a diagnostic marker (according to previous definition, i.e., false-positive test results in <4% of controls; Hubers ). In addition, novel biomarkers were examined, which have not been tested in sputum before (PRDM14, PHACTR3, FAM19A4; Steenbergen ). We developed and evaluated two panels, each consisting of three biomarkers. Both panels had in common the inclusion of biomarkers RASSF1A and 3OST2, and dependent of the panel application (high sensitivity or high specificity), the third biomarker of the panel was either PRDM14 or PHACTR3, respectively. In both learning and validation sets, the first combination rule showed similar sensitivity of 80–82% and specificity of 65–66% for lung cancer diagnosis. In diagnostic setting, that is, with focus on high specificity, the second panel revealed 67% sensitivity and 90% specificity. Last, a risk model for lung cancer prediction was composed, incorporating RASSF1A, 3OST2 and PRDM14. Accuracy should be improved, but this model shows potential as clinical tool for application in a population at risk for lung cancer to categorise subjects in different risk groups.

Twenty-one controls tested false positive for RASSF1A hypermethylation in their first sputum canister. Of note, one of these controls died from liver metastases of an unknown primary source three years after sputum collection. Another control presented with weight loss and fatigue, showing a non-progressing infiltrate in the right upper lobe. No further diagnostic work up was performed. In addition, five patients presented with lung metastases from a primary tumour different from lung cancer. These were separately analysed for RASSF1A hypermethylation. Two showed RASSF1A hypermethylation in their sputum: one had a primary breast tumour, the other metastatic colon carcinoma. For both cancer types, RASSF1A hypermethylation has been reported in the literature (Pfeifer and Dammann, 2005). Thus, RASSF1A hypermethylation in subjects without a primary lung cancer should not be interpreted as a false-positive test at first glance, as it is not excluded that the positivity may be due to the presence of not established lung cancer (Hubers ) or to lung metastases originating from another primary tumour.

Methylation frequencies of 3OST2 in our data (49.7–56.2%) are comparable as reported previously (Leng ), although numbers on specificity are contrasting. Leng ) observed a lower specificity, whereas Shivapurkar ) found no false-positive controls. COPD increases the risk of lung cancer among smokers and COPD patients present with similar symptoms (Adcock ). In our study, COPD was no confounding factor.

For diagnostic use of hypermethylation markers, for example, to confirm malignancy after imaging of a solid lesion or ground glass opacity, one should strive after high specificity of the markers. From this point of view, cutoff values for all methylation markers were set at a specificity of 96%, based on a rationale as previously published (Hubers ). Logistic regression yielded a novel combination of RASSF1A, 3OST2 and PHACTR3 with a sensitivity of 67% and specificity of 90%. In this scenario, sensitivity is lower. As a consequence, the high specificity leads to reduction in false-positive results.

A risk classification model was constructed, based on the approach as introduced in the previous research (Belinsky ; Zöchbauer-Müller ; Shivapurkar ; Baryshnikova ; Leng ; Van der Drift ; Stidley ; Hubers ). Instead of the conventional way of interpreting dichotomised test results of biomarkers as the ‘absolute' presence or absence of hypermethylation, this novel approach assumes that ratio values of biomarkers can be divided into categories with corresponding risk probabilities. This could be a practical tool for the clinician, both for use during the diagnostic process and for screening of lung cancer. In the current study, markers were examined in comparison with RASSF1A, APC and CYGB (which were included in the proposed risk model before) and we observed that RASSF1A, 3OST2 and PRDM14 were more accurate in predicting chance of lung cancer. The risk model showed consistent results between learning and validation sets.

Irrespective of the statistical model used, about 40% of the lung cancer patients will not be diagnosed with the current hypermethylation markers. This is explained by several reasons. First, none of the genes is hypermethylated in 100% of lung cancer, emphasising the need for complementary biomarkers. Second, previous research showed a mean concordance of 78% between matched primary hypermethylated lung cancer tissue and sputum (Hubers ), which raises a question about the representativity: not all cases with hypermethylation in the primary tumour show detectable DNA methylation in their sputum samples.

Interestingly, we observed a high specificity of RASSF1A and 3OST2 in never-smokers as well as in smokers with limited number of pack years (<15). This may suggest to examine these markers in other subjects than those meeting the current inclusion criteria for lung cancer screening (i.e., heavy smokers).

When compared with sputum cytology, DNA hypermethylation analysis showed to be superior to sputum cytology in lung cancer detection. At the 99% specificity level (comparable to sputum cytology), sensitivity of hypermethylation RASSF1A is still 16%.

Spontaneous sputum is usually easily obtained from smokers. Although in former and non-smokers collection of sputum seems more difficult, careful instruction may still lead to representative sputum samples. Alternatively, induced sputum may be an option (Chanez ; Anjuman ). In this study, we only collected spontaneous sputum, because this is more easily accomplished as collection can be performed at home. Detailed instructions in the information brochure increases patient compliance. Induced sputum requires additional logistics, such as an extra visit to the hospital and efforts of the patients.

A limitation of the study is that the impact of sputum testing on clinical decision making, clinical outcomes of patients to whom testing is applied and costs are not assessed.

Future research is needed to optimise the marker panels. Given the heterogeneous nature of lung cancer and the numerous cellular pathways involved (Hansen ), it is likely that a panel of biomarkers will yield a higher sensitivity compared with a single marker. Promising additional (diagnostic) markers such as microRNAs and tumour-specific proteins in sputum may further improve the efficiency for lung cancer diagnosis (Sun ; Xing ; Yu ).

Overall, test characteristics of sputum methylation have been reproduced. RASSF1A hypermethylation in sputum is validated as diagnostic marker for lung cancer. The panel of RASSF1A, 3OST2 and PHACTR3 hypermethylation revealed a 67% sensitivity and high specificity (90%) in a diagnostic setting.