Clinical validation of an autoantibody test for lung cancer.

BACKGROUND: Autoantibodies may be present in a variety of underlying cancers several years before tumours can be detected and testing for their presence may allow earlier diagnosis. We report the clinical validation of an autoantibody panel in newly diagnosed patients with lung cancer (LC). PATIENTS AND METHODS: Three cohorts of patients with newly diagnosed LC were identified: group 1 (n = 145), group 2 (n = 241) and group 3 (n = 269). Patients were individually matched by gender, age and smoking history to a control individual with no history of malignant disease. Serum samples were obtained after diagnosis but before any anticancer treatment. Autoantibody levels were measured against a panel of six tumour-related antigens (p53, NY-ESO-1, CAGE, GBU4-5, Annexin 1 and SOX2). Assay sensitivity was tested in relation to demographic variables and cancer type/stage.
RESULTS: The autoantibody panel demonstrated a sensitivity/specificity of 36%/91%, 39%/89% and 37%/90% in groups 1, 2 and 3, respectively, with good reproducibility. There was no significant difference between different LC stages, indicating that the antigens included covered the different types of LC well.
CONCLUSION: This assay confirms the value of an autoantibody panel as a diagnostic tool and offers a potential system for monitoring patients at high risk of LC.

introduction

Lung cancer (LC) is the worldwide leading cause of cancer-related mortality [1]. Tobacco smoking is estimated to cause upwards of 90% of cases, and other recognised risk factors include passive smoking, occupational exposure, especially to asbestos and radon exposure [1]. Outcomes are substantially better with early localised disease compared with locally advanced and metastatic disease, with 5-year survival rates of 53%, 23.7% and 3.5%, respectively [2].

Although the latent period of LC in smokers is reported to be at least 20 years [1], ∼85% of patients with LC remain undiagnosed until the disease is symptomatic and has reached an advanced stage [2]. At present, there is nothing to offer for early diagnosis, although ongoing clinical trials are investigating the use of spiral computed tomography (CT) in ‘at-risk’ individuals [3-12]. However, the radiation dose delivered and the substantial costs limit its widespread application as a screening procedure [13]. Furthermore, the high rate of false positives (as high as 50% in a prevalence round) [5] dictates that many individuals require follow-up examinations and a substantial proportion of individuals undergo unnecessary thoracotomy [14]. Application of a filter such as a blood-based marker to identify smokers at the highest risk for LC may improve the positive predictive value (PPV) of these screening tools [11, 15].

There is a considerable body of evidence documenting the presence of circulating antibodies to autologous cellular antigens [referred to as tumour-associated antigens (TAA)] in serum samples from patients with a variety of cancers, including LC [16-24]. Monitoring persons at increased risk of cancer for the presence of serum autoantibodies may allow earlier detection of the disease.

The panel of proteins selected for investigation comprised a number of well-recognised TAA, four of which (p53, NY-ESO-1, CAGE and GBU4-5) have been described by ourselves in a previous publication to induce the production of autoantibodies or immune biomarkers in LC [24]. In brief, p53 is a tumour suppressor gene, which is often mutated in cancer and to which autoantibodies were first described [25], autoantibodies to this protein have also been detected in some cases, even before the cancer diagnosis [26, 27]. NY-ESO-1 and CAGE are both cancer testis antigens whose expression has been described in a number of solid tumours [28, 29] and with GBU4-5, a protein of unknown function that encodes a DEAD box domain, have also been described as inducing autoantibodies in LC [24, 30, 31].

The remaining antigens SOX2, a member of the SOXB1 family of proteins that is described as inducing an autoantibody responses in small-cell lung cancer (SCLC) [32, 33], and Annexin I, a phospholipid-binding protein to which autoantibodies, have also been described [18].

The selection of these antigens was confirmed following screening of a panel of >20 potential antigens as being of greatest diagnostic utility for the diagnosis of all non-small-cell lung cancer (NSCLC) and SCLC cancer (C. Chapman, unpublished observations).

This manuscript reports the clinical validation set for these autoantibodies in the serum of patients with newly diagnosed LC (before any treatment) and matched controls.

patients and methods

patients

Findings from three separate groups of patients with newly diagnosed LC are reported. The third group is the final validation set where the data were run in a blinded manner. All patients with LC were as far as possible individually matched by gender, age and smoking history to a control individual with no previous history of malignant disease. In patients with LC, blood samples were obtained after diagnosis but before receiving any anticancer treatment. Demographic characteristics of the control versus the study population are given in the Appendix 1.

Group 1 comprised 145 patients with stage I/II LC (including NSCLC and SCLC) and 146 controls treated in centres in the United States and Russia. All subjects in this group were smokers; baseline patient characteristics are shown in Table 1. Group 2 comprised 241 patients with LC treated at a single centre in Germany as part of a collaborative study (Table 1). Tumour pathological information was available for the patients with LC, including tumour, node, metastasis staging and NSCLC histology (Table 2). In group 2, an additional 88 sera from unmatched individuals (25 normal and 63 with benign lung conditions) supplied by the single centre were analysed (Appendix 1).

Table 1.

Lung cancer patient characteristics

	Group 1 (n = 145)	Group 2 (n = 241)	Group 3 (n = 269)
Median age, years (range)	66 (41–87)	63 (28–87)	65 (38–87)
Patients >60 years, n (%)	96 (66.2)	140 (58.1)	171 (63.6)
Gender, n (%)
Male	81 (55.9)	172 (71.4)	199 (74.0)
Female	64 (44.1)	69 (28.6)	70 (26.0)
Smoking history, n (%)
Current	145	0	132 (49.1)
Previous	0	0	76 (28.3)
Never	0	0	24 (8.9)
Not determined	0 (0.0)	241 (100.0)	37 (13.8)

Table 2.

Tumour stage and histology according to gender

	Group 1 (n = 145)		Group 2 (n = 241)		Group 3 (n = 255a)
	Male (n = 81)	Female (n = 64)	Male (n = 172)	Female (n = 69)	Male (n = 188)	Female (n = 67)
Tumour type, n (%)
NSCLC	71 (87.7)	52 (81.3)	125 (72.7)	46 (66.7)	141 (75.0)	41 (61.2)
SCLC	10 (12.3)	12 (18.8)	47 (27.3)	23 (33.3)	47 (25.0)	26 (38.8)
NSCLC stage, n (%)
I	41 (57.7)	40 (76.9)	0 (0.0)	0 (0.0)	71 (50.4)	15 (36.6)
II	30 (42.3)	12 (23.1)	0 (0.0)	1 (2.2)	42 (29.8)	11 (26.8)
III	0 (0.0)	0 (0.0)	38 (30.4)	11 (23.9)	13 (9.2)	1 (2.4)
IV	0 (0.0)	0 (0.0)	63 (50.4)	25 (54.3)	1 (0.7)	2 (4.9)
Unknown	0 (0.0)	0 (0.0)	24 (19.2)	9 (19.6)	14 (9.9)	12 (29.3)
NSCLC histology, n (%)
Squamous	16 (22.5)	5 (9.6)	38 (30.4)	4 (8.7)	78 (55.3)	10 (24.4)
Adenocarcinoma	16 (22.5)	13 (25.0)	37 (29.6)	19 (41.3)	44 (31.2)	23 (56.1)
Large cell	2 (2.8)	0	4 (3.2)	2 (4.3)	5 (3.5)	0 (0.0)
Bronchoalveolar	1 (1.4)	18 (34.6)	0 (0.0)	0 (0.0)	1 (0.7)	5 (12.2)
Tubular adenocarcinoma	0	0	0	0	2 (1.4)	0
Not determined	4 (5.6)	12 (23.1)	46 (36.8)	21 (45.6)	9 (6.4)	1 (2.4)
Other	32 (45.1)	4 (7.7)	0	0	2 (1.4)	2 (4.9)
SCLC stage, n (%)
Limited SCLC	0	0	21 (44.7)	6 (26.1)	14 (29.8)	15 (57.7)
Extensive SCLC	0	0	19 (40.4)	14 (60.9)	8 (17.0)	3 (11.5)
Not determined	10 (100.0)	12 (100.0)	7 (14.9)	3 (13.0)	25 (53.2)	8 (30.8)

Tumour histology and stage data available for 255 of the 269 patients comprising group 3.

NSCLC, non-small-cell lung cancer; SCLC, small-cell lung cancer.

Group 3 comprised 269 patients with LC treated at centres in the United States, UK and Ukraine (Table 1). This group was assembled to validate the calibration and control scheme for the autoantibody assay. Tumour pathological information was available for the patients with LC (Table 2). The timeline for collection of samples from patients is shown in supplemental Table S1 (available at Annals of Oncology online).

Serum samples in group 1 were evaluated for autoantibodies against p53, NY-ESO-1, CAGE and GBU4-5. Serum samples in groups 2 and 3 were evaluated for autoantibodies against the same four antigens plus Annexin 1 and SOX2. In groups 2 and 3, samples from patients with cancers, matched normals, benign lung disease and control sera for the assay were interspersed in the order samples were assayed so that any batch effects would be spread over all sample types. The laboratory staff running the assay was blinded to the disease state of individual samples. Group 2, therefore, was a validation set for the results seen in group 1 for four of the antigens (i.e. p53, NY-ESO1, CAGE and GBU4-5) with the added value of Annexin 1 and SOX2. Group 3 validated a calibrated and controlled assay on the whole panel of six antigens.

autoantibody assay

Autoantibodies were determined by a quality-controlled, semi-automated indirect enzyme-linked immunosorbent assay in which samples were allowed to react with a titration series of antigen concentrations. All liquid handling steps were carried out using an automated liquid handling system. Briefly, purified recombinant antigens were diluted to provide a semi-log titration series for each antigen from 160 to 1.6 nM [34]. Control antigens consisting of the purified BirA or NusA tags were also included to allow subtraction of the signal due to nonspecific binding to bacterial contaminants. Antigen dilutions were adsorbed to the surface of microtitre plate wells in phosphate buffer at room temperature. After washing in phosphate-buffered saline containing 0.1% Tween 20 (pH 7.6), microtitre plates were blocked with a gelatine-based blocking buffer. Serum samples (diluted 1 in 110 in a blocking buffer) were then added to the plates and allowed to incubate at room temperature with shaking for 90 min. Following incubation, plates were washed and horseradish peroxidase-conjugated rabbit anti-human IgG (Dako, Glostrup, Denmark) was added. After a 60-min incubation, the plates were washed and 3,3′,5,5′-tetramethylbenzidine was added. Colour formation was allowed to proceed for 15 min before the optical density (OD) of each well was determined spectrophotometrically at 650 nm [35].

Calibration standards of known potency are not available for assays to measure autoantibodies against TAAs. Therefore, a calibration system was devised which utilised fluids drained from pleural or ascitic cavities of patients with LC [36]. The calibration system was only evaluated for group 3 samples. A reportable dilution range for each antigen, giving acceptable calibration precision, was determined at 7.5%–92.5% of the upper asymptote of the average calibration curve, equivalent to ∼5.0 natural log reference units (RU). These data were used to construct a calibration curve of OD versus log dilution to which a four-parameter model plot was fitted [37]. The background-corrected OD value for each unknown sample was then converted to a calibrated log RU.

Samples were judged to be positive if they fulfilled two criteria—i.e. they showed a dose response to the antigen titration series and the measured autoantibody signal to one or more of the antigens was above the accepted cut-off set for that antigen assay. The autoantibody signal for a sample was defined as above the cut-off when the result was greater than the calculated cut-off for the control population at either of the two highest points on the titration curve. All assays were carried out as two replicates and the mean value taken as the overall assay measurement.

optimisation of assay cut-offs

A specificity of 90% was selected in order to produce a test which could be used for early detection in a high-risk population and which would be health economically viable. For all groups, cut-offs based on mean + 3 standard deviations (SDs) of the normal population were used. In addition, for groups 2 and 3, the cut-offs were optimised using a Monte Carlo direct search method [38] to find a set of antigen-specific cut-offs yielding the maximum sensitivity for the fixed specificity of 90%.

For a set of possible cut-offs for the six panel antigens chosen by Monte Carlo sampling over the feasible range, the specificity/sensitivity was first estimated from the data. This was carried out 100 000 times. All combinations with a specificity of ∼90% were then extracted and the combination yielding the maximum sensitivity used. This is a process dependent on assay conditions and when new batches of proteins, or new types of protein, are introduced to the panel, new cut-offs will have to be calculated.

To support the quoted specificity/sensitivity panel results, the area under the curve (AUC) and standard error (SE) for the respective receiver operating characteristic (ROC) curve was calculated for each group. The ROC curve was constructed by calculating the specificity and sensitivity of the test for a succession of deviations from the original cut-offs, with the same deviation for each antigen in the panel.

adjustment for LCs in the control populations

The cut-offs are best set by comparing the results in a group of patients with known LC and a group of high-risk individuals (e.g. smokers and ex-smokers) who are known to not have the disease. However, the latter population is difficult to identify since the CT screening studies have clearly shown there are a percentage of smokers/ex-smokers who at any one time are ‘harbouring’ an asymptomatic LC. In the prevalence round, the percentage of undiagnosed occult cancers has been reported to be between 0.5% and 2.7% in heavy smokers, while in incidence rounds, it has been reported to be up to 2.3% [3-12]. For this reason, adjusted specificity and sensitivity values assuming some degree of occult LCs in the control populations were also calculated.

The method used to calculate and adjust for the presence of undiagnosed cancers in the controls used LC prediction models for which the most important predictors are age, current smoking status and smoking history, and family history of smoking-related cancers [39].

results

autoantibody expression

In group 1, autoantibodies to four antigens (p53, NY-ESO-1, CAGE and GBU4-5) were measured as raw OD values. Using cut-offs based on mean + 3 SDs gave a sensitivity of 36% with a specificity of 91% (50 of the 137, 8 unassessable). The sensitivities and specificities for each of these four antigens and the reproducibility of these assays have been reported elsewhere [35]. The sensitivity and specificity of the panel was similar for males and females. The ROC curve AUC was 0.71 (SE = 0.03).

In group 2, autoantibodies to six antigens (p53, NY-ESO-1, CAGE, GBU4-5 plus Annexin 1 and SOX2) were measured as raw OD values, with cut-offs based on mean + 3 SDs producing sensitivity and specificity values of 34% (80 of the 234, 7 unassessable) and 91%, respectively. Again, individual sensitivities and specificities for these six antigens have been reported elsewhere [35]. Using individually optimised cut-offs for each antigen, the overall sensitivity was 39% (33%–45%) (91 of the 234), with a specificity of 89%. In an at-risk population of 20 LCs per 1000 population, this would result in a PPV of 7.2% (i.e. 1 in 13.9 persons with a positive test would have a LC) and a negative predictive value (NPV) of 98.6%. The ROC curve AUC was 0.63 (SE = 0.03).

Of the 88 unmatched sera received from the group 2 centre, 8 of the 88 (9%) were positive, of which none of the 25 (0%) normal sera had raised autoantibodies, while 8 of the 63 (13%) of individuals with benign lung disease had raised autoantibodies. Follow-up data could only be obtained for one of these eight individuals who was found to have developed a gastric cancer, giving a specificity of at least 89% (55 of the 62).

In group 3, autoantibodies to the same six antigens as group 2 were measured as raw OD values and converted into calibrated RU. Using the ODs and applying cut-offs based on mean + 3 SDs gave a sensitivity of 32% (85 of the 269) with a specificity of 91%. Using RU values with individually optimised cut-offs for each antigen, the sensitivity was 37% (31%–43%) (100 of the 269), with a specificity of 90%. In an at-risk population of 20 LCs per 1000 population, this would result in a PPV of 7.0% (i.e. 1 in 14.3) and an NPV of 98.6%. The ROC curve AUC was 0.64 (SE = 0.02).

Individual antigen sensitivity and specificity are shown in supplemental Table S2 (available at Annals of Oncology online).

adjustment for occult LCs within the control population

Adjustment generated specific cut-offs for each antigen for the different methods. The sensitivities for each antigen for a fixed specificity of 90% are shown for the unadjusted and adjusted method in Table 3. The most conservative estimate for adjusted sensitivity is 40%, which in an at-risk group of 20 LCs per 1000 population would give a PPV of 7.5% (i.e. 1 in 13.3) and an NPV of 98.7%.

Table 3.

Comparison of performancea before and after adjustment for the presence of undiagnosed occult cancers in the control population

Adjustment method		Group 2	Group 3
Unadjusted	Sensitivity	39% (91/234)	37% (100/269)
	Specificity	89% (207/232)	90% (242/269)
Occult cancer rate (5%)	Sensitivity	42% (99/234)	40% (108/269)
	Specificity	90% (197/220)	90% (230/255)
Occult cancer rate (11%)	Sensitivity	46% (108/234)	43% (115/269)
	Specificity	89% (184/206)	90% (214/238)

Sensitivity for specificity of 90% ± 1%, based on optimised cut-offs for individual antigens.

effect of patient and disease characteristics on autoantibody assay sensitivity and specificity

The calibrated group 3 dataset with an unadjusted sensitivity and specificity of 37% and 90%, respectively, was used to assess whether patient characteristics, tumour type or stage gave rise to significant variation in the specificity/sensitivity (Figure 1). Statistical comparison of subgroups with remaining controls demonstrated no significant difference in sensitivity according to patient gender, smoking status and age or tumour type or stage (P > 0.10). There was also no significant difference in sensitivity between those NSCLC tumours where the subtype was known and those where it was unknown.

Figure 1.

Forest plot showing the sensitivity at a fixed 90% specificity by patient demographics, tumour characteristics and lung cancer stage. Line shows unadjusted sensitivity of 37% (all stages of cancers) in group 3 (n = 269). NSCLC, non-small-cell lung cancer; SCLC, small-cell lung cancer.

discussion

This report confirms a validated assay for the detection of autoantibodies to selected cancer-associated antigens in the peripheral blood. The value of a test for early cancer detection is usually defined via a number of related parameters, including sensitivity, specificity, PPV and NPV. A percentage of smokers/ex-smokers are ‘harbouring’ an asymptomatic LC at any one time. Even with the most conservative estimation of occult LCs, the panel of autoantibodies can identify 40% of primary LCs, including early stage of disease, with a specificity of 90% against age-matched, gender-matched and smoking history-matched controls. The specificity was similar (at least 89%) for patients with benign disease.

Autoantibodies to p53 [26, 27, 40, 41], NY-ESO-1 [30, 31], CAGE [29, 42], GBU4-5 [31], Annexin 1 [16, 18, 43] and SOX2 [44] have all been shown to be capable of inducing autoantibodies in patients with LC. The data in this manuscript further confirm the value of a panel of autoantibodies over a single autoantibody assay [19, 23, 24, 35]. Recent publications have reported autoantibodies to a natural form of Annexin 1 [43] and other antigens (e.g. 14-3-3 theta [43, 45] and LAMR1 [43]), which are elevated in LC and up to 1 year before clinical diagnosis. The combination of 14-3-3 theta, Annexin 1 and LAMR1 gave an AUC on a combined ROC curve of 0.73. While these results were based on a research assay, it is possible that adding 14-3-3 theta and/or LAMR1 to the current panel might increase the sensitivity.

Group 3 data confirm that there was no significant difference between different stages of LC, although due to sample size the confidence intervals were sometimes wide. Further evaluation of the data was, therefore, carried out by comparing early-stage (stage I/II NSCLC plus limited SCLC) with late-stage (stage III/IV NSCLC plus extensive SCLC) disease, which again showed no difference. The presence of such a signal in early-stage disease is precisely what would be expected of an in vivo amplification signal such as the humoural immune response. This is in contrast to cancer-associated antigens, which are markers of tumour burden and not useful for the early detection or screening of breast [46, 47] or colorectal cancer [48, 49].

Previous publications [16–24, 50] have highlighted the potential value of a panel of autoantibodies for the early detection of cancer. Using a panel of antigens, autoantibodies have been reported up to 5 years before screening CT scans [22] in LC and up to 4 years before screening mammography in young women at increased risk [21, 23]. Other authors have highlighted individual autoantibodies such as p53 autoantibodies detected before diagnosis of cancer in smokers with chronic obstructive pulmonary disease [27] or in patients with asbestosis [41]. In the latter publication, the average lead time (time from first positive sample to diagnosis) was 3.5 years (range 1–12 years). There are similar publications on other single autoantibodies [45, 51, 52]. These findings all indicate the induction of autoantibodies happening relatively early in the process of carcinogenesis.

This panel assay is the first to show reproducible results with a calibration and control system and offers a potential system for monitoring a population at high risk of LC, either alone or in conjunction with imaging modalities (e.g. CT). The similar sensitivities and specificities measured for these three datasets and with different batches of proteins utilised emphasise the robustness of these autoantibody assays and also confirm the value of a panel of autoantibodies over a single autoantibody assay.

At a fixed 90% specificity, the sensitivity of 40% is a conservative estimate of the performance of the assay both in terms of estimating the level of clinically occult LCs (supplemental Table S2, available at Annals of Oncology online) and also the sensitivity reported for SCLC (n = 73) in group 3. The latter is lower than the 55% sensitivity and 90% specificity, which the authors will report in a larger consecutive series (n = 242) from a single centre (C. J. Chapman, A. J. Thorpe, A. Murray et al., unpublished data).

The sensitivity of 40% with a specificity of 90% are similar to mammography in high-risk young women [53], while the incidence of LCs in heavy smokers is at least three times the incidence of breast cancer in a typical cohort of high-risk young women [5, 7]. Therefore, in terms of absolute number of cancers, this test should detect more LCs for every 1000 high-risk persons tested than screening mammography would detect breast cancers in a high-risk group of young women, even if mammography were 100% sensitive rather than its current 40% [53]. This has to be seen in the context of a disease (i.e. LC), which has a mortality rate between 85% and 95%. By way of contrast, annual CT in the Mayo CT screening trial had a specificity of 49% (with a sensitivity of 67%) in the prevalence round. In an at-risk group of 20 of the 1000, CT gave a PPV of 2.5% (i.e. 1:40) and an NPV of 98.7%. The autoantibody test with a sensitivity of 40% and a specificity of 90% would have a PPV of 7.5% (i.e. 1 in 13.3) and an NPV of 98.7% in a similar-risk group.

While such comparisons serve to highlight the potential value of an autoantibody test for LC that has a specificity of 90%, the authors envisage the autoantibody technology and imaging as being complementary.

funding

Oncimmune Ltd and the University of Nottingham.

disclosure

CJC and JFRR are consultants to Oncimmune Ltd, a University of Nottingham spinout company and JFRR holds stock. AM and GH are full-time employees of Oncimmune Ltd. CR holds stock option and is also a consultant to Oncimmune Ltd. WCW is the scientific advisor for Oncimmune Ltd. GH is an employee of Oncimmune Ltd. GHF is the Chairman of Oncimmune Ltd and holds stock. ACB holds stock and options in Oncimmune Ltd and has a significant conflict.