Elevated tumor markers for monitoring tumor response to immunotherapy.

Background: As the immune-related response evaluation criteria in solid tumors (irRECIST) by imaging greatly underestimated the objective response to immunotherapy, we established the response evaluation criteria in solid tumors based on tumor markers (RecistTM) to explore whether RecistTM can compensate for the deficiencies of the irRECIST criteria.
Methods: This was an observational study, which consisted of two parts. The first part (Group A) was a retrospective study including the patients with malignant solid tumors. The second part (Group B) was a prospective study, which were EGFR-negative and ALK-negative patients with stage IIIB-IV non-small cell lung cancer receiving first-line treatment. From January 2017 to September 2020, one hundred and ten patients with a three-time increase in tumor markers receiving immunotherapy were recruited. The treatment response to immunotherapy was evaluated by irRECIST and RecistTM. Efficacy, overall survival (OS), first evaluation time and earliest response time under the different evaluation criteria were compared by statistics. Findings: The treatment response evaluated by the RecistTM criteria was not consistent with that evaluated by the irRECIST criteria (Kappa = 0.386, p < 0.001). RecistTM had a higher completed response (CR) rate compared to irRECIST criteria (20.9% vs 1.8%, p < 0.001). The earliest response time under the RecistTM criteria was 3.42 weeks earlier than that under the irRECIST criteria (u = -5.233, p < 0.001). There were significant differences in median OS between tumor marker-related complete response (tmCR) and tumor marker-related partial response (tmPR), as well as between tmPR and tumor marker-related stable disease (tmSD) (χ2 = 15.572, p < 0.001; χ2 = 7.720, p = 0.005), but not between tmSD and tumor marker-related progressive disease (tmPD) (χ2 = 1.596, p = 0.206). When applying both criteria together, for patients with immune-related CR / immune-related PR (irCR/irPR) (n = 54) under irRECIST criteria, there was a significant difference in median OS between achieving tmCR (n = 22) and tmPR (n = 32) (χ2 = 14.011, p < 0.001). RecistTM criteria can predict 1-year and 2-year OS more accurately than irRECIST criteria (AUCs:0.862 vs 0.552, 0.649 vs 0.521, respectively;both p < 0.001). In RecistTM, 4 patients had been observed with pseudoprogression in tumor markers. Interpretation: The RecistTM criteria could effectively distinguish CR, PR, and SD, which may help resolve the shortcomings of the RECIST criteria in evaluating the treatment response to immunotherapy, especially in assessing whether patients can achieve deep or even complete response as soon as possible. Funding: This work was supported by the Key projects of Chongqing Health and Family Planning Commission (to Xueqin Yang, 2019ZDXM011).

Evidence before this study

The immune-related response evaluation criteria in solid tumors (irRECIST) improved the guidelines for assessing progressive disease (PD); however, with the recent development of clinical research on immunotherapy as preoperative neoadjuvant therapy, the irRECIST criteria for evaluating preoperative clinical response by imaging greatly underestimated the objective response to immunotherapy, such as complete response (CR) and partial response (PR), compared with postoperative pathological response.

We searched PubMed for previous studies published until December 21, 2021, using the terms “tumor marker” and ”immunotherapy”. We found a few studies reported that decreasing levels of tumor markers positively correlated with treatment response and overall survival (OS) after immunotherapy. However, no studies established the response evaluation criteria in solid tumors based on tumor markers (RecistTM) and explored its role in compensating for the deficiencies of the irRECIST criteria.

Added value of this study

In this study, we established the RecistTM criteria based on tumor markers with initial levels exceeding the baseline level by >3 times. The RecistTM criteria could effectively distinguish CR, PR, and stable disease (SD) according to the median patient survival time. It had a higher CR rate than that evaluated by the irRECIST criteria and could further distinguish patients with better benefits from PR patients under the irRECIST criteria. Due to earlier evaluation time, tumor markers have the shorter response time than those of imaging, which may help confirmed the efficacy as soon as possible. Pseudoprogression of tumor markers was also observed in the treatment response to immunotherapy in a small number of patients.

Implications of all the available evidence

This study can help clinicians identify the population benefiting from immunotherapy in advance, which may help resolve the shortcomings of the irRECIST criteria in evaluating the treatment response to immunotherapy, especially in assessing whether patients can achieve deep or even complete response as soon as possible.

Alt-text: Unlabelled box

Introduction

Immunotherapy against cancer has rapidly developed in recent years. Immune checkpoint inhibitors (ICIs), such as monoclonal antibodies against CTLA4, PD-1, and PD-L1, are changing the treatment modalities of cancer patients by restoring the activity of anti-tumor T cells and inducing lasting responses. So much so that immunotherapy has become the standard of care for various types of cancers. While immunotherapy brought new hope to cancer patients, it also caused new problems and challenges in clinical practice; treatment response evaluation is one challenge.

Apart from the classical response mode, numerous studies have shown that immunotherapy also includes non-classical response modes such as pseudoprogression, dissociated-response, and hyperprogression., pseudoprogression is caused by initial T cell infiltration, resulting in an observable objective response [complete response (CR) and partial response (PR)] in patients although its incidence is < 10% and is far lower than that of true progression., A dissociated-response occurs in patients who simultaneously have an objective response and disease progression. Its frequency is similar to that of pseudoprogression, and its prognosis is better than that of true progression; therefore, continuous treatment should be considered for such patients., In contrast, hyperprogression occurs when patients have accelerated tumor growth at the early stages of ICIs treatment and indicates that patients cannot benefit from treatment. Its incidence varies from 4% to 29%, patient prognosis is often very poor, and in most cases related to a wide range of metastatic diseases. Thus such patients need to stop ICI treatment. Therefore, new treatment response evaluation criteria, including immune-related Response Criteria (irRC), immune-related Response Evaluation Criteria in Solid Tumors (irRECIST), immune Response Evaluation Criteria in Solid Tumors (iRECIST), and immune-modified Response Evaluation Criteria in Solid Tumors (imRECIST), were developed to adapt to immunotherapy response modes, which differ from those of previous treatment methods. The main changes include (1) the response must be reconfirmed ≥ 4 weeks after the first record of progressive disease (PD), and (2) the appearance of new lesions is not simply attributed to PD but included in the total tumor burden for evaluation. Therefore, the new treatment response evaluation criteria improved the guidelines for assessing PD.

However, with the recent development of clinical research on immunotherapy as preoperative neoadjuvant therapy, the RECIST criteria for evaluating preoperative clinical response by imaging greatly underestimated the objective response to immunotherapy compared with postoperative pathological response.13, 14, 15 For example, in the newly reported NADIM study in 46 patients, 20 patients underwent surgical treatment after completing preoperative treatment by nivolumab combined with three cycles of chemotherapy. Before surgery, the clinical treatment response based on the RECIST criteria was evaluated as 5% CR and 65% PR. However, in the postoperative pathological evaluation, the pathological complete response (pCR) was 65%, and the major pathological response (MPR), defined as <10% viable tumor cells in the resected specimen, was 15%. There was a great difference in assessing whether patients achieved CR between the two methods. In addition, in the CM159 study, after two cycles of nivolumab immunotherapy, an MPR was observed in 45% of the patients in the postoperative pathological evaluation. However, the preoperative computed tomography (CT) evaluations were assessed to have PR in only 10% of the patients, while most patients were assessed to have stable disease (SD). In two patients whose preoperative CT evaluation showed a significantly enlarged tumor, one patient was evaluated to have an MPR after surgery, and the other was evaluated to have pCR. Simultaneously, the study found many infiltrating lymphocytes and macrophages in primary tumors with a major pathological response, consistent with the immune response mechanism, and necrotic tumors consistent with fibrotic tissue repair. The above research fully demonstrates the important difference between clinical radiological response and actual pathological response, and that the existing RECIST criteria underestimate the actual treatment response of patients and cannot accurately distinguish between CR, PR, and SD, which further affects the doctors' ability to accurately predict when patients can benefit from this type of treatment.

Tumor markers have been widely applied in tumor diagnosis and treatment, including early diagnosis, treatment response assessment, and recurrence monitoring.16, 17, 18, 19 Theoretically, local immune cell infiltration in tumor tissues or fibrous tissue proliferation should not affect circulating tumor markers. In this study,we established the response evaluation criteria in solid tumors based on tumor markers (RecistTM) to explore whether RecistTM criteria can compensate for the deficiencies of the irRECIST criteria, and to highlight the importance of tumor markers in evaluating the treatment response to immunotherapy. The results showed that RecistTM criteria could effectively distinguish CR, PR, and SD. In addition, it had a higher CR rate than that evaluated by the irRECIST criteria and could further distinguish patients with better benefits from PR patients under the irRECIST criteria.

Methods

Study design

This was an observational study, which consisted of two parts. The first (Group A) was a retrospective study including the patients with malignant solid tumors. The second part was a prospective study (Group B), which were EGFR-negative and ALK-negative patients with stage IIIB–IV non-small cell lung cancer (NSCLC) receiving first-line treatment. All included patients underwent immunotherapy, and their tumor markers before immunotherapy were >3 times higher than the baseline levels. All patients were residents of the oncology department of Daping Hospital, Chongqing, China, between January 2017 and September 2020. The study was approved by the Ethics Committee of Daping Hospital and conducted according to the Declaration of Helsinki. Written informed consent was obtained from the patients in group B and waived in group A due to the study's retrospective nature.

Group A included patients treated with immunotherapy alone or a combination of immunotherapy with other treatments. The combined therapies included combining of chemotherapy, combining of anti-angiogenesis therapy, and combining of radiotherapy, while the ICIs included pembrolizumab, nivolumab, sintilimab, camrelizumab, tislelizumab, and toripalimab. Chemotherapy was administered as either a single drug or a dual drug combination. The included patients must be administered with ICIs at least two cycles and the patients underwent resection of the target lesions after immunotherapy was excluded. Group B trial was registered under chictr.org.cn/ChiCTR1900027270 and is ongoing but no longer recruiting patients; the enrollment was completed before September 2020. All patients in Group B received a combination of immunotherapy and chemotherapy, and chemotherapy was administered as either a single drug or a dual drug combination. All patients had to be treated with combination therapy at least 2 cycles before subsequent maintenance on single-drug immunotherapy could be considered. Further inclusion and exclusion criteria are available at chictr.org.cn.

The treatment response to immunotherapy was evaluated by two criteria, namely, the irRECIST and the RecistTM. Tumor markers were assessed before each ICIs treatment in Group A and Group B. Serum tumor marker testing was performed by Luminex xMAP assays. Radiological evaluation was assessed at the initial treatment (baseline) in both groups, thereafter every three to twelve weeks in group A and every six weeks in group B.

First evaluation time was defined as the time of the first image or tumor marker evaluation after the initial treatment. The earliest response time for RecistTM was defined as the time from the initial treatment to the time when the major tumor markers decreased >20%, while no other tumor markers increased >20%. The earliest response time for irRECIST was defined as the time from initial treatment to the time when measurable lesions decreased ≥10% while no non-measurable lesions increased. OS was calculated from the start of ICIs to the date of death or last follow-up.

irRECIST and RecistTM criteria

The treatment response evaluation by irRECIST included the immune-related complete response (irCR), immune-related partial response (irPR), immune-related stable disease (irSD), and immune-related progression disease (irPD). For specifics, refer to the research by Bohnsack et al. The treatment response evaluation by RecistTM included tumor marker-related complete response (tmCR), tumor marker-related partial response (tmPR), tumor marker-related stable disease (tmSD), and tumor marker-related progression disease (tmPD).

Under RecistTM, we selected a single tumor marker with levels exceeding the baseline level by > 3 times as the primary marker, and others not exceeding this threshold were considered as secondary markers. If there were > 1 tumor markers meeting the requirements, the one with better correlation with the tumor was selected as the primary marker. If not, select the higher times increase as the primary marker. Tumor markers included: carcinoembryonic antigen (CEA) > 15 ng/ml (cutoff: 5 ng/ml), carbohydrate antigen 19–9 (CA199) > 105 U/ml (cutoff: 35 U/ml), carbohydrate antigen 125 (CA125) > 105 U/ml (cutoff: 35 U/ml), carbohydrate antigen 153 (CA153) >105 U/ml (cutoff: 35 U/ml), alpha-fetoprotein (AFP) > 60 ng/ml (cutoff: 20 ng/ml), squamous cell carcinoma antigen (SCCAg) > 7.5 ng/ml (cutoff: 2.5 ng/ml), cytokeratin fragment antigen 21-1 (CYFRA21–1) > 9.9 ng/ml (cutoff: 3.3 ng/ml), neuron-specific enolase (NSE) > 48.9 ng/ml (cutoff: 16.3 ng/ml), and gastrin > 195.6 (cutoff: 65.2 ng/ml). Specific criteria are detailed in Table 1.

Table 1

Summary of RecistTM criteria.

	Primary markers*	Secondary markers	Markers with normal baseline levels	Other conditions to be met
tmCR	Fall to normal	Fall to normal	normal	All the above conditions are met and maintained for ≥6 weeks.
tmPR	Decrease ≥ 30%	Decrease, or increase ≤ 20%.	≤ 1.5 times the cutoff value	All the above conditions are met and maintained for ≥6 weeks.
tmSD	None of the tmPR, tmCR, and tmPD criteria are fulfilled.			Maintained for ≥6 weeks.
tmPD	Increase ≥ 30%	Increase ≥ 50%	≥ 2 times the cutoff value	Any of the conditions are met, and it needs to be confirmed again after 3-4 weeks.

RecistTM, response evaluation criteria in solid tumors based on tumor markers; tmCR, tumor marker-related complete response; tmPR, tumor marker-related partial response; tmSD, tumor marker-related stable disease; tmPD, tumor marker-related progression disease.

This has to be >3 times the cutoff value. Because of different detection methods in different hospitals, the cutoff values of tumor markers may vary. In addition, due to the influence of examination errors, in patients whose absolute value of the primary marker baseline levels was <20, the increasing or decreasing levels of primary markers were divided by 50% instead of 30%.

The treatment response evaluation was conducted independently by two investigators (YY and YL). Disagreements were resolved by discussion with a third investigator (XY).

Statistical analysis

Variables were summarized as median (range) for continuous variables and number (%) for categorical variables. McNemar's tests were used to compare the efficacy evaluated between irRECIST and RecistTM. Concordance between RecistTM and irRECIST was assessed with Cohen's kappa coefficient. Agreement between the 2 assessments was categorized as poor (Kappa <0.4), moderate (0.4 ≤Kappa<0.75), and almost perfect (kappa≥0.75). Paired Wilcoxon signed rank test was used to evaluate the differences between first evaluation time and earliest response time in group A and group B. Median OS was estimated by Kaplan-Meier method. OS survival curves were stratified by different evaluation criteria and compared using the Log-rank test. The accuracies of irRECIST and RecistTM for prognosis of OS were determined with time-dependent receiver operating characteristic (ROC) curve analysis using the “timeROC” function, and comparisons between two time-dependent area under curves (AUCs) were performed with the “compare” function implemented in the R package “timeROC” (version 0.3 published in 2015-03-25). All statistical tests were bilateral with significance level 0.05. Most of the statistical analyses were performed using SPSS software package (version 16.0) (SPSS Inc., Chicago, IL, USA). Graphs were drawn using Excel software (v.2017).

Role of the funding source

This study was funded by the Key projects of Chongqing Health and Family Planning Commission. However, there was no role of the funding in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Patient baseline characteristics

In the retrospective study (Group A), all patients had solid malignant tumors and received immunotherapy. From 392 patients receiving immunotherapy, we selected 292 patients with complete follow-up data. However, only 77 patients (26.4%) had tumor markers >3 times higher than baseline. The median age of the 77 patients was 61 years (range: 22–78 years), and the median number of immunotherapy cycles was 5 (range: 2–28). The mainly RecistTM markers used to assess the treatment response included CEA, SCCAg, CA199, and CA125; the others included CYFRA21–1(n = 4), Gastrin (n = 3), NSE (n = 2), CA153 (n = 2), and AFP(n = 2) (Table 2).

Table 2

Patients’ characteristics.

	Group A (n = 77)	Group B (n = 33)	Total (n = 110)
Sex
Male	54 (70.1%)	27 (81.8%)	81 (73.6%)
Female	23  (29.9%)	6 (18.2%)	29 (26.4%)
Age
≥65	22 (28.6%)	15 (45.5%)	37 (33.6%)
<65	55 (71.4%)	18 (54.5%)	73 (66.4%)
Tumor types
Lung cancer	46 (59.7%)	33 (100%)	79 (71.8%)
Other squamous cancers	13 (16.9%)	0	13 (11.8%)
Other non-squamous cancers	18 (23.4%)	0	18 (16.4%)
Combination or not
Yes	66 (85.7%)	33 (100%)	99 (90.0%)
No	11 (14.3%)	0	11 (10.0%)
Drugs
Toripalimab	22 (28.6%)	6 (18.2%)	28 (25.5%)
Pembrolizumab	15 (19.5%)	5 (15.2%)	20 (18.2%)
Nivolumab	10 (13.0%)	0	10 (9.1%)
Sintilimab	23 (29.9%)	13 (39.4%)	36 (32.7%)
Tislelizumab	7  (9.1%)	6 (18.2%)	13 (11.8%)
Camrelizumab	0	3 (9.1%)	3 (2.7%)
Cycles
≤3	23 (29.9%)	8 (24.2%)	31 (28.2%)
4–9	37 (48.1%)	15 (45.5%)	52 (47.3%)
≥10	17 (22.1%)	10 (30.3%)	27 (24.5%)
Treatment line
1st	47 (61.0%)	33 (100%)	80 (72.7%)
2nd	8 (10.4%)	0	8 (7.3%)
≥3rd	22 (28.6%)	0	22 (20.0%)
Markers
CEA	38 (49.4%)	12 (36.4%)	50 (45.5%)
SCCAg	7 (9.1%)	0	7 (6.4%)
CA199	9 (11.7%)	8 (24.2%)	17 (15.5%)
CA125	10 (13.0%)	4 (12.1%)	14 (12.7%)
Other markers	13 (16.9%)	9 (27.3%)	22 (20.0%)

CEA, carcinoembryonic antigen; CA199, carbohydrate antigen 19–9; CA125, carbohydrate antigen 125; SCCAg, squamous cell carcinoma antigen.

In the prospective study (Group B), 37 NSCLC patients receiving first-line immunotherapy were enrolled, of which four were excluded because they did not conform to the study protocol. The median age of the remaining 33 patients was 63 years (range: 25–74 years). The median number of immunotherapy cycles was 7 (range: 2–22). The mainly RecistTM markers used to assess the treatment response included CEA, CA199, and CA125; the others included CYFRA21–1 (n = 5), NSE (n = 2), CA153 (n = 1), and AFP (n = 1) . The markers used in group B had a little bit different from that used in group A, as the group A included more types of tumors (Table 2 and Supplemental material Table S1; STROBE flow diagram see Figure 1).

Figure 1

STROBE flow diagram.

Comparison of treatment response assessment by different evaluation criteria

The treatment response in group A was as follows: tmCR: 15(19.5%), tmPR: 31 (40.3%), tmSD: 11 (14.3%), and tmPD 20 (20.6%) under the RecistTM criteria, and irCR: 1 (1.3%), irPR: 34 (44.2%), irSD: 20 (26.0%) and irPD: 22(28.6%) under the irRECIST criteria. The treatment response in group B was as follows: tmCR: 8 (24.2%), tmPR: 18 (54.5%), tmSD: 3 (9.1%), and tmPD 4 (12.1%) under the RecistTM criteria, and irCR: 1 (3.0%), irPR: 20 (60.6%), irSD: 8 (24.2%) and irPD: 4 (12.1%) under the irRECIST criteria. There were significant differences between RecistTM criteria and irRECIST criteria on the CR rate (A: p < 0.001; B: p = 0.016, respectively). However, no significant differences were found on the disease control rate (CR+PR+SD) (A: p = 0.754; B: p = 1, respectively). Therefore, the results of group A and group B were similar.

The consistency between the RecistTM and irRECIST criteria was 57.1% and 57.6% in Group A and Group B, respectively, and 57.3% in the total population (n = 110). In the total population, 20 (37.0%) out of 54 patients classified as irPR by the irRECIST criteria were classified as tmCR by the RecistTM criteria, while 16 (57.1%) out of 28 patients classified as irSD by the irRECIST criteria were classified as tmPR or tmCR by the RecistTM criteria. A kappa test showed a poor consistency of assessment between the two methods (Kappa = 0.386, p < 0.001). However, the two types of evaluation criteria were 90.9% consistent in evaluating whether patients had achieved disease control, and the Kappa test showed that the two methods were moderately consistent (Kappa = 0.741, p < 0.001; Figure 2).

Figure 2

Consistency between RecistTM and irRECIST in all patients (n = 110). The treatment response to immunotherapy was evaluated by the irRECIST and RecistTM criteria. The consistency between these two criteria was 57.3% in the total population. A kappa test showed a poor consistency of assessment between the two methods. RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors; irCR, immune-related complete response; irPR, immune-related partial response; irSD, immune-related stable disease; irPD, immune-related progressive disease; tmCR, tumor marker-related complete response; tmPR, tumor marker-related partial response; tmSD, tumor marker-related stable disease; tmPD, tumor marker-related progressive disease.

First evaluation time and earliest response time of patients with different evaluation criteria

We compared the first evaluation time of patients using the RecistTM and irRECIST criteria. The first evaluation time with the RecistTM and irRECIST criteria in Group A was 3.39 ± 1.00 weeks vs. 6.93 ± 3.37 weeks (u = −6.543, p < 0.001), respectively; in Group B, since it was a prospective study, the first evaluation time was the 3rd and the 6th week for the RecistTM and irRECIST criteria, respectively.

Among all patients, 54 patients achieved an objective response (CR or PR) under both criteria. The first evaluation time of the RecistTM and irRECIST criteria was 3.29 ± 0.82 weeks and 6.31 ± 2.41 weeks after treatment, respectively (u = −8.470, p < 0.001), and the earliest response time was 4.64 ± 3.31 and 8.06 ± 4.26 weeks, respectively (u = −5.233, p < 0.001); the earliest response time under the RecistTM criteria was 3.42 weeks earlier than that under the irRECIST criteria. Among the patients, 46 had the first evaluation time within 3 weeks after treatment according to RecistTM criteria, and the earliest response time of 35 patients (76.1%, 35/46) was the first evaluation time. According to irRECIST criteria, 44 patients had the first evaluation time within 6 weeks after treatment, while the earliest response time of 34 patients was within 6 weeks after treatment (77.3%, 34/44), without significant differences between the two (χ2 = 0.018, p = 0.894; Figure 3).

Figure 3

The first evaluation time and the earliest response time of the patients with objective response under both criteria (n = 54). Arrow: The cases that were assessed as pseudoprogression by RecistTM or irRECIST. RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

Correlation between treatment response and survival time

In this study, the median follow-up time of Group A was 30 months, and at the time of submitting this article, 22 of 77 patients were still alive. The median follow-up time of patients in Group B was 26 months, and at the time of submitting this article, 15 of the 33 patients were still alive.

Under the RecistTM criteria, the median overall survival (OS) of different treatment responses in Group A was as follows: tmCR: not estimated (NE), tmPR: 18 m (95% CI 13.70–22.3) ; tmSD: 12 m (95% CI 5.53–18.47) ; tmPD: 9 m (95% CI 6.82–11.18)  (χ2 = 39.666; p < 0.001). There were significant differences in OS between tmCR and tmPR as well as between tmPR and tmSD (χ2 = 7.923, p = 0.005; χ2 = 4.294, p = 0.038), but not between tmSD and tmPD (χ2 = 1.330, p = 0.249). This indicates that our established RecistTM criteria can distinguish relatively well among CR, PR and SD treatment responses, but could not distinguish between patients achieving SD and PD (Figure 4A). Under the irRECIST criteria, as only two patients achieved irCR in both groups, we combined patients achieving irCR and irPR for statistical analysis in this study. The median OS of patients in Group A was as follows: irCR/irPR: 24 m (95% CI: 19.12–28.88); irSD: 15 m (95% CI: 9.74–20.26); irPD: 8 m (95% CI: 5.70–10.30) (χ2 = 31.635; p < 0.001). There were significant differences in median OS between irSD and irPD (χ2 = 10.943, p = 0.001), but not found between irCR/irPR and irSD (χ2 = 1.745, p = 0.187; Figure 4B). In order to illustrate more accuracy for prognosis achieved by RecistTM than irRECIST criteria, time-dependent ROC analysis was performed. The AUCs for 1-year and 2-year OS were 0.891 (95 (95% CI: 0.426–0.661), 0.520 (95% CI: 0.410–0.631) for irRECIST criteria% CI: 0.787–0.995), 0.678 (95% CI: 0.568–0.787) for RecistTM criteria, and 0.544, respectively. At these two time points, the AUCs achieved by RecistTM criteria were all significantly higher than those with irRECIST criteria (both p < 0.001; Figure 5).

Figure 4

Median overall survival (OS) of different efficacy evaluations by RecistTM (A) or irRECIST (B) in group A (n = 77). RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

Figure 5

Time-dependent receiver operating characteristic (ROC) analysis for predicting overall survival (OS) by RecistTM or irRECIST in group A (n = 77). A: ROC curves of 1-year OS; B: ROC curves of 2-year OS. RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

We further combined patients in Group A and Group B. According to RecistTM criteria, among the 23 patients who achieved tmCR, 19 patients were still alive by the time this article was submitted. The median OS of patients with different treatment responses under RecistTM criteria was as follows: tmCR: NE; tmPR: 20 m (95% CI: 14.16–24.85); tmSD: 12 m (95% CI: 7.42–16.58); tmPD: 9 m (95% CI: 6.61–11.39) (χ2 = 64.575; p < 0.001). There were still significant differences between tmCR and tmPR, as well as between tmPR and tmSD (χ2 = 15.572, p < 0.001; χ2 = 7.720, p = 0.005), but not between tmSD and tmPD (χ2 = 1.596, p = 0.206). Under irRECIST criteria, the median OS were: irCR/irPR: 26 m (95% CI: 17.58–34.42); irSD: 15 m (95% CI: 12.74–12.26); irPD: 8 m (95% CI: 5.55–10.45) (χ2 = 54.232; p < 0.001). There were significant differences between irCR/irPR and irSD, as well as between irSD and irPD (χ2 = 8.060, p = 0.005; χ2 = 15.034, p < 0.001; Figure 6). Consistent with the results from Group A, the AUCs achieved by RecistTM criteria were significantly higher than those by irRECIST criteria for 1-year and 2-year OS (both p < 0.001). The AUCs of RecistTM criteria were 0.862 (95% CI: 0.773–0.951) and 0.649 (95% CI: 0.563–0.736); whereas, the AUCs of irRECIST criteria were 0.552 (95% CI: 0.449–0.654) and 0.521 (95% CI: 0.433–0.610; Figure 7), respectively.

Figure 6

Median overall survival (OS) of different efficacy evaluations by RecistTM or irRECIST in all patients (n = 110). RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

Figure 7

Time-dependent receiver operating characteristic (ROC) analysis for predicting overall survival (OS) by RecistTM or irRECIST in all patients (n = 110). A: ROC curves of 1-year OS; B: ROC curves of 2-year OS. RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

When applying both criteria together, for irCR/irPR patients (n = 54), there was a significant difference in median OS between achieving tmCR (n = 22) and tmPR (n = 32) (NE vs. 22 m (95% CI: 17.49–26.51), χ2 = 14.011, p < 0.001), indicating that patients ultimately truly benefiting from immunotherapy could be further distinguished by the RecistTM criteria from patients who achieved remission according to irRECIST criteria. However, for patients with irSD, there was only a trend of difference in median OS between achieving tmCR/tmPR (n = 16) and tmSD/tmPD (n = 12) (16 m (95% CI: 6.20–25.80) vs. 15 m (95% CI: 9.67–20.33), χ2 = 3.842, p = 0.050), indicating that among patients achieving SD according to irRECIST criteria, RecistTM criteria might not be good to identify patients who could further benefit from the therapy (Figure 8).

Figure 8

Median overall survival (OS) of combined efficacy evaluations by RecistTM and irRECIST. RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

We conducted a separate analysis on the relationship between treatment response and survival time of NSCLC patients according to CEA levels. A total of 40 patients met the requirements for this analysis. The differences and trends of the relationship between treatment response and survival time among the groups were similar to those in the general population (Figure 9).

Figure 9

Median overall survival (OS) of different efficacy evaluations by RecistTM with carcinoembryonic antigen (CEA) or irRECIST in non-small cell lung cancer (NSCLC) patients (n = 40). RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors.

Non-classical response mode under the RecistTM and irRECIST criteria

In this study, there were 12 cases with non-classical response mode under the irRECIST criteria, all of which were NSCLC, accounting for 17.1% (12/70) of the NSCLC patients. Among them, 6 (8.6%, 6/70) had a dissociated-response, and 6 (8.6%, 6/70) had pseudoprogression (5 of them belonged to Group B).

In RecistTM, 4 patients with NSCLC (5.7%, 4/70) had pseudoprogression, and the tumor markers involved included CEA, CA199, and CA125. Among them, three patients showed a short-term increase in tumor markers within 3 weeks after treatment, followed by a rapid decrease, accompanied by pseudoprogression in radiology. In another case, CEA and CA199 showed different responses; CEA decreased rapidly after treatment while CA199 showed pseudoprogression after treatment (Table 3, Figs. 3 and 10).

Table 3

The patients with atypical response pattern evaluated by RecistTM and irRECIST.

	Disease	RecistTM			irRECIST
		Tumor marker	Efficacy	Response pattern	Efficacy	Response pattern
1	LUSC	SCCAg	PR	–	SD	DR
2	LUAD	CEA	PR	PsPD	PR	PsPD
3	LUAD	CEA	PD	–	SD	DR
4	LUAD	CEA	PR	–	SD	DR
5	LUSC	CA125	PR	–	SD	DR
6	LUSC	CEA	SD	–	SD	DR
7	LUAD	CEA	SD	–	SD	DR
8	LUSD	CYFRA21–1	CR	–	PR	PsPD
9	LUAD	CA199	PR	PsPD	PR	PsPD
10	LUAD	CA125	PR	PsPD	PR	PsPD
11	LUAD	CEA/CA199	CR	PsPD (CA199)	PR	–
12	LUAD	CEA	CR		PR	PsPD
13	LUSC	CA199	CR		PR	PsPD

RecistTM, response evaluation criteria in solid tumors based on tumor markers; irRECIST, immune-related Response Evaluation Criteria in Solid Tumors; LUSC, Lung squamous cell carcinoma; LUAD, Lung Adenocarcinoma; PsPD, Pseudoprogression; DR, Dissociated response. CR, Completed response; PR, partial response; SD, stable disease; PD, progressive disease; CEA, carcinoembryonic antigen; CA199, carbohydrate antigen 19–9; CA125, carbohydrate antigen 125; SCCAg, squamous cell carcinoma antigen; CYFRA21–1, Cytokeratin fragment antigen 21–1.

Figure 10

Tumor marker pseudoprogression in four patients. Cases 1–3 showed a short-term increase in tumor markers within 3 weeks after treatment, followed by a rapid decrease, accompanied by pseudoprogression in radiology. In Case 4, CEA and CA199 showed different responses; CEA decreased rapidly after treatment while CA199 showed pseudoprogression after treatment. PR, partial response; SD, stable disease; PD, progressive disease. CT, computed tomography; CEA, carcinoembryonic antigen; CA199, carbohydrate antigen 19–9; CA125, carbohydrate antigen 125.

Discussion

Due to immune cell infiltration and the presence of necrotic tumors resulting from fibrotic tissue repair, the existing RECIST criteria cannot truly reflect the actual patient treatment response. Although multi-spot biopsy assists in identifying the actual treatment response of patients administered with ICIs, the best timing for biopsy is not yet clear, and this invasive procedure is not convenient except for preoperative neoadjuvant immunotherapy. At present, most studies reported that circulating tumor DNA (ctDNA) and circulating immune cells can monitor the treatment response to immunotherapy at an early stage and distinguish pseudoprogression from true disease progression.20, 21, 22, 23 Goldberg et al. examined 182 consecutive plasma samples from 49 patients with metastatic NSCLC who received anti-PD-1 or anti-PD-L1 therapy and found that patients with a ctDNA level decrease >50% (n = 14) showed longer survival than those in whom the ctDNA level decreased by <50% (n = 14) (205.5 vs. 69 days, p < 0.001). The recent study from phase 3 CheckMate −816 trial also proved that ctDNA clearance may indicate the benefit of pathological response. In that study, 33.3% (13/39) of patients with ctDNA clearance achieved pathological complete response (pCR), while in patients without ctDNA clearance, only one patient (2.1%, 1/48) was found to achieve pCR. However, there were still 66.7% (13/24) of patients with ctDNA clearance that did not achieved pCR. Lee et al. reported a sensitivity, specificity, positive predictive value, and negative predictive value in predicting the recurrence of melanoma of 78%, 95%, 79%, and 51%, respectively. Therefore, due to the influence of detection techniques and the amount of free ctDNA in blood, the detection of tumor DNA in the blood has a relatively high false-negative rate, which can easily mislead doctors.

FDG PET/CT can be also used for the prediction of response to immunotherapy, Kaira et al. reported that using early PET / CT evaluation after the beginning of ICIs, the uptake kinetics of NSCLC patients after 1 month of immunotherapy can better predict the further development of the disease than the corresponding CT measurement. However, Enhanced 18F-FDG uptake can also be triggered by an increased influx and activity of immune cells infiltrates induced by ICIs therapy itself., In a noteworthy study, Humbert et al. Investigated NSCLC patients treated with ICIs who had PD according to PET / CT at the 7th week after treatment initiation. These patients continued to receive treatment. Of the 19 patients, 42% had a disease progression, which was defined as ICIs continuation of six months. However, 26% were confirmed as dissociated response and 32% were confirmed as pseudoprogression. In addition, many studies found that immune-related adverse events such as pneumonitis, thyreoiditis or hepatitis, may lead to abnormal patterns of 18F-FDG tracer uptake in various involved organs. These findings represent possible defects in PET/CT interpretation.29, 30, 31 Therefore, its application in the evaluation of ICIs response has great limitations. Moreover, the standardization of FDG PET/CT interpretation is missing and different criteria, such as EORTC, PERCIST, and PERCIMT, have been investigated until now.

While monitoring recurrence and progression, tumor marker responses often appear earlier than imaging findings, as shown by biochemical recurrence in ovarian cancer. Recent studies have shown that decreasing levels of tumor markers positively correlated with treatment response and overall survival (OS) after immunotherapy., Zhang et al. enrolled 308 patients with advanced NSCLC and evaluated four tumor markers (CEA, CA125, CYFRA21-1, and SCCAg), patients with a decrease in ≥2 markers had higher ORR and longer progression-free survival (PFS). Bello et al. included 71 patients with NSCLC, all treated with nivolumab monotherapy. As a result, the OS of patients whose tumor markers decreased >20% was significantly longer than that of patients whose tumor markers decreased <20%. However, those studies have included ≥50% of the patients with baseline tumor marker levels within normal ranges. Thus measuring tumor marker levels may not be appropriate to evaluate treatment responses and can affect the reliability of the conclusions.

In the present study, 110 patients with a three-time increase in tumor markers receiving immunotherapy were recruited. The results showed that the treatment response evaluated by the RecistTM criteria was not consistent with that evaluated by the irRECIST criteria, with significant differences between the two. However, according to the median patient survival time, the RecistTM criteria could effectively distinguish CR, PR, and SD. It had a higher CR rate than that evaluated by the irRECIST criteria and could further distinguish patients with better benefits from PR patients under the irRECIST criteria. RecistTM criteria can predict OS more accurately than irRECIST criteria according to time-dependent ROC analysis for 1- year and 2-year OS. In addition, because tumor marker detection is convenient and affordable, its first evaluation time is significantly shorter than those of imaging. Therefore its earliest response time is also shorter, and the treatment response to immunotherapy can be observed earlier. However, the main defect of this criteria is that it cannot effectively distinguish patients with SD from those with PD, which means that if the patient has no significant decrease in tumor markers, it suggests that the treatment is ineffective and the patient cannot benefit from immunotherapy. However, whether to change the treatment regimen still needs to refer to irRECIST criteria, not RecistTM criteria, as the latter needs further clinical study to verify. In addition, due to the low number of cases, this study failed to prove that the RecistTM criteria could identify the patients who benefited from the immunotherapy among the SD patients under irRECIST criteria.

Tumor markers mainly originate from tumor cell metabolites or their disintegration products after necrosis. Usually, after surgical tumor resection, the elevated tumor markers quickly drop to normal levels. Therefore, in theory, tumor markers will not be affected by local immune cells in tumor tissues, and the tumor marker changes in patients receiving immunotherapy will reflect the tumor status better than imaging findings. However, in this study, four patients showed pseudoprogression of tumor markers, accounting for 5.7% of the NSCLC patients. Although this ratio is <8.6% incidence rate under the irRECIST criteria in this study, it still exceeds our initial expectations. We speculate this could be explained by tumor marker levels being affected by benign diseases, including systemic or local immune inflammation. This is consistent with our previous finding that autoimmune diseases could lead to increased CEA. The peritoneal, pleural, and pericardial effusion caused by mesothelial cell activation resulting from inflammation can also lead to increased CA125 (the increased CA125 in case 3 was accompanied by pericardial and pleural effusion)., In addition, benign diseases related to immunity such as autoimmune hepatitis, interstitial lung disease associated with rheumatoid arthritis, and systemic lupus erythematosus can also lead to increased CA199.39, 40, 41 Therefore, in cases where tumor markers increase early after treatment, we do not recommend changing the treatment plan immediately. Instead, we need to dynamically observe the subsequent changes in tumor markers and comprehensively assess the changes by including imaging findings.

As this study consisted of two parts: retrospective study and prospective study, the results can represent a relatively wider population. However, they can only represent the population of patients with tumor markers >3 times higher than baseline. Thus the results does not apply to patients with normal or only slightly higher baseline tumor markers. Also, there are other limitations to the present study. First, there were certain detection errors in detection tests. When in doubt, repeated detection was needed to be performed. Second, this is a single center study with small sample sizes and the major focus was directed to lung cancer, which may affect the reliability of the conclusion and the possibility of application to all solid tumors. Third, although this study proved that the RecistTM criteria may be superior to the irRECIST criteria in some aspects, however, this superiority needs to be further confirmed by comparison with pathological response. Thus the criteria we established based on tumor markers still need to be further validated in multicenter study with large sample sizes and refined based on different tumor markers and different tumor types. In addition, we also need to evaluate the correlation between RecistTM criteria and pathological response in neoadjuvant studies.

In summary, this study can help clinicians identify the population benefiting from ICIs therapy in advance, which may help resolve the shortcomings of the RECIST criteria in evaluating the treatment response to immunotherapy, especially in assessing whether patients can achieve deep or even complete response as soon as possible.

Authors’ contributions

Yi Yang, Xiaolin Jiang and Yun Liu undertook most of the work, including data collection and analysis. they should be regarded as co-first author. Xuemei Li and Xunjie Kuang participated in data collection. He Xiao participated in data analysis. Kan Gong, Huan Huang, and Yanli Xiong participated in patient management. Xueqin Yang designed the study. Yi Yang participated in writing - original draft and Xueqin Yang participated in writing -review & editing. All authors have contributed to the last version of the manuscript. The authors read and approved the final manuscript.

Funding

This work was supported by the Key projects of Chongqing Health and Family Planning Commission (to Xueqin Yang, 2019ZDXM011)

Data sharing statement

The data presented in this study are available on request from the corresponding author.

Declaration of interests

The authors declare that they have no competing interests.