Endoscopic ultrasound as a reliable tool for assessment of pancreatic adenocarcinoma treatment: Example of in situ gene therapy.

Background and study aims  In pancreatic cancer, the antitumor effect can only be assessed by means of a computed tomography (CT) scan using RECIST (Response Evaluation Criteria in Solid Tumours) criteria. The aim of this study was to assess the intra-observer and interobserver agreement of endoscopic ultrasound (EUS) imaging in assessing tumor volume in primary pancreatic cancer. Patients and methods  During a Phase 1 gene therapy trial, 21 patients had EUS before the first and second EUS-guided in situ gene therapy injections. All anonymized EUS files were then randomly distributed to three gastroenterologists/endosonographers and three radiologists (blind status). The largest tumor diameter was measured and the intraclass correlation coefficient (ICC) was determined. Results  Intra-observer and interobserver agreements were good to excellent, regardless of operator experience (junior versus senior member of staff) (ICC: 0.65 to 0.84). A comparison of pretreatment and post-treatment measurements by the investigators highlighted a significant antitumor effect (-11 %; P  = 0.0098), similar to that obtained during the generic protocol (-10 %; P  = 0.0045). Conclusions  Interobserver agreement regarding primary pancreatic adenocarcinoma measurements appears good to excellent, thus paving the way for the future inclusion of EUS assessments, particularly in trials assessing local therapies for pancreatic tumors. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

Pancreatic ductal adenocarcinoma (PDAC) is the fifth leading cause of cancer-related deaths in Western countries and will be the second cause of cancer-related deaths within 5 years 1 2 . Surgical resection is the only curative treatment available. Unfortunately, curative surgery is possible in only 10 % to 15 % of cases. From a palliative perspective, chemotherapy can be administered to patients with locally advanced (gemcitabine) or metastatic (FOLFIRINOX – gemcitabine + abraxane) tumors. These treatments improve some clinical parameters but culminate in modest median survival rates 1 2 3 4 5 . Gene therapy constitutes an innovative approach for treatment of PDAC 6 .

We conducted a pilot Phase I gene therapy trial (Thergap-1 trial) in unresectable patients with unresectable pancreatic cancer a gene therapy product CYL-02 GMP grade (i. e. plasmid encoding for SST2 and DCK::UMK genes complexed to a non-viral synthetic vector, polyethylenimine. – Cayla-InvivoGen Company, Toulouse, France). The Thergap-1 protocol was based on two EUS-guided direct intratumoral injections (1 month apart) of increasing doses of DNA, followed by gemcitabine infusions over a 2-month period 7 . After enrolling 22 patients with unresectable pancreatic cancer, we noted the excellent feasibility of this protocol with no serious adverse events directly imputable to the investigational medicinal product. A significant reduction in tumor size was noted at 1 month and stable condition at 2 months 7 .

The antitumor effect, which was not described in our report, was measured by comparing tumor size before each EUS-guided CYL-02 intra-tumor injection. The larger diameter of the primary tumor was recorded during EUS. Significant regression in PDAC tumor volume was not mentioned in the final report (published as an original manuscript) because RECIST criteria could not be applied to the EUS assessment. According to the relevant rationale, a CT scan is standardized, reproducible, easily exportable, and shared via a network or CD-ROM (with imaging quality intact). To date, EUS has not been included in therapeutic assessment of any nonsurgical treatment of PDAC. Interest is obviously limited because the window extends only to the primary tumor and surrounding structures, such as vessels, peritoneum, proximal lymph nodes, duodenum, stomach, and spleen. Conversely, PDAC often presents as a hypoechoic/hypodense structure with unclear limits, considerable infiltration, and pseudopods. Measurement of it would benefit from high-resolution EUS examination. Concerning the development of intratumoral treatment such as radiofrequency or gene therapy, it is important to recognize the role of EUS in the range of investigations used to assess and follow up on PDAC treatment, and locally advanced tumors in particular.

The aim of this study, therefore, was to assess: 1) the intra-observer and interobserver agreement of EUS in assessing primary tumor volume in PDAC patients; and 2) antitumor effect of in situ gene therapy recorded during the interobserver agreement protocol.

Patients and methods

Patients

During the Thergap-1 trial, 21 patients (mean age 61 years; 14 men, 7 women) (full description given in Table 1 ) underwent EUS. EUS records were available at baseline and after 1 month for 17 of the patients. With regard to the four remaining patients, two had received only one baseline injection of gene therapy product CYL-02 (septicemia due to chronic biliary-stent obstruction, rapid disease progression) and no EUS records were available for the other two patients at 1 month (incomplete records during the Thergap-1 protocol) (i. e. 21/21 EUS records at baseline and 17/21 EUS records at 1 month). The Thergap-1 trial (gene therapy for advanced pancreatic adenocarcinoma) protocol was approved by the Ethics Committee (Comité de Protection des Personnes Sud-Ouest N°1, number 1–10–21) on August 2010 (the sponsor was the CHU of Toulouse), and by AFSSAPS (French Health Products Safety Agency) (No. TG.10.05.01) and the HCB (High Council for Biotechnologies) (No. 4883) on November 2010 (EUDRACT number: 2006-005317-35-A; Clinical Trial http://clinicaltrials.gov/ct2/show/NCT01274455 ). Each patient gave their written informed consent to receive treatment and undergo additional investigations including EUS 7 .

Clinical characteristics of 21 pancreatic cancer patients enrolled in the study.

Variables	N (%)
Age (yr)	61 ± 9
Male gender, n (%)	14 (67)
Tumor status, n (%)
Locally advanced	13 (62)
Metastasis	8 (38)
Tumor site
Head	9 (43)
Body	10 (48)
Caudal region	2 (9)
Earlier treatments prior to enrolment in the Thergap-1 trial
Chemotherapy ( 1 )	10 (48)
Biliary stent	4 (19)
Surgery	1 (5)

FOLFIRINOX n = 6; gemcitabine n = 1; FOL-FOX n = 1; radiochemotherapy n = 2.

EUS recordings, investigators and assessment

Prior to injection of the gene therapy product (i. e. at Visits 1 and 7) ( Fig. 1 ), anonymized EUS examinations were performed, each one lasting at least 3 minutes (Olympus Professional CD recorder ref. – mpg4 format – one CD-ROM per examination) including systematically small and high magnifications of the target tumor to avoid bias and difficulty in assessment during subsequent video analysis. In addition, the same echoendoscope (Olympus GFUCT-140 – Hamburg, Germany) and the same ultrasound device (Aloka Alpha5 Ultrasound – Landsberg am lech, Germany) were used during the protocol to avoid any bias in image quality.

Fig. 1 

Flowchart of the Thergap-1 trial. Vertical blue arrows indicate the times of EUS recordings immediately before administration of the CYL-02 (gene therapy product) intratumoral injection (V1 and V7); Each EUS-guided intratumoral injection of CYL-02 was followed by Gemcitabine infusions (1000 mg/m 2 each week for 3 weeks per month – indicated by vertical grey arrows). CT, computed tomography.

Ultimately, a total of 38 records were available for the present study. Ten additional anonymized EUS examinations (so-called “training records”- performed outside the Thergap-1 protocol) were included (recorded under the same conditions as for the Thergap-1 protocol): five patients with unresectable PDAC, confirmed by histological examination, three patients with well-defined cystic tumors (positive control for a clear measurement), and two patients with a normal pancreas (negative control with no tumor). All of these training records were used as baseline training assessments for the present protocol. Overall, 48 recordings were assessed by each investigator. The team comprised three gastroenterologists/endosonographers (two senior [BN, BB] and one junior [AC]) and three radiologists performing gastrointestinal imaging (US, CT, angiography, and interventional – two senior [FM, OM] and one junior [BM]). The two senior gastroenterologists/endosonographers had at least 10 years of experience in EUS while the junior endosonographer was an accomplished operator with at least 5 years of experience in endoscopy, including at least 150 EUS that comprised at least 50 EUS-guided fine-needle aspiration biopsies. The two senior radiologists had at least 10 years of experience in gastrointestinal diagnostic and interventional radiology, while the junior radiologist had at least 5 years of experience in gastrointestinal ultrasonography and radiology.

Randomization

All anonymized EUS recording files from the THERGAP-1 trial, from numbers 1 to 38, were randomly distributed to each investigator, regardless of treatment group and time of injection (blind status). Each investigator was randomly given a set of records to review. All of the investigators had access to the 10 training records, which were read at the start of the assessment.

Examination of the EUS videotapes to assess tumor volume

Each investigator assessed the 10 training records before looking at their own set of THERGAP-1 EUS recordings. After reading the EUS records on computer (using Window Media Player or VLC player), each investigator had to choose three pictures per record which, in their opinion, provided the optimum view for measuring the maximum diameter of the primary PDAC tumor. These three views were then transferred to PowerPoint slides created especially for the study (3 tagged slides per recording). After pasting in the selected EUS images, the maximum size was indicated using the “insert lines and connectors” function with an arrow depicting the maximum tumor size. Each set of PowerPoint slides was saved, subsequently printed out, and all measurements taken as follows (LB, CC). The estimated maximum tumor diameter was measured with a ruler on a printed sheet (i. e. drawn arrow). The value was divided by the value measured between the two consecutive bars of the centimeter scale shown in the vertical section of each image ( Fig. 2 ). We selected the larger diameter among the three selected images for the statistical analysis. In addition, investigators had to assess the quality of each EUS video and image according to three criteria: “poor, mild, and good.” In terms of intra-observer agreement, three endosonographers (AC, BB, and BN) and two radiologists (FM, OM) took a wash-out period of 15 to 21 days after the initial examinations before doing second readings of the same EUS records.

Fig. 2 

Method used to record the maximum volume of primary pancreatic adenocarcinoma on EUS images captured from the video recordings. On the printed sheet showing the selected image, a ruler was used to measure the maximum volume as drawn by the expert (M1 in cm). This value was divided by the ruler measurement between two bars of the centimeter scale shown vertically to the right of each picture.

Statistical analysis

The statistical analyses were carried out using STATA 14.2 software (STATA Corp., College Station, Texas, United States). Patient characteristics were summarized with descriptive statistics appropriate for the type of variable in question. These descriptive statistics included the mean value with standard deviation (SD) for continuous variables and the number (N) and frequency (%) for categorical variables. Interobserver agreements between radiologists and gastroenterologists, and between trainees and senior investigators, as well as intra-observer agreement between three gastroenterologists and two radiologists, were assessed by calculating the intraclass correlation coefficient (ICC) with the associated 95 % confidence interval. The agreement values were interpreted as follows: ICC < 40, poor agreement; 0.41 to 0.60, fair agreement; 0.61 to 0.80, good agreement; and 0.81 to 100 excellent agreement. The size of the primary pancreatic tumor at V1 and V7 ( Fig. 3 ) was compared using the paired Wilcoxon test. P values reported were two-sided and the significance threshold was set at < 5%.

Fig. 3 

Assessment of the antitumor effect 1 month after in situ gene therapy plus gemcitabine infusions. Primary pancreatic adenocarcinoma volumes were measured before CYL-02 intra-tumoral injection at baseline (D1,V1) and after 1 month (D28, V7). The volume (maximum tumor size) at 1 month was compared to the baseline reading. a Measurements recorded in 17 patients by the principal investigator of the Thergap-1 trial (LB). b Measurements recorded in the same 17 patients by the six investigators in this study (AC, BB, BN, FM, OM, BM). Mean ± SD; the median value is shown in italics. *** P  = 0.0045. ** P  = 0.0098 (the paired Wilcoxon test).

Results

All investigators were able to analyze the 38 videos at least once (interobserver analysis) or twice (intra-observer analysis). The largest tumor diameter could be measured systematically (100 % feasibility). The quality of the recordings was deemed to be poor, average, and good in 18 %, 44 % and 39 % cases, respectively ( Table 2 ). Intra-observer and interobserver agreements are shown in Table 3 (whatever the time of examination during the Thergap-1 protocol). These were found to be good to excellent, regardless of operator experience (junior versus senior). In Table 4 , the results of the interobserver agreement are presented by separating the examinations performed before the intratumoral treatment with gene therapy followed by gemcitabine infusions (Visit 1 – Day 1) ( Fig. 1 ) from the examinations performed 1 month after the start of treatment (Visit 7 – Day 28) ( Fig. 1 ). The interobserver agreement was good to excellent for V1, but decreased at V7, particularly during the evaluation by the radiologists (junior or senior). On the other hand, the results for radiologists were generally worse than those for gastroenterologists, whatever the analysis (i. e. all times combined in Table 3 or separating V1 and V7 in Table 4 ).

Image quality of 38 EUS recordings assessed by six investigators during the Thergap-1 gene therapy trial.

Quality/investigators	A	B	C	D	E	F	All
Good n (%)	16 (42)	9 (24)	23 (61)	14 (37)	7 (18)	19 (50)	88 (39 %)
Average n (%)	18 (48)	19 (50)	11 (29)	20 (53)	20 (53)	12 (32)	100 (44 %)
Poor n (%)	4 (10)	10 (26)	4 (10)	4 (10)	12 (32)	7 (18)	41 (18 %)

A, junior gastroenterologist; B and C, senior gastroenterologists; D and E, senior radiologists; F, junior radiologist.

EUS, endoscopic ultrasound.

Interobserver and intra-observer agreement regarding the maximum volume of primary pancreatic tumours based on 38 EUS video examinations recorded during the Thergap-1 gene therapy trial.

Class	ICC value	95 % CI	Interpretation
Interobservers: gastroenterologists versus radiologists	0.73	0.63–0.81	Good
Interobservers: junior versus senior gastroenterologists	0.84	0.76–0.90	Excellent
Interobservers: junior versus senior radiologists	0.65	0.50–0.76	Good
Intra-observers: first versus second examination 1	0.76	0.65–0.84	Good

EUS, endoscopic ultrasound; ICC, interclass correlation coefficient; CI, confidence interval.

The same EUS records were evaluated again by three gastroenterologists and two radiologists after a 15– to 21-day washout period.

Interobserver agreement regarding the maximum volume of primary pancreatic tumours based on video examinations recorded before and after gene therapy plus gemcitabine treatment.

Class	ICC value	95 % CI	Interpretation
Interobservers: gastroenterologists versus radiologists before treatment 1	0.83	0.73–0.89	Excellent
Interobservers: junior versus senior gastroenterologists before treatment 1	0.85	0.74–0.91	Excellent
Interobservers: junior versus senior radiologists before treatment 1	0.75	0.57–0.85	Good
Interobservers: gastroenterologists versus radiologists after treatment 2	0.56	0.35–0.72	Fair
Interobservers: junior versus senior gastroenterologists after treatment 2	0.82	0.67–0.90	Excellent
Interobservers: junior versus senior radiologists after treatment 2	0.52	0.22–0.72	Fair

ICC, interclass correlation coefficient; CI: confidence interval.

Visit 1 – Day 1.

Visit 7 – Day 28.

In addition, the comparison of pretreatment and post-treatment measurements (V1 versus V7) by the investigators revealed a significant antitumor effect (–10 %; P  = 0.0045 – right panel b), similar to results obtained in the Thergap-1 protocol (measured by the principal investigator LB; –11 %; P  = 0.0098 – left panel a) ( Fig. 3 ).

Discussion

Although EUS only assesses locoregional disease, it is more accurate than conventional cross-sectional imaging (CT or magnetic resonance imaging) for displaying the primary PDAC tumor. The latter often infiltrates adjacent structures, has irregular contours, and is associated with pseudopods; hence, it is difficult to obtain an accurate measurement of the tumor diameter. Despite the fact that several interobserver agreement studies have been conducted with EUS for diagnosis of pancreatic diseases such as chronic pancreatitis and accumulation of pancreatic fluid, very few have included solid pancreatic masses, especially PDAC 8 9 10 11 12 13 14 . Moreover, only endosonographers have taken part in these pilot or multicenter studies, and no radiologists were included in the panel of experts.

To our knowledge, our study is the first to demonstrate the feasibility and good interobserver agreement of EUS assessments of maximum tumor diameter in pancreatic adenocarcinoma. These encouraging results, including determination of the antitumor effect, are based on standardized recordings and straightforward measurement of the largest tumor diameter. Furthermore, EUS video recordings can be accurately read not only by gastroenterologists but also by radiologists familiar with ultrasound imaging. However, the performance of radiologists is generally worse than that of gastroenterologists and this seems to be even more evident after treatment ( Table 4 ). This may explain the good agreement (and not excellent) in the overall analysis mixing records evaluated by gastroenterologists and radiologists. On the other hand, the agreement within gastroenterologists was systematically excellent. This can be explained by the fact that radiologists are less used to (or even naive about) static and dynamic echo-endoscopy images. Finally, if an independent EUS evaluation will be included in a future pancreatic cancer treatment trial, review by gastroenterologists/endosonographers would be preferable.

Regardless of the good agreement between investigators in terms of tumor diameter, there was also perfect agreement with regard to the antitumor effect noted in the gene therapy trial by the principal investigator 6 . Although the effect was modest and is currently being assessed on a larger scale during the Phase 2 study (Thergap-2 trial), this method is indicative of an important area of investigation in line with the EUS-guided administration of local antitumor treatments such as gene therapy, radiofrequency, and chemotherapy 6 15 16 17 18 . A centralized system, therefore, can be established for future protocols, provided that a clearly defined, standardized EUS recording is obtained. Moreover, contrast EUS could be used to accurately assess necrotic areas (observed elsewhere in the Thergap-1 study).

Study limitations include the small sample size. However, the design of Phase 1 with “first-in-human administration” of an innovative treatment (such as gene therapy product) is generally restricted to a small number of inclusions. Another limitation is the risk of investigator selection bias, although we purposely chose junior and senior staff. Looking at the results in Table 3 (all recordings regardless of the time of treatment) and Table 4 (separate pretreatment and post-treatment recordings), there is xcellent agreement between the junior and senior gastroenterologist reviewers. In addition, there is the lack of comparison with RECIST criteria on the CT scan. This was not feasible because CT scans were performed 1 month later according to the protocol ( Fig. 1 ). On the other hand, given that the images were judged to be of poor quality in 18 % of cases, it will be important to better establish rules for recording and image quality (and therefore for recorders) for this type of study. Subsequent trials will benefit from new-generation ultrasound devices and EUS endoscopes may be more efficient than those used in the present study.

It is noteworthy that the antitumor effect observed by the principal investigator (LB), who was not an investigator in the present protocol and who had measured the tumor volume during the examination of the patient (and without review) was confirmed by the examiners a few months later on the video recordings. This corresponds, in some way, with another concordance. In addition to the role of EUS in molecular diagnosis of pancreatic cancer and subsequent gene therapy 19 , another important indication for it can be added, namely assessment of pancreatic tumor volume during prospective trial follow-up.

Conclusions

In conclusion, based on a unique gene therapy protocol, interobserver agreement between experts, endosonographers, and radiologists appears to be good to excellent in terms of primary pancreatic tumor volume. This paves the way for the inclusion of EUS assessments in future trials focusing on local treatment of non-metastatic tumors in particular.