Endoscopic ultrasound versus computed tomography in determining the resectability of pancreatic cancer: A diagnostic test accuracy meta-analysis.

BACKGROUND/AIM: Endoscopic ultrasound (EUS) and contrast-enhanced computed tomography (CT) with pancreas protocol are used in assessing the resectability of neoplastic pancreatic lesions. Here, we performed a diagnostic test accuracy (DTA) meta-analysis, comparing the diagnostic accuracy of EUS and CT in evaluating the resectability of pancreatic cancer using surgical assessment as the reference standard. PATIENTS AND METHODS: A comprehensive electronic search was conducted up to March 2020. Studies comparing EUS and CT in assessing the resectability of pancreatic cancer using surgical assessment as reference standard were included. QUADAS-2 tool was used to assess the quality of the included studies. After data extraction, an analysis was done using DerSimonian Laird method (random-effects model) to estimate the overall diagnostic odds ratio (DOR) and determine the best-fitting receiver operating characteristics (ROC) curve.
RESULTS: Two studies, with 77 subjects combined, were included in the analysis. Overall, the risk of bias was moderate. EUS and CT were comparable in determining the resectability of pancreatic cancer with AUC = 75% (95% confidence interval (CI) 66%- 84%) for EUS as compared to 78% (95% CI 69%- 87%) for CT (P > 0.05). Pooled sensitivity and specificity was 87% (95% CI 70%- 96%) and 63% (95% CI 48%- 77%), respectively for EUS and 87% (95% CI 70%- 96%) and 70% (95% CI 55%- 83%), respectively for CT. DOR was 11.51 (95% CI 3.55- 36.81) for EUS as compared to 15.91 (95% CI 4.83- 51.62) for CT (P > 0.05).
CONCLUSIONS: Both EUS and CT provide reasonable sensitivity and specificity to detect the resectability of pancreatic cancer.

INTRODUCTION

Pancreatic cancer is the second most common cause of death from gastrointestinal malignancy and the seventh most common cause of death from cancer worldwide, with an overall estimated 5-year relative survival rate of 8.5%.[1] As symptoms are uncommon in the early stages, the disease is usually advanced at the time of diagnosis and is most often fatal.[1] Surgical resection with curative intent is the only treatment option that offers a chance of cure and is possible in less than 25% of cases at diagnosis.[2] The decision to operate with curative intent is dependent on the determination of resectability, therefore the challenge is to accurately determine resectability, to avoid unnecessary surgeries in patients with unresectable disease and to avoid denying surgery to those with resectable disease. Thus, while there are several applications of endoscopic ultrasound (EUS) in addition to determining resectability, including staging, sampling of tissues for diagnosis and atypical lymph nodes, in this review, we focus on the assessment of resectability of pancreatic cancer by EUS and computed tomography (CT).[345]

Available guidelines recommend EUS as an add-on investigation following CT scan to confirm the position of the tumor, further assess resectability and obtain biopsies, though it is not recommended as a routine staging tool.[34] Multiple studies have assessed and compared different imaging modalities in the assessment of pancreatic cancer resectability. Very few of these, however, directly compared EUS and CT and reported the diagnostic test accuracy data with separate data on pancreatic cancer. Two previous conventional meta-analyses compared EUS and CT,but these did not compare the diagnostic accuracy using the proper methodology recommended by the Cochrane Collaboration on performing a diagnostic test accuracy meta-analysis.[567] A meta-analysis of diagnostic test accuracy differs from a conventional meta-analysis in the assessment of article quality, statistical analysis and reported outcomes. This study aims to determine and compare from direct comparative studies, the diagnostic test accuracy of EUS and CT in evaluating the resectability of pancreatic cancer using surgical assessment as the reference standard.

METHODS

Registration

The study protocol was designed in accordance with PRISMA guidelines and registered (CRD42018076984) with the International Prospective Register of Systematic Reviews.[8]

Study selection

We included retrospective and prospective studies directly comparing EUS and CT in assessing the resectability of pancreatic cancer with surgical assessment being the reference standard. We accepted any criteria for resectability utilized in the studies. There were no restrictions on the ultrasound frequency or type of EUS or CT used. We did not exclude studies based on language, location or quality of the studies. We excluded studies with insufficient data, pediatric studies, duplicate publications, studies with no reference standards, and case-control studies as there is a high risk of bias.

Electronic searches

Two authors (MR and BC) completed a comprehensive literature search using OVID (EMBASE, HealthStar, MEDLINE), PubMed, EBSCO (CINAHL, e-journals), Web of Science, and Google Scholar to search for eligible publications up to March 2020. The following search terms were used: pancreatic, pancreas, cancer, carcinoma, adenocarcinoma, malignancy, tumour, EUS, endoscopic ultrasound, endosonography, CT, computed tomography, computerized tomography. No restriction was applied in terms of language, location or quality of the studies during the literature search. Recursive searches and cross-referencing were carried out by using a “similar articles” function. We also manually reviewed references of articles identified after the initial search.

Three authors (MR, BC, PF) independently reviewed the full text of studies deemed appropriate by at least one author after initial literature search and abstract review. The full-text articles that fulfilled the inclusion criteria and provided the necessary data were compared and reviewed. Where insufficient or conflicting data was presented in the articles which fulfilled the inclusion criteria otherwise, we attempted to communicate with the authors of the primary papers. Those included were by consensus. Discrepancies were resolved through discussion. A fourth author (MY) was involved in the discussion of discrepancies.

Data extraction and management

Two review authors (MR and BC) independently extracted the data from the included studies. True positive, true negative, false negative, and false-positive values for the determination of resectability by EUS and CT respectively were extracted separately by each reviewer and results were compared. Discrepancies were resolved through discussion. A third author (MY) was involved in the discussion of discrepancies and in extracting and assessing the data. The raw data was then computed with the assistance of LM and LT.

Assessment of methodological quality

The quality of each included study was assessed using the Quality Assessment of Diagnostic Accuracy Studies- 2 (QUADAS-2) assessment tool as recommended by the Cochrane collaboration.[7] It comprises two main categories; risk of bias and applicability, each with its own set of domains. We considered any study classified as low risk of bias and applicability concerns in all domains to have been of high methodological quality.

Outcome measure

The main outcome of interest was the diagnostic test accuracy of EUS and CT in assessing the resectability of pancreatic cancer. Secondary objectives were to compare the sensitivity and specificity of EUS and CT for determining resectability, to compute diagnostic odds ratio (DOR), positive and negative likelihood ratio (LR), and investigate the source of heterogeneity in the final analysis based on the methodology of included studies.

Statistical analysis and data synthesis

DerSimonian Laird method (random-effects model) was used to estimate the overall DOR and hence to determine the best-fitting receiver operating characteristics (ROC) curve. This allowed us to calculate the summary ROC (sROC) and area under curve (AUC). A perfect test has an AUC close to 1, and poor tests have AUCs close to 0.5. In STATA version 12 (College Station, Texas), we used the roccomp command to test for equality of ROC areas for EUS and CT, the diagt command to derive summary statistics and the mcc command to compare sensitivities and specificities. We compared DORs using the approach recommended by Altman and Bland via WinPEPI software.[910] Some commands required the use of patient-level data. The published data were transformed into datasets using the reported true positive, true negative, false negative, and false-positive values for each staging. We used RevMan version 5.3 to create forest plots and risk of bias graphs. We planned to assess the risk of publication bias by using the Funnel plot and conduct a meta-regression using the Moses-Shapiro-Littenberg approach if the number of included studies was more than 10. We report pooled sensitivities and specificities, DORs, and AUCs, alongside 95% confidence intervals (CIs) and P values where appropriate. A comparative AUC graph is shown.

RESULTS

Literature search

A total of two out of 3793 records met the inclusion criteria and were included in the diagnostic test accuracy meta-analysis. Both studies were prospective and included a total of 77 patients with pancreatic cancer, who had both EUS and CT. Figure 1 depicts the PRISMA flowchart for the detail of study selection, and Table 1 presents the characteristics of included studies. The quality of included studies and risk of bias using the QUADAS-2 tool is represented in Figure 2.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of study identification, inclusion, and reasons for exclusion

Table 1

Characteristics of included studies

Study	DeWitt et al.	Ramsay et al.
Publication Year	2004	2004
Patients	53	25
Mean age of patients	64	57
Female gender %	43	44
Enrolment	Consecutive	Consecutive
Study Type	Prospective	Prospective
Reference Standard	Intraoperative examination with a pathological assessment if resection attempted	Surgical staging, consensus opinion
EUS	Radial and linear Frequency not stated	Radial 7.5 and 12 MHz
CT	Multidetector CT with a quad-channel scanner	Single array spiral CT scanner.
Blinded	No	Yes

EUS: Endoscopic ultrasound, CT: Computed tomography, MHz: Megahertz

Figure 2

QUADAS-2 analysis risk of bias and applicability concerns summary. Recommended by the Cochrane Collaboration for the assessment of risk of bias in included studies

Comparison of EUS and CT

In the determination of resectability, EUS had an area under the curve (AUC) of 75% (95% CI 66% – 84%) as compared to 78% (95% CI 69% – 87%) for CT [Figure 3]. In head to head comparison, EUS and CT were comparable (χ2= 0.3294, degrees of freedom [df] = 1, and P > 0.05) [Table 2].

Figure 3

Summary receiving operating characteristic curve, comparing the diagnostic accuracy of CT to EUS. CT: Computed tomography; EUS: Endoscopic ultrasound

Table 2

Head to head comparison of pooled AUCs for EUS and CT

	Observed	Area Under the ROC Curve	Standard Error	95% Confidence Interval
EUS	77	0.75	0.05	0.66-0.84
CT	77	0.78	0.05	0.69-0.87

H0: AUC (EUS) = AUC (CT) χ2=0.95, degrees of freedom [df] = 1, Prob> χ2=0.3294. EUS: Endoscopic ultrasound, CT: Computed tomography, ROC: Receiver operating characteristic, H0: Null hypothesis, AUC: Area under curve

Pooled sensitivity and specificity of EUS and CT

Pooled sensitivity and specificity were calculated for the determination of resectability. Pooled sensitivity and specificity for EUS was 87% (95% CI 70% – 96%) and 63% (95% CI 48% – 77%) respectively, while pooled sensitivity and specificity for CT was 87% (95% CI 70% – 96%) and 70% (95% CI 55% – 0.83%) respectively [Figure 4]. The sensitivity and specificity of EUS and CT were not statistically different. [McNemar's χ2= 0.0 (P = 1.0) and 1.8 (P = 0.18) respectively].

Figure 4

Forrest plots of included studies on CT and EUS in determining the resectability of pancreatic cancer. CT: Computed tomography; EUS: Endoscopic ultrasound

DOR for EUS was 11.5 (95% CI 3.55 – 36.8) and DOR for CT was 15.9 (95% CI 4.83 – 51.6). The DORs of EUS and CT were also not statistically significant: Ratio 0.72 (95% CI 0.14 to 3.82; P = 0.703). See Table 3 for additional performance characteristics and diagnostic accuracy data.

Table 3

Performance characteristics of EUS and CT in determining pancreatic cancer resectability

Diagnostic measures	Resectability 2 studies n=77
	EUS	95% CI	CT	95% CI
Prevalence	40.3%	29.2-52.1%	39.7%	28.8-51.5%
Sensitivity	87.1%	70.2-96.4%	87.1%	70.2-96.4%
Specificity	63.0%	47.5-76.8%	70.2%	55.1-82.7%
ROC Area	0.75	0.66-0.84	0.79	0.70-0.88
LR+	2.36	1.58-3.52	2.92	1.85-4.63
LR-	0.20	0.08-0.52	0.18	0.07-0.47
DOR	11.51	3.55-36.81	15.91	4.83-51.62
PPV	61.4	45.5-75.6%	65.9%	49.4-79.9%
NPV	87.9	71.8-96.6%	89.2%	74.6-97%

EUS: Endoscopic ultrasound, CT: Computed Tomography, LR+/-: positive or negative Likelihood ratio

DISCUSSION

To our knowledge, this is the first diagnostic test accuracy meta-analysis comparing EUS and CT, head-to-head in the assessment of pancreatic cancer resectability using surgical assessment as the reference standard, utilizing appropriate methodology by Cochrane Collaboration.[7] Both EUS and CT showed reasonable diagnostic accuracy and were found to be comparable with an AUC of 75% (95% CI 66% – 84%) as compared to 78% (95% CI 69% – 87%) for EUS and CT, respectively (χ2= 0.3294, degrees of freedom [df] =1, and P > 0.05).

Two previous reviews attempted comparing EUS and CT in determining the resectability of pancreatic cancer. Both studies compared EUS and CT, using surgery as the reference standard.[56] There was an overlap of 3 articles between these two analyses and 2 articles with this study. The first one was a systematic review of available literature without pooled statistics and included 4 studies.[6] The authors reported 2 studies suggesting comparable outcomes and two suggesting the superiority of EUS and CT respectively in assessing resectability. The more recent review was a meta-analysis in which the authors included 6 studies comprising 280 patients and showed that both tests performed similarly in the assessment of resectability in a group that included peri-ampullary carcinoma.[5] The first was not a statistical analysis, while the methodology of the latter included trials which were not necessarily head-to-head comparisons and the authors calculated the accuracy of EUS and CT from different studies, comparing the numerical results. In that methodology, the populations included in the statistical analysis of each modality are not necessarily the same, as the performance characteristics of each imaging modality are potentially determined on different groups of patients with different baseline characteristics and disease burden. Therefore, the level of evidence derived from the results of such study would not be as much as the one based on our head-to-head analysis given the significance of confounding factors. Our meta-analysis differs in methodology which necessitates that head-to-head comparisons and statistical analysis. Despite similarities, our study provided stronger evidence on the comparability of EUS and CT in determining resectability of pancreatic cancer.

It is evident that EUS has definite advantages over CT including tissue sampling via fine needle aspiration (FNA) for diagnosis, atypical lymph node sampling, and sampling of incidental hepatic metastasis.[3] However, sufficient information is often inferred from radiological data in clinical practice. Given this, our finding of comparable accuracy of EUS and CT in determining the resectability of pancreatic cancer, and that CT is generally considered to be the gold standard and the preferred modality for pancreatic imaging, the decision to do an EUS may ultimately be determined by access to resources, local expertise, and institutional policy, which is often influenced by cost.[4] A cost analysis comparing EUS FNA, CT FNA, and surgery in the management of non-metastatic pancreatic head adenocarcinoma deemed resectable by CT, determined EUS FNA to be the least costly strategy primarily due to obviated unnecessary surgeries.[11]

The results of our study should be interpreted with caution given the inherent limitations in performing a meta-analysis. Our meta-analysis included only two studies, comprising a total population of 77 patients. More studies and a higher number of patients would provide a more accurate estimate and comparison of results. The scarcity of data reflects to some extent a paucity of the head-to-head studies comparing EUS and CT in determining resectability of pancreatic cancer not including peri-ampullary carcinoma as well as a dearth of publication of raw data. This may in some part be due to the human, infrastructural, and financial resources required to perform such studies as well as varied reporting styles. Many studies looking at EUS or CT alone or in sequence in determining resectability varying widely in methodology and reporting. However, head-to-head analysis where all patients receive both modalities regardless of the outcome of the other provides the most accurate comparison between the two modalities.

An additional factor limiting the applicability of our findings is the recent advancements in both CT and EUS imaging modalities in the past two decades which have substantially improved our ability to visualize and assess pancreatic lesions for resectability criteria.[12] The two papers included in this review, utilized conventional EUS for assessment of the tumor, which does not reflect the full potential of this imaging modality. One such advanced endoscopic technique is contrast-enhanced-EUS (CE-EUS), which was first introduced in 1995 and relied on direct injection of carbon dioxide gas mixed with heparinized saline and the patient's blood into the gastroduodenal or celiac artery.[13] Over the years, second-generation microbubble ultrasound contrast agents have been developed, which can be injected peripherally and easily reach the entire vascular system due to their small size.[14] Given the great potential of CE-EUS in assessing vascular invasion and tumor characteristics in comparison to conventional EUS, a large body of evidence has emerged, which now advocates for CE-EUS as the standard of care for assessing ductal pancreatic adenocarcinoma.[15] Another advanced EUS imaging technique is tridimensional-EUS, which allows for better spatial visualization of the tumor and can be used with contrast-enhancing agents to assess vasculature of the mass and its surrounding structures.[16] Lastly, EUS-elastography is a modern technique that improves staging and assessment of resectability by providing information about tissue strain and hardness.[17] Since its introduction in 2006, EUS-elastography has undergone improvements and can be used to distinguish solid pancreatic lesions based on their hardness characteristics.[18] Overall, endoscopic ultrasound techniques have undoubtedly underwent dramatic improvements since the publication of the two included studies in our review. The same can be stated for CT imaging with the recent advent of dual source, dual-energy multidetector CT, new approaches for the timing of the image, and perfusion CT which allows for better examination of the pancreas.[1920] As more head-to-head comparisons of CT and EUS become available, it would be crucial to perform another meta-analysis to update our findings. Despite this, our paper is the first meta-analysis to date with appropriate statistical analysis and head to head comparison of EUS and CT with surgery as the reference standard.

It should be recalled that EUS is operator-dependent, and the interpretation of its findings is subject to bias. Therefore, the results of the included studies may not be generalizable to all centers, dependant on local expertise. Of the included studies, there was heterogeneity in the type of CT (single array, multidetector) and type of sonographic probe used (radial and linear, linear alone), which must be taken into account when interpreting our results.[21] Furthermore, although surgical assessment is considered to be accurate in determining resectability, microscopically positive histologic margins could have been missed in one of the two included studies that did not report universal pathological confirmation/assessment.[22] Also, a surgeon's assessment of resectability while reflecting “real life” conditions, can be subjective, and although the included studies clearly defined their criteria for resectability and this was comparable between studies, it represents a potential limitation of the reference standard. One of the included studies also included 3 patients who were determined to have the unresectable disease by consensus after discussion of imaging without verification by surgical assessment.[22]

Current guidelines categorize resectability into three groups, resectable, borderline resectable and unresectable.[4] Borderline unresectable cases are offered neo-adjuvant regimens before reassessment for surgical resection. Both the determination of borderline resectability and subsequent reassessment of resectability may represent other potential targets for outcome assessment comparison where further benefit or clear superiority may be shown. Of note, no studies have compared the diagnostic accuracy of CT to that of combined EUS and CT, which may be beneficial if shown in future studies. Moreover, due to the large diversity of protocols and techniques available for EUS and CT which differ from center to center, future investigations should be thorough with a description of the exact imaging modalities and protocols they have used to aid in the development of clinical guidelines.

In conclusion, our analysis shows that both EUS and CT provide reasonable diagnostic accuracy in determining the resectability of pancreatic cancer. It seems that neither is superior in this regard. Given the results of our study, it is extremely difficult to make a recommendation towards either test for the definitive assessment of resectability in patients with pancreatic cancer. Another review recently published by the Cochrane group based on two individual studies did not find any strong evidence to suggest EUS should be routinely performed prior to laparotomy in patients found to have resectable pancreatic cancer based on CT scan alone.[23] However, due to the paucity of data and until higher-quality evidence emerges, we recommend EUS to be considered in addition to CT to provide further information on the resectability of pancreatic cancer based on local availability and expertise. This may avoid erroneously classifying a lesion as resectable or not especially given the other established benefits of EUS in accordance with current guidelines such as sampling via FNA as well as the major improvements it has undergone in the past decade.[34]

Financial support and sponsorship

Mohammad Yaghoobi is supported by an Internal Career Award by the Department of Medicine, McMaster University, Hamilton, Ontario, Canada.

Conflicts of interest

There are no conflicts of interest.