Literature DB >> 29029521

Contrast-enhanced endoscopic ultrasound for differential diagnosis of pancreatic cancer: an updated meta-analysis.

Xing-Kang He1,2, Yue Ding1,2, Lei-Min Sun1,2.   

Abstract

AIM: We aim to assess the diagnostic value of contrast-enhanced endoscopic ultrasound (CE-EUS) for pancreatic cancer and inflammatory lesions by pooling current evidence.
MATERIALS AND METHODS: A systematical search of PubMed, Web of Science and the Cochrane Library was performed from inception to January 2016. Two authors independently screened and extracted detailed data from included studies. A random effect model was adopted to estimate the pooled sensitivity, specificity in order to determine the diagnostic ablitity of CE-EUS. Furthermore, we conducted the meta-regression and subgroup analyses to explore possible heterogeneity.
RESULTS: Eighteen eligible studies enrolling 1668 patients were finally included in the study. The pooled sensitivity of CE-EUS for distinguishing pancreatic cancers from solid inflammatory masses was 0.93 (95% CI, 0.91-0.94), and the specificity was 0.88 (95% CI, 0.84-0.90). The area under summary receiver operating characteristic curve yielded 0.97. No publication bias was observed by Deeks' funnel plot in current meta-analysis.
CONCLUSIONS: We provided evidence that CE-EUS is a promising modality for differential diagnosis of pancreatic adenocarcinomas. Further multicenter prospective studies should be carried out to certify its utility.

Entities:  

Keywords:  contrast-enhanced EUS; contrast-enhanced doppler EUS; contrast-enhanced harmonic EUS; pancreatic adenocarcinomas

Year:  2017        PMID: 29029521      PMCID: PMC5630421          DOI: 10.18632/oncotarget.18915

Source DB:  PubMed          Journal:  Oncotarget        ISSN: 1949-2553


INTRODUCTION

Endoscopic ultrasonography (EUS) is considered as a valuable diagnostic technology for pancreatic diseases with good spatial resolution [1, 2]. Although the sensitivity of EUS is high, its ability to characterize and differentiate solid masses is still limited [3]. However, it is crucial for clinicians to establish or exclude pancreatic malignancy in clinical works. Distinguishing pancreatic cancer from inflammatory lesions remains challenging with conventional EUS. The development of EUS-guided fine needle aspiration (EUS-FNA) makes it possible for characterization of pancreatic lesions with high accuracy [4-6]. EUS-FNA is effective in differentiation of pancreatic masses. However, EUS-FNA had its own limitations, including sampling errors and invasive procedure [7, 8]. In addition, the sensitivity of EUS-FNA significantly reduced to 54%–73% with the setting of chronic pancreatitis [9, 10]. It is still imperative for endoscopists to seek for effective and noninvasive technologies that could differentiate pancreatic cancer accurately. Contrast-enhanced ultrasonography was used in percutaneously abdominal ultrasound examination since 1995, Contrast-enhanced endoscopic ultrasonography (CE-EUS) is also been performed to determine the pancreatic parenchymal perfusion and microvessels inside lesions of interest with better delineation [11]. The application of contrast agents in EUS had improved characterization ability of pancreatic masses and aided in the differentiation of pancreatic diseases. Several studies had evaluated the diagnostic ability of CH-EUS for pancreatic lesions (sensitivity, 80%–100%, specificity, 64%–100%) [12-28]. In 2012, results from a meta-analysis with limited population showed that pooled sensitivity and specificity of CE-EUS were 94% and 89%, respectively [29]. Recently, with the advent of second-generation contrast agents and quantitative analyses [26, 28], a large amount of trials assessed pancreatic solid masses using CE-EUS [16, 19, 20, 23, 26]. Given this background, we perform an updated meta-analysis based on current lectures to assess diagnostic value of CE-EUS for characterization and differentiation pancreatic lesions.

RESULTS

Characteristics and quality assessment of included study

Our initial search identified 1475 articles from databases. After applied inclusion and exclusion criteria, 18 eligible studies comprising 1668 participants were included in final analysis. Detailed selection flow was presented in Figure 1, The main characteristics are presented in Table 1. Ten studies were performed in Europe, seven were conducted in Asia and one was in USA. The gold diagnostic standard was based on pathology histology, or follow-up. According to QUADAS-2 criteria, the overall methodological quality of included articles was moderate to high (Figure 2).
Figure 1

Flow diagram of the literature selection procedure

Table 1

Characteristics of the selected studies

StudyCountryNo. of patientsSex (M/F)Age (mean, y)Diagnostic standardContrast agentContrast modeGold standard
Becker et al. 2001 [12]Germany2316/758.3HypoenhancementOptisonColor/power DopplerHistology, follow-up (6 m)
Hocke et al. 2008 [17]Germany194119/7564Irregular arterial vessels, no venous vesselsSonovuePower DopplerHistology,
Dietrich et al. 2008 [13]Germany93UnclearUnclearHypoenhancementLevovistColor DopplerHistology,
Sakamoto et al. 2008 [27]Japan156UnclearUnclearHypoenhancementLevovistPower DopplerHistology
Saftoiu et al. 2010 [25]Romania5443/1156.9Contrast-enhanced PDVI cut-off < 20%SonovuePower DopplerHistology, follow-up (> 6 m)
Seicean et al. 2010 [28]Romania3025/557Cut-off < 0.17SonovueHarmonicHistology, follow-up (9 m)
Napoleon et al. 2010 [22]France3519/1660HypoenhancementSonovueHarmonicHistology, follow-up (> 12 m)
Fusaroli et al. 2010 [14]Italy9044/4667Inhomogeneous hypoenhancementSonovueHarmonicHistology, follow-up (> 12 m)
Matsubara et al. 2011 [21]Japan9161/3061.4HypoenhancementSonazoidHarmonicHistology, follow-up (> 12 m)
Romagnuoloet al. 2011 [24]USA21UnclearUnclearHypoperfusion or perfusion defectsDefinityHarmonicHistology, follow-up (6 m)
Kitano et al. 2011 [19]Japan277173/10464.3HypoenhancementSonazoidHarmonicHistology, follow-up (> 12 m)
Imazu et al. 2012Japan3022/866.9maximum intensity gain cut-off < 12.5SonazoidHarmonicHistology, follow-up (> 12 m)
Lee et al. 2013 [20]Korea3724/1362.3HypoenhancementSonovueHarmonicHistology,
Gheonea et al. 2013 [15]Roumania5125/26UnclearTime intensity curve analysisSonovueHarmonicHistology, follow-up (6 m)
Gincul et al. 2014 [16]France10051/4964.6HypoenhancementSonovueHarmonicHistology, follow-up (12 m)
Park et al. 2014 [23]Korea9062/2863.5HypoenhancementSonovueHarmonicHistology, follow-up
Saftoiu et al. 2015 [26]Multicenter (Romania, Denmark, Germany, Spain).167127/4062HypoenhancementSonovueHarmonicHistology, follow-up (6 m)
Yamashita et al. 2015Japan14792/5569hypovascular pattern and lower intensity of enhancementSonazoidHarmonicHistology

Abbreviation: M/F, male/female; PDVI, Power Doppler Vascularity Index.

Figure 2

Quality assessment of included studies according to the quality assessment of diagnostic accuracy studies criteria-2

Red color indicated high risk of bias, Yellow color indicated unclear risk of bias, Green color indicated low risk of bias.

Abbreviation: M/F, male/female; PDVI, Power Doppler Vascularity Index.

Quality assessment of included studies according to the quality assessment of diagnostic accuracy studies criteria-2

Red color indicated high risk of bias, Yellow color indicated unclear risk of bias, Green color indicated low risk of bias.

Diagnostic value of contrast-enhanced EUS

For CE-EUS, the pooled estimates of sensitivity and specificity were 93% (95% CI, 0.91–0.94) and 88% (95% CI, 0.84–0.90), respectively (Figure 3). There was no significant heterogeneity in sensitivity (P = 0.39, I2 = 5.4%), while significant heterogeneity was observed in specificity (P < 0.001, I2 = 66.1%) (Figure 3). The area under the SROC was 0.97 (Figure 4). The pooled positive and negative likelihood ratios were 7.05 (95% CI, 4.65–10.71) and 0.09 (95% CI, 0.08–0.11) in diagnosis of pancreatic cancer (Figure 5). Diagnostic odds ratio was 91.05 (95% CI, 59.98–138.21), indicating a high value of diagnostic efficacy of CE-EUS (Figure 6). No significant risk of publication bias was observed in our study for CE-EUS by Deeks’ funnel plot (P = 0.967) (Figure 7).
Figure 3

Forest plot of pooled sensitivity and specificity for diagnostic value of CE-EUS

(A) Sensitivity; (B) Specificity. Low heterogeneity across pooled sensitivity (I2 < 30%) and High heterogeneity across pooled specificity (I2 > 50%).

Figure 4

Summary receiver operating characteristic (SROC) curve for the diagnostic accuracy of CE-EUS

AUC (Area Under Curve) of 0.97 indicated a a perfect test. SE, standard error.

Figure 5

Forest plot of positive likelihood ratio and negative likelihood ratio for CE-EUS

(A) forest plots of the positive likelihood ratio; (B) forest plots of negative likelihood ratio. High heterogeneity across pooled positive likelihood ratio (I2 > 50%)and Low heterogeneity across pooled negative likelihood ratio (I2 < 30%).

Figure 6

Forest diagnostic odds ratio of CE-EUS

Low heterogeneity across pooled diagnostic odds ratio (I2 < 30%).

Figure 7

Publication bias of selected studies by Deeks’ funnel plot

Forest plot of pooled sensitivity and specificity for diagnostic value of CE-EUS

(A) Sensitivity; (B) Specificity. Low heterogeneity across pooled sensitivity (I2 < 30%) and High heterogeneity across pooled specificity (I2 > 50%).

Summary receiver operating characteristic (SROC) curve for the diagnostic accuracy of CE-EUS

AUC (Area Under Curve) of 0.97 indicated a a perfect test. SE, standard error.

Forest plot of positive likelihood ratio and negative likelihood ratio for CE-EUS

(A) forest plots of the positive likelihood ratio; (B) forest plots of negative likelihood ratio. High heterogeneity across pooled positive likelihood ratio (I2 > 50%)and Low heterogeneity across pooled negative likelihood ratio (I2 < 30%).

Forest diagnostic odds ratio of CE-EUS

Low heterogeneity across pooled diagnostic odds ratio (I2 < 30%).

Meta-regression and sensitivity analysis

To explore possible heterogeneity, we performed a meta-regression analysis and results showed that the characteristics of studies were not significantly associated with diagnostic odds ratio (Table 2). It is illustrated by Figures 3–6 that two studies of Park et al. 2014 and Fusaroli et al. 2010 were outliers. After exclusion of them, sensitivity analysis still demonstrated the consistence of main results (Table 3).
Table 2

Meta-regression for the potential source of heterogeneity

Study characteristicRelative Diagnostic odd ratio (95% CI)P value
patient (< 60 patients vs. ≥ 60 patients)1.56 (0.56, 4.33)0.37
contrast mode (color/power Doppler vs. harmonic)0.92 (0.33, 2.58)0.87
country (Europe vs. other)0.87 (0.34, 2.21)0.76
analysis of images (quality vs. quantity)0.93 (0.30, 2.92)0.89

CI, Confidence interval.

Table 3

Subgroup analysis by exclusion of outliers

The pooled resultsPooled value (95% CI)P valueI2 (%)
Sensitivity0.93 (0.91, 0.94)0.3311.2
Specificity0.91 (0.88, 0.93)0.0637.6
Positive likelihood ratio8.10 (5.74, 11.42)0.1231
Negative likelihood ratio0.09 (0.07, 0.11)0.570
Diagnostic OR110.44 (73.42, 166.11)0.580

CI, Confidence interval; I, inconsistency; I > 50% was considered significant for heterogeneity.

CI, Confidence interval. CI, Confidence interval; I, inconsistency; I > 50% was considered significant for heterogeneity.

DISCUSSION

It is well known that pancreatic cancer is a lethal disease with dismal prognosis duo to low eraly detection rate. The 5-year survival rate was sharply decreased by advanced stage [30]. Patients were free of symptoms until they were diagnosed by advanced pancreatic cancer [31]. There is an imperative need to diagnose pancreatic cancer at earlier stages. Regarding accurate differentiation of pancreatic cancers from inflammatory tumor-like lesions still remains a big challenge for clinicians and is crucial for therapeutic decisions [32]. It is well demonstrated by several studies [33-35] that EUS is super to other modalities in detection and diagnosis of pancreatic diseases with high sensitivity. However, the ability to characterize solid lesions accurately still remains limited, particularly in the setting of chronic pancreatitis. CE-EUS, with the intravenously infusion of contrast agents, is a newly developed technology. It can characterize and differentiate pancreatic lesions non-invasively [11]. In general, CE-EUS could be classified as contrast-enhanced Doppler EUS (CD-EUS) and contrast-enhanced harmonic EUS (CH-EUS) according to the method of sonographic assessment [36]. For CD-EUS, intravenous contrast agents would enhance the Doppler signals from vascularity of targeted lesions [37, 38]. However, the disadvantage of this technique included the flash and blooming artifacts. Furthermore, the poor ability to depict microvessels with slow flow and parenchymal perfusion also limited its application widely [27, 39]. For these limitations, CH-EUS was developed to overcome them. It depicts harmonic signals from contrast agents selectively and filters them from surrounding tissues [40, 41]. Thereby, it provides more detailed images of fine vessels with slow flow and parenchymal perfusion in the target lesions [42, 43]. This allows pancreatic lesions to be visualized and characterized more accurately. Besides, the second-generation of contrast agents also demonstrated to be relatively safe for patients, even with liver and renal dysfunctions [43-46]. In our meta-analysis, the pooled results supported a great diagnostic value of CE-EUS for characterization and differentiation of pancreatic masses, which were consistent with previous meta-analysis and studies [29]. Compared with previous meta-analysis, the strength of our study is included comprehensive lectures. CE-EUS is extremely useful for patients with negative results of EUS-FNA. Although EUS-FNA still is considered as a gold standard for pancreatic cancer diagnosis, the sensitivity and accuracy is still suboptimal, particularly in the setting of chronic pancreatitis [47]. Several trials reported that CH-EUS could complements EUS-FNA by delineating the outline of the target lesions clearly, thus facilitating EUS-FNA [14, 19, 48]. Moreover, it could not only improve the sensitivity of EUS-FNA, but also avoid repeated biopsy or surgery. For CH-EUS, a well-known and sensitive diagnostic standard of pancreatic adenocarcinomas is a hypoenhanced image of lesion [19, 22]. age olesion [19, 22]. However, it is somehow operator-dependent and subjective to the analysis of the enhanced pattern, which might affect the diagnostic accuracy. To avoid this disadvantage, a quantification analysis, time–intensity curve for region of interest, was developed recently [15, 18, 21, 28]. Five of the included studies demonstrated the values of maximum intensity, median intensity, time to peak, intensity reduction rate, the ratio of uptake inside the mass to uptake of the surrounding parenchyma in discrimination of malignances from pancreatitis, solid-pseudopapilliary neoplasm and neuroendocrine tumors [15, 18, 21, 26, 28], which makes objective definition of lesion characteristics possible. CH-EUS is also a reproducible method in the evaluation of pancreatic lesions with good interobserver agreement, even for endosonographers with no or limited experience in EUS [16]. The present meta-analysis has some limitations. Significant heterogeneity in specificity and positive likelihood ratio might affect interpretation of the data and conclusions. Serval diagnostic criterion for CE-EUS were adopted in included studies, which might introduce some bias into our conclusion. Furthermore, we cannot exclude the presence of publication bias, although the analysis of the funnel plot indicated that it could not be detected. In conclusion, CE-EUS, especially for CH-EUS, is a promising tool for differential diagnosis of pancreatic cancer. CE-EUS should be regarded as a promising tool for pancreatic masses characterization, especially when EUS-FNA findings were negative. Further multicenter trials should be carried out to certify its utility.

MATERIALS AND METHODS

Search strategy

We searched PubMed, Web of Science and the Cochrane Library from inception to January 2016 for relevant articles comprehensively. Following search terms were adopted: (“contrast-enhanced” OR “contrast medium” OR “echo-enhanced”) AND (“pancreatic mass*” OR “pancreatic cancer” OR “pancreatitis” OR “pancreatitis” OR “pancreatic lesion*” OR “pancreatic adenocarcinoma”) AND (“ultrasonograph*” OR “ultrasound” OR “endosonograph*” OR “endosonography” OR “EUS”). We also searched bibliography of articles and reviews to identify additional articles.

Inclusion and exclusion criteria

Articles were considered as eligible if they used CE-EUS for the diagnosis, provision of data for true positive (TP), false positive (FP), false negative and true-negative (TN), the reference standard based on histopathology of samples by EUS-FNA, surgery or a follow-up of at least 6 months. Following studies were excluded: (1) complete data unavailable; (2) overlapping with the selected articles; (3) Case reports, reviews, editorials, comments, abstracts.

Data extraction

Two authors independently extracted following data from each study: authors, year, country, numbers of patient, sex, age, diagnostic standard, contrast agent, contrast mode, gold standard. We adopted the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) for quality assessment [49]. Any discrepancies were resolved by discussions or consensus.

Statistical methods

The pooled estimates of sensitivity, specificity, diagnostic odds ratio were performed by Meta-Disc, version 1.4 (Ramony Cajal Hospital, Madrid, Spain). Heterogeneity across studies was evaluated by the Cochrane Q test and I2 statistic. We constructed a summary receiver operating characteristic (sROC) curve and calculated the area under sROC (AUC). The meta-regression and sensitivity analyses were performed with the following covariates such as numbers of patient (< 60 patients vs. ≥ 60 patients), contrast mode (color/power Doppler vs. harmonic), country (Europe vs. other), analysis of images (quality vs. quantity). Deeks’ asymmetry test was used to detect publication bias by Stata version 13.0 (Stata Corporation, College Station, Texas). The two-tailed P value is statistically significant at less than 0.05.
  49 in total

Review 1.  Contrast-enhanced endoscopic ultrasonography.

Authors:  Nischita K Reddy; Ana Maria Ioncică; Adrian Săftoiu; Peter Vilmann; Manoop S Bhutani
Journal:  World J Gastroenterol       Date:  2011-01-07       Impact factor: 5.742

2.  Combined contrast-enhanced power Doppler and real-time sonoelastography performed during EUS, used in the differential diagnosis of focal pancreatic masses (with videos).

Authors:  Adrian Săftoiu; Sevasti A Iordache; Dan Ionu Gheonea; Carmen Popescu; Anca Maloş; Florin Gorunescu; Tudorel Ciurea; Alexandru Iordache; Gabriel Lucian Popescu; C T Lin Manea
Journal:  Gastrointest Endosc       Date:  2010-08-02       Impact factor: 9.427

Review 3.  Endoscopic ultrasound in the diagnosis and treatment of pancreatic disease.

Authors:  Christopher W Teshima; Gurpal S Sandha
Journal:  World J Gastroenterol       Date:  2014-08-07       Impact factor: 5.742

4.  Novel quantitative perfusion analysis with contrast-enhanced harmonic EUS for differentiation of autoimmune pancreatitis from pancreatic carcinoma.

Authors:  Hiroo Imazu; Keisuke Kanazawa; Naoki Mori; Keiichi Ikeda; Hiroshi Kakutani; Kazuki Sumiyama; Shoryoku Hino; Tiing Leong Ang; Salem Omar; Hisao Tajiri
Journal:  Scand J Gastroenterol       Date:  2012-04-17       Impact factor: 2.423

Review 5.  Pancreatic cancer: current concepts in imaging for diagnosis and staging.

Authors:  E Tamm; C Charnsangavej
Journal:  Cancer J       Date:  2001 Jul-Aug       Impact factor: 3.360

6.  Effectiveness of contrast-enhanced harmonic endoscopic ultrasound for the evaluation of solid pancreatic masses.

Authors:  Jin-Seok Park; Hyung Kil Kim; Byoung Wook Bang; Sang Gu Kim; Seok Jeong; Don Haeng Lee
Journal:  World J Gastroenterol       Date:  2014-01-14       Impact factor: 5.742

7.  [Contrast enhanced endosonography for improving differential diagnosis between chronic pancreatitis and pancreatic cancer].

Authors:  M Hocke; C Schmidt; B Zimmer; T Topalidis; C F Dietrich; A Stallmach
Journal:  Dtsch Med Wochenschr       Date:  2008-09-01       Impact factor: 0.628

8.  Comparison of endoscopic ultrasonography and multidetector computed tomography for detecting and staging pancreatic cancer.

Authors:  John DeWitt; Benedict Devereaux; Melissa Chriswell; Kathleen McGreevy; Thomas Howard; Thomas F Imperiale; Donato Ciaccia; Kathleen A Lane; Dean Maglinte; Kenyon Kopecky; Julia LeBlanc; Lee McHenry; James Madura; Alex Aisen; Harvey Cramer; Oscar Cummings; Stuart Sherman
Journal:  Ann Intern Med       Date:  2004-11-16       Impact factor: 25.391

9.  Clinical role of contrast-enhanced harmonic endoscopic ultrasound in differentiating solid lesions of the pancreas: a single-center experience in Korea.

Authors:  Tae Yoon Lee; Young Koog Cheon; Chan Sup Shim
Journal:  Gut Liver       Date:  2013-06-11       Impact factor: 4.519

10.  Contrast-enhanced harmonic endoscopic ultrasonography in pancreatic diseases.

Authors:  Can Xu; Zhaoshen Li; Michael Wallace
Journal:  Diagn Ther Endosc       Date:  2012-11-01
View more
  8 in total

1.  Shear Wave Elastography Can Differentiate between Radiation-Responsive and Non-responsive Pancreatic Tumors: An ex Vivo Study with Murine Models.

Authors:  Hexuan Wang; Bradley Mills; Reem Mislati; Rifat Ahmed; Scott A Gerber; David Linehan; Marvin M Doyley
Journal:  Ultrasound Med Biol       Date:  2019-11-11       Impact factor: 2.998

Review 2.  Advanced Imaging of the Biliary System and Pancreas.

Authors:  Selena Zhou; James Buxbaum
Journal:  Dig Dis Sci       Date:  2022-02-16       Impact factor: 3.199

Review 3.  Role of Endoscopic Ultrasonography and Endoscopic Retrograde Cholangiopancreatography in the Diagnosis of Pancreatic Cancer.

Authors:  Yasutaka Ishii; Masahiro Serikawa; Tomofumi Tsuboi; Ryota Kawamura; Ken Tsushima; Shinya Nakamura; Tetsuro Hirano; Ayami Fukiage; Takeshi Mori; Juri Ikemoto; Yusuke Kiyoshita; Sho Saeki; Yosuke Tamura; Sayaka Miyamoto; Kazuaki Chayama
Journal:  Diagnostics (Basel)       Date:  2021-02-04

4.  Predicting Factors for Pancreatic Malignancy with Computed Tomography and Endoscopic Ultrasonography in Chronic Pancreatitis.

Authors:  Jian-Han Lai; Keng-Han Lee; Chen-Wang Chang; Ming-Jen Chen; Ching-Chung Lin
Journal:  Diagnostics (Basel)       Date:  2022-04-15

Review 5.  Comparison of endoscopic ultrasonography with and without contrast enhancement for characterization of pancreatic tumors: a meta-analysis.

Authors:  Yasunobu Yamashita; Toshio Shimokawa; Reiko Ashida; Bertrand Napoléon; Andrea Lisotti; Pietro Fusaroli; Rodica Gincul; Christoph F Dietrich; Shunsuke Omoto; Masayuki Kitano
Journal:  Endosc Int Open       Date:  2022-02-25

Review 6.  Diagnosis, treatment, and current concepts in the endoscopic management of gastroenteropancreatic neuroendocrine neoplasms.

Authors:  Giuseppe Iabichino; Milena Di Leo; Monica Arena; Giovanni Giuseppe Rubis Passoni; Elisabetta Morandi; Francesca Turpini; Paolo Viaggi; Carmelo Luigiano; Luca De Luca
Journal:  World J Gastroenterol       Date:  2022-09-14       Impact factor: 5.374

Review 7.  Endoscopic ultrasound (EUS) and the management of pancreatic cancer.

Authors:  Muhammad Nadeem Yousaf; Fizah S Chaudhary; Amrat Ehsan; Alejandro L Suarez; Thiruvengadam Muniraj; Priya Jamidar; Harry R Aslanian; James J Farrell
Journal:  BMJ Open Gastroenterol       Date:  2020-05

Review 8.  Current role of endoscopic ultrasound in the diagnosis and management of pancreatic cancer.

Authors:  Federico Salom; Frédéric Prat
Journal:  World J Gastrointest Endosc       Date:  2022-01-16
  8 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.