Predictive performance of factors associated with malignancy in intraductal papillary mucinous neoplasia of the pancreas.

BACKGROUND: Estimation of the risk of malignancy in intraductal papillary mucinous neoplasia (IPMN) of the pancreas is a clinical challenge. Several routinely used clinical factors form the basis of the current consensus guidelines. This study aimed to determine the predictive values of the most commonly assessed risk factors.
METHODS: A meta-analysis of individual risk factors of malignancy in IPMN was performed. Contingency tables were derived from these data, and sensitivity, specificity, negative and positive predictive values, and diagnostic odds ratios (DOR) were determined. Hierarchical summary receiver operating characteristic (HSROC) curves for each factor were calculated and the respective area under the curve (AUC) was assessed.
RESULTS: A total of 3443 studies were screened initially. Analysis of recent literature revealed 60 studies with 13 relevant risk factors including clinical, serological and radiological parameters. The largest area under the HSROC curve was found for weight loss (0·84) and jaundice/raised bilirubin level (0·80), followed by increased carcinoembryonic antigen (CEA) (0·79) or carbohydrate antigen (CA) 19-9 (0·78) levels. The most sensitive factors were patient age (71 per cent) and mural nodules (65 per cent), and jaundice/raised bilirubin level (97 per cent) and increased CEA level (95 per cent) were most specific. None of the analysed factors reached a positive or negative level of prediction beyond 90 per cent.
CONCLUSION: None of the established criteria safely distinguishes malignant from non-malignant lesions.

Introduction

The clinical management of intraductal papillary mucinous neoplasia (IPMN) is still controversial. The major reason is the absence of factors that clearly predict malignancy. To overcome this issue, consensus conferences in Sendai1 and Fukuoka2 have defined combinations of risk factors that may predict malignancy more sensitively and specifically. A large number of mainly single‐centre analyses based on these criteria have been published, but a recent meta‐analysis3 of data from these publications demonstrated that both overall sensitivity and specificity of the most recent (Fukuoka) criteria were relatively low.

The present study aimed to assess the predictive values of individual factors that have been associated with malignancy in IPMN. Studies including branch duct (BD), main duct (MD) and mixed‐type IPMN were considered, focusing on those that reported sensitivity and specificity of individual risk factors of malignancy. Data were pooled and meta‐analysed, allowing for a determination of statistical classifiers.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines4 were followed. Two investigators screened two databases, PubMed and Web of Science, independently. In cases of disagreement, a third investigator decided on inclusion of the study. The search strategy consisted of the following terms: ‘intraductal papillary mucinous neoplasm’ AND biomarker OR marker OR predictor OR malignancy OR serum OR CA19‐9 OR CEA OR ‘pancreatic enzymes’ OR amylase OR lipase OR PLR OR NLR OR Ca24‐2 OR bilirubin OR platelet OR neutrophil and ‘pancreatic cancer’ AND enzymes OR ‘serum amylase’ OR ‘serum lipase’ OR amylase OR lipase OR ‘serum enzymes’ and ‘cancer AND platelet lymphocyte ratio OR neutrophil lymphocyte ratio’. The search was conducted to cover articles published between 2006 (publication of the Sendai consensus) and April 2016 (date of search).

Criteria for study inclusion were as follows: patients with histologically confirmed IPMN; studies that analysed one or more of the factors of the consensus guidelines or one of the other factors defined in the primary literature search; and studies that allowed for clear assignment of presence of the respective factor to the histological outcome. Invasive carcinoma and high‐grade dysplasia (formerly carcinoma in situ) were considered as malignant lesions.

Articles with abstracts that did not fit the scope of the search were excluded, along with non‐English‐language articles, case reports, small case series with ten or fewer patients, reviews and meta‐analyses. Only studies that allowed for a quantitative analysis of the results into a 2 × 2 contingency table were included in the meta‐analysis.

Studies eligible for inclusion were grouped according to the respective factor of interest. All continuous exposures (for example laboratory parameters such as carbohydrate antigen (CA) 19‐9) were then converted into a binary form using widely used cut‐off values. In the next step, 2 × 2 tables were designed for all studies. Sensitivities, specificities, negative predictive values (NPVs), positive predictive values (PPVs) and diagnostic odds ratios (DORs) were calculated. Results were pooled using a random‐effects model. Final results for each analysed factor were depicted using forest plots. Heterogeneity was assessed using I 2 statistics. Study quality and publication bias were investigated using funnel plots. The open‐source statistical software R 3.3 and the meta‐analysis package metafor 1.9‐9 (R Foundation for Statistical Computing, Vienna, Austria) were used for the analysis. The mada 0·5·7 package was used for calculation of the hierarchical summary receiver operating characteristic (HSROC) curves and the corresponding area under the curve (AUC).

Results

A total of 3443 studies were screened. Initial screening for markers derived from the differential blood count (neutrophil : lymphocyte ratio, platelet : lymphocyte ratio) revealed poor study quality for these factors, so these studies were excluded. After further exclusion of non‐relevant studies, 60 were included in the final analysis (Fig.  1). Of these studies, 33 investigated mural nodules5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 21 examined dilatation of the main pancreatic duct (MPD)7 9, 15 16, 18 21, 23 26, 27 30, 32 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 21 analysed cyst size5 7, 8, 9 14, 15 18, 20 23, 25 27, 28 31, 36 38, 40 41, 43 47, 48, 49, 17 assessed CA19‐9 increase14 15, 18 21, 23 26, 32 35, 38 43, 50, 51, 52, 53, 54, 55, 56 and 12 investigated the impact of increased bilirubin levels and/or jaundice8 15, 18 21, 31 38, 43 44, 50 54, 57 58. Other characteristics (age13, 14, 15 23, 27, carcinoembryonic antigen (CEA) increase19 50, 56, diabetes mellitus38 40, 48, lymphadenopathy6 32, 41 54, 58, male sex18 23, 54, pancreatitis15 20, 31 35, 59, 60, 61, thickened cyst wall6 8, 32 62 and weight loss10 57, 63 64) were assessed in between three and seven studies. The characteristics of included studies are shown in Table   1.

Figure 1

Flow chart depicting the search strategy

Table 1

Characteristics of included studies

Reference	Factor	Cut‐off value	No. of patients	Age (years)*	Male sex (%)	Study type	Statistical analysis	Type of IPMN
Baiocchi et al.53	CA19‐9	37 units/ml	44	69·3 (38–86)	45·5	Prospective uncontrolled case study	Uni	BD+MI+MD
Hwang et al.52	CA19‐9	37 units/ml	118	63·4(8·5) (41–85)	61	RCCS	Uni	BD
Roch et al.55	CA19‐9	37 units/ml	171	IPMA: 68·4	IPMA: 80·2	RCCS	Multi	BD+MI+MD
				IPMC: 71·2	IPMC: 53·8
Xu et al.51	CA19‐9	37 units/ml	86	62(9) (41–76)	72·1	RCCS	Uni	BD+MI+MD
Jang et al.47	Cyst size	20 mm	138	60·6(8·9) (32–82)	63	RCCS	Multi	BD+MI+MD
Nagai et al.49	Cyst size	30 mm	69	63(9)	62·3	RCCS	Uni	BD
Akita et al.17	MN	–	32	IPMA: 65·3(8·5)	IPMA: 65	RCCS	Multi	BD
				IPMC: 62·6(7·5)	IPMC: 50
Arima et al.29	MN	–	76	IPMA: 66·3(8·3)	IPMA: 72	RCCS	Uni	BD+MI+MD
				IPMC: 70·3(9·1)	IPMC: 65·4
Kawada et al.24	MN	10 mm	202	68(7)	54·5	RCCS	Multi	BD
Kwong et al.33	MN	–	284	67·3(10·8)	43	Retrospective multicentre case–control study	Uni	BD
Moris et al.34	MN	–	856	70·6	39	Retrospective international multicentre case–control study	Uni	BD
Ogawa et al.11	MN	> 3·6 mm	49	64·9 (41–81)	66·1	RCCS	Uni	BD
Ohno et al.12	MN on EUS	–	87	66·5(9·5)	60·9	RCCS	Uni	BD+MI+MD
Seo et al.37	MN	–	60	64·3(9)	63·3	RCCS	Multi	BD
Shimizu et al.22	MN	7 mm	310	67·1(8·7)	58·3	RCCS	Multi	BD+MI+MD
Kang et al.45	MPD	7 mm	375	63·8(9·0)	62·4	RCCS	Multi	BD+MI+MD
Ridtitid et al.46	MPD	5–9 mm	105	65·2(12·5)	53	RCCS	Uni	BD
Sadakari et al.39	MPD	5 mm	73	66(8) (46–82)	65·8	RCCS	Uni	BD
Ohno et al.42	MPD enlargement on EUS	–	142	65(9) (37–83)	53·8	RCCS	Uni	BD
Kim et al. 59	Pancreatitis	–	118	61·2 (37–78)	70·3	RCCS	Uni	BD+MI+MD
Morales‐Oyarvide et al. 61	Pancreatitis	–	325	68(10·9)	48·9	RCCS	Multi	BD+MI+MD
Tsutsumi et al. 60	Pancreatitis	–	150	Pancreatitis group: 70(8·2)	Pancreatitis group: 57·9	RCCS	Uni	BD+MI+MD
				Non‐pancreatitis group: 66(8·6)	Non‐pancreatitis group: 62·6
Carbognin et al. 62	Thickened cyst wall	–	29	IPMA: 64·7(9·9)	58·6	RCCS	Uni	BD
				IPMC: 62·2(12·2)
Correa‐Gallego et al. 63	Weight loss	–	123	68 (62–75)	40·7	RCCS	Multi	BD
Dortch et al. 64	Weight loss	10 lb	66	68(8·5)	33·36	RCCS	Uni	BD
Ammori et al. 25	Cyst size	30 mm	184	68 (34–88)	n.a.	RCCS	Uni	BD+MI+MD
	MN
Chiu et al. 6	Lymphadenopathy		40	60 (32–67)	69	RCCS	Multi	BD+MI
	MN	3 mm					Uni
	Thickened cyst wall	3 mm					Uni
Fritz et al. 50	CA19‐9	37 units/ml	142	n.a.	57·75	RCCS	Uni	BD+MI+MD
	CEA	5 ng/ml	142
	Jaundice (bilirubin)	2 mg/dl	160
Fritz et al. 54	CA19‐9	37 units/ml	233	66 (28–87)	39·91	RCCS	Uni	BD
	Jaundice (bilirubin total)	2 mg/dl
	Lymphadenopathy
	Male sex
Fujino et al. 38	CA19‐9	35 units/ml	64	IPMA: 66(1·2)	60·94	RCCS	Multi	BD+MI+MD
	Cyst size	42 mm		IPMC: 65·1(9·5)			Uni
	DM						Uni
	Jaundice						Uni
	MPD size						Uni
Goh et al. 44	Jaundice (obstructive)		39	63 (33–83)	66	RCCS	Uni	BD
	MPD	5 mm
Hirono et al. 13	Age	70 years	54	69 (44–81)	57·4	RCCS	Uni	BD+MI+MD
	MN	5 mm
Hirono et al. 21	CA19‐9	37 units/ml	134	68(9·7) (32–84)	55·2	RCCS	Uni	BD
	Jaundice (obstructive)						Uni
	MN	5 mm					Multi
	MPD	5 mm					Uni
Hwang et al. 19	CEA	5 ng/ml	237	63·1 (38–83)	57·8	RCCS	Multi	BD
	MN		237
Jang et al. 26	CA19‐9	37 units/ml	333	63·6(8·9)	61·7	RCCS	Multi	BD
	MN		350
	MPD	5 mm	350
Kato et al. 27	Age	65 years	47	66·2 (50–77)	63·8	RCCS	Uni	BD
	Cyst size (enlargement)		17				Uni
	MN		47				Multi
	MPD	5 mm	47				Uni
Kim et al. 32	CA19‐9	37 units/ml	367	63·7(9)	63	RCCS	Multi	BD+MI+MD
	Lymphadenopathy						Uni
	MN						Multi
	MPD	5 mm					Multi
	Thickened cyst wall						Uni
Kim et al. 30	MN	10 mm	93	n.a.	n.a.	RCCS	Uni	BD+MI+MD
	MPD	–
Kim et al. 18	CA19‐9	37 units/ml	324	62 (30–83)	55·2	RCCS	Multi	BD
	Cyst size		324				Uni
	Jaundice (bilirubin)	1·2 mg/dl	187				Uni
	Male sex		324				Multi
	MN	5 mm	324				Multi
	MPD		324				Multi
Kim et al. 31	Cyst size	30 mm	177	63 (30–87)	61	Retrospective multicentre case–control study	Uni	BD
	Jaundice (obstructive)		177				Uni
	MN on EUS	5 mm	110				Multi
	Pancreatitis		177				Uni
Kurahara et al. 35	MN	5 mm	55	IPMA: 64·8(9·2)	67·3	RCCS	Multi	BD
	Pancreatitis			IPMC: 63·4(8·8)
	CA19‐9	37 units/ml
Lee et al. 41	Cyst size	40 mm	129	60·9 (32–77)	72·9	RCCS	Uni	BD+MI+MD
	Lymphadenopathy						Multi
	MPD	7 mm					Multi
Lou et al. 57	Jaundice	–	51	63 (41–78)	64·7	RCCS	Uni	BD+MI+MD
	Weight loss
Maguchi et al. 16	MN	–	29	66 (37–85)	51·3	Retrospective multicentre case–control study	Uni	BD
	MPD dilated + MN
Mimura et al. 40	Cyst size	30 mm	43	IPMA: 66(1·84)	67·4	RCCS	Uni	BD+MI+MD
	DM		82	IPMC: 66·7(1·86)			Multi
	MPD	6 mm	43				Uni
Murakami et al. 7	Cyst size	28 mm	62	n.a.	IPMA: 79·5	RCCS	Uni	BD+MI+MD
	MN				IPMC: 65·3		Uni
	MPD	6 mm					Multi
Nagai et al. 10	MN	–	57	IPMA: 63 (46–80)	IPMA: 46·4	RCCS	Uni	BD+MI
	Weight loss			IPMC: 64 (41–85)	IPMC: 65·5
Nara et al. 14	Age	70 years	123	64·7 (40–84)	56·9	RCCS	Uni	BD+MI+MD
	CA19‐9	37 units/ml					Multi
	Cyst size	40 mm					Multi
	MN						Multi
Ohtsuka et al. 20	Cyst size	30 mm	99	67 (33–85)	IPMA: 57·1	RCCS	Uni	BD
	MN				IPMC: 75·7		Multi
	Pancreatitis						Multi
Okabayashi et al. 5	Cyst size	30 mm	23	66·4 (53–86)	69·6	RCCS	Multi	BD+MI+MD
	MN on EUS	5 mm	10
Rodriguez et al. 8	Jaundice		145	67 (35–90)	42·8	RCCS	Uni	BD
	MN
	Thickened cyst wall
	Cyst size	30 mm
Sahora et al. 23	Age	65 years	217	67 (21–92)	37·8	RCCS	Multi	BD
	CA19‐9	39 units/ml
	Cyst size	30 mm
	Male sex
	MN
	MPD	5 mm
Shin et al. 15	Age	60 years	204	61 (35–77)	68·1	RCCS	Multi	BD+MI+MD
	CA19‐9	37 units/ml					Multi
	Cyst size	30 mm					Uni
	Jaundice (bilirubin)	1·2 mg/dl					Uni
	MN						Multi
	MPD	6 mm					Multi
	Pancreatitis						Multi
Suzuki et al. 36	Cyst size	47 mm	96	67(10) (34–81)	66·7	RCCS	Multi	BD+MI+MD
	MN	9 mm					Multi
	MPD	9 mm					Uni
Takeshita et al. 9	Cyst size + MPD max. diameter	–	46	65 (43–78)	52·8	RCCS	Multi	BD
	MN						Uni
	MPD dilated + max. cyst size						Multi
Walter et al. 28	MN	30 mm	60	64(12·2)	60·3	RCCS	Multi	BD+MI+MD
	Cyst size
Woo et al. 48	Cyst size	30 mm	85	63 (40–82)	58·8	RCCS	Uni	BD
	DM
Xu et al. 43	CA19‐9	37 units/ml	54	IPMA: 61·4(8·24) (43–81)	66·7	RCCS	Multi	BD+MI+MD
	Cyst size	30 mm					Uni
	Jaundice						Uni
	MPD						Uni
Yamada et al. 58	Jaundice (obstructive)	–	166	66·6(8·5)	60·2	RCCS	Multi	BD+MI+MD
	Lymphadenopathy
You et al. 56	CA19‐9	37 units/ml	87	61·5(9·2)	64·4	RCCS	Multi	BD+MI+MD
	CEA	5 ng/ml

Values are mean(s.d.) (range). IPMN, intraductal papillary mucinous neoplasia; CA, carbohydrate antigen; Uni, univariable; BD, branch duct; MI, mixed‐type IPMN; MD, main duct; RCCS, retrospective controlled cohort study; IPMA, benign IPMN; IPMC, malignant IPMN; Multi, multivariable; MN, mural nodules; EUS, endoscopic ultrasonography; MPD, main pancreatic duct; n.a., not available; CEA, carcinoembryonic antigen.

AUC values derived from HSROC curves were calculated for each factor (Table   2). The largest AUCs were 0·84 for weight loss (Fig.  2) and 0·80 for jaundice (Fig.  3), followed by the serological markers CEA (0·79) and CA19‐9 (0·78) (Fig.  3). The radiological criteria of lymphadenopathy (0·51) and thickened cyst wall (0·56) had the lowest AUC values (Fig. 4).

Table 2

Results of the pooled analysis

	No. of studies	No. of patients	AUC	Sensitivity (%)	Specificity (%)	DOR	I 2 (%)
Age	5	645	0·67 (0·62, 0·72)	71 (53, 89)	59 (47, 71)	3·64 (2·20, 6·03)	21
CA19‐9	17	2747	0·78 (0·75, 0·82)	49 (41, 57)	89 (86, 92)	7·29 (5·36, 9·91)	44
CEA	3	456	0·79 (0·70, 0·86)	35 (21, 48)	95 (91, 99)	8·37 (4·27, 16·42)	0
Cyst size	21	2375	0·68 (0·65, 0·72)	64 (56, 72)	69 (61, 77)	3·62 (2·75, 4·76)	35
Diabetes	3	231	0·71 (0·62, 0·79)	46 (37, 56)	83 (76, 90)	4·42 (2·20, 8·90)	0
Jaundice	12	1689	0·80 (0·76, 0·84)	26 (18, 33)	97 (96, 99)	7·98 (5·24, 12·15)	0
Lymphadenopathy	5	945	0·51 (0·41, 0·61)	20 (8, 32)	93 (84, 100)	4·74 (2·18, 11·14)	52
Male sex	3	774	0·62 (0·56, 0·68)	59 (48, 71)	59 (47, 71)	2·14 (1·47, 3·12)	0
Dilatation of MPD	21	2991	0·77 (0·73, 0·80)	60 (52, 68)	80 (75, 86)	6·59 (4·69, 9·26)	55
Mural nodules	33	5068	0·77 (0·75, 0·80)	65 (60, 71)	81 (76, 85)	7·89 (6·34, 9·82)	32
Pancreatitis	7	1127	0·67 (0·63, 0·72)	32 (21, 43)	86 (80, 91)	2·67 (1·94, 3·68)	2
Thickened cyst wall	4	581	0·56 (0·46, 0·66)	23 (10, 36)	95 (88, 100)	4·93 (1·98, 11·35)	11
Weight loss	4	297	0·84 (0·78, 0·89)	53 (34, 72)	90 (83, 96)	8·72 (4·21, 18·07)	0

Values in parentheses are 95 per cent confidence intervals. AUC, area under the curve; DOR, diagnostic odds ratio; CA, carbohydrate antigen; CEA, carcinoembryonic antigen; MPD, main pancreatic duct.

Figure 2

Receiver operating characteristic (ROC) curves for clinical parameters associated with malignancy in intraductal papillary mucinous neoplasia: a pancreatitis, b weight loss, c male sex, d age, e diabetes mellitus

Figure 3

Receiver operating characteristic (ROC) curves for serological parameters associated with malignancy in intraductal papillary mucinous neoplasia: a carbohydrate antigen (CA) 19‐9, b jaundice, c carcinoembryonic antigen (CEA)

Figure 4

Receiver operating characteristic (ROC) curves for radiological parameters associated with malignancy in intraductal papillary mucinous neoplasia: a dilatation of main pancreatic duct (MPD), b thickened cyst wall, c mural nodules, d lymphadenopathy, e cyst size

Factors with the highest sensitivities were patient age (71 per cent), presence of mural nodules (65 per cent) and cyst size (64 per cent). Jaundice (26 per cent), thickened cyst wall (23 per cent) and lymphadenopathy (20 per cent) were the least sensitive. Specificity was highest for jaundice (97 per cent), raised CEA level (95 per cent) and thickened cyst wall (95 per cent), and lowest for patient age (59 per cent) and male sex (59 per cent) (Table   2; Fig.  S1, supporting information). Jaundice (82 per cent) and lymphadenopathy (71 per cent) had the highest PPVs, and male sex the lowest (26 per cent); NPVs ranged from 86 per cent for male sex to 60 per cent for diabetes mellitus (Fig.  S1, supporting information).

The pooled DOR was highest for weight loss (8·72), CEA increase (8·37) and jaundice (7·98), and lowest for male sex (2·14) and pancreatitis (2·67) (Table   2; Figs S2–S4, supporting information).

Analysis of heterogeneity of the included factors revealed low heterogeneity for the majority (below 30 per cent), with moderate heterogeneity (30–60 per cent) for CA19‐9 level, cyst size, lymphadenopathy, dilatation of the MPD and the presence of mural nodules (Table   2). Funnel plots of study quality are shown in Fig. S5 (supporting information).

Discussion

High reliability in the identification or exclusion of malignancy is an important characteristic of a diagnostic test that is clinically useful in patients with suspected cancer. This meta‐analysis assessed the diagnostic accuracy of a number of established clinical, radiological and serological markers, and revealed that no single clinically established factor (or the absence of such a factor) sufficiently predicted or excluded malignancy. Several factors provided high specificity, but sensitivity was generally poor.

Although it provides a comprehensive overview of all established factors in the stratification of IPMN of the pancreas, this analysis has several limitations. Studies evaluating BD, MD and mixed‐type IPMN were all included. It is conceded that many surgeons would feel that MD IPMN should generally be resected and might wonder why those different entities were investigated in one analysis. Although the dogma that all MD IPMN should be resected is based on an estimated malignancy rate of 61·6 per cent, compared with only 25·5 per cent for BD IPMN2, IPMN with only minimal MD involvement can be followed up safely without surgical intervention in some patients65. On the other hand, BD IPMN with high‐risk signs according to the Fukuoka consensus should be resected2. Future biomarkers might provide safe exclusion of malignancy in MD and BD IPMN. Until such reliable biomarkers have been established, the risk of malignancy in MD IPMN, mixed‐type IPMN and BD IPMN might be estimated incorrectly. The authors chose to include all three subtypes of IPMN of the pancreas to gain a thorough overview of the current literature, although it is accepted that this approach might represent a source of bias. Other limitations include the retrospective nature of most of the included studies, the conversion of continuous variables into binary variables, and heterogeneity of the studies.

The absence of a single valid criterion to predict malignancy leads to the conclusion that several factors need to be combined to identify sufficiently patients likely to benefit from intervention, or observation. The presence or absence of combinations of factors including jaundice, presence of mural nodules, dilatation of the MPD and others might then be used to guide treatment decisions.

The present study did identify factors with relatively high AUCs, such as the presence of increased levels of tumour markers. CEA and CA19‐9 are not included in the recommendations of the current consensus guidelines, but they might be valuable adjuncts where there is diagnostic uncertainty owing to their relatively high specificity. Several potential scoring formulas to improve diagnostic accuracy have been developed over the past decade19 36, 66 67, but none has been validated prospectively.

Until the identification of biomarkers with an adequate ROC curve (such as troponin T for the diagnosis of myocardial infarction), decisions regarding intervention or observation remain largely dependent on the ‘gut feeling’ of treating clinicians. An international prospective study using highly standardized clinical pathways with the collection of high‐quality biomaterial should be undertaken.

Appendix S1. Supporting information

Click here for additional data file.