An international study of interobserver variability of "string sign" of pancreatic cysts among experienced endosonographers.

BACKGROUND AND OBJECTIVES: No single optimal test reliably determines the pancreatic cyst subtype. Following EUS-FNA, the "string sign" test can differentiate mucinous from nonmucinous cysts. However, the interobserver variability of string sign results has not been studied.
METHODS: An experienced endosonographer performed EUS-FNA of pancreatic cysts on different patients and was recorded on video performing the string sign test for each. The videos were shared internationally with 14 experienced endosonographers, with a survey for each video: "Is the string sign positive?" and "If the string sign is positive, what is the length of the formed string?" Also asked "What is the cutoff length for string sign to be considered positive?" Interobserver variability was assessed using the kappa statistic (κ).
RESULTS: A total of 112 observations were collected from 14 endosonographers. Regarding string sign test positivity, κ was 0.6 among 14 observers indicating good interrater agreement (P < 0.001) while κ was 0.38 when observers were compared to the index endosonographer demonstrating marginal agreement (P < 0.001). Among observations of the length of the string in positive samples, 89.8% showed >5 mm of variability (P < 0.001), indicating marked variability. There was poor agreement on the cutoff length for a string to be considered positive.
CONCLUSION: String sign of pancreatic cysts has a good interobserver agreement regarding its positivity that can help in differentiating mucinous from nonmucinous pancreatic cysts. However, the agreement is poor on the measured length of the string and the cutoff length of the formed string to be considered a positive string sign.

INTRODUCTION

Pancreatic cystic lesions (PCLs) have increasingly been diagnosed in the past few years. This is a consequence of extended usage of high quality cross-sectional imaging, including computed tomography and magnetic resonance imaging and increasing clinician awareness about PCLs. Pancreatic cysts are being diagnosed incidentally in up to 20% of patients undergoing abdominal imaging for other etiologies.[12] It is estimated that PCLs are present in 2%–45% of the population with a high prevalence in more advanced age.[3] Pancreatic cysts have a wide spectrum of differential diagnoses ranging from lesions that are completely benign in nature to lesions with variable degrees of premalignant nature. The three major types of PCLs include pseudocysts, nonmucinous cystic lesions, and mucinous cystic lesions.[24]

Pancreatic cyst fluid sample is obtained through EUS-FNA. The “string sign” test usually involves placing a sample of the aspirated fluid from the PCL between the thumb and index finger and separating the fingers to measure the distance between the fingers before the sample string breaks [Figures 1 and 2]. A positive result is indicated by the formation of a long string due to a high concentration of mucus, a viscid, slippery secretion that is usually rich in mucin. Mucin has a higher surface tension than water, so it should form a longer string than water or nonmucinous fluid does. Bick et al. showed that string sign is highly specific in diagnosing mucinous pancreatic cysts (MPCs) and improves the diagnostic accuracy of cyst fluid analysis when used in combination with other cyst fluid tests.[5] Thus far, no study has studied interobserver variability between endosonographers in detecting and evaluating string sign. The aim of this study was, therefore, to evaluate the interobserver variability in string sign positivity and in the length of the formed string in a positive string sign test.

Figure 1

Image panel showing 2 pictures of positive string sign test. The length of the string is about 20 mm long in (a) and 10 mm long in (b) (based on the thickness of the index finger of the index endosonographer)

Figure 2

Image showing negative string sign test

METHODS

Study design

We recorded eight videos of an experienced senior endosonographer (M. S. B.) evaluating the string sign for 7 patients after obtaining pancreatic cyst fluid sample through EUS-FNA, performed by the same endosonographer (M. S. B.) since one patient had 2 different videos [Videos 1 and 2]. The index endosonographer defined a positive string sign if the length of the formed string was 5 mm or more. This index endosonographer assessed the positivity of the string sign and measured the formed string in positive string sign samples, and his measurements were used as a reference for the positivity of the string sign and for the length of the formed string. Under an Institutional Review Board–approved study protocol to retrospectively extract prospectively collected data from patients who underwent EUS-FNA, the 8 videos were shared internationally with 14 other experienced endosonographers in the United States, Europe, and Asia through the web-based storage-cloud box. All patient identifiers were removed from the videos. These observers were asked to complete a survey comprising the same questions for each video: “Is the string sign positive?” and “If the string sign is positive, what is the length of the formed string in positive string sign samples?” The observers were also asked: “What is the cutoff length for the formed string to be considered a positive string sign test?” All the endosonographers were informed that the measured anteroposterior/lateral thickness of the tip of the index finger of the performing endosonographer is about 1.5 cm, which was used as a 'reference length' to guide the measurement of the formed string.

Pancreatic cystic lesion subtypes

The first patient for Videos 1 and 2 had a mucinous cyst, which was confirmed by surgical pathology after distal pancreatectomy. Patients for Videos 3, 6 and 7 had a confirmed mucinous cyst based on cytological evaluation showing mucin and mucinous epithelium after pancreatic cyst aspiration. The patient for video 4 had a mucinous cyst, which was confirmed by surgical pathology after Whipple's procedure. Patients for Videos 5 and 8 had a possible mucinous cyst based on elevated carcinoembryonic antigen (CEA) level; however, no mucin or mucinous epithelium was found on cytological evaluation and these 2 patients did not undergo surgical resection with no availability of surgical pathology [Table 1 shows patient demographics and pancreatic cyst characteristics].

Table 1

Patient demographics and pancreatic cyst characteristics

Patient	Video	Race	Age	Sex	Cyst characteristics	Cyst studies	Cytology	Surgery	Surgical pathology	Reference string sign length	Final diagnosis
1	1 and 2	African American	44	Female	46×39 mm. pancreatic tail	Amylase=144. CEA=NA (due to high viscosity). Glucose=NA	Scant mucinous epithelium and mucin	Distal pancreatectomy and splenectomy	MCN with focal low grade dysplasia	35 mm	Mucinous due to surgical pathology
2	3	White	74	Female	34×23.5 mm. uncinate process	Amylase >21,000. CEA=52.8. Glucose=165	Ductal epithelium with mucin	No	NA	10 mm	Mucinous due to cytology
3	4	White	74	Male	13×22 mm. pancreatic head	Amylase >21,000. CEA=30.4. Glucose=NA	Mucinous ductal epithelium and mucin	Whipple surgery	IPMN with focal high grade dysplasia	10 mm	Mucinous due to surgical pathology
4	5	White	74	Male	26×27 mm. pancreatic head	Amylase >21,000. CEA=2860. Glucose=NA	Ductal epithelium in a degenerated acellular background	No	NA	0 mm	Possibly mucinous based on cyst fluid markers
5	6	White	81	Male	25×17 mm. pancreatic body	Amylase=1387. CEA=377. Glucose=NA	Mucinous epithelium and mucin	No	NA	5 mm	Mucinous due to cytology
6	7	Asian	76	Female	17×16 mm. uncinate process	Amylase >7500. CEA=207. Glucose=60	Mucin	No	NA	10 mm	Mucinous due to cytology
7	8	White	74	Male	22×22 mm. pancreatic head	Amylase=1726. CEA=2180. Glucose=159	Rare Acinar epithelium. no mucin was seen	No	NA	0 mm	Possibly mucinous based on cyst fluid markers

MCN: Mucinous cystic neoplasm, IPMN: Intraductal papillary mucinous neoplasms, NA: Not available, CEA: Carcinoembryonic antigen

Outcomes

The primary outcome of this study is to evaluate the interobserver variability in assessing the positivity of string sign in aspirated fluid from PCLs. The secondary outcome is to assess the interobserver variability in measuring the length of the formed string in positive string sign samples.

Statistical analysis

The kappa statistic (κ) was used to measure the interrater agreement for the positivity of the string sign and the variability of the measured length of the samples with a positive string test using the eight videos. A κ value of 1 represents perfect agreement between the raters. A κ of 0 indicates no more rater agreement than that expected by chance. In addition, κ >0.75 denotes excellent agreement, 0.40≤ κ ≤0.75 denotes good agreement, and 0.00< κ <0.40 denotes marginal agreement. A one-sample test of proportion was used to determine whether any variation of >5 mm was greater than or equal to 50%. Statistical analysis was performed using Stata/SE version 16.0 statistical software (Stata Corp LLC, College Station, TX, USA).

RESULTS

There were 112 observations, or one measurement per observer (n = 14 observers) per video (n = 8 videos). The index endosonographer measurements were used only as a reference and were not included in the 112 observations. Forty-nine (43.75%) observations were string sign positive. One string measurement out of the 49 positive string signs was 45–50 mm long. Two measurements were 40–45 mm long. Eleven measurements were 30–35 mm long. Nine measurements were 25–30 mm long. Six measurements were 20–25 mm long. Five measurements were 15–20 mm long. Ten measurements were 10–15 mm long. Four measurements were 5–10 mm long and one measurement out of the 49 string sign positive measurements was <5 mm long [Table 2].

Table 2

String sign results for all the observations

	n (%)
String sign positive
No	63 (56.25)
Yes	49 (43.75)
If yes, what is the length of the string in mm?
<5	1 (2.04)
5-10	4 (8.16)
10-15	10 (20.41)
15-20	5 (10.20)
20-25	6 (12.24)
25-30	9 (18.37)
30-35	11 (22.45)
40-45	2 (4.08)
45-50	1 (2.04)

Table 3 shows the distribution of string sign positivity for each video. Video 1 had 100% agreement on the positivity of string sign. Video 8 had 100% agreement on the negativity of the string sign. For video 3, only 9 observers agreed on the same result, while the remaining 5 observers had a different opinion. For each of the remaining videos, 12 or more observers had good agreement.

Table 3

String sign positive results for each video

	String sign positive, n (%)
	No	Yes
Video
1	0 (0.00)	14 (28.57)
2	1 (1.59)	13 (26.53)
3	9 (14.29)	5 (10.20)
4	12 (19.05)	2 (4.08)
5	13 (20.63)	1 (2.04)
6	12 (19.05)	2 (4.08)
7	2 (3.17)	12 (24.49)
8	14 (22.22)	0 (0.00)

When we evaluated string sign positivity, we observed good interobserver agreement during the assessment of interrater agreement for all observers without including the index endosonographer measurements in the comparison, the κ value was 0.60, which was statistically significant (P < 0.001) and pairwise comparisons between observers are shown in Table 4. Second, when we assessed the interrater agreement between all observers and the index endosonographer, the results were statistically significant (P < 0.001) with a κ value of 0.38 indicating marginal agreement [Table 5]. Pairwise comparisons between each observer and the index endosonographer were also provided.

Table 4

Interrater agreement overall and observer’s pairwise comparisons

	Observers	Agreement	Kappa	SE	P
Interrater agreement	14 videos		0.60		<0.001
Observer 2 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 3 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 4 versus observer 1	8	0.75	0.50	0.31	0.102
Observer 5 versus observer 1	8	0.75	0.50	0.31	0.102
Observer 6 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 7 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 1	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 1	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 1	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 1	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 1	8	0.50	0.00	0.35	0.999
Observer 3 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 4 versus observer 2	8	0.75	0.50	0.31	0.102
Observer 5 versus observer 2	8	0.75	0.50	0.31	0.102
Observer 6 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 7 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 2	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 2	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 2	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 2	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 2	8	0.50	0.00	0.35	0.999
Observer 4 versus observer 3	8	0.75	0.50	0.31	0.102
Observer 5 versus observer 3	8	0.75	0.50	0.31	0.102
Observer 6 versus observer 3	8	0.88	0.75	0.34	0.028
Observer 7 versus observer 3	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 3	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 3	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 3	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 3	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 3	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 3	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 3	8	0.50	0.00	0.35	0.999
Observer 5 versus observer 4	8	0.75	0.50	0.31	0.102
Observer 6 versus observer 4	8	0.88	0.75	0.34	0.028
Observer 7 versus observer 4	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 4	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 4	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 4	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 4	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 4	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 4	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 4	8	0.50	0.00	0.35	0.999
Observer 6 versus observer 5	8	0.88	0.75	0.34	0.028
Observer 7 versus observer 5	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 5	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 5	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 5	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 5	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 5	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 5	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 5	8	0.50	0.00	0.35	0.999
Observer 7 versus observer 6	8	0.88	0.75	0.34	0.028
Observer 8 versus observer 6	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 6	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 6	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 6	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 6	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 6	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 6	8	0.50	0.00	0.35	0.999
Observer 8 versus observer 7	8	1.00	1.00	0.35	0.005
Observer 9 versus observer 7	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 7	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 7	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 7	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 7	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 7	8	0.50	0.00	0.35	0.999
Observer 9 versus observer 8	8	0.88	0.75	0.34	0.028
Observer 10 versus observer 8	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 8	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 8	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 8	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 8	8	0.50	0.00	0.35	0.999
Observer 10 versus observer 9	8	0.88	0.75	0.34	0.028
Observer 11 versus observer 9	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 9	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 9	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 9	8	0.50	0.00	0.35	0.999
Observer 11 versus observer 10	8	0.75	0.50	0.31	0.102
Observer 12 versus observer 10	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 10	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 10	8	0.50	0.00	0.35	0.999
Observer 12 versus observer 11	8	1.00	1.00	0.35	0.005
Observer 13 versus observer 11	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 11	8	0.50	0.00	0.35	0.999
Observer 13 versus observer 12	8	0.88	0.75	0.34	0.028
Observer 14 versus observer 12	8	0.50	0.00	0.35	0.999
Observer 14 versus observer 13	8	0.63	0.25	0.34	0.465

SE: Standard error

Table 5

Interrater agreement overall and observer’s pairwise comparisons compared to the reference

	Observers	Agreement	Kappa	SE	P
All observers versus reference	112	0.67	0.38	0.08	<0.001
Observer 1 versus reference	8	0.75	0.50	0.31	0.102
Observer 2 versus reference	8	0.63	0.33	0.26	0.206
Observer 3 versus reference	8	0.63	0.33	0.26	0.206
Observer 4 versus reference	8	1.00	1.00	0.35	0.005
Observer 5 versus reference	8	0.50	0.20	0.21	0.346
Observer 6 versus reference	8	0.88	0.71	0.34	0.035
Observer 7 versus reference	8	0.63	0.33	0.26	0.206
Observer 8 versus reference	8	0.75	0.50	0.31	0.102
Observer 9 versus reference	8	0.63	0.33	0.26	0.206
Observer 10 versus reference	8	0.63	0.33	0.26	0.206
Observer 11 versus reference	8	0.50	0.20	0.21	0.346
Observer 12 versus reference	8	0.75	0.50	0.31	0.102
Observer 13 versus reference	8	0.63	0.33	0.26	0.206
Observer 14 versus reference	8	0.50	0.00	0.31	0.999

SE: Standard of error

Variation within 5 mm of the measured length of the formed string in positive string sign samples was considered by us as a good interrater agreement range. However, as illustrated in Table 6, 89.8% of the measurements in positive samples were variable by more than 5 mm among observers (P < 0.001). This indicates poor interrater agreement with marked interobserver variability when it comes to the measured length of the formed string in positive string signs among the observers only without including the reference endosonographer. These results were also similar when the measured length of the formed string in positive string signs between observers and the reference endosonographer measurements were compared as illustrated in Table 7.

Table 6

Variation between the observers regarding the length of the formed string in string sign positive samples

	Variation ≤5 mm, n (%)	Variation >5 mm, n (%)	P
All observers	5 (10.20)	44 (89.80)	<0.001
Observer 2 versus observer 1	2 (50.00)	2 (50.00)	0.999
Observer 3 versus observer 1	1 (25.00)	3 (75.00)	0.317
Observer 4 versus observer 1	2 (50.00)	2 (50.00)	0.999
Observer 5 versus observer 1	2 (50.00)	2 (50.00)	0.999
Observer 6 versus observer 1	1 (25.00)	3 (75.00)	0.317
Observer 7 versus observer 1	1 (25.00)	3 (75.00)	0.317
Observer 8 versus observer 1	2 (50.00)	2 (50.00)	0.999
Observer 9 versus observer 1	3 (75.00)	1 (25.00)	0.317
Observer 10 versus observer 1	1 (25.00)	3 (75.00)	0.317
Observer 11 versus Observer 1	2 (50.00)	2 (50.00)	0.999
Observer 12 versus observer 1	3 (75.00)	1 (25.00)	0.317
Observer 13 versus observer 1	2 (50.00)	2 (50.00)	0.999
Observer 14 versus observer 1	4 (100.00)	-	-
Observer 3 versus observer 2	3 (100.00)	-	-
Observer 4 versus observer 2	3 (100.00)	-	-
Observer 5 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 6 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 7 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 8 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 9 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 10 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 11 versus observer 2	2 (66.67)	1 (33.33)	0.564
Observer 12 versus observer 2	3 (100.00)	-	-
Observer 13 versus observer 2	3 (100.00)	-	-
Observer 14 versus observer 2	1 (33.33)	2 (66.67)	0.564
Observer 4 versus observer 3	1 (33.33)	2 (66.67)	0.564
Observer 5 versus observer 3	3 (100.00)	.	0.083
Observer 6 versus observer 3	1 (33.33)	2 (66.67)	0.564
Observer 7 versus observer 3	1 (33.33)	2 (66.67)	0.564
Observer 8 versus observer 3	3 (100.00)	-	-
Observer 9 versus observer 3	3 (100.00)	-	-
Observer 10 versus observer 3	1 (33.33)	2 (66.67)	0.564
Observer 11 versus observer 3	3 (100.00)	-	-
Observer 12 versus observer 3	3 (100.00)	-	-
Observer 13 versus observer 3	3 (100.00)	-	-
Observer 14 versus observer 3	3 (100.00)	-	-
Observer 5 versus observer 4	2 (33.33)	4 (66.67)	0.414
Observer 6 versus observer 4	3 (50.00)	3 (50.00)	0.999
Observer 7 versus observer 4	2 (33.33)	4 (66.67)	0.414
Observer 8 versus observer 4	6 (100.00)	-	-
Observer 9 versus observer 4	1 (16.67)	5 (83.33)	0.102
Observer 10 versus observer 4	2 (33.33)	4 (66.67)	0.414
Observer 11 versus observer 4	1 (16.67)	5 (83.33)	0.102
Observer 12 versus observer 4	4 (66.67)	2 (33.33)	0.414
Observer 13 versus observer 4	3 (50.00)	3 (50.00)	0.999
Observer 14 versus observer 4	2 (33.33)	4 (66.67)	0.414
Observer 6 versus observer 5	2 (100.00)	-	-
Observer 7 versus observer 5	1 (50.00)	1 (50.00)	0.999
Observer 8 versus observer 5	1 (50.00)	1 (50.00)	0.999
Observer 9 versus observer 5	1 (50.00)	1 (50.00)	0.999
Observer 10 versus observer 5	2 (100.00)	-	-
Observer 11 versus observer 5	2 (100.00)	-	-
Observer 12 versus observer 5	2 (100.00)	-	-
Observer 13 versus observer 5	2 (100.00)	-	-
Observer 14 versus observer 5	1 (50.00)	1 (50.00)	0.999
Observer 7 versus observer 6	1 (20.00)	4 (80.00)	0.180
Observer 8 versus observer 6	5 (100.00)	-	-
Observer 9 versus observer 6	5 (100.00)	-	-
Observer 10 versus observer 6	3 (60.00)	2 (40.00)	0.655
Observer 11 versus observer 6	1 (20.00)	4 (80.00)	0.180
Observer 12 versus observer 6	3 (60.00)	2 (40.00)	0.655
Observer 13 versus observer 6	2 (40.00)	3 (60.00)	0.655
Observer 14 versus observer 6	2 (40.00)	3 (60.00)	0.655
Observer 8 versus observer 7	1 (33.33)	2 (66.67)	0.564
Observer 9 versus observer 7	1 (33.33)	2 (66.67)	0.564
Observer 10 versus observer 7	1 (33.33)	2 (66.67)	0.564
Observer 11 versus observer 7	3 (100.00)	-	-
Observer 12 versus observer 7	2 (66.67)	1 (33.33)	0.564
Observer 13 versus observer 7	2 (66.67)	1 (33.33)	0.564
Observer 14 versus observer 7	1 (33.33)	2 (66.67)	0.564
Observer 9 versus observer 8	3 (75.00)	1 (25.00)	0.317
Observer 10 versus observer 8	4 (100.00)	-	-
Observer 11 versus observer 8	1 (25.00)	3 (75.00)	0.317
Observer 12 versus observer 8	2 (50.00)	2 (50.00)	0.999
Observer 13 versus observer 8	2 (50.00)	2 (50.00)	0.999
Observer 14 versus observer 8	4 (100.00)	-	-
Observer 10 versus observer 9	3 (100.00)	-	-
Observer 11 versus observer 9	1 (33.33)	2 (66.67)	0.564
Observer 12 versus observer 9	2 (66.67)	1 (33.33)	0.564
Observer 13 versus. observer 9	2 (66.67)	1 (33.33)	0.564
Observer 14 versus observer 9	3 (100.00)	-	-
Observer 11 versus observer 10	1 (33.33)	2 (66.67)	0.564
Observer 12 versus observer 10	2 (66.67)	1 (33.33)	0.564
Observer 13 versus observer 10	2 (66.67)	1 (33.33)	0.564
Observer 14 versus observer 10	2 (66.67)	1 (33.33)	0.564
Observer 12 versus observer 11	2 (100.00)	-	-
Observer 13 versus observer 11	2 (100.00)	-	-
Observer 14 versus observer 11	2 (100.00)	-	-
Observer 13 versus observer 12	3 (75.00)	1 (25.00)	0.317
Observer 14 versus observer 12	1 (25.00)	3 (75.00)	0.317
Observer 14 versus observer 13	1 (33.33)	2 (66.67)	0.564

Table 7

Variation between the observers and reference endosonographer regarding the length of the formed string in string sign positive samples

	n (%)	P
Overall
Variation ≤5	22 (44.90)	0.475
Variation >5	27 (55.10)
Video 1
Variation ≤5	5 (35.71)	0.285
Variation >5	9 (64.29)
Video 2
Variation ≤5	5 (38.46)	0.405
Variation >5	8 (61.54)
Video 3
Variation ≤5	3 (60.00)	0.655
Variation >5	2 (40.00)
Video 4
Variation ≤5	2 (100.00)	0.157
Variation >5	0 (0.00)
Video 5
Variation ≤5	0 (0.00)	0.317
Variation >5	1 (100.00)
Video 6
Variation ≤5	1 (50.00)	0.999
Variation >5	1 (50.00)
Video 7
Variation ≤5	6 (50.00)	0.999
Variation >5	6 (50.00)
Video 8
Variation ≤5	0 (0.00)	-
Variation >5	0 (0.00)

In regard to the cutoff length of the formed string used to interpret a positive test, some observers used 3.5 mm, and other observers used 10 mm while the index endosonographer used 5 mm as a cutoff length.

DISCUSSION

Non MPCs encompass a wide, heterogeneous variety that can be neoplastic with variable degrees of premalignant nature, as serous cystadenoma, cystic neuroendocrine tumors, and lymphangioma or they can be completely nonneoplastic with no malignant potential, as lymphoepithelial cyst, retention cyst, or endometrial cyst. MPCs include mucinous cystic neoplasm and intraductal papillary mucinous neoplasms (IPMN).[36] The third PCL subtype is pseudocyst, which always develops after trauma or pancreatitis. Pseudocysts are completely benign with no premalignant nature, and the consequences of misdiagnosing MPC for pseudocyst can be serious since mucinous cysts can be premalignant.[7]

Many efforts are underway to differentiate between mucinous and nonmucinous PCLs. It is crucial to determine the pancreatic cyst subtype, as that impacts follow-up, surveillance, and management. The proper diagnosis and management of pancreatic cysts, according to subtype may prevent progression to pancreatic cancer. At the same time, accurate diagnosis can minimize the lifelong screening and health-related costs if the PCL subtype does not have premalignant potential. However, no single optimal test is available to determine the cyst subtype yet.[7] The yield of cytological analysis is variable, with pooled meta-analysis showing only 63% sensitivity in differentiating mucinous from non-MPCs.[8]

Managing pancreatic cysts can be challenging, especially for MPCs, given their premalignant behavior. Until the beginning of this decade, many more PCLs with malignant potential underwent surgical resection, but following the release of International Association of Pancreatology Sendai guidelines in 2006[9] and the release of Fukuoka guidelines in 2012,[10] a more conservative approach has been followed and the majority of newly diagnosed PCLs are not being surgically resected. Additional guidelines have been released since, including the American Gastroenterological Association guidelines in 2015,[11] the revision of the Fukuoka guidelines in 2017,[12] the American College of Gastroenterology clinical guidelines in 2018,[13] and the European guidelines in 2018,[3] establishing criteria by which to stratify MPCs according to their risk of malignant transformation and to help guide their follow-up and management.

EUS-FNA plays a pivotal role in evaluating pancreatic cysts, both to identify PCL subtype and to further assess a PCL with an established diagnosis that has developed worrisome features.[13] The most commonly used pancreatic cyst fluid marker is CEA, which is helpful in differentiating mucinous and nonmucinous cysts. Median CEA is higher in MPCs compared to nonmucinous cysts. In a large systematic review and meta-analysis, Thornton et al. found that cyst fluid CEA level has a pooled sensitivity of 63% and a pooled specificity of 88%.[14] However, the diagnostic accuracy, sensitivity, and specificity of the CEA level in cyst fluid vary according to the CEA cutoff.[1516] Brugge et al. showed that with cyst fluid CEA cutoff level of 192 ng/ml has sensitivity of 75%, specificity of 83.6%, and diagnostic accuracy of 79% in differentiating mucinous from non-MPCs. The mucinous cysts in this study included benign and malignant mucinous cysts.[17]

As a cost-free, easy-to-perform measure of cyst fluid viscosity, the string sign test is widely used in clinical practice; however, the literature on string sign is limited. In a study by Leung et al., the median length of the formed string from the pancreatic cyst is 0 mm in benign nonmucinous cysts compared to 3.5 mm length in MPC; however, Bick et al. used 10 mm as a cutoff length for string sign to be considered positive.[518] Oh et al. demonstrated that string sign alone for diagnosing MPCs has sensitivity of 54.6%, specificity of 100%, and diagnostic accuracy of 72.3%, whereas the combination of cyst fluid CEA analysis, cyst fluid cytology, and string sign test increased the overall diagnostic accuracy to 94%, higher than that of any of these tests individually.[19] Results by Bick et al. revealed that the overall diagnostic accuracy of cyst fluid analyses improved to 89% with the combination of string sign test and cyst fluid CEA level by using 200 ng/ml as a cutoff value.[5]

Our study demonstrated good interrater agreement on the positivity of string sign tests among the observers and marginal agreement between the observers and index endosonographer. Majority of the disagreement of the observers among themselves or with the index endosonographer was due to variability on the agreement of the cutoff length of the formed string to be considered a positive string sign. Furthermore, the weaker agreement between observers and index endosonographer can be explained by assessing string sign in real time versus assessing its positivity in videos. This study also showed that there is the poor interrater agreement with marked interobserver variability regarding the measurement of the length of the formed string in positive string sign samples. The poor interrater agreement was found in both, among the observers and between the observers and the index endosonographer.

One limitation of our study is that all the observations and measurements were obtained from videos and not in real time, which may make the measurements more difficult to interpret and may increase the variability among the observations. Since all the observers that were being compared to each other with a similar method of measurement in a video, some of the potential error in comparing a real-time measurement (by the index endosonographer) versus a video recording is decreased. The reference scale used was the thickness of the thumb of one person (operator) where the apparent thickness can vary in each observation/video recording due to parallax (depending on the angle of the camera and its distance from thumb). To overcome this drawback, the same videos were being compared to each other with a similar method of measurement. In real-world practice, it is actually unclear how various endosonographers measure the length of the string sign since no standardized method for measurement of the length of the string sign has been described. Many use lengths of unbroken string on the separation of the two fingers in contact (index finger and thumb). The use of two glass slides, after keeping a drop of cyst fluid in between, kept against a reference scale at eye level has been suggested as an alternate method by some. While using the two-finger method, do endosongraphers use the tip of their finger as a reference as used in this study? Do they just estimate the length on the visual inspection? Do they put a ruler behind their hand as a reference while performing string sign test? Bick et al. suggested there are 2 methods for assessing string sign: Placing a drop of fluid between 2 gloved fingers and slowly spreading them apart (already discussed above) or watching the fluid drip from the EUS-FNA needle.[5] Interestingly, all the test endosonographers appeared to be familiar with the gloved two-finger method, and none of them commented that they use another method. It is our sense that most of the endosonographers in the real world make a visual estimation of the length of the string between gloved fingers and do not use objective measurements such as a ruler or a guide such as the thickness of the tip of their finger. Another practical issue is that the string appears for a fleeting moment, especially when it is shorter in length or when the fluid is less viscous, and it can be difficult to quickly determine the length before the string collapses. These practical issues with measuring the string sign should be realized.

As shown in Table 1, only 2 patients representing three videos in our study had surgical resection with pathology confirming mucinous cysts. Other 3 patients in our study had mucin and/or mucinous epithelium seen on cytological examination confirming mucinous cysts. The remaining 2 patients representing [Videos 5 and 8] have a possible mucinous cyst based on cyst fluid CEA level >192 ng/ml. However, the main intent of the study was not to address the correlation of a positive string sign with mucinous pathology but rather the interobserver variability in what is considered a positive string sign by trained endosonographers. In fact, when the string sign videos were sent to the test endosonographers, no clinical information about patient demographics, cyst size and morphology, results of cytopathology or biochemical markers or surgical pathology were shared with them to objectively assess the string sign interobserver assessment and reliability in and of itself.

A very important finding from our study is that various expert endosonographers from across different continents use different cutoff lengths, ranging from 3.5 mm to 10 mm, to define a positive string sign, which definitely affected the κ value. This is an extremely important finding of this study that is not affected by the fact that the measurement was not done in real time by the observers, since even if they performed the measurement in real time, they would still be using a different cut-off values. Our study could provide important guidance in interpreting the positivity of a string sign. The longer the string sign (e.g., >10 mm), the less disagreement would be expected regarding its positivity. Furthermore, a very long (a few centimeters) string could potentially be enough information to diagnose a mucinous lesion without ancillary studies such as CEA, glucose levels, cytopathologic conformation or any emerging molecular markers. If the aspirated cyst fluid is visibly “thick” and opaque, appearing like “glue” similar to what we see at the ampulla with a “fish mouth” appearance in main duct IPMN, and the string sign is a few centimeters long, do we really need another test to tell us that this fluid is mucin and the cyst is mucinous in nature? All would agree in that situation that the string sign is positive.

Our study results are very important in cases of less viscous and clearer mucin, where the string sign may be between 3.5 mm and 15 mm, which can cause disagreement among endosonographers regarding, first, the minimum length considered positive and second, the actual length of the string sign. We found that 89.8% of observations in positive samples showed variability >5 mm in the length of the formed string (P < 0.001), and this variability can make the interpretation of positivity or negativity challenging and more likely to be discordant among two endosonographers. When fluid is more viscous with string sign of 20 mm or more, this is less relevant since these string signs would be at least around 10 mm despite the variability among endosonographer observations and would most likely be considered positive by all the endosonographers in this study.

CONCLUSION

String sign is a useful and reliable test that can be used to improve the diagnostic accuracy of other pancreatic cyst fluid studies when used in combination. There is a good interrater agreement among different experienced endosonographers in assessing its positivity. However, there is disagreement on the minimum length of the string sign to be considered positive, ranging from 3.5 mm to 10 mm, and poor interrater agreement with marked interobserver variability (>5 mm) in the measured length of the formed string. Therefore, a positive string sign, especially when it is <2 cm, should be interpreted with caution and not used as a single test but in combination with other tests to differentiate mucinous from nonmucinous cysts. More studies are required to determine the optimal cutoff length at which a formed string should be considered a positive string sign, which should have a sufficiently high sensitivity, specificity, and diagnostic accuracy that would be internationally accepted as a criterion for differentiating mucinous from nonmucinous PCLs.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.