Triage amalgamated dermoscopic algorithm (TADA) for skin cancer screening.

IMPORTANCE: Dermoscopic triage algorithms have been shown to improve beginners' abilities for identifying pigmented skin lesions requiring biopsy.
OBJECTIVE: To estimate the diagnostic accuracy of the Triage Amalgamated Dermoscopic Algorithm (TADA) for pigmented and nonpigmented skin cancers. Secondarily, to compare TADAs performance to those of existing triage algorithms for the identification of pigmented skin cancers.
DESIGN: Cross-sectional, observational, reader study that took place at a beginner and intermediate level dermoscopy course. PARTICIPANTS: Two hundred medical professionals of various specialties attended the course and 120 voluntarily joined the study (60% participation rate). EXPOSURES: After receiving basic dermoscopy training, participants evaluated 50 polarized, dermoscopic images of pigmented (22 benign, 18 malignant) and nonpigmented (1 benign, 9 malignant) skin lesions using TADA. Pigmented lesions were also evaluated using the Three-Point Checklist and AC Rule. With TADA, participants first determined if a lesion was an unequivocal angioma, dermatofibroma, or seborrheic keratosis, which would exclude it from further evaluation. All other lesions were assessed for architectural disorder, starburst pattern, blue-black or gray color, shiny white structures, negative network, ulcer/erosion, or vessels. Any one feature indicated suspicion for malignancy.
RESULTS: Most participants were dermatologists (n=64, 53.3%) or primary care physicians (n=41, 34.2%), and many lacked previous dermoscopy training (n=52, 43.3%). TADA's sensitivity and specificity for all skin cancers was 94.6% (95% CI=93.4-95.7%) and 72.5% (95% CI=70.1-74.7%), respectively. For pigmented skin cancers, the sensitivity and specificity were 94.0% (95% CI=92.9-95.0%) and 75.5% (95% CI=73.8-77.2%). This compared to 71.9% (95%CI=69.8-73.9%) and 81.4% (95%CI=79.7-83.0%) for the Three-Point Checklist and 88.6% (95%CI=87.1-89.9%) and 78.7% (95%CI=76.9-80.3%) for the AC Rule.
CONCLUSIONS: These results suggest that TADA compares favorably to existing triage algorithms and might be a useful triage tool with high sensitivity and specificity for pigmented and nonpigmented skin cancers. Further studies are needed to validate these preliminary observations.

Introduction

Dermoscopy allows skilled observers to more accurately identify pigmented skin cancers compared to clinical exam alone [1-3]. In some cases, it can also help identify nonpigmented malignancies [4]. Despite the potential for improved skin cancer detection, a number of barriers are preventing many dermatologists, dermatology residents, and other medical professionals interested in skin cancer management from adopting dermoscopy. Lack of training has been cited as a major hindrance [5,6]. Nonetheless, the use of dermoscopy is increasing [7], and with it, interest in educational materials that provide novices an entry point into dermoscopy [8].

Teaching beginners the numerous and often nuanced dermoscopic patterns and structures required for diagnosis can be daunting. This has led some authors to suggest that triage and not diagnosis be the goal of the dermoscopic evaluation when performed by non-experts [9,10]. Triage in the context of skin lesion evaluations requires the examiner to determine if a lesion is suspicious for malignancy, thus requiring a biopsy or specialist referral; it does not require that a specific diagnosis be made. Triage algorithms may be easier to teach, learn, and implement by allowing for the nonspecific identification of concerning lesions using limited dermoscopic criteria. The validation of two triage algorithms, the Three-Point Checklist (asymmetry, atypical network, blue-white color) and the AC Rule (asymmetry, color variation), have demonstrated the feasibility of this approach by quickly improving novices’ abilities to recognize pigmented lesions requiring biopsy [11,12]. A limitation of both methods is that they were designed for subsets of pigmented skin cancers.

The triage amalgamated dermoscopic algorithm (TADA) was designed to identify common pigmented and nonpigmented skin cancers (Figure 1). Although TADA does not ask users to preselect and apply the algorithm to exclusively pigmented lesions, it does require that users first determine if a lesion is a dermoscopically unequivocal example of one of three commonly encountered benign neoplasms (angioma, dermatofibroma, seborrheic keratosis). If a lesion is determined to be one of these three neoplasms, it is excluded from further evaluation. If not, it is then evaluated for architectural disorder (i.e., disorganized or asymmetric distribution of colors and/or structures)—a robust criterion that is strongly associated with malignancy and has good interobserver agreement, making it easier to teach and learn and lending itself to the inclusion in a triage algorithm [13,14]. To improve sensitivity for organized and symmetric skin cancers, including certain melanomas (e.g., spitzoid, desmoplastic, nodular, and amelanotic) as well as non-melanoma skin cancers, TADA includes six additional criteria (starburst, blue-black or gray color, shiny white structures, negative network, ulcer/erosion, vessels) previously validated to be associated with different subtypes of melanoma and non-melanoma skin cancers and are independent predictors of malignancy [15-21]. The presence of any one feature included in TADA warrants a biopsy or specialist referral. Since shiny white structures can only be seen with polarized light, TADA requires the use of polarized dermoscopy. TADA was not tested on facial, mucosal, volar, or nail lesions.

Figure 1

The Triage Amalgamated Dermoscopic Algorithm (TADA)—Illustrated diagram outlining TADAs step-wise approach to evaluating and managing pigmented and nonpigmented skin lesions. [Copyright: ©2017 Rogers et al.]

The aim of this study was to determine the sensitivity and specificity of TADA for the detection of common skin cancers (melanoma, basal cell carcinoma, squamous cell carcinoma). A secondary aim was to compare the performance of TADA, the Three-Point Checklist, and AC Rule when identifying pigmented study lesions.

Materials and Methods

Study Design

This was a cross-sectional, observational study performed in an experimental setting.

Participants

This study was approved by the Memorial Sloan Kettering Cancer Center Institutional Review Board without requirement of written informed consent in accordance with the Helsinki Declaration. The study was performed on August 14, 2015, at a dermoscopy course for beginner and intermediate level dermoscopy users. All registered attendees were invited to participate. Participation was voluntary. There was no compensation or inducement. Participant characteristics (age, sex, medical specialty, prior dermoscopy training, cumulative dermoscopy experience) were recorded on data collection forms.

Image Selection

The image records of AAM were retrospectively and sequentially reviewed, starting from the most recent dermoscopic image on file, to identify an approximately equal proportion of representative examples of common benign and malignant skin lesions. Facial, mucosal, volar, and nail lesions were excluded. Sixty-two skin neoplasms were selected, of which twelve were excluded due to image quality or lack of polarized dermoscopic images. The resulting 50 lesions included 27 malignant (16 melanomas, 7 basal cell carcinomas, and 4 squamous cell carcinomas) and 23 benign neoplasms (8 nevi, 5 angiomas, 5 seborrheic keratoses, 4 dermatofibromas, and 1 clear cell acanthoma). All but one of the 16 melanomas measured less than 0.5 mm thick (nodular melanoma >1mm). Ten of the 50 lesions (20%) were clinically and dermoscopically nonpigmented (2 melanomas, 5 basal cell carcinomas, 2 squamous cell carcinomas, and 1 clear cell acanthoma). All malignant lesions were pathologically verified. Benign lesions were either evaluated pathologically or were required to be unchanged compared to baseline images. Images were captured with contact polarized dermoscopy (x10 magnification factor) using a digital camera (Canon Powershot G16; Canon Inc., Tokyo, Japan) and a dermoscopy lens attachment (DermLite FOTO system; 3Gen Inc., San Juan Capistrano, CA, USA).

Dermoscopy Training

The study was conducted during the latter half of the second day of a three-day dermoscopy course. On day one of the course, participants were lectured on basic dermoscopic criteria of common benign and malignant skin lesions. Participants were also introduced to the idea of dermoscopy algorithms as part of a lecture on dermoscopic teaching methodologies. On the morning of day two, participants reviewed the material covered on day one via unknown lesion identification sessions with feedback. Instruction on how to apply the Three-Point Checklist, AC Rule, and TADA algorithms occurred during a 30-minute training session immediately prior to the study.

Evaluation of Study Lesions

Dermoscopic images were displayed in PowerPoint® and projected onto two large screens. Participants used worksheets to evaluate the study lesions. The worksheets separately listed the dermoscopic criteria included in the three algorithms. For the Three-Point Checklist, the criteria evaluated were asymmetry (monoaxial or biaxial), atypical network, and blue-white color, with two of the three being required for biopsy. For the AC Rule, the criteria evaluated were asymmetry and color variation, which were ranked on a scale of 1 to 10. Based on the evaluation of these two criteria, users then determined if a lesion was suspicious for malignancy (yes or no) [22]. For TADA, participants were first asked to determine if a lesion was an unequivocal angioma, dermatofibroma, or seborrheic keratosis. If the lesion was determined to be one of these three, they were instructed to stop filling out the worksheet and wait for the next case. Otherwise, participants assessed the lesion for architectural disorder. Lesions demonstrating this feature were considered to be suspicious for malignancy without need for further evaluation for the remaining TADA criteria. Lesions lacking architectural disorder (i.e., organized, symmetric lesions) were evaluated further for the presence of starburst pattern, blue-black or gray color, shiny white structures, negative network, ulcer/erosion, or vessels, with the presence of any one feature indicating suspicion for malignancy. Lesions lacking all TADA criteria were considered equivocal and required monitoring for morphological changes or symptoms (i.e., itching, bleeding). Clinical images were not provided. However, information regarding textural features (i.e., firm, keratotic, smooth, dimpling) was given. The lesions were displayed in random order.

Statistical Analysis

Descriptive statistics were used to describe the study participants, study lesions, and participant evaluations. Three separate dichotomous outcome measures were created with the data to reflect the participants’ lesion evaluations for TADA, the Three-Point Checklist, and AC Rule. The primary independent variable for these analyses is the benign or malignant nature of the lesion based on histologic evaluation. Separate cross-classifications of the benign/malignant nature of a lesion by participant algorithm outcome were created and used to calculate overall estimates of diagnostic accuracy. Since study participants evaluated multiple study lesions, a general estimating equations approach was used to estimate model-based diagnostic accuracy measures while evaluating the effect of participant characteristics, such as previous dermoscopy training and/or years practicing dermatology. Separate models were independently used to estimate sensitivity and specificity. Algorithm performance comparisons of sensitivity, specificity and area under the receiver operating characteristic (ROC) curve were made. For the Three-Point Checklist and AC Rule, only participants’ responses for pigmented study lesions were recorded and used for statistical analysis. All statistical analyses were performed with Stata v14.1 (Stata Corporation, College Station, TX).

Results

Two hundred individuals attended the dermoscopy course and 120 (60%) participated in the study. Participant characteristics, including age, sex, medical specialty, previous dermoscopy training, and years of dermoscopy experience, are indicated in Table 1.

TABLE 1

Characteristics of study participants (n=120). [Copyright: ©2017 Rogers et al.]

Variable	Coding	n (%)
Age	<20	7 (5.8)
	21–30	28 (23.3)
	31–40	25 (20.8)
	41–50	37 (30.8)
	51–60	15 (12.5)
	61–70	4 (3.3)
	71–80	0 (0.0)
	>81	0 (0.0)
	Did Not Respond	4 (3.3)
Sex	Male	52 (43.3)
	Female	64 (53.3)
	Did Not Respond	4 (3.3)
Specialty	Dermatology	64 (53.3)
	Internal Medicine	22 (18.3)
	Family Medicine	19 (15.8)
	Emergency Medicine	2 (1.7)
	General Surgery	2 (1.7)
	Pathology	1 (0.8)
	Dentistry	1 (0.8)
	Instrumental Physics	1 (0.8)
	Medical Student	1 (0.8)
	Did Not Respond	7 (5.8)
Previous Dermoscopy Training	Yes	63 (52.5)
	No	52 (43.3)
	Did Not Respond	5 (4.2)
Years of Previous Dermoscopy Experience	0	28 (23.3)
	≤1	24 (20.0)
	2–5	35 (29.2)
	6–10	18 (15.0)
	>10	9 (7.5)
	Did Not Respond	6 (5.0)

In total, 5,646 lesion evaluations were performed, 3,036 malignant (53.8%) and 2,610 benign (46.2%), with a mean of 47 evaluations per participant (out of a possible 50). In the first step of TADA, 25.6% (n=1,443/5,646) lesion evaluations resulted in the diagnosis of angioma, dermatofibroma, or seborrheic keratosis (Figure 2). Of these lesions, 94% (n=1,357) were histologically benign and 90% (n=1,301) were one of the three, above-named, benign lesions. The diagnosis of angioma was made on 427 evaluations (29.6% of the 1,443 lesion evaluations in step one), 96% of which were correctly classified (n=411). Angioma had a false positive rate (malignant lesions erroneously identified as angioma) of 0.2% (n=13). Of the 458 dermatofibroma diagnoses made (31.7% of the 1,443 lesion evaluations in step one), 94% (n=431) were correct. The false positive rate for dermatofibroma was 1% (n=29). The diagnosis of seborrheic keratosis was made on 558 occasions (38.7% of the 1,443 lesion evaluations in step one). Of these, 82% (n=459) were correctly classified. Seborrheic keratosis had a false positive rate of 1.4% (n=44). Among the 3,036 evaluations of histologically malignant lesions, 2,950 were performed in the second step of TADA, denoting that 97% of malignant study lesions were correctly triaged in step one.

Figure 2

Breakdown of participants’ responses for TADA Step 1 and Step 2—Results of the 5,646 lesion evaluations performed by study participants as a function of the true diagnoses of study lesions. Benign lesions correctly identified in TADA step 1 are classified as “true benign” and are further broken down into either “correctly classified,” for lesions with true diagnoses of angioma, dermatofibroma, or seborrheic keratosis, or “incorrectly classified,” for all other benign lesions. Malignant study lesions incorrectly identified as benign in TADA step 1 are classified as “false benign.” Lesions identified as malignant in TADA step 2 (positive for any one criteria) are classified as either “true malignant,” for lesions with true malignant diagnoses, or “false malignant,” for lesions with true benign diagnoses. Lesions identified as not malignant, or equivocal, in TADA step 2 (negative for all criteria) are either classified as “true benign,” for lesions with true benign diagnoses, or “false benign,” for lesions with true malignant diagnoses.

Abbreviations: DF, dermatofibroma; SK, seborrheic keratosis; CCA, clear cell acanthoma; AMM, amelanotic melanoma; BCC, basal cell carcinoma; MM, malignant melanoma; NM, nodular melanoma; SCC, squamous cell carcinoma

[Copyright: ©2017 Rogers et al.]

Participants performed 4,203 lesion evaluations (74.4%) in step two of TADA, of which 3,590 (85.4%) were identified as suspicious for malignancy based on the presence of any single TADA criterion. Eighty percent of these lesions were true malignancies. The features that most strongly discriminated benign and malignant study lesions were disorganized architecture, ulcers/erosions, and shiny white structures. Disorganized architecture was identified in 57% of histologically malignant lesions versus 21% of benign lesions (p<0.001). Ulcers/erosions were identified in 29% versus 11% of malignant and benign lesions, respectively (p<0.001). Shiny white structures were identified in 25% versus 11% of malignant and benign lesions, respectively (p<0.001). Of the 613 lesion evaluations lacking any TADA criteria (14.6% of the 4,203 lesion evaluations in step two), 87% were truly benign.

TADA had a sensitivity of 94.6% and a specificity of 72.5% for all malignant study lesions. Sensitivity and specificity estimates for the individual study lesions, as well as for participants with and without previous dermoscopy training, are listed in Table 2. In order to compare the diagnostic performance of TADA to the Three-Point Checklist and AC Rule, nonpigmented study lesions (n=10) were excluded from the analysis. In this evaluation, TADA performed with the highest sensitivity (94.0%, 95% CI: 92.9%–95.0%), followed by the AC Rule (88.6%, 95% CI: 87.1%–89.9%) and the Three-Point Checklist (71.9%, 95% CI: 69.8%–73.9%). The Three-Point Checklist had the highest specificity (81.4% (95% CI: 79.7%–83.0%), followed by the AC Rule (78.7%, 95% CI: 76.9%–80.3%) and TADA (75.5%, 95% CI: 73.8%–77.2%). ROC curves for the three algorithms highlight these results (Figure 3).

TABLE 2

Model-based estimates of sensitivity and specificity for TADA for all study lesions. [Copyright: ©2017 Rogers et al.]

Variable	Sensitivity			Variable	Specificity
	Coding	Estimate (95% CI)	P-value		Coding	Estimate (95% CI)	P-value
Overall		94.6 (93.4—95.7)	—	Overall		72.5 (70.1—74.7)	—
Diagnosis	AMM	95.6 (91.5—99.8)	0.942	Diagnosis	Angioma	76.4 (72.6—80.4)	<0.001
	BCC	95.2 (92.9—97.6)	0.651		CCA	39.1 (37.0—41.4)	<0.001
	MM	94.4 (92.3—96.6)	0.251		DF	93.6 (90.0—98.1)	<0.001
	NM	91.6 (88.2—95.1)	0.020		Nevus	69.4 (67.0—71.9)	—
	SCC	95.7 (93.8—97.7)	—		SK	82.9 (79.2—86.7)	<0.001
Previous Dermoscopy Training	No	93.6 (91.8—95.5)	—	Previous Dermoscopy Training	No	69.0 (64.6—73.7)	—
	Yes	95.4 (94.8—99.6)	0.14		Yes	73.2 (71.4—84.5)	0.450

Abbreviations: AMM, amelanotic melanoma; BCC, basal cell carcinoma; MM, malignant melanoma; NM, nodular melanoma; SCC, squamous cell carcinoma; CCA, clear cell acanthoma; DF, dermatofibroma; SK, seborrheic keratosis

Figure 3

Receiver operating characteristic (ROC) curves for TADA, the Three-Point Checklist, and AC Rule—ROC curves demonstrate the diagnostic performance of the three algorithms compared for the identification of pigmented skin lesions. [Copyright: ©2017 Rogers et al.]

Discussion

In this pilot study, we tested a novel triage algorithm to determine its sensitivity and specificity for common skin cancers. A significant proportion of study participants lacked previous dermoscopy training (43%, n=52) and/or experience (23%, n=28). This allowed us to evaluate the potential utility of TADA as a skin cancer detection aid for inexperienced dermoscopists, who comprise our target audience. Our study population had a greater sample of beginners than the pilot studies for the Three-Point Checklist and AC Rule, which included 6 inexperienced dermoscopists and 17 laypersons, respectively [11,12]. Our study population also compares favorably to the participant profile in a study that reevaluated the Three-Point Checklist, of which 24 individuals (14%) lacked previous dermoscopy experience. 9 The Three-Point Checklist was also evaluated in a prospective trial with 73 primary care physicians; however, the previous dermoscopic experience or training of participants was not indicated [23]. A limitation of our study is that non-participant characteristics were not recorded and we are unable to report on any differences between participants (n=120) and non-participants (n=80).

The overall sensitivity of TADA for pigmented and nonpigmented skin cancers was 94.6%. This value was marginally influenced by participants’ previous dermoscopy training (95.4% vs. 93.6%). The first criterion included in TADA is architectural disorder, which is not an objective criterion in that it cannot be defined by any given shape or color. It is rather the result of the overall impression, or gestalt, of an asymmetric or chaotic lesion. The subjective interpretation of disorganization within a lesion has been shown to have better interobserver agreement than most objectively defined criteria [13]. It has also been shown to be one of the dermoscopic criteria with the highest discriminatory power [13-15]. Indeed in the present study, architectural disorder allowed for the correct identification of greater than 50% of malignant study lesions. In order to identify malignancies with ordered and symmetric appearances, participants needed to be able to recognize six additional features, three of which (blue-black or gray color, ulcer/erosion, and vessels of any morphology) are colors and structures not specific to dermoscopy and, in our experience, beginners have been able to quickly recognize. Facial, acral, nail, and mucosal lesions were not evaluated and the algorithm states that TADA cannot be used for lesions on these sites. While our results for non-melanoma skin cancer can likely be generalized to facial lesions, more robust studies across multiple ages and skin color cohorts are needed to validate the dermoscopic features of early special sites melanomas.

Notably, untrained participants achieved an overall specificity of 69% using TADA. Additionally, the specificities for the three types of benign lesions included in the algorithm ranged from 76% to 94%. This finding substantiates our view that beginners can be quickly trained to accurately identify classic examples of certain benign lesions. In many instances, these benign neoplasms can have dermoscopic characteristics attributable to malignant lesions, such as the blue, black, or gray colors commonly observed in seborrheic keratoses or the shiny white structures or scar-like areas seen in dermatofibromas [24,25]. However, when these features are viewed in the context of the global lesion pattern as a whole, the diagnosis can become apparent. Additionally, the frequency with which these lesions are encountered in clinical practice can allow one to rapidly gain experience in their identification. While requiring users to gain additional dermoscopic knowledge in order to identify these lesions is arguably a limitation of TADA, it also seemed to strengthen the algorithm, as indicated by the high specificities achieved for these lesions.

Pre-selection of lesions is not something unique to TADA. The Three-Point Checklist has reported sensitivities and specificities ranging from 91.0% to 96.3% and 32.8% to 71.9%, respectively, for the identification of pigmented melanoma and basal cell carcinoma [9,11]. The AC Rule has reported sensitivities and specificities of 94% and 62%, respectively, for pigmented melanoma. 12 Two of the three criteria used in the Three-Point Checklist are for pigmented lesions and at least two of these criteria must be present for a lesion to warrant a biopsy. This greatly decreases the likelihood that the algorithm will identify nonpigmented malignancies. Regarding the AC Rule, the final determinant of whether or not a lesion requires a biopsy is the user’s level of suspicion, which reflects the presence of one or both of the algorithms criteria. One of these criteria, asymmetry, is not specific to pigmented lesions. The AC Rule thus might be applicable for nonpigmented lesions, however, only those with disorganized architecture. Similarly, the algorithm might miss organized, homogenously pigmented skin cancers, such as some nodular melanomas.

When comparing the results of TADA to that of the Three-Point Checklist and AC Rule for the detection of pigmented lesions, we found that TADA performed with the highest sensitivity by as much as 22%. The fact that the sensitivity estimates for TADA were the same on the entire data set and on the subset of pigmented lesions (94.6% vs. 94.0%) might reflect the inclusion of sufficient dermoscopic criteria for the identification of both pigmented and nonpigmented skin cancers. Regarding specificity, we found that all three algorithms performed well. The specificity seen with TADA was slightly lower than for the other two algorithms. Certain features included in TADA, such as vessels of any morphology, will invariably lead to biopsies of some benign lesions, like clear cell acanthomas or some intradermal nevi. TADA knowingly sacrifices this specificity for simplicity and sensitivity. Additional training in the identification of certain benign lesions would likely increase TADAs specificity.

The results of this study suggest that TADA might be a useful triage tool by providing a simplified dermoscopic method with high sensitivitity for common skin cancers. However, our study population consisted of individuals attending a dermoscopy course and were thus a motivated group with an interest in learning dermoscopy. As such, our results might not be generalizable to all novices. To make definitive statements about the efficacy of TADA in clinical practice, our findings would need confirmation with a greater number of inexperienced dermoscopists evaluating a larger and more diverse sample of neoplasms with a more balanced proportion of pigmented and nonpigmented lesions. This would ideally be achieved with a prospective study. Since clinical-dermoscopic correlation can be crucial for certain diagnoses, the addition of gross images would more closely correlate to the clincal scenerio. Our use of clinical descriptives (i.e., firm, keratotic) could have introduced an information bias not reproducible in clinical settings, however, this limitiation would have more relevance to a study evaluating teladermoscopy. Further, in the present study, all participants received standardized training. Randomizing participants to various levels and durations of training would allow us to determine if the teaching modality for TADA can be streamlined. An upcoming study will address the latter limitation and also determine if the inclusion of step one (identifying common benign lesions) truly strengthens the algorithm.