Clinical performance of the Nevisense system in cutaneous melanoma detection: an international, multicentre, prospective and blinded clinical trial on efficacy and safety.

BACKGROUND: Even though progress has been made, the detection of melanoma still poses a challenge. In light of this situation, the Nevisense electrical impedance spectroscopy (EIS) system (SciBase AB, Stockholm, Sweden) was designed and shown to have the potential to be used as an adjunct diagnostic tool for melanoma detection.
OBJECTIVES: To assess the effectiveness and safety of the Nevisense system in the distinction of benign lesions of the skin from melanoma with electrical impedance spectroscopy.
METHODS: This multicentre, prospective, and blinded clinical study was conducted at five American and 17 European investigational sites. All eligible skin lesions in the study were examined with the EIS-based Nevisense system, photographed, removed by excisional biopsy and subjected to histopathological evaluation. A postprocedure clinical follow-up was conducted at 7 ± 3 days from the initial measurement. A total of 1951 patients with 2416 lesions were enrolled into the study; 1943 lesions were eligible and evaluable for the primary efficacy end point, including 265 melanomas - 112 in situ and 153 invasive melanomas with a median Breslow thickness of 0·57 mm [48 basal cell carcinomas (BCCs) and seven squamous cell carcinomas (SCCs)].
RESULTS: The observed sensitivity of Nevisense was 96·6% (256 of 265 melanomas) with an exact one-sided 95% lower confidence bound estimated at 94·2% and an observed specificity of 34·4%, and an exact two-sided 95% confidence bound estimated at 32·0-36·9%. The positive and negative predictive values of Nevisense were 21·1% and 98·2%, respectively. The observed sensitivity for nonmelanoma skin cancer was 100% (55 of 48 BCCs and seven SCCs) with an exact two-sided 95% confidence bound estimated at 93·5-100·0%.
CONCLUSIONS: Nevisense is an accurate and safe device to support clinicians in the detection of cutaneous melanoma.

Early detection of melanoma is vital for treatment outcome and survival. Treatment of early-stage melanoma is mostly curative, whereas thicker melanomas are associated with a poor 5-year survival rate due to increased metastatic potential.1,2 In most instances, physicians feel fairly confident when distinguishing nonsuspicious from suspicious lesions by relying on unaided eye examination, dermoscopy assessment and patient history. However, cutaneous melanomas can be misdiagnosed as benign, and a significant proportion of benign lesions are unnecessarily excised. The sensitivity of the clinical diagnosis of cutaneous melanoma with unaided eye examination is around 60% and can improve significantly with the use of dermoscopy.3–37 Although progress has been made in the detection of melanoma it still poses a challenge, especially in its earlier stages. Therefore, a number of technologies utilizing either visual or nonvisual techniques, such as total body photography,38,39 confocal microscopy,40,41 Raman spectroscopy,42,43 multispectral imaging,44 automated dermoscopy image analysis,45 genomic detection of melanoma by stratum corneum stripping46 and electrical impedance spectroscopy (EIS),47,48 have been developed to support physicians in detecting melanomas at an earlier stage. In a previous study, the EIS-based Nevisense system (SciBase AB, Stockholm, Sweden) was shown to have the potential to be used as an adjunct diagnostic tool, although it was concluded that more clinical data were necessary to ensure the safety and effectiveness of the system.48

The aim of this clinical investigation was to assess the safety and effectiveness of the Nevisense system, which has been designed to aid in the discrimination between benign lesions and primary cutaneous melanoma. In this article, results are presented and discussed in the context of clinical utility.

Materials and methods

Ethical conduct

The guidelines of the revised Declaration of Helsinki, the Guidelines of Good Clinical Practice (ISO-14155), and the demands of national and data protection laws and other applicable regulatory requirements were followed. The clinical trial registration number is NCT01077050 (www.ClinicalTrials.gov).

Study design and data acquisition

Recruitment into this blinded multicentre prospective study was conducted at five American and 17 European investigational sites (Sweden, Germany, Austria, Hungary, U.K. and Spain). Potential study participants were screened according to the inclusion and exclusion criteria. Subsequent to written informed consent, patients were asked about their medical history and a clinical evaluation was performed. A photograph and dermoscopic image of each included lesion was taken before and after Nevisense measurements to document evaluation according to the protocol. In accordance with standard clinical practice, eligible and evaluable lesions were excised and subjected to the investigational site's histopathology evaluation and managed accordingly.

A further histopathological evaluation was completed by a panel of three experienced histopathologists who evaluated each lesion independently and were blinded from the investigational site's original histopathology diagnosis. In the case of agreement among the experts, the diagnosis was considered as the study's histopathological gold standard (HGS). If there was significant disagreement among the pathology reviewers on whether the lesion represented a malignancy, the respective slides were submitted to two additional experts whose diagnosis was then chosen as the HGS if they reached agreement. In case of disagreement by the two additional reviewers, the corresponding lesion was excluded from the efficacy analysis.

A postprocedure follow-up either by a telephone call or at a participant's visit to the investigational site was conducted at 7 ± 3 days after the Nevisense evaluation, at which time the patient was evaluated for any adverse events.

Inclusion and exclusion criteria

Patients with skin lesions selected for total excision to rule out melanoma were asked to participate in the study. To minimize selection bias, all lesions destined for excision were eligible for inclusion in the study. To ensure a broad spectrum of excised lesions, dermatologists were encouraged to enroll a mix of lesions with an even distribution of low-, medium- and high-risk lesions. The exclusion criteria were derived from previous studies conducted with the investigational device and are listed in Table1.47,48

Table 1

Exclusion criteria

Men or women of any ethnic group aged < 18 years

Patient not willing or able to read, understand and sign the study-specific informed consent form

Metastases or recurrent lesions

Lesion < 2 mm or > 20 mm in diameter

Lesion located on acral skin, e.g. sole or palm

Lesion located on areas of scars, crusts, psoriasis, eczema or similar skin conditions

Lesion on hair-covered areas, e.g. scalp, beards, moustaches or whiskers

Lesion located on genitalia

Lesion located in an area that has been previously biopsied or subjected to any kind of surgical intervention or trauma

Lesion located on mucosal surfaces

Lesion with foreign matter, e.g. tattoo or splinter

Lesion and/or reference located on acute sunburn

Skin surface not measurable, e.g. lesion on a stalk

Skin surface not accessible, e.g. inside ears, under nails

Skin not intact (measurement area), e.g. bleeding or with clinical noticeable ulceration

Review of images

The photographs and dermoscopic images were taken with a Sony DSC-W290 (Sony, Tokyo, Japan) and a hand-held dermoscope (DermLite II PRO HR®, 3Gen; DermLite, San Juan Capistrano, CA, U.S.A.). The images were reviewed by three dermatologists with 2–5 years of experience in dermoscopy assessment. The option to reach out to additional experienced dermoscopists in difficult cases was allowed. Lesions were classified according to the clinical ABCD rule,49,50 the dermoscopic ABCD rule,51 the seven-point checklist52 and the overall suspicion of malignancy classified by the visual classification board from 0 (benign) to 10 (malignant). This was conducted to ascertain a standardized clinical and dermoscopic classification of the degree of suspicion of malignancy of each study lesion.

Blinding

The case report forms, the Nevisense measurements and the histopathological reports were kept blinded from the sponsor by a contract research organization until classification of all lesions in the pivotal study had been made.

The investigators were blinded to the entire diagnostic information of the device to ensure that the device could not bias the clinical judgement nor affect the clinical management of the patient in any way.

Electrical impedance spectroscopy measurements

Electrical impedance was measured with the Nevisense system equipped with a spring-loaded probe and a disposable electrode having an active area of approximately 5 × 5 mm2. Prior to measurement, the skin was moistened for 30 s with physiological saline solution, after which a reference measurement of healthy skin close to the lesion was performed. The procedure was then repeated on the lesion under study. The system measures the overall electrical resistance and reactance at 35 different frequencies logarithmically distributed between 1·0 kHz and 2·5 MHz at four depth settings with a total of 10 permutations. The applied voltage and resulting current is limited to 150 mV and 75 μA, respectively, and is not sensed by the patient. Measurements take approximately 8 s, and within seconds the system computes both a score (0–10) and a dichotomous output (EIS negative/positive) at a fixed cut-off. The fixed threshold is set at 4, i.e. scores < 4 are EIS negative and scores of ≥ 4 are EIS positive. The dichotomous output was used in the study to demonstrate the sensitivity and specificity end points.

Study objective and end point

The objective of this clinical investigation was to determine the safety and effectiveness of the Nevisense device, which has been designed to help distinguish between cutaneous melanoma and benign lesions of the skin, using EIS relative to the HGS.

This study had two coprimary analyses, aiming to demonstrate the accuracy of the Nevisense device: (i) a one-sided exact 95% confidence bound of the sensitivity in detecting cutaneous melanoma of > 90% (sensitivity ≥ 0·90 to detect melanoma); (ii) nonrandom result at the given sensitivity, i.e. sensitivity + specificity > 1·0.

The safety of the Nevisense analysis was measured by the occurrence and incidence of all adverse events reported for study participants throughout their participation in the study. The primary safety end point was achieved if no serious adverse events related to the device had occurred.

Additional analysis

Clinical histopathological gold standard

To estimate the sensitivity and specificity of the clinical HGS used for treatment the investigational site's histopathological diagnoses were compared with the study HGS. The analysis was conducted on eligible and evaluable lesions.

Unaided lesion evaluation and dermoscopy assessment

The observed sensitivity and specificity of the visual reference standard was calculated using the cut-offs prespecified from literature and the outcome of the different visual classification algorithms.

Results

Study participants and skin lesions

A total of 1951 participants with 2416 lesions were recruited between March 2010 and November 2011. The demographic characteristics of the study population are presented in Table2. The median age of the patients was 48 years (range 18–91). Women comprised 51·9% of the study group. Most (97·1%) patients were white and the majority were either of Fitzpatrick skin type 2 or 3. No significant differences in the demographic characteristics between the enrolled and the eligible lesions were observed.

Table 2

Demographic characteristics of the enrolled and eligible participants

Characteristics	Patients enrolled (n = 1950)	Patients with eligible lesions (n = 1611)
Sex
Male	929 (47·6)	765 (47·5)
Female	1013 (51·9)	846 (52·5)
Missing	8 (0·4)	0 (0)
Age (years), median (range)	48 (18–91)	48 (18–91)
Race and ethnicity
Asian	5 (0·3)	5 (0·3)
White	1893 (97·1)	1571 (97·5)
Black or African American	2 (0·1)	2 (0·1)
Hispanic or Latino	29 (1·5)	25 (1·6)
Other	12 (0·6)	8 (0·5)
Missing	9 (0·5)	0 (0)
Fitzpatrick skin type
1. Always burns easily; never tans	136 (7)	117 (7·3)
2. Always burns easily; tans minimally	945 (48·5)	783 (48·6)
3. Burns moderately; tans gradually	635 (32·6)	526 (32·7)
4. Burns minimally; always tans well	192 (9·8)	158 (9·8)
5. Rarely burns; tans profusely	29 (1·5)	23 (1·4)
6. Never burns; deeply pigmented	1 (0·1)	1 (0·1)
Missing	12 (0·6)	3 (0·2)

Data presented as number (percentage) of patients, except for age. For one subject, the signed informed consent form could not be located at the site and the data were thus deleted.

Table3 presents the distribution of reasons for exclusion from the effectiveness analysis. Out of the 2416 registered lesions a total of 473 were excluded, mainly owing to investigator oversight or the inability to render a final histopathological diagnosis. Approximately one-quarter of the excluded lesions were device-related (inadequate reference measurement quality or general device failures).

Table 3

Reasons for exclusion of lesions from the analysis

Reason for exclusion	No. of lesions		Source
	n	%
Lesions included	2416
Signed informed consent form missing	1	< 0·1	Investigator: 11·0%
Withdrawal	17	0·7
Not eligible (i.e. inclusion/exclusion)	61	2·5
Major protocol violation	29	1·2
Measurement not acquired	60	2·5
Coveragea	98	4·1
Ineligible histopathology (preparation quality)	8	0·3	Pathology: 4·1%
Missing histopathologyb	39	1·6
Inaccurate mapping of histopathologyc	7	0·3
No consensusd	44	1·8
Poor reference qualitye	95	3·9	Device-related: 4·5%
Device failure	14	0·6
Eligible lesions	1943

Less than 75% of the lesion was covered with measurements, e.g. a 10 × 10-mm2 lesion only measured once was excluded.

No histology slides were/could be provided by the site.

Provided histology slides were not mapped accurately to the lesion measured.

The consensus board could not agree on a final diagnosis.

Inability to obtain a reference measurement with adequate quality after four consecutive attempts.

Performance of Nevisense

The dichotomous outcome of the Nevisense system was compared with the HGS. Of the 1943 eligible and evaluable lesions (Table4), 265 (13·2%) were cutaneous melanoma, 55 (2·8%) were nonmelanoma skin cancer (NMSC), including basal cell carcinomas (BCCs) and squamous cell carcinomas (SCCs), of which Nevisense correctly identified 256 melanomas and all 55 NMSCs, yielding an observed sensitivity of 96·6% and 100·0%, respectively. A total of 157 naevi with severe dysplasia were included, of which Nevisense gave a positive reading for malignancy in 132 cases. Seven out of eight actinic keratoses gave a positive reading. One Merkel cell carcinoma was included, which was correctly identified as malignant. Out of the remaining 1457 lesions, 501 were diagnosed as negative, yielding an observed specificity of 34·4%. No significant difference in the presented sensitivity and specificity was encountered, when the possible dependency in outcome between the lesions of the same patient was accounted for through a generalized linear mixed model. The positive predictive value (PPV) of Nevisense was 21·1% and the negative predictive value (NPV) was 98·2%. The Nevisense score was compared with lesion severity and, as can be discerned from Figure1, a clear step function is evident for the score outcome with increasing lesion severity.

Table 4

Observed sensitivity (Sens) and specificity (Spec) for Nevisense combined with lower (LCB) and upper 95% confidence bounds (UCB), and true and false positives/negatives in the efficacy analysis population differentiated by histopathological lesion type and melanoma thickness

Type	Sens	TPa	FNa	Total	LCBb	UCBb
Melanomac	96·6	256	9	265	93·7d	98·4
pTis	93·8	105	7	112	87·6	97·5
pT1a	97·9	92	2	94	92·5	99·7
pT1b	100	19	0	19	82·4	100
pT2a	100	24	0	24	85·8	100
pT2b	100	11	0	11	71·5	100
pT3a	100	1	0	1	2·5	100
pT3b	100	3	0	3	29·2	100
pT4a	100	1	0	1	2·5	100
pT4b	NA	0	0	0	NA	NA
Severe dysplasiae	84·1	132	25	157	77·4	89·4

TP, true positive; TN, true negative; FP, false positive; FN, false negative; NA, not applicable; NMSC, nonmelanoma skin cancer; BCC, basal cell carcinoma; SCC, squamous cell carcinoma.

TP correctly identified as positive, FN incorrectly identified as negative, TN correctly identified as negative and FP incorrectly identified as positive by the investigational device.

Exact LCB and UCB calculated using the Clopper–Pearson method.

American Joint Committee on Cancer staging system for melanoma sixth edition was used with ad hoc adoption of the seventh edition during the course of the study when pronounced mitosis was present.2,58

Exact Clopper–Pearson and mid-P one-sided 95% LCB estimated at 94·2% and 94·4%, respectively.

Severe cytological atypia or architectural disorder where diagnosed as severe dysplasia.

No majority board agreement on degree of dysplasia.

Fig 1

The study's histopathological gold standard plotted against the Nevisense score outcome. EIS, electrical impedance spectroscopy.

Nevisense false negatives

Of the nine melanomas classified by Nevisense as false negative, seven had sufficient image quality to render an outcome for the ABCD dermoscopy rule and seven-point checklist by the reviewing dermatologists. The overall visual malignancy grading scored two of this subset of seven melanomas as positive with a score of 4, and five as negative with a score of 0–2.

Three of the nine FN lesions were identified on patients aged < 30 years. The median diameter of these nine lesions was 4 mm (range 2–8), of which five had a diameter < 6 mm. Seven were in situ and two were early invasive melanomas (T1a) with Breslow thicknesses of 0·4 mm and 0·6 mm, respectively. The lesions were located on the following anatomical locations: lower extremities (n = 4), upper extremities (n = 1), upper back (n = 2), buttocks (n = 1) and facial area (n = 1).

Performance of the unaided examination and dermoscopy assessment

To determine the diagnostic uncertainty of the study lesions, a postexcisional performance study of unaided examination and dermoscopy was completed. Of the 1943 eligible and evaluable lesions, 1701 (238 melanomas) had sufficient image quality to enable a classification according to the ABCD rule, ABCD dermoscopy rule and the seven-point checklist.

The result of the dermoscopic and investigational site's histopathological evaluations in conjunction with the Nevisense are presented in Table5, and grouped according to the study's HGS.

Table 5

Observed sensitivity and specificity for the dermoscopic, investigational site's histopathology evaluations, as well as the Nevisense result grouped according to the study's histopathological gold standard (HGS). The results are derived from the cohort of eligible and evaluable lesions that had sufficient image quality to render a dermoscopic evaluation

Type	Seven-point	Seven-point weighted	ABCD dermoscopy < 5·45	ABCD dermoscopy < 4·75	Malignancy gradinga	Investigational site's histopathology	Nevisense
Melanoma sensitivity	49·2	60·9	47·1	54·2	70·6	84·5	97·1
pTis	28·7	43·6	28·7	37·6	55·4	73·3	94·1
pT1a	57·1	65·5	51·2	57·1	75·0	89·3	98·8
pT1b	76·5	76·5	76·5	82·4	88·2	100·0	100·0
pT2a	63·6	86·4	72·7	72·7	90·9	95·5	100·0
pT2b	88·9	100·0	88·9	88·9	100·0	100·0	100·0
pT3a	100·0	100·0	100·0	100·0	100·0	100·0	100·0
pT3b	100·0	100·0	66·7	100·0	100·0	100·0	100·0
pT4a	100·0	100·0	0	100·0	100·0	100·0	100·0
Severe dysplasia sensitivityb	12·1	24·8	12·8	20·8	38·3	NA	83·9
Overall specificity	94·2	89·2	94·0	90·1	81·4	98·0	35·8

All values given as a percentage. NA, not applicable.

Overall malignancy grading as determined by the visual classification board with a fixed cut-off at 4.

Severe cytologic atypia or architectural disorder where diagnosed as severe dysplasia.

The observed sensitivity with the ABCD dermoscopy rules for melanoma detection with a cut-off of 4·75 and 5·45 was 54·2% and 47·1%, with an observed specificity of 90·1% and 94%, respectively.

The observed sensitivity for melanoma detection with the original and weighted seven-point checklists was 49·2% and 60·9% with an observed specificity of 94·2% and 89·2%, respectively.

The observed sensitivity for the reviewing dermatologist, when considering the combined features for malignancy (overall visual board malignancy grading), was 70·6%, with an observed specificity of 81·4%.

Performance of the investigational site's histopathologists

The observed sensitivity and specificity for melanoma of the investigational site's histopathologists was 84·9% (225/265) and 98·1% (1429/1457), respectively. The observed sensitivity increased with melanoma thickness, from 73·2% for Tis to 100% for T2b–T4.

Safety

A total of 36 adverse events (AEs) were observed in 28 patients (1·5%), out of which only three AEs that occurred on three patients (0·2%) were defined as definitely related to the device. All AEs related to the device were of mild severity. No serious AEs, serious adverse device effects or unanticipated adverse device effects were observed throughout the entire study.

Discussion

Melanoma detection often poses a challenge in equivocal lesions or in patients with many atypical naevi. Therefore, a component of the clinical work-up and interpretation incorporates not only lesion-specific information, but also patient-derived melanoma risk factors, which includes a comparative analysis of all lesions present on a patient.53–56 Even if the clinical decision is based on a collective interpretation of all the presently available clinical risk factors, melanomas can still be misdiagnosed as benign lesions.37 As early detection of melanoma is vital for treatment outcome and survival,1,2 additional objective information that could assist in the early detection of melanoma could significantly reduce the morbidity related to the unnecessary removal of benign lesions and has the potential to improve mortality through early diagnosis.

In this study, a postexcision evaluation of the isolated performance of dermoscopy was conducted according to three dermoscopic classification algorithms in which the evaluator was presented with the clinical and dermoscopic images of all excised lesions from a patient. The results show that a considerable amount of the included melanomas were equivocal lesions with insufficient dermoscopic criteria to be classified as malignant.

There are several plausible explanations for the relatively low sensitivity (47·1–70·6%) observed in the postexcisional dermoscopic evaluation with limited clinical data. Firstly, the evaluation was conducted on a cohort of 85·0% in situ and early invasive melanomas. In addition, the evaluations were completed by dermatologists with 2–5 years of experience in dermoscopy, with additional support in difficult cases. Most importantly, the evaluators did not have the added benefit of comparative analysis of all lesions present on the patients, i.e. they could only base their judgement on the excised lesions.

While the outcome of the dermoscopy assessment might have been different if the lesions had been evaluated by more experienced dermoscopists, the results of the study likely reflect the dermoscopic acumen of an average dermatologist. It has to be stressed that almost all lesions were removed owing to some clinical concern for melanoma, as it would not be ethical to excise lesions deemed benign clinically, except for functional or cosmetic reasons. As mostly preselected equivocal lesions destined for excision were included in the study, the dermoscopic diagnostic performance is not a true reflection of the performance in normal clinical practice. However, it gives an insight into the isolated performance of dermoscopy on equivocal lesions submitted for biopsy, which is the intended use of Nevisense.

Nevisense was able to achieve a high sensitivity (96·6%) in a melanoma cohort consisting mostly of in situ and early invasive melanoma without fully compromising the specificity (34·4%). The observed sensitivity of the device increased with Breslow thickness, and no invasive melanoma at stages T1b–T4 were missed by the device. Cases where the investigational device gave a false negative reading occurred mostly in small lesions with few or no dermoscopic features, and with low cellularity.

Two early-stage invasive melanomas were inaccurately classified as negative, but in both cases compliance with the measurement procedure could not be fully verified. In fact, for one lesion the verification data were missing, and the other lesion had not been fully covered with measurements. As the system only detects changes that occur directly underneath the electrode, it is important that the lesions are measured completely and/or that the most suspicious malignant part of the lesion is measured to ensure as accurate a reading as possible. A review of the 98 lesions excluded owing to coverage issues included a total of 22 melanomas, of which the device still accurately classified 20 as positive. These are good results, as for some cases only 25·0% of the surface had been measured with the device, suggesting that even though coverage is important, it is not vital in most cases.

The overall observed specificity was 34·4%. Approximately one-third of the equivocal lesions submitted for biopsy in the study were accurately identified as negative by the device and thus would not have needed a biopsy. Furthermore, the high observed NPV of 98·2% – equal to that of the observed NPV of histology in the study – ensures that few melanomas will be inaccurately left untreated when given a low score by the device.

There are two additional results that are important to highlight. Firstly, a high proportion of seborrhoeic keratoses were inaccurately classified as positive. The high ratio of positive readings was anticipated as this has been observed in previous studies, mainly owing to the typically high degree of structural changes compared with normal skin. However, the system is intended to be used by dermatologists or clinicians trained in the diagnosis of skin cancer who will almost always recognize seborrhoeic keratoses clinically and would therefore seldom apply Nevisense to these lesions. Secondly, the observed sensitivity of 100·0% in NMSC is extremely valuable as these malignancies should not be left untreated. Few other noninvasive technologies are able to identify accurately NMSC, as well as melanoma.

The observed sensitivity and specificity presented do not take into account the clinical information regarding the full extent of the patient's history, as well as the comparative analysis with other lesions, which has been shown to be critical in the clinical decision of whether to excise a lesion or not. However, in reality, clinicians often end up performing single lesion examinations owing to factors such as time constraints; as such, the isolated performance of the dermoscopy evaluation would reflect their diagnostic accuracy.57 As can be discerned from Table5, a large number of melanomas included in the study were equivocal lesions and hard to differentiate from nonmalignant equivocal lesions.

The Nevisense system is intended for use on cutaneous lesions with one or more clinical or historical characteristics of melanoma. The system is designed to be used when a clinician chooses to obtain additional information when considering excision and is not meant to be used to confirm a clinical diagnosis of melanoma. It should be used by physicians trained in the clinical diagnosis of skin cancer to ensure the system result is one element of the overall clinical assessment. Not only should the negative or positive EIS outcome be incorporated into the assessment, but also the EIS score, as it is coupled with the stage and severity of a lesion.

In summary, Nevisense has been shown to be an accurate and safe device that should be used in conjunction with the clinical risk assessment for patients with suspicion of melanoma in the intended use population. A negative or positive EIS reading in combination with the score outcome should be used as guidance for whether a lesion should be excised or not.