Effect of adding a diagnostic aid to best practice to manage suspicious pigmented lesions in primary care: randomised controlled trial.

OBJECTIVES: To assess whether adding a novel computerised diagnostic tool, the MoleMate system (SIAscopy with primary care scoring algorithm), to current best practice results in more appropriate referrals of suspicious pigmented lesions to secondary care, and to assess its impact on clinicians and patients.
DESIGN: Randomised controlled trial.
SETTING: 15 general practices in eastern England. PARTICIPANTS: 1297 adults with pigmented skin lesions not immediately diagnosed as benign.
INTERVENTIONS: Patients were assessed by trained primary care clinicians using best practice (clinical history, naked eye examination, seven point checklist) either alone (control group) or with the MoleMate system (intervention group). MAIN OUTCOME MEASURES: Appropriateness of referral, defined as the proportion of referred lesions that were biopsied or monitored. Secondary outcomes related to the clinicians (diagnostic performance, confidence, learning effects) and patients (satisfaction, anxiety). Economic evaluation, diagnostic performance of the seven point checklist, and five year follow-up of melanoma incidence were also secondary outcomes and will be reported later.
RESULTS: 1297 participants with 1580 lesions were randomised: 643 participants with 788 lesions to the intervention group and 654 participants with 792 lesions to the control group. The appropriateness of referral did not differ significantly between the intervention or control groups: 56.8% (130/229) v 64.5% (111/172); difference -8.1% (95% confidence interval -18.0% to 1.8%). The proportion of benign lesions appropriately managed in primary care did not differ (intervention 99.6% v control 99.2%, P=0.46), neither did the percentage agreement with an expert decision to biopsy or monitor (intervention 98.5% v control 95.7%, P=0.26). The percentage agreement with expert assessment that the lesion was benign was significantly lower with MoleMate (intervention 84.4% v control 90.6%, P<0.001), and a higher proportion of lesions were referred (intervention 29.8% v control 22.4%, P=0.001). Thirty six histologically confirmed melanomas were diagnosed: 18/18 were appropriately referred in the intervention group and 17/18 in the control group. Clinicians in both groups were confident, and there was no evidence of learning effects, and therefore contamination, between groups. Patients in the intervention group ranked their consultations higher for thoroughness and reassuring care, although anxiety scores were similar between the groups.
CONCLUSIONS: We found no evidence that the MoleMate system improved appropriateness of referral. The systematic application of best practice guidelines alone was more accurate than the MoleMate system, and both performed better than reports of current practice. Therefore the systematic application of best practice guidelines (including the seven point checklist) should be the paradigm for management of suspicious skin lesions in primary care. TRIAL REGISTRATION: Current Controlled Trials ISRCTN79932379.

Introduction

Differentiating melanomas from other pigmented skin lesions in primary care is challenging.1 Worldwide the incidence of melanoma is increasing faster than any other cancer, with an approximate doubling of rates every 10-20 years in countries with white populations.2 In the United Kingdom the incidence of melanoma has quadrupled over the past 40 years; data from Cancer Research UK for 2008 reported 11 770 new cases and 2070 deaths. Early detection is critical in reducing mortality and morbidity from melanoma, as stage 1 disease has five year survival rates of over 95% compared with 10-20% for stage 4 disease.3 As pigmented lesions are commonly presented in primary care consultations, general practitioners need to be able to reassure people with benign lesions and rapidly refer those with suspicious lesions. In 2003 the UK All-Party Parliamentary Group on Skin reported that 95% of lesions referred to a UK specialist were benign4; furthermore, a recent UK study showed that general practitioners recognised only 66.7% of skin malignancies.5 This difficulty in distinguishing benign from potentially malignant lesions is consistent with international evidence that general practitioners can be as sensitive but less specific than dermatologists at diagnosing melanoma.6 The appropriate referral of patients to secondary care has important clinical, safety, quality, and economic ramifications, not only for general practitioners working in the UK’s “gatekeeper” system but also globally.7 Therefore, novel interventions to improve the accuracy of identification of suspicious pigmented lesions have a potentially important role, especially in primary care, where such management initially occurs.

Interventions to improve general practitioners’ diagnostic performance and efficiency of referral have included the use of checklists and educational and technical approaches. The seven point checklist8 has been widely evaluated and revised,9 and although it has never been tested in a primary care trial, was recommended for use by all primary care professionals in the assessment of pigmented skin lesions by the 2005 English National Institute for Health and Clinical Excellence (NICE) guidelines on referral for suspected cancer.10 Evidence about the usefulness of brief educational approaches such as face to face11 and internet12 training courses is equivocal. Technical approaches have included the use of dermoscopy13 and digital monitoring; a recent Australian study in primary care found that the combination of these techniques could increase general practitioners’ sensitivity for the diagnosis of melanoma and thus significantly reduce the proportion of benign lesions excised, but learning these techniques took considerable time and was completed by only 62% of the trial doctors.14

The MoleMate system (see acknowledgments for manufacturer details) is a computerised diagnostic tool that applies the innovative technology of Spectrophotometric Intracutaneous Analysis (SIAscopy). It is easy and quick to learn and use.15 This non-invasive scanning technique uses light reflected from the skin in the visible and infrared spectra to produce images of the epidermal and dermal melanin and vasculature and the collagen content of the papillary dermis within the lesion. Patterns within these images indicate histopathological features consistent with melanoma and are highly predictive of melanoma in the experimental setting16 and in secondary care when applied in a scoring algorithm.17 The original secondary care algorithm was refined for primary care use to account for the higher prevalence of seborrhoeic keratoses and haemangiomas seen in this setting, and the increasing prevalence of these lesions with age (fig 1).18 The primary care scoring algorithm was integrated with a hand held SIAscopy scanner to create the MoleMate system (fig 2). We determined whether use of the MoleMate system in UK general practice would result in more appropriate referrals of suspicious pigmented lesions to secondary care than current best practice. We hypothesised that the system would increase the proportion of appropriate referrals without increasing the total number of referrals.

Fig 1 Primary care scoring algorithm17

Fig 2 Screenshot of MoleMate system. MedX Health (http://simsys-molemate.com/simsys-molemate/siascopy/)

Methods

We carried out a prospective, randomised open trial with pragmatic ascertainment of an endpoint, in 15 general practices in eastern England. The protocol for the MoleMate UK Trial has been published elsewhere.19

Participants

All general practice team members identified potential participants. Adults were eligible for enrolment if they were aged 18 or over and had a suspicious pigmented lesion. For the purpose of the trial the definition of a suspicious pigmented lesion was any lesion presented by a patient, or opportunistically seen by a family doctor or practice nurse, that could not immediately be diagnosed as benign and about which the patient could not be reassured. We excluded patients who were unable to give informed consent or were considered inappropriate to include by their family doctor. Potentially eligible patients were internally referred within the general practice to attend a trial appointment within one week.

Procedures

Lead clinicians

In each practice we trained two lead clinicians (a total of 28 general practitioners and two nurse practitioners) in consent procedures for the trial, data collection, and best practice assessment. They also learned to use and interpret the MoleMate system by completing a two hour training CD-ROM to identify relevant SIAscopic features of various pigmented skin lesions; this has been shown to significantly improve the ability of general practitioners to interpret SIAscopic images.15 At the trial appointment the lead clinician confirmed eligibility of the patients and obtained consent. We did not recruit practices already using a MoleMate system, and we excluded general practitioners with known dermatological expertise from being lead clinicians—that is, current hospital practitioners, clinical assistants in dermatology, and general practitioners with a specialist interest in dermatology.

Randomisation

The lead clinician randomised participants to either the best practice (control) group or the MoleMate (intervention) group on the basis of a block randomisation method, using computer generated, randomly permuted blocks of size 2, 4, and 6, established by the trial statistician (ATP). Sets of numbered, sealed envelopes were prepared, with the order of the sequences verified at completion of the trial (ATP). Randomisation was stratified by lead clinician and patient’s age (≤45 years, ≥46 years) to account for a potential differential effect on referral owing to the inclusion of age in the intervention system’s primary care scoring algorithm.

Best practice (control) group

Lesions of participants allocated to the control group were clinically assessed according to the Cambridge University Hospitals NHS Foundation Trust guidelines. This included taking a clinical history and naked eye examination.8 9

MoleMate (intervention) group

Lesions of participants allocated to the intervention group were clinically assessed as for the control group. Lead clinicians then used the MoleMate system to support their assessment and management of the lesion.

For all lesions, lead clinicians then decided whether to refer patients through the fast track skin cancer pathway or to manage them in primary care.

Reference standards

We recorded a reference standard final diagnosis for all lesions in the trial. For referred lesions we defined this as expert opinion by a histologist or dermatologist, and for non-referred lesions as review by two other dermatology experts of the recorded clinical history and examination, a digital photograph, and MoleMate image where available. For lesions where there was uncertainty based on this initial review (for example, poor quality digital photograph), the lead clinician collected these data on a second occasion 3-6 months later. All participants with non-referred lesions were also offered a follow-up consultation with the lead clinician 3-6 months later to collect these data, including a second photograph, for review by the dermatology experts to identify change over time and to confirm a benign diagnosis.20

Outcomes

The primary outcome was the appropriateness of referral, defined as the proportion of referred lesions that secondary care experts decided to biopsy or monitor; it was a measure of the diagnostic accuracy of the general practitioner with or without the aid of the MoleMate system.

Secondary outcomes included those related to the clinicians (diagnostic performance, confidence, learning effects) and patients (satisfaction, anxiety). We assessed the diagnostic performance of the lead clinician—namely, the proportion of benign lesions appropriately managed in primary care, the percentage agreement with the expert decision to biopsy or monitor (sensitivity), and the percentage agreement with the expert assessment that the lesion was benign (specificity)—using data from all lesions in the trial (histology result or expert clinical diagnosis). We assessed the confidence and attitudes of the lead clinician towards the intervention two weeks after the trial was set-up and at trial completion using a modified measure (1-7 scale) based on the theory of planned behaviour.21 We measured potential contamination in the comparison group due to the learning effects of the lead clinician by comparing differences in the appropriateness and volume of referrals between groups for the first 10 intervention consultations of each lead clinician’s data collection, when contamination would be minimal (naive period), with these differences for the remaining consultations (potentially contaminated period).

We measured patients’ satisfaction using dimensions of care items from EUROPEP (a 23 item validated and internationally standardised measure of patients’ evaluations of care in general practice).22 Patients’ anxiety was measured by questionnaire, including the Spielberger state trait anxiety inventory23 and a modified cancer worry scale,24 completed within one week and at three months after the consultation with the lead clinician.

Other secondary outcomes

Other secondary outcomes in the trial protocol included: examination of the association between the index of suspicion scale and the seven point checklist, and their predictiveness of lesion outcomes (the index of suspicion scale was not found to have clinical utility and so was omitted from analysis, whereas the predictiveness of lesion outcome using the seven point checklist will be reported separately); economic evaluation (this will be reported separately as it used the denominator of person rather than lesion); and five year follow-up of melanoma incidence, which is being undertaken in collaboration with the Eastern Cancer Registration and Information Centre and will report in due course.

Statistical analysis

We reviewed sample size estimates at a planned, blinded interim analysis. This confirmed a recruitment rate of 6-8 participants for each practice each month, and a lesion referral rate of 28%. We also confirmed a mean cluster size among the referred sample of 1.06, and that the dermatology experts chose to obtain a biopsy or to monitor 55% of the referred lesions in the combined arm data.19 The primary and secondary care clinicians (general practitioners, dermatologists, plastic surgeon) agreed by consensus that a 15% reduction in referrals to the fast track skin cancer clinic would be the minimum clinically important difference to justify such a significant change in practice. Therefore a sample size of 400 referred lesions from about 380 patients, with overall 1450 study lesions from 1150 randomised participants, would be required to detect an increase of 15% in the rate of biopsy or monitoring, with 80% power at the 5% level of significance.

The intention to treat population comprised all lesions from all randomised patients. Analysis of the primary outcome was based on all lesions referred to secondary care. We compared the proportion of referred lesions that were biopsied or monitored between randomised groups principally using Donner’s test for clustered proportions through a linear mixed effects model (using R with the nlme package) with patient as a random term. To examine whether the results were sensitive to clustering of lesions within patients, we additionally reported a planned secondary analysis that was unadjusted for clustering using the χ2 test. The same approach was used to compare the proportion of benign lesions appropriately managed in primary care, the percentage agreement with the expert decision to biopsy or monitor, and the percentage agreement with the expert assessment that the lesion was benign, and the volume of referrals (defined as the proportion of lesions from randomised participants that were referred to secondary care). For the principal analysis (adjusted for clustering), multi-lesion clusters were insufficient to provide a comparison of the percentage agreement with the expert decision to biopsy or monitor.

As the proportion of benign lesions appropriately managed in primary care and the percentage agreement with the expert decision to biopsy or monitor had extreme proportions with large denominators, for the clinician diagnostic performance analysis we used the exact confidence interval and corresponding P value from Wilson’s score method.25 We report the analyses unadjusted for clustering because, compared with the cluster adjusted results, the estimated differences in proportions are the same, with negligibly wider and therefore conservative confidence intervals and with the same conclusions. In each randomised group we summarised the histological and expert clinical diagnosis of melanoma as a rate.

We used a mean score and 95% confidence interval to summarise confidence and attitudes towards the MoleMate system. Assessment for contamination through learning effects was undertaken by testing for a difference in the intervention effect for the primary outcome across naive and potentially contaminated periods. Using a t test we compared patients’ measures (anxiety, worry about cancer, satisfaction with the consultation): analyses were unadjusted cross sectional comparisons between groups. Change over time was summarised, reporting means at baseline and mean changes between time points. We carried out analyses using SPSS version 17 and R version 2.12.0.

Results

Participants were recruited from March 2008 to May 2010 and data collection was completed in September 2010. Overall, 1580 lesions on 1297 participants from 15 general practices (range 45-151 patients per practice) were randomly assigned to study groups (fig 3). Four participants (total seven lesions) withdrew from the control group after randomisation. With the exception of these four participants, all other randomised patients were included in the intention to treat analyses for the primary outcome. The groups were well matched on baseline characteristics (table 1).

Fig 3 Flow of participants through study

Table 1

 Baseline characteristics of 1293 participants with suspicious pigmented skin lesions allocated to best practice (control) group or MoleMate (intervention) group.* Values are numbers (percentages) of participants unless stated otherwise

Characteristics	Control group (n=650)	Intervention group (n=643)	Total (n=1293)
Mean (SD) age (years)	44.8 (16.9)	44.5 (16.7)	44.6 (16.8)
Age group:
≤45	358 (55.1)	355 (55.2)	713 (55.1)
≥46	292 (44.9)	288 (44.8)	580 (44.9)
Men	235 (36.2)	230 (35.8)	465 (36.0)
Women	415 (63.8)	413 (64.2)	828 (64.0)
Ethnicity:
White†	608 (93.5)	606 (94.2)	1214 (93.9)
Mixed, Asian, Black, Chinese, other	25 (3.9)	20 (3.2)	45 (3.5)
Missing	17 (2.6)	17 (2.6)	34 (2.6)
No of lesions assessed:
1	535 (82.3)	516 (80.2)	1051 (81.3)
2	97 (14.9)	109 (17.0)	206 (15.9)
3	16 (2.5)	18 (2.8)	34 (2.6)
4	2 (0.3)	0 (0.0)	2 (0.2)
Occupation:
Employed	289 (44.5)	292 (45.4)	581 (44.9)
Looking after home or family	29 (4.5)	30 (4.7)	59 (4.6)
Unemployed	13 (2.0)	7 (1.1)	20 (1.5)
Retired	91 (14.0)	85 (13.2)	176 (13.6)
Full time education	27 (4.2)	31 (4.8)	58 (4.5)
Long term sickness	7 (1.1)	6 (0.9)	13 (1.0)
Other	7 (1.1)	11 (1.7)	18 (1.4)
Missing	187 (28.8)	181 (28.1)	368 (28.5)
Education:
No qualifications	47 (7.2)	46 (7.2)	93 (7.2)
GCSE or similar	64 (9.8)	76 (11.8)	140 (10.8)
GCE A level or similar	56 (8.6)	54 (8.4)	110 (8.5)
Higher education or similar	58 (8.9)	75 (11.7)	133 (10.3)
Degree or similar	249 (38.3)	224 (34.8)	473 (36.6)
Missing	176 (27.1)	168 (26.1)	344 (26.6)

*Four participants withdrew after randomisation.

†All melanomas diagnosed in white population.

Primary outcome: appropriateness of referral

The appropriateness of referrals did not differ significantly between the intervention and control groups (56.8% v 64.5%): absolute difference −8.1% (95% confidence interval −18.0% to 1.8%); P=0.12 (table 2). The results from the per protocol and intention to treat analysis were similar, and evidence was lacking of a difference in the effect of the intervention between the two age subgroups.

Table 2

 Appropriateness of referrals and clinician’s diagnostic performance in best practice (control) and MoleMate (intervention) groups. Values are percentages (number/number in group)

Variables	Control group	Intervention group	% difference (95% CI)	P value
No of lesions assessed	785	788
% appropriate referral rate*	64.5 (111/172)	56.8 (130/229)	−8.1 (−18.0 to 1.8)	0.11
% appropriately managed in primary care†	99.2 (588/593)	99.6 (535/537)	0.5 (−0·6 to 2·0)	0.46
% agreement with expert decision to biopsy or monitor (sensitivity)†	95.7 (111/116)	98.5 (130/132)	2.8 (−1.8 to 7.4)	0.26
% agreement with expert assessment that lesion benign (specificity)†	90.6 (588/649)	84.4 (535/634)	−6.2 (−9·9 to −2·6)	<0.001
Volume referred†	22.4 (176/785)	29.8 (235/788)	7.4 (3·1 to 11·7)	0.001

*Difference adjusted for clustering of lesions within patients; difference unadjusted for clustering is −7.8% (95% confidence interval −17.4% to 1.8%, P=0.12).

†Unadjusted for clustering of lesions within patients.

Of the 1573 lesions analysed in the control and intervention groups, 411 were referred (176 and 235, respectively) and 1162 were not referred (609 and 553). Table 3 shows the reference standard diagnoses of the referred lesions from both groups. Of the 241 lesions considered appropriately referred by the experts, 215 were biopsied and 22 were monitored. For the lesions that were neither biopsied nor monitored, and therefore were considered not appropriately referred, the reference standard was based on the expert clinical diagnosis.

Table 3

 Diagnoses of 411 referred lesions: 176 in best practice (control) group and 235 in MoleMate (intervention) group*

Reference standard diagnosis	Appropriately referred				Not appropriately referred
	Control group	Intervention group	Total		Control group	Intervention group	Total
Total	111	130	241		61	99	160
Method of diagnosis†:
Histology	100	115	215		—	—	—
Monitored	9	13	22		—	—	—
Expert	—	—	—		61	99	160
Melanoma‡:	17	18	35
Superficial spreading	12	11	23		—	—	—
Nodular	0	2	2		—	—	—
Lentigo maligna melanoma	3	1	4		—	—	—
In situ§ or lentigo maligna	2	4	6		—	—	—
Other skin cancers:	1	5	6
Squamous cell carcinoma	0	1	1		—	—	—
Basal cell carcinoma	1	4	5		—	—	—
Other lesions:	82	92	174		61	99	160
Dysplastic (atypical) naevus	26	31	57		2	6	8
Seborrhoeic keratosis	8	14	22		12	31	43
Dermatofibroma and haemangioma	0	3	3		2	6	8
Lentigo	5	4	9		0	1	1
Other benign¶	43	40	83		45	55	100

*Participants totalling 10 referred lesions did not attend for dermatology assessment (four in control group, six in intervention group).

†Missing histology (two in control group, two in intervention group).

‡Invasive and pre-invasive.

§Clark level 1.

¶For example, benign naevus.

Thirty six histologically confirmed melanomas were diagnosed during the trial (tables 3 and 4). More than half were less than 1 mm thick with a good prognosis, and none were subsequently found to have metastases or lymph node involvement (control group: nodular melanoma 0, superficial spreading melanoma 13, lentigo maligna melanoma 3, melanoma in situ/lentigo maligna 2; intervention group: nodular melanoma 2, superficial spreading melanoma 11, lentigo maligna melanoma 1, melanoma in situ/lentigo maligna 4). Of the 18 melanomas diagnosed in the control group, 17 were considered suspicious by the lead clinicians and the patients appropriately referred: the non-referred lesion, identified by the two dermatology experts who reviewed all images in the trial, was a superficial spreading melanoma (Breslow thickness 1.2 mm). All of the 18 melanomas diagnosed in the intervention group were considered suspicious and the patients appropriately referred. In the referred participants six non-melanoma skin cancers were detected: one basal cell carcinoma in the control group and four basal cell carcinomas and one squamous cell carcinoma in the intervention group.

Table 4

 Diagnoses of lesions in 1162 non-referred patients: 609 in best practice (control) group and 553 in MoleMate (intervention) group*

Diagnoses	Control group	Intervention group	Total
Total	603	539	1142
Benign	588	535	1123
Outcome unknown†	10	2	12
Dermatology expert review‡:	5	2	7
Monitor	1	0	—
Melanoma	1	0	—
Superficial spreading melanoma	1		—
Other skin cancers	3	2	—
Basal cell carcinoma	3	2	—

*19 non-referred lesions were excluded from follow-up due to violation of recruitment criteria or discontinued protocol (five in control group, 14 in intervention group), and one patient with one lesion died (control group).

†Did not attend lead clinician review at 3-6 months when uncertain reference diagnosis based on data from first lead clinician consultation.

‡Following lead clinician review at three months.

Among the referred but benign lesions the control and intervention groups did not differ significantly for diagnosis of seborrhoeic keratoses (histology: 7% (8/111) v 11% (14/130), P=0.34; expert opinion: 20% (12/61) v 31% (31/99), P=0.11) and dysplastic naevi (histology: 23% (26/111) v 24% (31/130), P=0.94; expert opinion: 3% (2/61) v 6% (6/99), P=0.71; table 3). Most of the non-referred lesions were considered at expert review to have been appropriately assessed (table 4). Reference diagnosis procedures for non-referred lesions identified six skin cancers. Both lesions in the intervention group were basal cell carcinomas, whereas the control group yielded one melanoma and three basal cell carcinomas.

Clinician outcomes

The median age of the lead clinicians (28 general practitioners, two nurse practitioners; 16 men) was 44 (range 35-56) years, and they had a median 15 (4-27) years’ experience. Seven had undergone some training in dermatology (three had a short dermatology training post, three were on clinical attachment to an out-patient clinic, and one was unspecified).

Diagnostic performance

Addition of the MoleMate system made no significant difference to the proportion of benign lesions appropriately managed in primary care or the percentage agreement with the expert decision to biopsy or monitor (table 2). The intervention reduced the percentage agreement with the expert assessment that the lesion was benign, and so resulted in a higher proportion of lesions referred.

Clinicians’ confidence

At completion of the trial the lead clinicians were confident and positive about using the MoleMate system. Compared with at the start of the trial at trial completion more clinicians thought that using MoleMate during a consultation would be simple (mean 4.8 (SD 1.56) v mean 5.9 (SD 0.65), P=0.001) and cost effective (mean 4.9 (SD 1.51) v mean 5.7 (SD 1.02), P=0.08). Furthermore, they thought that using MoleMate would be less likely to increase patients’ anxiety (mean 3.1 (SD 1.53) v mean 2.4 (SD 1.22), P=0.07) or to prolong consultations (mean 6.2 (SD 0·86) v mean 5.2 (SD 1.22), P<0.001). The clinicians agreed that MoleMate had been easy to use (median 6, interquartile range 5-6; Likert type scale 1=disagree strongly to 7=agree strongly) and fast enough to use (median 6, 5-6) in a consultation, and that the use of the seven point checklist (median 5, 4-6) or MoleMate (median 5, 4-6) had “often helped their decision about the referral.”

Clinician learning effects

There was no evidence of any learning effects and therefore contamination between the groups. The initial naive period and the remaining potentially contaminated period did not differ significantly for either the volume of referrals (naive period: control group 25.9% (85/328) v intervention group 29.9% (109/364), difference 4.0%; potentially contaminated period: control group 19.9% (91/457) v intervention group 29.7% (126/424), difference 9.8%; difference between periods 5.8% (95% confidence interval −3.0% to 14.5%), P=0.20) or the appropriateness of the referral (naive period: control group 66.7% (56/84) v intervention group 57.5% (61/106), difference −9.1%; potentially contaminated period: control group 62.5% (55/88) v intervention group 56.1% (69/123), difference −6.4; difference between periods 2.7% (−16.5% to 21.9%), P=0.78).

Patient outcomes

Patients’ satisfaction

The response rate to the patient questionnaire, completed within one week of the consultation with the lead clinician, was 74% (968/1293), and three months later was 80% (904/1135). The scores for patients’ satisfaction (five responses: poor, fair, good, very good, excellent) showed significant differences between the groups across all 12 items, although only a few responded to each item as poor or fair. Compared with the control group, more patients in the intervention group ranked their consultation as very good or excellent for thoroughness (control group 71.2% (n=475) v intervention group 83.1% (n=485), P<0.001), communication (control group 70.6% (n=479) v intervention group 82.1% (n=485), P<0.001), and reassuring care (control group 66.4% (n=474) v intervention group 77.2% (n=481), P<0.001).

Patients’ anxiety

General and skin cancer specific anxiety scores did not differ between groups immediately after the consultation or over time. Immediately after the consultation patients with non-referred lesions in the intervention group had lower mean general anxiety compared with people in the control group: 32.56 (SD 0.58) v 34.72 (SD 0.64), P=0.013; table 5.

Table 5

 General and skin cancer specific anxiety within one week of trial consultation and three month follow-up. Values are mean (SD) responses

Variables	Within 1 week of trial consultation					3 month follow-up
	Control group	Intervention group	Difference (95% CI)	P value		Control group	Intervention group	Difference (95% CI)	P value
Speilberger state trait anxiety inventory*:
All	34.91 (0.56) n=459	33.77 (0.50) n=472	−1.14 (−2.62 to 0.34)	0.13		34.59 (0.51) n=452	33.96 (0.52) n=450	−0.63 (−2.07 to 0.83)	0.39
Referred	35.45 (1.15) n=118	36.18 (0.93) n=158	0.73 (−2.16 to 3.62)	0.62		33.61 (0.98) n=118	33.85 (0.92) n=154	0.24 (−2.43 to 2.90)	0.86
Non-referred	34.72 (0.64) n=341	32.56 (0.58) n=314	−2.16 (−3.87 to 0.46)	0.013		34.94 (0.60) n=334	34.02 (0.63) n=296	−0.92 (−2.64 to 0·80)	0.29
Referred minus non-referred	0.73 (1.28)	3.62 (1.05)	2.89 (−0.34 to 6.13)	0.08		−1.32 (1.17)	−0.17 (1.10)	1.16 (−1.99 to 4.31)	0.47
Skin cancer worry scale†:
All	10.06 (0.13) n=471	10.25 (0.15) n=478	0.19 (−0.20 to 0.58)	0.34		9.45 (0.14) n=449	9.34 (0.14) n=458	−0.12 (−0.51 to 0.27)	0.56
Referred	9.90 (0.25) n=119	10.30 (0.27) n=161	0.41 (−0.35 to 1.16)	0.29		9.68 (0.34) n=115	9.53 (0.27) n=156	−0.16 (−1.01 to 0·68)	0.71
Non-referred	10.12 (0.16) n=352	10.22 (0.17) n=317	0.11 (−0.35 to 0.56)	0.56		9.37 (0.15) n=334	9.24 (0.16) n=302	−0.13 (−0.56 to 0.30)	0.55
Referred minus non-referred	−0.22 (0.30)	0.08 (0.31)	0.30 (−0.56 to 1.15)	0.50		0.31 (0.33)	0.28 (0.29)	−0.03 (−0.89 to 0.83)	0.95

*Six item response (1-4 Likert type responses); short form scaled up to 20 item range of 20-80.

†Six items; score 6-24.

Discussion

The MoleMate UK Trial assessed whether this novel computerised diagnostic tool, comprising a primary care scoring algorithm integrated with SIAscopy, could improve management of suspicious pigmented lesions in primary care; results in both study groups showed strong agreement with expert assessment of lesions. Adding the MoleMate system to best practice did not improve the appropriateness of referral; in fact, it resulted in lower agreement with expert assessment that the lesion was benign, resulting in a higher proportion of referrals overall. No melanomas were missed in the MoleMate group and only one was missed in the best practice group. Nevertheless, lead clinicians were confident that the MoleMate system enhanced their practice, and patients ranked satisfaction with consultations higher with the MoleMate system than with best practice alone, and were not anxious by the addition of this new diagnostic aid to best practice. By being perceived more positively, the novel technology provided false reassurance, as the systematic application of best practice guidelines ultimately proved more accurate.

Strengths and weaknesses of the study

An important strength of this rigorously conducted trial was its strong internal reliability. Unusually for skin cancer studies carried out in primary care, expert clinical diagnoses on all lesions were obtained, including lesions that were managed in primary care. We applied a rigorous randomisation method based on sealed envelopes produced by an independent statistician. Our sample size was met and interim analysis showed no significant clustering effects to reduce the power of the trial. We collected data between three and six months after the trial consultation to identify change over time and to confirm a benign diagnosis.20 Of 566 lesions reassessed, only two were found to be clinically significant: a dysplastic (atypical) naevus and a lentigo maligna. The choice of reference standard diagnosis was inevitably pragmatic as, for ethical reasons, we could not obtain histology for every lesion in the trial. From a total of 1573 lesions, only 2.7% did not have a reference standard diagnosis. Although there were eight more lesions in the best practice group managed in primary care for which we do not have a final diagnosis, we do not believe these would alter the key findings of this trial. Furthermore, all participants have been flagged with the Eastern Cancer Registry and Information Centre to identify any melanomas diagnosed over the next five years. In addition to gaining data on diagnostic performance we also obtained information on the acceptability of the intervention by patients and health professionals, which has not previously been examined in similar studies.

Several aspects of the trial design have inherent limitations. Our primary outcome measure was chosen as the one most relevant to an intervention designed to improve referral patterns of pigmented skin lesions. The definition of appropriateness of referral was based on the subsequent clinical management by an expert dermatologist. Thus any lesion considered by a dermatologist worthy of biopsying or monitoring would, from a primary care perspective, be an appropriate referral. As this was the first trial of the MoleMate system in primary care it would have been premature to power the trial to assess differences in detection rates of melanoma given the low prevalence. In fact, this trial is the largest done in primary care worldwide to date to test a new melanoma diagnostic aid.

In the absence of comparable data about people presenting with suspicious lesions in primary care, it is difficult to comment on the representativeness of the sample. The over-representation of people with higher education levels and under-representation of ethnic minority groups compared with the UK general practice population may slightly limit generalisability across the United Kingdom. As a new diagnostic test,26 the MoleMate system was implemented as part of a novel service model in which patients with a suspicious pigmented lesion were referred internally to a trained lead clinician. The trial population therefore represented patients referred to this new service model with a suspicious lesion. This internal referral pathway is consistent with similar service models and subspecialisation increasingly occurring within general practice.

We chose to compare the MoleMate system with standardised best practice rather than usual care for well considered reasons in the design of a trial in primary care27: firstly, a usual care arm would not have allowed us to obtain data on all lesions for reference standard diagnostic purposes because of a significant potential Hawthorne effect incurred through the required data collection procedures. We therefore would not have been able to assess appropriateness of management in primary care. Secondly, the risk of contamination between groups would have been much greater and therefore would have meant using a practice level, cluster randomised design, with consequent effects on the scale and feasibility of the trial. While this means that we do not have directly comparable data for usual care, the findings relating to best practice have important clinical implications.

Strengths and weaknesses in relation to other studies

Both the MoleMate and best practice groups showed impressively high diagnostic performances compared with other studies. Previous international estimates of sensitivity for the diagnosis of melanoma are in the order of 29-41% using history and naked eye assessment alone in primary care.14 28 29 Best practice performed much better, with a sensitivity of 95.7% for suspicious lesions, such that 17 out of 18 melanomas were correctly identified in this trial group. While there are no comparable data on appropriateness as defined in this trial, the ratio of melanomas to benign skin lesions was much higher in both trial groups than previously reported.4 Although the lead clinicians had no specialist training in dermatology before the trial, they were briefly trained in data collection procedures designed to systematically use best practice local guidelines, including the application of the seven point checklist. We did not detect any improvement in their performance during the trial and therefore do not believe that contamination between trial groups through learning effects is the explanation for the high performance in the best practice group. We propose that the formal implementation of the best practice guidelines contributed to the high performance in the comparison group.

SIAscopy has been shown to be an effective diagnostic aid in nurse led screening of pigmented lesions in secondary care17 but may not help expert dermatologists to distinguish melanomas from benign lesions.30 Dermoscopy is the most widely used non-invasive approach to aid diagnosis of melanoma; in the secondary care setting its use increases diagnostic accuracy compared with clinical visual inspection31 and SIAscopy.32 Dermoscopy has also been shown to improve accuracy of primary care doctors to triage lesions suggestive of skin cancer13; however, it is a difficult technique to learn, and takes time to become proficient.14 30 Computerised skin imaging devices based on dermoscopy have also been developed, such as MoleMax, SolarScan, and MelaFind.33 Few have yet been assessed in primary care and none using our rigorous approach of a randomised controlled trial. In a smaller non-randomised trial in Australian primary care, dermoscopy and short term sequential digital dermoscopy imaging showed a significant reduction in referrals or excisions of benign pigmented lesions while doubling the sensitivity for the diagnosis of melanoma.14

Implications for clinicians and policy makers

For melanoma, as for other cancers and any serious condition in primary care where prevalence is low, the inherent diagnostic problem remains of aiming for high sensitivity without resultant poor specificity. It has been suggested recently that tighter gatekeeping, as occurs in British general practice, may contribute to later cancer diagnosis because of a higher focus on specificity of referral or investigation.34 In this trial the lower specificity of the MoleMate system resulted in an increased volume of referrals, but the detection rates of melanoma in both arms were high.

By introducing an internal referral system within the practice, we tested two approaches to assess pigmented skin lesions in a more systematic way in primary care. The implementation of best practice guidelines performed better than the MoleMate system, and both performed better than reports of current practice. Improving the management of suspicious pigmented skin lesions could be rapidly effected by changes in general practice systems that ensure the routine application of the seven point checklist, perhaps further enhanced with internal referral and subsequent follow-up of all non-referred patients with lesions.

Skin malignancy is an important cause of mortality in the United Kingdom and the incidence is rising every year; early detection and management will improve outcomes

Differentiating melanomas from other pigmented skin lesions is challenging, and diagnostic technology could improve primary care performance and appropriate referral of high risk pigmented skin lesions

Adding the MoleMate system (SIAscopy with primary care scoring algorithm) to the systematic application of best practice guidelines did not increase the proportion of patients appropriately referred with lesions; instead, the intervention led to a higher proportion of patients referred with lesions

Systematic application of best practice guidelines and the MoleMate system both performed much better than reports of current practice

Although the novel diagnostic technology was less accurate than best practice, clinicians and patients rated it higher than best practice for reassuring and thorough care, suggesting false reassurance

On current evidence the systematic application of best practice guidelines (including the seven point checklist) is the paradigm for management of suspicious skin lesions in primary care