Diagnosing malignant melanoma in ambulatory care: a systematic review of clinical prediction rules.

OBJECTIVES: Malignant melanoma has high morbidity and mortality rates. Early diagnosis improves prognosis. Clinical prediction rules (CPRs) can be used to stratify patients with symptoms of suspected malignant melanoma to improve early diagnosis. We conducted a systematic review of CPRs for melanoma diagnosis in ambulatory care.
DESIGN: Systematic review. DATA SOURCES: A comprehensive search of PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS was conducted in May 2015, using combinations of keywords and medical subject headings (MeSH) terms. STUDY SELECTION AND DATA EXTRACTION: Studies deriving and validating, validating or assessing the impact of a CPR for predicting melanoma diagnosis in ambulatory care were included. Data extraction and methodological quality assessment were guided by the CHARMS checklist.
RESULTS: From 16 334 studies reviewed, 51 were included, validating the performance of 24 unique CPRs. Three impact analysis studies were identified. Five studies were set in primary care. The most commonly evaluated CPRs were the ABCD, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) dermoscopy rule (at a cut-point of >4.75; 8 studies; pooled sensitivity 0.85, 95% CI 0.73 to 0.93, specificity 0.72, 95% CI 0.65 to 0.78) and the 7-point dermoscopy checklist (at a cut-point of ≥1 recommending ruling in melanoma; 11 studies; pooled sensitivity 0.77, 95% CI 0.61 to 0.88, specificity 0.80, 95% CI 0.59 to 0.92). The methodological quality of studies varied.
CONCLUSIONS: At their recommended cut-points, the ABCD dermoscopy rule is more useful for ruling out melanoma than the 7-point dermoscopy checklist. A focus on impact analysis will help translate melanoma risk prediction rules into useful tools for clinical practice. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisability and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. Quality assessment criteria were used to assess risk of bias and the majority of studies were at low risk in relation to the randomisation procedure and monitoring of loss to follow-up.

A large proportion of studies did not provide sufficient information and data to perform stratified meta-analysis according to different levels of risk.

Current research shows that dermoscopic clinical prediction rules (CPRs) may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients but future impact analysis research is necessary to establish their impact on patient outcomes.

Introduction

The incidence of malignant melanoma in most developed countries has been steadily rising (faster than other cancer types) in recent decades.1 2 Increases in the age-standardised incidence of at least 4–6% per annum have been reported internationally in many fair-skinned populations including Australia, the USA and most of Europe.3–5 Simultaneously, there has been a significant rise in overall 5-year survival in melanoma patients, largely attributable to earlier detection and diagnosis of thinner tumours.6 While the majority of patients may survive melanoma, the disease has a significant impact on patient quality of life7 and healthcare expenditure, with the average annual total treatment costs for melanoma in the USA increasing to US$3.3 billion in 2011.8 Melanoma is potentially preventable since a significant risk factor, exposure to ultraviolet (UV) radiation, is modifiable.9 However, other risk factors (eg, number naevi, eye and hair colour, freckles, familial history and genetic predisposition) also play an important role in the risk of developing melanoma.10 11

Early detection followed by curative surgery greatly improves melanoma prognosis. However, early detection may be affected by the challenging natures of differential diagnosis of pigmented lesions. Particularly in primary care where the evaluation of suspected skin lesions is imposing an increasing burden due to rising incidences of skin cancer.12 It has been suggested that primary care practitioners' skills of diagnosing skin lesions could be improved.13 A number of clinical prediction pules (CPRs) and computer-assisted diagnostic tools have been developed to assist in distinguishing malignant melanoma from benign pigmented skin lesions. The UK National Institute for Clinical Excellence (NICE) guidelines advise against routine use of computer-assisted diagnostic tools in the initial evaluation of a pigmented skin lesion (NICE guidelines) and promote use of the weighted 7-point checklist in primary care to guide referral (NG12). When used by dermatologists for the diagnosis of melanoma, certain CPRs have demonstrated high sensitivity and specificity.6 Although each CPR has its own unique elements, there is significant overlap in terms of their content (see online supplementary appendix 1), and while their use is promoted, it is unclear which rules are most suitable for use in primary care.

CPRs may be for use in clinical (ie, naked eye) examination, or in conjunction with dermoscopy. Dermoscopy, dermatoscopy or epiluminescent microscopy refers to the examination of pigmented skin lesions using surface microscopy.14 15 The use of dermoscopy, primarily by dermatologists, has been found to increase diagnostic accuracy compared with naked-eye inspection, as it allows the visualisation of features that are not visible to the naked eye.14–16 However, the effectiveness of dermoscopy depends on clinical experience and training. Dermatologists with formal training in dermoscopy have higher melanoma detection rates compared with untrained dermatologists and primary care physicians.16–18

As primary care or ambulatory care physicians are frequently and increasingly confronted with the care of skin lesions suspected of malignancy,12 it is essential to identify tools to aid primary care practitioners to differentiate patients with clinically significant lesions, requiring referral, from those who can be treated and monitored in primary care. The aim of this study was to perform a systematic review of CPRs for the diagnosis of malignant melanoma, to evaluate their diagnostic accuracy in primary care and specialist outpatient settings, among patients with a pigmented skin lesion. Secondary aims were to review studies that have examined the implementation of CPRs in clinical practice through impact analysis studies.

Methods

The protocol for this systematic review was published on PROSPERO (CRD42015020898) and was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.19

Search strategy and data sources

A systematic literature search was conducted (May 2015) including the following databases: PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS, using combinations of the following keywords and MeSH terms: melanoma/diagnosis, melanoma, prediction, score, model, decision, sensitivity, specificity, validate, derived. Hand searches of references of retrieved full-text articles and key author searches supplemented the search. No date or language limits were imposed.

Study selection

All articles were initially screened for inclusion according to title and abstract by two reviewers (NW, EH). Full-text articles of studies considered eligible for inclusion were independently read by both reviewers, with any disagreements resolved by a third reviewer (BC).

Validation studies

Validation studies were eligible for inclusion if they met the following criteria;

Population: Adults (age ≥18 years) with a pigmented skin lesion in ambulatory care settings in general practice/family medicine, dermatology, plastic surgery and other relevant specialties.

Risk: Derivation and/or validation of a CPR for melanoma diagnosis to aid decision making about referral or investigation of a pigmented skin lesion. CPRs were defined as ‘a clinical tool that quantifies the individual contributions that various components of the history, physical examination and investigations make toward the diagnosis, prognosis or likely response to treatment in a patient’.

Comparison: Usual clinical judgement for decision making about referral or investigation OR another CPR for melanoma diagnosis.

Primary outcome: Performance of a CPR for predicting diagnosis of malignant melanoma (in terms of sensitivity, specificity, negative predictive values and positive predictive values).

Observational study designs (eg, cohort, cross-sectional, case–control) were included. Studies were excluded where they had undergone derivation only, reported individual predictors only, or used computer-assisted diagnostic tools, following the NICE guideline recommendation against the routine use of computer-assisted diagnostic tools.20

Impact analysis

The following study designs were included for impact analysis: (cluster) randomised controlled trials (RCTs), controlled before–after studies or interrupted time series studies. We excluded uncontrolled study designs. We included studies where a melanoma CPR was used to predict melanoma compared with usual care in the clinical setting. The outcomes of interest included physician behaviour, process of care, patient outcomes and/or cost-effectiveness. A requirement for inclusion was that the CPR comprised the entire intervention. Studies where the CPR was implemented as part of a broader guideline, protocol or decision aid were excluded. Studies that used a CPR to determine eligibility for trial inclusion but were not part of the intervention were also excluded.

Data extraction

Data were extracted by four reviewers (LA, HB, HS, EH) using a data form based on the CHARMS checklist.21 Data extracted included study design and setting, patient demographics and inclusion criteria, CPR name, CPR type (clinical or dermoscopic), predictive accuracy of the CPR (sensitivity/specificity) and for impact analysis, the impact on the primary outcome.

Critical appraisal of studies

Two reviewers (EH, NW) critically appraised included studies using the CHARMS checklist, developed to provide guidance on data extraction and critical appraisal of prediction modelling studies.21 The checklist contains 11 domains of critical appraisal. The methodological quality of each study was independently evaluated by two reviewers and by a third reviewer if consensus was not reached. The methodological quality of each impact analysis study was also independently assessed, using an appropriate quality assessment checklist. RCTs were assessed using the Cochrane risk of bias tool and controlled before–after studies were evaluated using Cochrane criteria for these study designs.22

Statistical analysis

Statistical analysis was conducted using Stata V.12 (StataCorp, College Station, Texas, USA), in particular the metandi and midas commands. For each CPR, a standard cut-point was identified (table 1). From each included study we extracted (where available) the numbers of true positives, false positives, true negatives, false negatives, sensitivity and specificity and their corresponding 95% CIs. Where sensitivity/specificity for more than one observer was reported, the mean value was included in the analysis. Studies were grouped for analysis by CPR type (ie, clinical or dermoscopic). Summary estimates of sensitivity and specificity and their corresponding 95% CIs were calculated using the bivariate random effects model (midas). The bivariate model has the benefits of being easily interpretable, is technically straightforward to undertake and takes into account the sample and heterogeneity beyond chance between studies.23

Table 1

CPRs identified for inclusion with cut-points for identification of melanoma

Rule name	Cut-point used	Number of validation studies
Clinical rule
ABCDE clinical rule	≥1 or ≥2	4
ABCD clinical rule	≥1	4
Revised 7-point checklist (clinical)	≥3	4
7-point checklist (clinical)	≥3	4
Dermoscopic rules
ABCD rule of dermoscopy*	≥4.75	15
	≥5.45	6
	≥4.2	1
	Not reported	1
7-point checklist for dermoscopy	≥3	17
Menzies 1996 dermoscopy for melanoma	≥1, no negative features	8
3-point checklist for dermoscopy	≥1	6
Seven features for melanoma (7FFM)	≥2	5
CASH dermoscopy algorithm	≥8	3
ABCDE rule (dermoscopy)	Not reported	2
The 3-colour dermoscopy test	≥3	2
Revised 7-point checklist for dermoscopy	≥1	1
Kreusch 1992 dermoscopy	Not reported	1
Nilles 1994 dermoscopy	Not reported	1
Menzies 2008 dermoscopy for melanoma	≥1	1
DynaMel algorithm	≥3	1
Menzies 2008 dermoscopy for skin cancer	≥0 (high sensitivity); ≥1 (high specificity)	1
Simplified ABC-point list for dermoscopy	≥4	1
AC rule for dermoscopy	Not reported	1
Emery 2010 SIAscopy	≥6	1
Guitera RCM 2012	Not reported	1
Digital dermoscopy algorithms	Multiple algorithms, different cut-offs	1

*Score = (A score×1.3)+(B score×0.1)+(C score×0.5)+(D score×0.5).

ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6mm) Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, color, architecture, symmetry, and homogeneity; CPR, clinical prediction rules RCM, reflectance confocal microscopy.

Individual and summary estimates of sensitivity and specificity were plotted on a hierarchical summary receiver operating characteristic (HSROC) graph. This approach incorporates sensitivity and specificity, while taking into account the correlation between the two.24 Sensitivity (true positive) was graphed on the y-axis and 1-specificity (false negative) on the x-axis. The 95% confidence region and the 95% prediction region were also plotted around the pooled estimates in order to depict the precision with which the pooled estimates were determined (confidence ellipse around the mean value) and to illustrate the amount of between-study variation (prediction ellipse).

Results

The search strategy yielded a total of 25 816 articles. Of these 9481 were duplicates and 16 166 were deemed irrelevant based on title/abstract. The remaining 171 were reviewed in full with 51 meeting the inclusion criteria (see online supplementary appendix 2). From these, 24 unique melanoma CPRs were identified (table 1). Twelve papers reported derivation and validation studies, 36 were validation studies only and three were impact analyses.

Summary of studies

Table 2 summarises the characteristics of the included studies. The majority (11, 22%) were conducted in Italy14 15 25–34 and ranged from an analysis of 40 lesions to 1580 lesions. From 13 studies providing information, mean age of included patients ranged from 36.7 to 53.25 28 31 35–44 From the 14 studies that reported gender, the proportion of males ranged from 22% to 60%.25 31 33 35–45 In total, 31 of the 50 studies were published in/or after 2000.14 25 28 29 31–37 42–44 46–62 Five studies were set in primary care,36 44 49 62 63 with the remainder undertaken in specialist outpatient settings.

Table 2

Characteristics of validation and impact analysis studies included

Validation studies
Author year, country	Setting	CPR used	Lesions	Patient: n, sex, mean age	CPR applied by: nExperience	Reported sensitivity/specificity
Annessi 2007,25 Italy	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopy	19896 melanomas, 102 non-melanoma	N=19554% maleMean age: 43	2ELM-experienced dermatologists	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 84.4Sp: 74.57-point checklist for dermoscopy (cut-point ≥3)Se: 78.1Sp: 64.7
Argenziano 1998,26 Italy	Department of dermatology	7-point checklist for dermoscopyABCD rule of dermoscopy	342117 melanoma, 225 non-melanoma	NR	53 experienced2 less experienced	7-point checklist for dermoscopy (cut-point ≥3)Expert user:Se: 95.0Sp: 75.0Non-expert user (mean):Se: 89.0Sp: 61.5ABCD rule of dermoscopy (cut-point ≥4.75)Expert user:Se: 85.0Sp: 66.0Non-expert user (mean):Se: 91.5Sp: 31.0
Argenziano 2003,14 9 countries	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopyMenzies 1996 dermoscopy for melanoma	108	NR	40Experienced	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 82.6Sp: 70.07-point checklist for dermoscopySe: 85.7Sp: 71.1Menzies 1996 dermoscopy for melanomaSe: 85.7Sp: 71.1
Argenziano 2011,27 Italy	Department of dermatology	7-point checklist for dermoscopyRevised 7-point checklist for dermoscopy	300100 excised melanoma, 100 excised non-melanoma, 100 non-excised non-melanoma	NR	8Experienced	7-point checklist for dermoscopy (cut-point ≥3)Se: 77.9Sp: 85.6Revised 7-point checklist for dermoscopy (cut-point ≥1)Se: 87.8Sp: 74.5
Benelli 1999,15 Italy	Department of dermatology	7FFM (seven features for melanoma) dermoscopyABCDE clinical rule	40160 melanomas, 341 non-melanoma	NR	2Research team	7FFM (seven features for melanoma) dermoscopy (cut-point of ≥2)Se: 80.0Sp: 89.1ABCDE clinical rule (cut-point ≥2)Se: 85.0Sp: 44.5
Benelli 2000,28 Italy	Department of dermatology	7FFM (seven features for melanoma) dermoscopyABCDE clinical rule	60076 melanomas, 524 non-melanoma	Mean age: 53	3	7FFM (seven features for melanoma) dermoscopy (cut-point of ≥2)Se: 68.8Sp: 86.0ABCDE clinical rule (cut-point of ≥2)Se: 47.3Sp: 56.0
Binder 1999,66 Austria	Department of dermatology	ABCD rule of dermoscopy	250	NR	1712 experienced5 trainee	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 81.0Sp: 77.0ABCD rule of dermoscopy (cut-point ≥5.45)Se: 73.0Sp: 90.0
Blum 2003,71 Germany	Department of dermatology	The 3-colour dermoscopy test	249	NR	NR	The 3-colour dermoscopy testSe: 76.9Sp: 90.1
Blum 2004,47 Germany	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopyMenzies 1996 dermoscopy for melanomaSimplified ABC-point list for dermoscopy7FFM (seven features for melanoma) dermoscopy	26984 melanomas, 185 non-melanoma	NR	NR	ABCD rule of dermoscopySe: 90.5Sp: 72.47-point checklist for dermoscopySe: 90.5Sp: 87.0Menzies 1996 dermoscopy for melanomaSe: 95.2Sp: 77.87FFM (seven features for melanoma) dermoscopySe: 94.0Sp: 74.6Simplified ABC-point list for dermoscopySe: 90.5Sp: 87.0
Blum 2004,48 Germany	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopyMenzies 1996 dermoscopy for melanoma7FFM (seven features for melanoma) dermoscopy	26984 melanomas, 185 non-melanoma	NR	NR	ABCD rule of dermoscopySe: 90.5Sp: 72.47-point checklist for dermoscopySe: 90.5Sp: 87.0Menzies 1996 dermoscopy for melanomaSe: 95.2Sp: 77.87FFM (seven features for melanoma) dermoscopySe: 94.0Sp: 74.6
Buhl 2012,35 Germany	Department of dermatology	DynaMel Algorithm7-point checklist for dermoscopy	675	N=68857% maleMean age: 42	Dermatology residents	DynaMel AlgorithmSe: 77.1Sp: 98.17-point checklist for dermoscopy (cut-point ≥3)Se: 47.5Sp: 99.0
Carli 2002,29 Italy	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopy	20044 melanomas, 156 non-melanoma	NR	5Dermatology residents	ABCD rule of dermoscopy (cut-point ≥5.45)Se: 88.1Sp: 45.77-point checklist for dermoscopy (cut-point ≥3)Se: 91.9Sp: 35.2
Dal Pozzo 1999,30 Italy	Department of dermatology	7FFM (seven features for melanoma) dermoscopy	713168 melanomas, 545 non-melanoma	NR	3	7FFM (seven features for melanoma) dermoscopySe: 94.6Sp: 85.5
Dolianitis 2005,49 Australia	Primary care and dermatology department	7-point checklist for dermoscopyABCD rule of dermoscopyMenzies 1996 dermoscopy for melanoma	4020 melanomas, 20 non-melanoma	NR	6135 primary care physicians, 10 dermatologists, 16 trainee dermatologists	7-point checklist for dermoscopySe: 81.4Sp: 73.0ABCD rule of dermoscopy (cut-point ≥5.45)Se: 77.5Sp: 80.4Menzies 1996 dermoscopy for melanomaSe: 84.6Sp: 77.7
Emery 2010,36 UK	Family practice	Emery 2010 SIAscopy in primary care for melanoma	1211	N=85852% maleMean age: 50	1SIAscopy expert	Emery 2010 SIAscopy in primary care for melanomaSe: 50.0Sp: 84.0
Feldman 1998,67 Austria	Department of dermatology	ABCD rule of dermoscopy	50030 melanomas, 470 non-melanoma	NR	NR	ABCD rule of dermoscopy (cut-point ≥4.2)Se: 88.0Sp: 64.0
Gereli 2010,50 Turkey	Department of dermatology	7-point checklist for dermoscopy3-point checklist for dermoscopy	9648 melanoma, 48 non-melanoma	NR	32 experienced1 inexperienced	7-point checklist for dermoscopy (cut-point ≥3)Se: 87.5Sp: 16.23-point checklist for dermoscopy (cut-point ≥2)Se: 89.6Sp: 31.2
Guitera 2012,51 Multiple	Skin cancer clinic	Guitera 2012 confocal microscopy for melanoma	710216 melanomas, 494 non-melanoma	N=663	NR	Guitera 2012 confocal microscopy for melanomaSe: 87.6Sp: 70.8
Haenssle 2010,37 Germany	Department of dermatology	7-point checklist for dermoscopy	1219127 melanomas, 1092 non-melanoma	N=68857% maleMean age: 42	Inexperienced	7-point checklist for dermoscopy (cut-point ≥3)Se: 62.0Sp: 97.0
Healsmith 1993,64 UK	Pigmented lesion clinic	Revised 7-point checklist (clinical) ABCDE clinical rule	16565 melanoma, 100 non-melanoma	NR	NR	Revised 7-point checklist (clinical)Se: 100Sp: NRABCDE clinical ruleSe: 92.3Sp: NR
Henning 2008,52 USA	Department of dermatology	CASH dermoscopy algorithmABCD rule of dermoscopy7-point checklist for dermoscopyMenzies 1996 dermoscopy for melanoma	15050 melanoma, 100 non-melanoma	NR	2Inexperienced	CASH dermoscopy algorithmSe: 87.0Sp: 67.0ABCD rule of dermoscopySe: 86.0Sp: 74.07-point checklist for dermoscopySe: 76.0Sp: 57.0Menzies 1996 dermoscopy for melanomaSe: 92.0Sp: 38
Higgins 1992,38 UK	Department of dermatology	7-point checklist (clinical)7-point checklist (clinical) revised	1000 melanoma, 100 non-melanoma	N=10030% maleMean age: 36.7	NR	7-point checklist (clinical) revisedSe: NRSp: 70.0
Kittler 1999,39 Austria	Department of dermatology	ABCD rule of dermoscopyABCDE rule (dermoscopy)	35673 melanomas, 283 non-melanoma	N=35243% maleMean age: 52	NR	NR
Keefe 1989,40 Scotland	Hospital dermatology clinic	7-point checklist (clinical)	222	N=19522% maleMean age: 43	Dermatologists195 patients	7-point checklist (clinical) (cut-point ≥3)Dermatologists:Se: 85.7Sp: 66.5Patients:Se: 71.4Sp: 66.2
Kreusch 1992,84 Germany	Department of dermatology	Kreusch 1992 dermoscopy for melanoma	31796 melanomas, 221 non-melanoma	NR	21 experienced1 inexperienced	Kreusch 1992 dermoscopy for melanomaExperienced:Se: 98.9Sp: 94.1Inexperienced:Se: 97.0Sp: 94.2
Lorentzen 1999,68 Denmark	Department of dermatology	ABCD rule of dermoscopy	232	NR	84 experienced4 inexperienced	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 59.0Sp: 92.0ABCD rule of dermoscopy (cut-point ≥5.45)Se: 41.0Sp: 98.0
Lorentzen 2000,53 Denmark	Department of dermatology	ABCD rule of dermoscopy	25864 melanoma, 194 non-melanoma	NR	3Experienced	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 70.7Sp: 88.0
Luttrell 2012,54 Austria	Department of dermatology	AC rule for dermoscopy	20025 melanoma, 178 non-melanoma	NR	17Lay persons	AC rule for dermoscopySe: 91.2Sp: 94.0
Mackie 2002,55 Scotland	Pigmented lesion clinic	The 3-colour dermoscopy test	12669 melanoma 57 non-melanoma	NR	3Experienced	The 3-colour dermoscopy testSe: 97.0Sp: 55.0
McGovern 1992,41 USA	Dermatology clinic	7-point checklist (clinical)ABCD clinical rule	23716 malignant, 221 non-melanoma	N=17950% maleMean age: 44	NR	7-point checklist (clinical)Se: 0.44Sp: 0.94
Menzies 1996,85 Australia	Melanoma unit	Menzies 1996 dermoscopy for melanoma	385107 melanomas	NR	NR	Menzies 1996 dermoscopy for melanomaSe: 92.0Sp: 71.0
Menzies 200856		7-point checklist for dermoscopy3-point checklist of dermoscopyMenzies 1996 dermoscopy for melanomaMenzies 2008 dermoscopy for melanomaMenzies 2008 dermoscopy for skin cancer	497105 melanomas, 392 non-melanoma	NR	12Experienced	7-point checklist for dermoscopySe: 41.0Sp: 83.03-point checklist of dermoscopySe: 50.0Sp: 71.0Menzies 1996 dermoscopy for melanomaSe: 54.0Sp: 76.0Menzies 2008 dermoscopy for melanomaSe: 70.0Sp: 56.0Menzies 2008 dermoscopy for skin cancerSe: 95.0Sp: 80.0
Menzies 201357		ABCD rule of dermoscopy7-point checklist for dermoscopy3-point checklist of dermoscopyMenzies 1996 dermoscopy for melanomaCASH dermoscopy algorithmMenzies 2013 dermoscopy for nodular melanoma	465217 melanomas, 248 non-melanoma	NR	12	ABCD rule of dermoscopySe: 81.5Sp: NR7-point checklist for dermoscopySe: 94.4Sp: NR3-point checklist of dermoscopySe: 83.9Sp: NRMenzies 1996 dermoscopy for melanomaSe: 98.4Sp: NRCASH dermoscopy algorithmSe: 41.0Sp: 83.0Menzies 2013 dermoscopy for nodular melanomaSe: 93.0Sp: 70.0
Nachbar 1994,69 Germany	Department of dermatology	ABCD rule of dermoscopy	19469 melanomas	NR	NR	ABCD rule of dermoscopy (cut-point ≥5.45)Se: 92.8Sp: 91.2
Nilles 1994,86 Germany	Department of dermatology	Nilles 1994 dermoscopy for melanoma	26072 melanomas, 188 non-melanoma	NR	NR	Nilles 1994 dermoscopy for melanomaSe: 90.0Sp: 85.0
Osborne 1999,45 UK	Department of Dermatology	Revised 7-Point Checklist (clinical)	778778 melanomas, 0 non-melanoma	N=73335% male	NR	Revised 7-Point Checklist (clinical)False negative rate: 18.5
Piccolo 2014,31 Italy	Department of dermatology	ABCD rule of dermoscopy	16533 melanomas, 129 non-melanoma	N=16559% maleMean age: 43.5	43 dermatologists 1 FP	ABCD rule of dermoscopySe: 91.0Sp: 52.0
Pizzichetta 2002,32 Italy	Department of oncology	ABCD rule of dermoscopy	129	N=123	2Experienced	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 90.0Sp: 43.0ABCD rule of dermoscopy (cut-point ≥5.45)Se: 90.0Sp: 53.5
Rao 199765	Department of dermatology	ABCD rule of dermoscopyABCD clinical rule	73	N=63	4Experienced dermatologists	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 90.0Sp: 57.0ABCD clinical ruleSe: 84.0Sp: 78.0
Skvara 2005,42 Austria	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopy	32563 melanomas, 262 non-melanoma	N=29744% maleMean age: 39	2Experienced dermatologists	ABCD rule of dermoscopy (cut-point ≥4.75)Se: 31.7Sp: 87.37-point checklist for dermoscopySe: 11.1Sp: 95.2
Soyer 2004,33 Italy	Department of dermatology	3-point checklist of dermoscopy	23168 melanomas, 163 non-melanomas	N=22549% male	6Inexperienced	3-point checklist of dermoscopySe: 96.3Sp: 32.8
Stolz 1994,70 Germany	Department of dermatology	ABCD rule of dermoscopy	157	NR	NR	ABCD rule of dermoscopy(cut-point ≥5.45)Se: 97.9Sp: 90.3
Strumia 2003,34 Italy	Department of dermatology	ABCD rule of dermoscopyABCDE rule (dermoscopy)	49	NR	2
Thomas 1998,6 France	Department of dermatology	ABCDE clinical rule	1140	NR	NR	ABCDE clinical rule (cut-point ≥2)Se: 89.3Sp: 65.3
Unlu 2014,43 Turkey	Department of dermatology	ABCD rule of dermoscopy7-point checklist for dermoscopy3-point checklist of dermoscopyCASH dermoscopy algorithm	11524 melanomas, 91 non-melanoma	N=11549% maleMean age: 39	3Experienced dermatoscopists	ABCD rule of dermoscopySe: 91.6Sp: 60.47-point checklist for dermoscopySe: 79.1Sp: 62.63-point checklist of dermoscopySe: 87.5Sp: 65.9CASH dermoscopy algorithmSe: 91.6Sp: 64.8
Wadhawan 2011,59 USA	Images from library of skin cancer	7-point checklist for dermoscopy	347	NR	NR	7-point checklist for dermoscopySe: 87.3Sp: 71.3
Walter 2013,44 UK	Family practice	7-point checklist (clinical)Revised 7-point checklist (clinical)	143636 melanomas, 1400 non-melanoma	N=118235.9% maleMean age: 44.7	NR	7-point checklist (clinical)Se: 80.6Sp: 61.7Revised 7-point checklist (clinical)Se: 91.7Sp: 33.1
Zalaudek 2006,60 29 Countries	Pigmented lesion clinic	3-point checklist for dermoscopy	15044 malignant, 106 non-melanoma	NR	150Varying levels of experience	3-point checklist for dermoscopySe: 94.0Sp: 71.9
Impact Analysis Studies
Author year, Country	Study design	Participant selection	Lesions	Intervention	Control	Outcomes
Westerhoff 2000,62 AustraliaPrimary care	Controlled before and after	74 FPs	n=100 (50 melanoma, 50 non-melanoma)Selected randomly from the Sydney Melanoma Unit image database	Educational intervention. FPs given educational material on Menzies 1996 rule, followed by a 1-hourPresentation on surface microscopy	Usual care	Correct diagnosis of melanoma, percent (SD):Intervention 75.9 (12)Control 54.8 (22)Correct diagnosis of non-melanoma, percent (SD):Intervention 57.8 (14)Control 55.8(15)
Walter 2012,63 EnglandPrimary care	RCT	15 FP practices	1580 from 1297 patients	Patients assessed using the MoleMate system (SIAscopy with primary care scoring algorithm)	Best practice (clinical history, naked eye examination, 7-checklist clinical)	Primary, appropriateness of referral (defined as the proportion of referred lesions that secondary care experts decided to biopsy or monitor): no statistically significant difference between intervention and control; 56.8% vs 64.5%; difference −8.1% (95% CI −18.0% to 1.8%).Secondary:Appropriate management of benign lesions in primary care: no statistically significant difference between intervention and control (99.6% vs 99.2%, p=0.46).Agreement with an expert decision to biopsy or monitor: no statistically significant difference between intervention and control (98.5% vs control 95.7%, p=0.26).Patient satisfaction: more intervention patients ranked their consultation very good/excellent for thoroughness than control (83.1% vs 71.2%, p<0.001).Patient anxiety: no statistically significant difference between intervention and control in anxiety scores (32.56 vs 34.72, p=0.013)
Argenziano 2006,72 Spain, ItalyPrimary Care	RCT	73 FPs	2548 lesions from 2522 patients presenting to primary care with a pigmented skin lesion.1203 lesions in dermoscopy group (six melanoma)1345 lesions in control group (six melanoma)	Use of dermoscopy in addition to ‘naked-eye’ lesion screening.Both groups received a 4 hours educational intervention incorporating clinical examination and use of the 3-point checklist (dermoscopy algorithm)	Naked-eye screening alone.	Primary outcome:Referral accuracy of PCPs (defined as the ability of the PCP to correctly determine a lesion may be malignant or benign, when the gold standard is diagnosis by a second expert clinician) reported as sensitivity, specificity, PPV, NPV.Significant difference in sensitivity (dermoscopy 79.2%, naked eye 54.1%, p=0.002) and negative predictive value (dermoscopy 9801%, naked eye 95.8%, p=0.004)Secondary outcome:Number of malignant tumours missed by PCPs using naked-eye examination (n=23) and using dermoscopy (n=6) (p=0.002)

ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Color; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6mm) Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, colour, architecture, symmetry, and homogeneity; CPR, clinical prediction rules, ELM, epiluminescence microscopy; FP, family physicians; PCP, primary care physicians; PPV, positive predictive value; NPV, negative predictive value; NR, Not reported; RCT, randomised controlled trials; Se, sensitivity; Sp, specificity.

Summary of CPRs identified

Of the 24 rules identified, four were clinical (ie, naked eye), 17 were dermoscopic and the remaining three used novel diagnostic technologies. The most commonly applied clinical CPR was the ABCDE rule (five studies),6 15 28 64 65 while for dermoscopy the most common were the Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) rule of dermoscopy (23 studies)14 25 26 29 31 32 39 42 43 47–49 52 53 57 65–70 and the 7-point checklist for dermoscopy (17 studies).14 25 26 29 35 37 42 43 46–50 52 56 57 59

Each of the elements included in the 24 rules identified are presented in table 3. All four clinical rules included the elements of diameter and colour variegation (table 3 and see online supplementary appendix 1). The most frequently included elements in the 17 dermoscopic rules were multiple colours (13 rules), asymmetry (12 rules) and streaks (10 rules) (table 3 and see online supplementary appendix 1).

Table 3

Comparison of elements in clinical prediction rules for malignant melanoma

(a) Clinical rules
	Clinical CPR name
Elements	ABCD	ABCDE	7-point checklist	Revised 7-point checklist
Asymmetry	X	X		X
Border irregularity	X	X	X
Colour variegation	X	X	X	X
Diameter (>6 mm)	X	X	X (>7 mm)	X (>7 mm)
Evolving (eg, size, shape, colour)		X	X (size)	X
Altered sensation			X	X
Inflammation			X	X
Crusting, bleeding			X	X
Cut-point	≥1	≥1 or ≥2	≥3	≥3

ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6mm)Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, color, architecture, symmetry, and homogeneity; CPR, clinical prediction rules; FFM, features for melanoma.

Methodological quality of validation studies

Based on the CHARMS checklist, the quality of included studies varied.21 All studies had weaknesses in study design and quality assessment was often hindered by poor reporting of methods. The studies had reasonable sample sizes and all provided adequate definitions of the outcome of interest. However, a number of important weaknesses were identified. None of the studies reported on missing data and key performance measures of model performance (eg, calibration) were often missing. Derivation studies typically reported information on model development, in terms of selection of candidate predictors, selection of predictors during modelling, and model evaluation. However, often the methods applied introduced a strong risk of bias, for example, a number of studies described splitting the original sample into a development and validation sample which is considered statistically inefficient and results in overfitting of the model.21 Full results of the quality assessment are shown in online supplementary appendix 3.

Predictive accuracy of melanoma CPRs

The results for the most commonly applied CPRs, the ABCD rule and the 7-point checklist are presented here. The sensitivity and specificity of all rules identified (including the ABCDE clinical rule, the seven features for melanoma rule and Menzies dermoscopy for melanoma rule) are summarised in table 4.

Table 4

Sensitivity and specificity of all clinical and dermoscopy CPRs

Rule name	Cut-point	Sensitivity*	Specificity*
Clinical rules
ABCDE	≥1	Two studies0.47–0.92 (mean 0.70)	One study0.56
	≥2	0.85	0.44
7-point checklist	≥3	Three studies0.44–0.86 (mean 0.70)	Three studies0.62–0.94 (mean 0.74)
Revised 7-point checklist	≥3	0.92	0.33
ABCD rule	≥1	0.84	0.78
Dermoscopic rules
ABCD rule	≥4.75	Meta-analysis (eight studies)0.85 (95% CI 0.73 to 0.93)	Meta-analysis (eight studies)0.72 (95% CI 0.65 to 0.78)
	≥5.45	Four studies0.73–0.98 (mean 0.85)	Four studies0.46–0.91 (mean 0.79)
	≥4.2	0.88	0.64
7-point checklist	≥3	Meta-analysis (11 studies)0.77 (95% CI 0.61 to 0.88)	Meta-analysis (11 studies)0.80 (95% CI 0.59 to 0.92)
Menzies 1996 for melanoma	≥1	Six studies0.85–0.95 (mean 0.91)	Six studies0.38–0.78 (mean 0.69)
3-point checklist	≥1	Five studies0.50–0.96 (mean 0.84)	Four studies0.31–0.72 (mean 0.55)
Seven features for melanoma (7FFM)	≥2	Five studies0.69–0.95 (mean 0.86)	Five studies0.74–0.86 (mean 0.82)
CASH algorithm	≥8	Three studies0.41–0.92 (mean 0.73)	Three studies0.65–0.97 (mean 0.82)
The 3-colour test	≥3	Two studies0.77–0.97 (mean 0.87)	Two studies0.55–0.90 (mean 0.73)
Revised 7-point checklist	≥1	0.88	0.28
Kreusch 1992	Not reported	0.99	0.94
Nilles 1994	Not reported	0.90	0.85
Menzies 2008 for melanoma	≥1	0.70	0.56
DynaMel algorithm	≥3	0.77	0.98
Menzies 2008 for skin cancer	≥0 (high sensitivity); ≥1 (high specificity)	0.95	0.80
Simplified ABC-point list	≥4	0.90	0.87
AC rule	Not reported	0.91	0.94
Emery 2010 SIAscopy	≥6	0.50	0.84
Guitera RCM 2012	Not reported	0.88	0.71
ABCDE rule	Not reported	Not reported	Not reported

*Where sensitivity and specificity are presented for more than one study, the range and mean are presented. Where meta-analysis was possible, values from meta-analysis are presented with 95% CIs.

ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Color, or a large (greater than 6mm) Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, color, architecture, symmetry, and homogeneity; CPR, clinical prediction rules; RCM, reflectance confocal microscopy.

Clinical (naked eye) CPRs for melanoma diagnosis

Four studies validating the ABCDE clinical rule6 15 28 64 and one validating the ABCD clinical rule65 were included. There was insufficient data to conduct any meta-analysis. Rao et al reported a sensitivity of 0.84 and specificity of 0.78, for an unspecified cut-point.65

Six studies validating the original and revised 7-point checklist were included. There was insufficient data to conduct a meta-analysis. Of the four studies validating the original 7-point checklist (cut-point ≥3), three reported sensitivity (range 0.44–0.86, mean 0.70) and specificity (range 0.62–0.94, mean 0.74).40 41 44 Only one of the four studies validating the revised 7-point checklist (cut-point ≥1) reported sensitivity (0.92) and specificity (0.33) (table 4).44

Dermoscopic CPRs for melanoma diagnosis

ABCD rule of dermoscopy

The ABCD rule of dermoscopy (also described as the ABCD rule of Stolz), was validated in 23 studies, 15 of which applied a cut-point of >4.75 (indicating a suspicious lesion) and six studies a cut-point of 5.45 (highly suggestive for melanoma). At a cut-point of >4.75, eight studies provided sufficient information for meta-analysis,42 43 47 52 65 71 resulting in a pooled sensitivity of 0.85 (95% CI 0.73 to 0.93) and specificity of 0.72 (95% CI 0.65 to 0.78) (figure 1A, B). This indicates that at this cut-point, the dermoscopy CPR is more useful for ruling out rather than ruling in melanoma, with a higher pooled sensitivity than specificity. I2 were high (>70%), indicating a high degree of heterogeneity. Of the seven studies excluded from meta-analysis, sensitivity ranged from 0.71 to 0.91 (mean 0.79) and specificity ranged from 0.43 to 0.92 (mean 0.72). None of the six studies that applied a cut-point of 5.45 were suitable for meta-analysis. From four studies that presented the information, sensitivity ranged from 0.73 to 0.98 (mean 0.85) and specificity ranged from 0.46 to 0.91 (mean 0.79) (table 4).

Figure 1

(A) Diagnostic accuracy ABCD rule with dermoscopy—pooled sensitivity and specificity (eight studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic.

Seven-point checklist for dermoscopy

The 7-point checklist for dermoscopy was validated in 18 studies, 17 of which applied a cut-point of 3. 11 studies provided sufficient information for meta-analysis, revealing a pooled sensitivity of 0.77 (95% CI 0.61 to 0.88) and pooled specificity of 0.80 (95% CI 0.59 to 0.92) (See figure 2A, B).25–27 35 37 42 43 47 50 52 71 There was a high degree of heterogeneity in the results (I2>90%). Removing two outliers27 50 made minimal difference to the pooled result. Only one study validated the revised 7-point checklist for dermoscopy and reported sensitivity 0.78 and specificity 0.65 for a cut-point of 3 (table 4).27

Figure 2

(A) Diagnostic accuracy of 7-point checklist with dermoscopy—pooled sensitivity and specificity (11 studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic; ROC, receiver operating characteristic.

Impact analysis

We identified three unique studies that examined the impact of a melanoma CPR on processes of care (melanoma diagnosis and referrals), however, no patient outcomes were examined (table 2).62 63 The methodological quality of these studies is presented in online supplementary appendix 4.

Using a controlled before–after design, Westerhoff et al investigated the impact of an educational intervention about the Menzies 1996 rule on melanoma diagnosis by family physicians (FP). The control group did not receive the training. Postintervention, there was a significant improvement in melanoma diagnosis (75.9% vs 62.7%, p<0.001). No significant improvement was seen in the control group (54.8% vs 53.7%, p=0.59).62

Walter et al conducted a RCT to compare the use of a new imaging device, the MoleMate system (SIAscopy with a primary care scoring algorithm), to current best practice (clinical history, naked-eye examination, 7-point checklist). The authors found no difference between these two approaches in terms of appropriate referrals (the proportion of referred lesions that secondary care experts biopsied or monitored) to urgent skin cancer clinics (intervention 56.8% vs control 64.5% p=0.11) or the proportion of benign lesions appropriately managed in primary care (intervention 99.6% vs control 99.2%, p=0.46).63

Argenziano et al's RCT,72 involved primary care physicians first attending a 1-day training course describing the ABCD rule (cut-point unspecified) and the 3-point checklist. They were then randomly assigned to assess patients with skin lesions, either by clinical (ie, naked eye) examination, or by dermoscopy using the 3-point checklist. The referral assessments were checked for accuracy by dermatologists. The dermoscopy arm demonstrated a 25% improvement in the sensitivity of primary care referrals of pigmented lesions compared with the naked-eye examination (79.2% vs 54.1%, p=0.002), without a reduction in specificity (71.8% vs 71.3%, p=0.915).72

Discussion

Summary of findings

This systematic review identified 48 studies validating a total of 24 CPRs for melanoma. Overall, the majority of validation studies used dermoscopic CPRs, with very few studies validating clinical CPRs. Meta-analysis of the dermoscopic CPRs demonstrated relatively high pooled estimates of sensitivity (0.77–0.86). The clinical implication is that applying dermoscopy CPRs will enable low-risk patients to be observed and kept under review in a primary care setting, without immediate referral for excision to secondary care. Meta-analysis was not possible for clinical CPRs but individual studies report variable sensitivity, ranging from 0.44 to 0.86. Three impact analysis studies were identified, with two reporting an improvement in melanoma diagnosis with the use of a CPR.

Context of previous research

The sensitivities and specificities we report indicate that currently available CPRs are reasonably good at ruling out melanoma. The pooled sensitivity of the ABCD rule for dermoscopy (cut-point of >4.75) was 0.85 (95% CI 0.73 to 0.93), higher than that of the 7-point checklist for dermoscopy (0.77, 95% CI 0.61 to 0.88). While this evidence would support the use of such rules in prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients, there are a number of important caveats that may prevent their adoption in primary care.

Melanoma is a high-stakes condition, one which doctors tend to be cautious in diagnosing, often preferring to excise a benign lesion rather than to miss a potentially fatal cancer.73 In such cases, a CPR with near perfect sensitivity would be desirable, however, it has been argued that a lower sensitivity should not prevent CPR use unless usual decisions, made without the rule, are demonstrably better.74 Our results are comparable with previous systematic reviews focused on melanoma diagnosis across healthcare settings in highlighting that dermoscopic CPRs are demonstrably better in terms of diagnostic accuracy in comparison with inspection by the naked eye.16 75 However, even a rule with almost 100% sensitivity may not be adopted. For instance, implementation of the Canadian CT Head Rule, despite 100% sensitivity in validation studies, did not result in a reduction in imaging rates, with clinicians' reporting unease with certain components of the rule and fear of missing a high-stakes diagnosis as reasons for not adopting the CPR.76

Before considering whether to use a CPR in clinical practice, it is essential that its performance be established through external validation (ie, in settings outside where it was derived). We identified a number of external validation studies in this review, however, in keeping with much CPR research, the reporting of these studies was often poor.77 78 In particular, the common issues of limited acknowledgement and handling of missing data and key performance measures of prediction models, that is, calibration, being omitted was encountered.77 The lack of available data in some papers meant not all studies could be combined in the meta-analysis, meaning the sensitivities and specificities reported here are not based on the totality of existing evidence. Furthermore, we were unable to assess diagnostic accuracy at different cut-point thresholds for respective CPRs. Improved reporting of CPRs at cut-point thresholds will enable pooling of diagnostic accuracy data, and will provide more robust measures of diagnostic accuracy. After validation, impact analysis studies are undertaken to determine the impact of the implementation of a CPR on processes and outcomes of care. Despite increasing interest in developing and validating CPRs relevant to primary care, relatively few have undergone impact analysis.79 Despite the large number of CPRs identified in this review, we identified only three impact analysis studies, with only two studies reporting an improvement in correct melanoma diagnosis in primary care as a result. Arguably, the dearth of well conducted and clearly reported external validation and impact analysis studies undermines trust in the use of such rules in practice.77

Current NICE guidelines for melanoma detection and management recommend dermoscopy of any suspicious lesion, advising against using computer-assisted diagnostic tools (NG14) while promoting use of the weighted 7-point checklist in primary care to guide referral (NG12).20 Based on the findings of this review, the ABCD rule for dermoscopy had a higher sensitivity than the seven point for dermoscopy checklist at their respective cut-points, indicating its potential for use in primary care. Dermoscopy, however, requires training and equipment, and is less commonly performed in primary care. Evidence suggests that dermatologists have better diagnostic accuracy than primary care physicians.18 Three studies retrieved in our search assessed dermoscopy CPR performance when applied by non-experts, with two studies reporting that the CPRs performed well overall when used by non-experts, mainly primary care physicians.49 66 72 Westerhoff et al62 and Blum et al80 demonstrated that training primary care physicians to use dermoscopy with CPRs showed significant improvement in the diagnosis of melanoma compared with naked eye inspection. Alongside the use of CPRs, training in dermoscopy would seem to be a strategy that will enhance diagnostic accuracy of melanoma in the future particularly in light of emerging evidence of differences in dermoscopic features of melanoma such as head and neck melanoma.81 It has also been highlighted that significant efforts are needed to standardise and improve dermoscopic terminology to more broadly promote the use of dermoscopy in the primary care setting.82 Of the 24 rules identified in this review, four were clinical (ie, naked eye) and 17 were dermoscopic. Owing to the limited number of studies and available data, no meta-analysis of clinical CPRs could be conducted. The range of reported sensitivities from individual studies indicates that there is insufficient evidence to recommend their use in practice.

Strengths and limitations of our study

The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisability and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. However, the findings of this systematic review need to be interpreted in the context of the limitations of the original studies. The lack of available data in some papers meant not all studies could be combined in the meta-analysis. A number of studies that validated CPRs and algorithms using novel diagnostic technologies which incorporated computerised image analysis and artificial intelligence were excluded from the review as routine use of these are not currently recommended in UK NICE clinical guidelines. Significant heterogeneity existed between the studies with respect to differences in the study populations and application of the CPR. Finally, individual patient data that enables pooling of risk scores at the different cut-points would enable researchers to explore the clinical use of applying risk scores at different cut-points with the purpose of assessing the role of melanoma CPRs at the different diagnostic thresholds of ‘ruling out’ (using highest pooled sensitivity) or ‘ruling in’ (using highest pooled specificity) of respective melanoma CPRs.

Implications for practice and future research

Early detection followed by curative surgery greatly improves the prognosis of malignant melanoma. As the incidence of melanoma skin cancer increases, primary care physicians are increasingly required to screen for melanoma.12 Therefore, efforts to increase the early detection of melanoma must focus on supporting primary care physicians in performing skin cancer screenings with recent evidence highlighting the benefits of developing targeted screening strategies in high-risk patients in primary care.18 83 This systematic review identified 24 separate clinical (naked eye) and dermoscopic CPRs, with some overlap in the included the elements. Our analysis highlights that dermoscopic CPRs have reasonable sensitivity, with the ABCD rule for dermoscopy having better sensitivity than the 7-point checklist for dermoscopy. Further development of new rules is unlikely to benefit the field of research. An increased emphasis on better reporting of validation studies, particularly at different cut-point thresholds, would allow for the conduct of more robust diagnostic accuracy meta-analysis to inform decision making. Further methodologically robust RCTs are necessary also to examine the impact of implementing CPRs in clinical practice, in terms of patient outcomes, physician behaviour, processes of care and cost-effectiveness. Finally, while guidelines promote the use of dermoscopy in the assessment of pigmented skin lesions, there needs to be greater emphasis on training in primary care on this examination technique.

Conclusion

This systematic review and meta-analysis shows that dermoscopic CPRs have reasonably high pooled estimates of sensitivity and may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients. The ABCD rule of dermoscopy has higher pooled sensitivity than the 7-point checklist for dermoscopy, when consideration about ruling out melanoma is being made. A focus on impact analysis may help translate melanoma CPRs into useful and effective triage tools for use in primary care.