The effect of a dermato-oncological training programme on the diagnostic skills and quality of referrals for suspicious skin lesions by general practitioners.

BACKGROUND: The rising incidence rates of skin cancer (SC) lead to an enormous burden on healthcare systems. General practitioners (GPs) might play an important part in SC care, but research has shown poor clinical recognition of SC, leading to a high rate of potentially unnecessary referrals.
OBJECTIVES: The aim of this study was to evaluate if a dermato-oncological training programme (DOTP) for GPs improved their diagnostic skills and quality of referrals.
METHODS: Out of 194 GPs in the Nijmegen area, 83 (42·8%) followed a DOTP on SC. Referrals from both a trained cohort (TC) and two cohorts of untrained GPs [untrained present cohort (UPC) and untrained historical cohort (UHC)] were included. Data on diagnostic skills, quality of referrals and the number of potentially unnecessary referrals were evaluated.
RESULTS: A total number of 1662 referrals were analysed. The referral diagnosis was correct more often in the TC (70·3%) compared with the UPC (56·2%; P < 0·001) and the UHC (51·6%; P < 0·001). Furthermore, the TC also provided a better lesion description, mentioned a diagnosis more often in their referral letters and more often performed diagnostics before referral. In addition, fewer potentially unnecessary referrals were identified in the TC compared with the UPC (62·7% vs. 73·7%; P < 0·001) and the UHC (75·2%; P < 0·001).
CONCLUSIONS: GPs who followed a DOTP had better diagnostic skills and quality of referrals than untrained GPs, leading to fewer potentially unnecessary referrals. This might enhance a more efficient use of the limited capacity in secondary dermatological care and consequently lead to lower healthcare costs.

Skin cancer (SC) is the most common malignancy worldwide and the incidence rates are rapidly rising, leading to an enormous and growing burden on healthcare systems.1, 2, 3, 4 Most patients with SC initially visit a general practitioner (GP), often followed by referral to a dermatologist. Therefore, GPs play an important part in SC care and optimal recognition and referral are considered of vital importance to optimize efficacy in SC care. Previous research identified a rather poor clinical recognition of (pre)malignant skin tumours by GPs, leading to a high rate of potentially unnecessary referrals.5 As previously suggested, better training of GPs might improve their diagnostic skills, diminish the number of potentially unnecessary referrals and thereby optimize efficacy in SC care.6, 7, 8, 9 The aim of this study was to evaluate if a dedicated dermato‐oncological training programme (DOTP) for GPs improves the diagnostic skills and quality of referrals by GPs.

Materials and methods

In this cohort study, we studied the effect of a DOTP on the diagnostic skills and quality of referrals by GPs.

The dermato‐oncological training programme

A DOTP was developed as part of a project called ‘Suspicious Skin Lesions’ in the Nijmegen area of the Netherlands. The aim of this project was to improve SC care by GPs and transfer low‐risk nonmelanoma SC care to primary care, while maintaining high‐quality care. Previous research showed GPs and dermatologists to be generally positive about this transfer of care.9 At the beginning of the project, regional guidelines were drawn up concerning the diagnostics and treatment of (pre)malignant skin lesions in primary care, and when a GP should refer a patient to a dermatologist (Appendix S1; see Supporting Information). These regional guidelines were based on existing (inter)national guidelines.10, 11, 12, 13, 14, 15 The DOTP consisted of: (1) instruction on the project (aims) and guidelines; (2) a mandatory online course on SC (including an examination); and (3) two optional live courses on SC and dermoscopy. The mandatory online course is a nationally developed course on SC, developed by the Dutch College of General Practitioners (NHG) together with SC experts,16 including sections on diagnostics (including punch biopsy), therapy and counselling of patients regarding SC care and prevention. The duration of this course was approximately 2 h.

Study population

All GPs with their own practice in the Nijmegen area (n = 194) received an invitation to participate in the project. In total, 83 GPs (42·8%) participated in the project and finished the DOTP, following the above‐mentioned criteria. Of these, 46 (55·4%) and 21 (25·3%) finished the optional SC and dermoscopy course, respectively, as well. Referrals from GPs to dermatologists in two hospitals in the Nijmegen area (Radboud University Medical Centre and Canisius‐Wilhelmina hospital) were included chronologically in two different time periods after the DOTP was finished (September–October 2018 and May–June 2019). For comparison, referrals from both trained and untrained GPs were included, forming a trained cohort (TC) and an untrained present cohort (UPC). Referrals in these two time periods were included to study the sustainability of the effect of the DOTP over time. In addition, a comparison was made using a historical control group from previous research on GP referrals in the same population (2008–2010),5 forming an untrained historical cohort (UHC). The following referrals were excluded: referrals by GPs in training under supervision by a GP who participated in the project, and referrals by locum GPs (some locum GPs participated in the courses, but could not participate in the project; therefore data could not be obtained). In addition, nine trained GPs were excluded because although they followed the obligatory part of the DOTP, they did not consent to further participation (also shown in Figure 1).

Figure 1

Flowchart showing the inclusion of GPs and referral letters (same geographical area in both time periods). DOTP, dermato‐oncological training programme; GP, general practitioner.

Study outcomes and data collection

Firstly, the agreement between the referral diagnosis made by GPs (primary diagnosis mentioned in the referral letter) and the final diagnosis made by the dermatologist was assessed. In cases in which no diagnosis was mentioned in the referral letter, letters were excluded from this specific analysis. When available, a histopathological diagnosis was used as the final diagnosis. In cases of no histopathological diagnosis, the clinical diagnosis made by the dermatologist was used as the final diagnosis. In addition, the positive predictive values were also assessed by comparing the full differential diagnosis provided by GPs in the referral letter with the final diagnosis made by the dermatologist.

Secondary outcomes were: (1) the proportion of proper dermatological descriptions used in the referral letters; (2) the number of diagnostic procedures (dermoscopy, punch biopsies) performed by the GPs; (3) the number of additional (pre)malignant lesions found by the dermatologist; and (4) the proportion of potentially unnecessary referrals.

The description of the lesion(s) was evaluated using the generally accepted Dutch model to describe skin lesions (PROVOKE), in which a skin disorder is supposed to be described by mentioning seven characteristics (anatomical localization, distribution, size, shape, border, colour and lesion morphology). Firstly, the description of all seven characteristics were separately analysed. In addition, the aggregated description was analysed on completeness. For this, a ‘proper description’ was defined as the description of three or more characteristics in the referral letter.

Potentially unnecessary referrals were lesions that probably could have been treated in primary care, namely low‐risk basal cell carcinoma (BCC), low‐risk Bowen disease (BD), actinic keratosis (AK) and all benign diagnoses without any complicating factors (see below) in accordance with the guideline of the Dutch College of General Practitioners and regional project guidelines.10 A full overview of referral criteria is provided in Appendix S1 (see Supporting Information).

Data were collected using both the referral letters and medical records (including histopathology reports). Data extraction was performed by the same researcher in all cases in both hospitals (E.M.). Data verification on accuracy and inconsistencies was performed by a separate researcher (S.F.K.L.) in 10% of the cases. In cases of inconsistency (0·19%) a consensus meeting was planned and the full dataset was additionally checked on this specific inconsistency.

Statistical analysis

A power analysis was performed to determine the minimum number of referrals to be included to expect to find 40% correct diagnoses in the referral letters compared with the final diagnoses made by dermatologists (as previously reported)5 and to consider a minimum difference of 10% in correct diagnoses between the groups as a clinically relevant result with power (1 – β) set at 0·80 and α = 0·05 (two‐tailed). At least 404 referrals per cohort needed to be included. Because most, but not all, variables analysed were included in the UHC, this cohort was excluded in case a necessary variable was missing for a specific analysis. Categorical and continuous variables were described using numbers and percentages or mean and standard deviation, respectively. Positive predictive values (percentage of GP diagnoses that were in line with the final diagnosis) were calculated per lesion type. To test for significant differences between cohorts, a Chi‐square test or independent‐samples Student's t‐test was applied for categorical and continuous variables, respectively. To compare the effect of the cohort on the likelihood for a correct diagnosis a mixed‐model logistic regression analysis was used, as this takes the clustering of multiple referrals per GP into account. As the specific GPs from the UHC are unknown, the UHC–TC and UHC–UPC comparisons were performed with standard logistic regression analysis. To give insight into the consequences of taking the clustering effect into account, for the UPC–TC comparison results from the standard as well as the mixed‐model logistic regression are presented. No imputation of missing data was performed and only the available data per variable were analysed, because the amount of missing data was small. Data collection and statistical analysis were performed by using the Statistical Package for Social Sciences (SPSS®) for Windows, version 25·0 (IBM Corp., Armonk, NY, USA).

Ethics

Due to the nature of this study, evaluating an existing healthcare project, no formal medical ethical approval was necessary according to Dutch law.

Results

Referral characteristics

A total of 1662 referral letters were included, of which 451 (27·1%) were in the TC, 476 (28·6%) were in the UPC and 735 (44·2%) were in the UHC (Figure 1). Referred patients had a mean age of 59·7 (± 18·8) years and the majority were women (n = 927; 55·8%). No statistically significant differences regarding sex and age distribution were detected between the three cohorts. The most common reasons for referral were lesions (clinically) diagnosed as BCC by GPs (n = 483; 29·1%), followed by melanocytic naevus (n = 309; 18·6%) and AK (n = 230; 13·8%), as shown in Table S1 (see Supporting Information). In 272 (16·4%) referral letters, no diagnosis was mentioned by the GP, and when comparing the different cohorts this was significantly lower in the TC (n = 30; 6·7%) compared with the UPC (n = 65; 13·7%; P < 0·001) and the UHC (n = 177; 24·1%; P < 0·001). A statistically significant difference was also seen between the UPC and UHC (P < 0·001). For an overview per group, refer to Tables S2–4 (see Supporting Information). The median number of included referral letters per GP was two (range 1–24) in the TC and UPC (not known for UHC, because personalized GP data were unavailable).

Lesion(s) description

Data on the description of each PROVOKE characteristic in the different cohorts is shown in Table 1. A proper lesion(s) description (at least three out of the seven PROVOKE characteristics) was present more often in the TC (n = 198; 43·9%), compared with the UPC (n = 167; 35·1%; P = 0·006) and UHC (n = 176; 23·9%; P < 0·001), respectively. When comparing the control groups, the UPC contained a higher number of proper lesion(s) descriptions compared with the UHC (n = 167; 35·1% vs. n = 176; 23·9%; P < 0·001).

Table 1

‘PROVOKE’ description characteristics

Description characteristic	TC, n (%)	UPC, n (%)	UHC, n (%)	P‐valuea
Anatomical localization	445 (98·7)	451 (94·7)	705 (95·9)	0·001 / 0·007 / 0·34
Distribution	7 (1·6)	2 (0·4)	9 (1·2)	0·08 / 0·63 / 0·15
Size	146 (32·4)	142 (29·8)	152 (20·7)	0·40 / < 0·001 / < 0·001
Shape	73 (16·2)	56 (11·8)	50 (6·8)	0·05 / < 0·001 / 0·002
Border	58 (12·9)	51 (10·7)	61 (8·3)	0·31 / 0·01 / 0·15
Colour	215 (47·7)	219 (46·0)	211 (28·7)	0·61 / < 0·001 / < 0·001
Morphology	128 (28·4)	92 (19·3)	127 (17·3)	0·001 / < 0·001 / 0·37

aTC vs. UPC/TC vs. UHC/UPC vs. UHC.

TC, trained cohort; UHC, untrained historical cohort; UPC, untrained present cohort.

Bold represents significant outcomes

Diagnostic procedures performed by general practitioners

There were 595 lesions referred as BCC, squamous cell carcinoma or BD (TC: n = 216, UPC: n = 170 and UHC: n = 209). The percentage of punch biopsies taken was significantly higher in the TC compared with the UPC (n = 67; 31·0% vs. n = 19; 11·2%; P < 0·001) and UHC (n = 9; 4·3%) (P < 0·001). Also, a punch biopsy was performed more often in the UPC compared with the UHC (P = 0·01). GPs mentioned the use of dermoscopy in 11·3% (n = 51) of referrals in the TC, and 4·4% (n = 21) in the UPC (P < 0·001). The level of dermoscopy use in the UHC was not known; therefore, comparison was not possible.

Final diagnosis

The most common final diagnosis was BCC (n = 377; 22·7%), followed by AK (n = 362; 21·8%), and melanocytic naevus (n = 257; 15·5%) (Table S5; see Supporting Information). Of all lesions diagnosed as (pre)malignancies with the exception of AK (n = 544), a histopathological diagnosis was present in 97·6% (n = 531).

Agreement on diagnosis between the general practitioner and dermatologist

As shown in Table 2, the percentage of correct primary diagnoses by GPs was significantly higher in the TC (n = 296/421; 70·3%) compared with the UPC (n = 231/411; 56·2%; P < 0·001) and the UHC (n = 288/558; 51·6%; P < 0·001). There was no significant difference between the UPC and UHC (P = 0·16). After exclusion of all referrals containing a primary diagnosis which was already histopathologically proven by the GP, the number of correct primary diagnoses in the TC (n = 222/345; 64·3%) remained significantly higher compared with both the UPC (n = 203/383; 53·0%; P = 0·002) and UHC (n = 276/548; 50·4%; P < 0·001), while the UPC and UHC remained comparable (P = 0·43). Also, after correction for a potential clustering effect based on the number of included referral letters per GP using a mixed‐model logistic regression analysis (with GP as level, correct or incorrect diagnosis as outcome and trained/untrained as determinant), comparable outcomes on agreement on diagnoses between GP and dermatologist were found (Table 2).

Table 2

Comparison of cohort effect on correct diagnoses, based on logistic regressiona

	TC	UPC	UHC
Proportion of correct diagnoses per cohort, n/n (%)	296/421 (70·3%)	231/411 (56·2%)	288/558 (51·6%)
Comparison between cohorts	OR	95% CI	P‐value
TC vs. UPC, mixed‐model LR	1·882	1·390–2·584	< 0·001
TC vs. UPC, standard LR	1·845	1·387–2·456	< 0·001
TC vs. UHC, standard LR	2·000	1·700–2·899	< 0·001
UPC vs. UHC, standard LR	1·203	0·931–1·554	0·157

aReferral letters without a diagnosis mentioned were excluded from this analysis.

CI, confidence interval; LR, logistic regression; OR, odds ratio; TC, trained cohort; UHC, untrained historical cohort; UPC, untrained present cohort

An overview of referral diagnoses and corresponding final diagnoses for the TC, UPC and UHC is shown in Tables S2–4 (see Supporting Information). In those cases where the GP diagnosis was considered correct when the right diagnosis was mentioned somewhere in the differential diagnosis, 74·3% (n = 313) were correct in the TC, and 62·8% (n = 258) in the UPC (P < 0·001). The full differential diagnosis was not known for the UHC; therefore, comparison was not possible. Positive predictive values per lesion type between the three cohorts are shown in Table 3.

Table 3

Positive predictive value per diagnosis

Diagnosis	TC (%)	UPC (%)	UHC (%)	P‐valuea
BCC	67·9	52·2	48·4	0·008 / < 0·001 / 0·50
SCC	40·8	19·2	31·6	0·01 / 0·48 / 0·06
Melanoma (in situ)	75·4	65·3	66·7	0·14 / 0·77 / 0·87
AK	60·0	50·0	73·5	0·70 / 0·79 / 0·25
BD	75·0	61·5	0·0	0·47 / – / –
(Atypical) naevus/lentigo	77·8	66·7	57·9	0·43 / 0·008 / 0·01
SK	76·5	71·4	82·1	0·22 / 0·67 / 0·22

aTC vs. UPC / TC vs. UHC / UPC vs. UHC.

AK, actinic keratosis; BCC, basal cell carcinoma; BD, Bowen disease; SCC, squamous cell carcinoma; SK, seborrhoeic keratosis; TC, trained cohort; UHC, untrained historical cohort; UPC, untrained present cohort.

Bold represents significant outcomes

Additional (pre)malignant lesions found by the dermatologist

In 15·9% (n = 264) of all referred patients, the dermatologist found 406 additional (pre)malignant lesions (Table S6; see Supporting Information). Additional lesions found mostly included BCC (n = 196; 48·3%), followed by AK (n = 157; 38·7%) and BD (n = 26; 6·4%). The number of patients in whom additional (pre)malignant lesions were found was comparable between the TC and UPC (n = 54; 12·0% vs. n = 64; 13·4%; P = 0·502). This number was higher in the UHC (n = 146; 19·9%) compared with the TC (P < 0·001) and UPC (P = 0·004).

Potentially unnecessary referrals

Table 4 shows the percentage of referrals that were potentially unnecessary based on the final diagnosis. The TC had fewer potentially unnecessary referrals (n = 283; 62·7%) than both the UPC (n = 351; 73·7%; P < 0·001) and the UHC (n = 553; 75·2%; P < 0·001). No significant difference was detected between the UPC and UHC (P = 0·56). Focusing on low‐risk BCC and low‐risk BD specifically, no significant difference was found between the TC and UPC (n = 19/134; 14·2% vs. n = 22/108; 20·3%; P = 0·20), as well as between the TC and UHC (36/167; 21·6%; P = 0·10) and the UPC and UHC (P = 0·81).

Table 4

Potentially unnecessary referrals per cohort

Cohort	Unnecessary referrals	P‐value
TC	283/451 (62·7%)	< 0·001a
UPC	351/476 (73·7%)	0·56b
UHC	553/735 (75·2%)

a P‐value TC vs. UPC and UHC; b P‐value UPC vs. UHC.

TC, trained cohort; UHC, untrained historical cohort; UPC, untrained present cohort

Effect of the dermato‐oncological training programme over time

No significant differences were found regarding proper descriptions of lesion(s), agreement between the referral diagnosis (GP) and the final diagnosis (dermatologist) and potentially unnecessary referrals comparing referrals from trained GPs in the two different time periods (Table S7; see Supporting Information).

Effect of the additional live course(s)

A subanalysis within the TC between GPs who followed and those who did not follow the additional live course(s) did not show any significant differences on proper descriptions of lesion(s), diagnostic accuracy and potentially unnecessary referrals in this study (data not shown).

Discussion

The current study demonstrates that GPs who followed a DOTP had better diagnostic skills and quality of referrals than untrained GPs.

As shown here, trained GPs more often mentioned a diagnosis and proper description of lesion(s) in their referral letters, performed more diagnostic procedures (dermoscopy and punch biopsy) before referring a patient to a dermatologist, and diagnosed skin lesions correctly more often (even after correcting for diagnostic procedures performed). This might have all contributed to the lower number of potentially unnecessary referrals found in the TC compared with both untrained cohorts, which enhances optimal efficacy in SC care and might eventually even lead to lower healthcare costs.17 These findings are generally in line with previous studies, also describing an improvement in diagnostic skills for GPs who followed a form of SC education,6, 7, 8 as well as performing more diagnostic procedures themselves.18 However, because the current study has a nonrandomized design, confirmation of these findings in a randomized controlled trial would be beneficial. Next to the improvement of efficacy in SC care, lowering the number of potentially unnecessary referrals and optimizing referral in case this is indicated (e.g. histopathological subtyping of a BCC by the GP before referral) might improve the patient's burden as well (e.g. by lowering the hospital visits needed and enhancing the one‐stop‐shop concept19). Furthermore, improved diagnostic skills of GPs might enable the option to refer patients back to the GP for follow‐up after treatment of (high‐risk) SC by the dermatologist at an earlier time and thereby further optimize the limited capacity of secondary SC care.

However, there still seems to be room for improvement. Although the TC referred fewer patients with benign lesions and AKs compared with both untrained cohorts, a major portion of the reason(s) for referral in the TC still comprised these lesion subtypes (45·6%). To enhance further improvement of SC care efficacy, reducing the number of these types of referral is desired and should be studied in more detail. In addition, no significant difference was seen between the percentage of patients with low‐risk BCC and BD referred by the TC compared with both untrained cohorts. This patient group comprises another important portion of potentially unnecessary referrals in the TC, even though transferring the care of patients with low‐risk SC to primary care was one of the main goals of the project ‘Suspicious Skin Lesions’. Another important finding was the number of additional skin lesions found by dermatologists in addition to the reason(s) for referral. Total‐body skin examination (TBSE) comprises an essential part of dermato‐oncological care, because additional (pre)malignant skin lesions are often found.20 This might be explained by fewer TBSEs performed by GPs (although strongly encouraged in the DOTP) and/or previously shown lower diagnostic skills in diagnosing (pre)malignant skin lesions compared with dermatologists.5 Lack of time in primary care could be a factor contributing to the above‐mentioned points for improvement.9, 21 Obviously, transfer of SC care puts more pressure on primary care and a sufficient amount of time, finances and healthcare providers seems essential to facilitate this. Another contributing factor could be that some patients have a lack of trust in their GP to treat SC, and therefore have a strong preference for referral to a dermatologist. Proper patient instruction on the roles of the GP and dermatologist in SC care might assist in gaining a patient's trust;22 therefore, we developed a patient leaflet on this topic which is currently being used.

We did not see a difference in the main outcomes comparing referrals from the two time periods (2 vs. 10 months after training). Nevertheless, it would be interesting to find out if these findings persist after a longer period of follow‐up. In a different study concerning the effect of a DOTP for GPs, only a qshort‐term effect was seen on the GPs’ competence, and this effect was negligible after 12 months.23

Next to proper training, using obtained knowledge and technical skills in daily practice on a regular basis seems essential to maintain quality, gain more experience and probably lower the threshold to perform SC treatment in daily practice. Therefore, having a GP with a special focus on SC care in a group practice might help to maintain the effects of training and further enhance treating more low‐risk SCs in primary care.24 Furthermore, a refresher course after a certain amount of time might be helpful in maintaining high‐quality SC care as well. Another option could be to establish dedicated primary care SC centres, occupied by specially trained GPs, which is already daily practice in some other countries with high SC incidence rates (e.g. Australia).

A limitation of this study is a potential bias in the group of trained GPs. GPs participating in the project could be more interested and motivated in treating SC than other GPs, and could therefore be more knowledgeable in SC beforehand. A randomized controlled trial comparing trained and untrained GPs could therefore be an interesting approach for future research. Furthermore, it should be noted that data regarding patients with SC treated in primary care who are not referred to a dermatologist are missing in this current study. More attention for primary SC care (without referral to a dermatologist) in future research is therefore encouraged.

In conclusion, GPs who followed a dedicated DOTP had better diagnostic skills and quality of referrals than untrained GPs, leading to fewer potentially unnecessary referrals. This might enhance a more efficient use of the limited capacity in secondary dermatological care and consequently lead to lower healthcare costs. Further optimization of SC care and additional transfer of care from dermatologists to GPs might be accomplished by further development of educational programmes and by appointing specially trained GPs.

Appendix S1 Regional guidelines between general practitioners and dermatologists in the Nijmegen area concerning diagnostics and treatment of (pre)malignant skin lesions.
Table S1 Overview of number of referral diagnoses per lesion type.
Table S2 Referral diagnoses and corresponding final diagnoses for TC.
Table S3 Referral diagnoses and corresponding final diagnoses for UPC.
Table S4 Referral diagnoses and corresponding final diagnoses for UHC.
Table S5 Overview of number of final diagnoses per lesion type.
Table S6 Subtype distribution of 406 additional (pre)malignant lesions in 264 patients found by the dermatologist.
Table S7 Comparisons: trained cohort over time.

Click here for additional data file.

Powerpoint S1 Journal Club Slide Set.

Click here for additional data file.

Video S1 Author video.

Click here for additional data file.