Clinical performance indicators for monitoring the management of cutaneous melanoma: a population-based perspective.

The prognosis of cutaneous malignant melanoma (CMM) is based on disease progression. The highly heterogeneous clinical-pathological characteristics of CMM necessitate standardized diagnostic and therapeutic interventions tailored to cancer's stage. This study utilizes clinical performance indicators to assess the quality of CMM care in Veneto (Northeast Italy). This population-based study focuses on all incidences of CMMs registered by the Veneto Cancer Registry in 2015 (1279 patients) and 2017 (1368 patients). An interdisciplinary panel of experts formulated a set of quality-monitoring indicators for diagnostic, therapeutic, and end-of-life clinical interventions for CMM. The quality of clinical care for patients was assessed by comparing the reference thresholds established by experts to the actual values obtained in clinical practice. The prevalence of stage I-CMM decreased significantly from 2015 to 2017 (from 71.8 to 62.4%; P < 0.001), and almost all the pathology reports mentioned the number of nodes dissected during a lymphadenectomy. More than 90% of advanced CMMs were promptly tested for molecular BRAF status, but the proportion of patients given targeted therapies fell short of the desired threshold (61.1%). The proportion of stage I-IIA CMM patients who inappropriately underwent computerized tomography/MRI/PET dropped from 17.4 to 3.3% ( P < 0.001). Less than 2% of patients received medical or surgical anticancer therapies in the month preceding their death. In the investigated regional context, CMM care exhibited both strengths and weaknesses. The evaluated clinical indicators shed essential insight on the clinical procedures requiring corrective action. It is crucial to monitor clinical care indicators to improve care for cancer patients and promote the sustainability of the healthcare system.

Introduction

Cutaneous malignant melanoma (CMM) is one of the most prevalent malignancies in fair-skinned populations [1]. Its incidence is increasing at a faster rate than that of any other neoplastic disease worldwide [2], and Italy is no exception. From 1970 to 2020, the incidence of CMM in Italy rose from 1.6 to 12.6 for males and from 2 to 12.0 for females [3,4]. Such an epidemiological profile necessitates adequate care strategies that encompass the entire disease management spectrum, from primary and secondary prevention to treatment (including patient rehabilitation procedures).

High-quality healthcare in oncology relies on both standardized diagnostic-therapeutic procedures (tailored to a ‘personalized’ patient profile) and the monitoring of clinical intervention outputs. Using this combined approach, the efficacy of diagnostic-therapeutic procedures can be quantified and any inefficient care strategies addressed. This leads to the best possible clinical performance [5].

Appropriate structure, process, and outcome ‘indicators’ can be used to consistently monitor the quality of care in various clinical settings [6]. In oncology, as in other clinical settings, quality indicators can reliably measure: (a) the appropriateness of diagnostic procedures and tests; (b) the efficacy of treatments; (c) critical areas requiring corrective action; and (d) the sustainability and relative priority of investments targeting the oncology-specific world of care [5].

This study uses a validated set of clinical care quality indicators to critically examine the quality of care for CMM patients in the Veneto region (Northeast Italy) in the years 2015 and 2017.

Materials and methods

Context

The Italian National Health System is a regionally structured public service primarily supported by general taxation. Its policies are founded on the fundamental values of universality, free access, freedom of choice, pluralism in provision, and equity [7].

In 2015, the Veneto Oncology Network (ROV) published a comprehensive document detailing the clinical procedures to be implemented at each stage of CMM patients’ clinical management, from initial diagnosis to end-of-life care [8]. The ROV document was based on current national and international literature [9-11]. It contained a detailed set of indicators for monitoring consistency between recommendations and real-world clinical practice.

Clinical data

Data were gathered using the high-resolution, population-based Veneto Cancer Registry (RTV), which covers the regional population (4.9 million residents) as well as the regional health service records. This study focuses on all incident cases of invasive CMM recorded by the RTV in 2015 (1279 cases) and 2017 (1368 cases), the 2 years for which data are available. Cancer registration procedures are based on information collected from various sources (e.g. pathology reports, death certificates, and the health service’s administrative records).

The variables recorded for this study were: demographics (age and sex); histological subtypes of CMM (not otherwise specified, superficial spreading, nodular, lentigo maligna, acral-lentiginous, desmoplastic, and spitzoid variant); CMM growth phase (radial vs. vertical); ulceration (absent vs. present); Breslow thickness (≤0.75, 0.76–1.50, 1.51–3.99, and ≥4.00); CMM regression (absent vs. present); tumor-infiltrating lymphocytes (absent vs. present); mitotic count (≤2 vs. >2/mm2); and, combined clinical-pathological TNM stage at diagnosis (I, II, III, and IV). Clinical information was also available on procedures [ultrasound, computerized tomography (CT), magnetic resonance imaging (MRI), and PET]; timing and type of surgical procedures (i.e. skin biopsy, wide-margin excision, sentinel lymph node biopsy [SLNB], and lymphadenectomy); and medical or radiation therapies (timing and type of drugs administered or radiotherapy sessions).

Indicators

In 2020, the Veneto Regional Health Administration established a Regional Working Group (RWG) on clinical care for CMM. The RWG (operatively run by the ROV with staff from the RTV and the University of Padua) set up a multidisciplinary panel of CMM experts (dermatologists, epidemiologists, oncologists, pathologists, radiotherapists, and surgeons). The RWG was tasked with identifying a set of indicators to support the reliable monitoring of CMM care management across all regional health units.

Based on the Manual of Melanoma Clinical Pathway Quality Indicators [11], the RWG identified indicators of the conformity of local oncological practices with the recommendations in international guidelines. These clinical indicators were largely consistent with those identified by internationally recognized scientific societies and institutions, including: the American College of Surgeons [12], the Melanoma Institute of Australia [13], the Danish Melanoma Group [14], the German S3 guidelines [15,16], the National Health Service Scotland [17], the Dutch Melanoma Treatment Registry [18], and the Istituto Toscano Tumori [19].

The RWG identified five main indicator categories for the different phases of a CMM patient’s clinical management. These categories covered: (A) diagnosis and staging (nine indicators); (B) surgical treatment(s) (10 indicators); (C) radiation therapies (one indicator); (D) medical treatments (one indicator); and, (E) end-of-life management (two indicators). All but two indicators were associated with threshold values. No thresholds were established for the remaining two due to the lack of solid scientific evidence to support them; however, these indicators were retained due to their potential utility in overall, describing a patient’s therapeutic pathway. Table 1 illustrates the indicators and thresholds.

Table 1

Main indicators of the quality of cutaneous malignant melanoma care

Clinical phase	Indicator	Threshold (%)
A. Diagnostics and Staging	1. New cases with TNM stage I as a percentage of all newly diagnosed melanomas	>70
	2. New cases with TNM stages III–IV as a percentage of all newly diagnosed melanomas	<10
	3. Percentage of new cases of invasive melanoma assessed for ulceration	≥90
	4. Percentage of diagnostic pathology reports indicating the number of lymph nodes removed	≥90
	5. Percentage of patients with lesions of 1–4 mm in thickness undergoing sentinel lymph node biopsy (SLNB)	≥90
	6. Percentage of patients undergoing SLNB, with lesions < 0.8 mm in thickness and no reported ulceration or mitoses	<10
	7. Percentage of TNM stages I–IIA patients undergoing head CT scans, chest CT/MRI scans, abdominal CT/NMRI scans, or PET scans within 180 days after diagnosis	<10
	8. Percentage of TNM stage IV patients screened for BRAF mutations	≥90
B. Surgery	9. Time elapsing between biopsy and complete excision: % of patients waiting <90 days	≥90
	10. Percentage of cases with pT1, pT2 disease ≤ 2.0 mm in thickness and surgical margins < 0.8 cm	<10
	11. Percentage of cases with pT3, pT4 disease > 2.0 mm in thickness and surgical margins < 1.6 cm	<10
	12. Percentage of patients found positive on SLNB	≥15
	13. Percentage of SLNB-positive patients undergoing lymphadenectomy	No threshold
	14. Percentage of patients undergoing axillary lymphadenectomy with ≥12 lymph nodes removed	≥90
	15. Percentage of patients undergoing inguinal lymphadenectomy with ≥6 lymph nodes removed	≥90
	16. Percentage of patients undergoing SLNB at a local reference center	≥90
	17. Percentage of patients whose treatment was completed at local healthcare facilities and at referral centers for surgical procedures	≥90
C. Radiotherapy	18. Percentage of patients given adjuvant radiotherapy or medical therapy after lymphadenectomy	No threshold
D. Anticancer medical therapy	19. Percentage of TNM stage IV patients treated with BRAF/MEK inhibitors and/or immunological checkpoint inhibitors within 12 months after diagnosis	≥90
E. End-of-life phase	20. Percentage of patients undergoing surgery in the 30 days before their death	<10
	21. Percentage of patients given radiotherapy or immunotherapy in the 30 days before their death	<10

The table also associates each indicator with the clinical phase to which it refers.

CT, computerized tomography; MEK, mitogen-activated protein kinase.

Statistics

We computed percentages and 95% confidence intervals for sarcoma cases that comply with quality indicators using the modified Wald method. We verified the percentage difference by year group (2015 and 2017) using the Chi-square test or Fisher’s exact test where the predicted frequency was less than 5. Results were considered statistically significant at the P < 0.05 level. Percentages were reported at a regional level, but subregional levels were also available for local use.

All analyses were performed using R version 4.0.4 (RStudio, Inc., Boston, Massachusetts, USA).

Results

Table 2 displays the TNM stage of CMM incidences as recorded by the Regional Cancer Registry in 2015 (1279 cases) and 2017 (1378 cases). In 2017, the overall number of CMM cases increased by 6%, and there was also a significant increase in both the proportion of high-stage disease (stage III/IV: 2015 = 11.6%; 2017 = 14.9%) and the number of unstaged CMMs.

Table 2

Cases of cutaneous malignant melanoma in Veneto (Italy) in 2015 and 2017

TNM stage	2015	2017
	N (%)	N (%)
I	918 (72)	854 (62)
II	161 (13)	215 (16)
III	117 (9)	141 (10)
IV	26 (2)	63 (5)
Missing	57 (4)	95 (7)
Total	1279 (100)	1368 (100)

Table 3 depicts the indicator values achieved in the 2015 and 2017 CMM incidences. As in Table 1, the indicators are listed by phase of clinical disease management: (A) diagnostics and staging; (B) surgical treatments; (C) radiotherapy; (D) medical anticancer treatments; and (E) end-of-life management.

Table 3

Performance indicators used to monitor the clinical management of cutaneous malignant melanoma cases recorded in the population-based, high-definition regional cancer registry in Veneto in 2015 and 2017

			2015 regional average	2017 regional average
Clinical phase	Indicators	Threshold	% (95% C.I.)	% (95% C.I.)	P value
A. Diagnostics and staging	1. New cases with TNM stage I as a percentage of all newly diagnosed melanomas	>70%	71.8 (69.22–74.23)	62.4 (59.80–65.00)	<0.001
	2. New cases with TNM stages III–IV as a percentage of all newly diagnosed melanomas	<10%	11.2 (9.51–13.04)	14.9 (13.07–16.91)	0.005
	3. Percentage of new cases of invasive melanoma assessed for ulceration	≥90%	94.2 (92.79–95.43)	93.3 (91.82–94.54)	0.360
	4. Percentage of diagnostic pathology reports indicating the number of lymph nodes removed	≥90%	99.6 (98.71–99.96)	98.6 (97.32–99.34)	0.070
	5. Percentage of patients with 1–4-mm thick lesions undergoing sentinel lymph node biopsy (SLNB)	≥90%	86.1 (81.08–90.26)	84.4 (79.85–88.34)	0.668
	6. Percentage of patients with lesions <0.8 mm in thickness and no reported ulceration or mitoses	<10%	4.0 (2.40–6.11)	4.1 (2.51–6.23)	1.000
	7. Percentage of TNM stage I–IIA patients undergoing head CT scans, chest CT/MRI scans, abdominal CT/MRI scans, or PET scans within 180 days after diagnosis	<10%	17.3 (15.02–19.82)	3.3 (2.24–4.63)	<0.001
	8. Percentage of TNM stage IV patients screened for BRAF mutations	≥90%	–	92.1 (82.44–97.37)	–
B. Surgery	9. Time elapsing between biopsy and complete excision: % of patients waiting <90 days	≥90%	–	86.2 (84.15–88.15)	–
	10. Percentage of cases with pT1, pT2 disease ≤ 2.0 mm in thickness and surgical margins < 0.8 cm	<10%	26.6 (23.67–29.59)	30.3 (27.32–33.47)	0.088
	11. Percentage of cases with pT3, pT4 disease > 2.0 mm in thickness and surgical margins < 1.6 cm	<10%	51.4 (43.77–59.04)	60.8 (53.93–67.46)	0.079
	12. Percentage of SLNB-positive patients	≥15%	17.5 (14.38–20.99)	18.1 (15.14–21.42)	0.846
	13. Percentage of SLNB-positive patients undergoing lymphadenectomy	No threshold	86.6 (77.26–93.11)	81.3 (72.00–88.49)	0.449
	14. Percentage of patients undergoing axillary lymphadenectomy with ≥12 lymph nodes removed	≥90%	92.4 (83.20–97.49)	85.3 (75.27–92.44)	0.290
	15. Percentage of patients undergoing inguinal lymphadenectomy with ≥6 lymph nodes removed	≥90%	96.7 (82.78–99.92)	97.4 (86.19–99.93)	1.000
	16. Percentage of patients undergoing SLNB at a local referral center	≥90%	–	48.1 (44.05–52.16)	–
	17. Percentage of patients whose treatment was completed at local healthcare facilities and at referral centers for surgical procedures	≥90%	–	65.7 (62.70–68.63)	–
C. Radiotherapy	18. Percentage of patients given adjuvant radiotherapy or medical therapy after lymphadenectomy	No threshold	4.5 (1.49–10.29)	2.5 (0.51, 7.07)	0.483
D. Anticancer medical therapy	19. Percentage of TNM stage IV patients treated with BRAF/MEK inhibitors or immunological checkpoint inhibitors within 12 months after diagnosis	≥90%	–	61.4 (45.50–75.64)	–
E. End-of-life phase	20. Percentage of patients undergoing surgery in the 30 days before their death	<10%	–	0.8 (0.02–4.12)	–
	21. Percentage of patients given radiotherapy or immunotherapy in the 30 days before their death	<10%	–	1.5 (0.02–5.33)	–

Bold indicates statistical significance of P values.

CT, computerized tomography; MEK, mitogen-activated protein kinase.

Clinical phase A: indicators focusing on diagnostics and staging

The proportion of CMM patients diagnosed at stage I (Indicator 1) fell just short of the threshold in 2015 but was significantly lower in 2017 (62.4%; P < 0.001). These results are largely consistent with the proportion of CMM patients diagnosed in advanced stages (Indicator 2), whose prevalence was higher than anticipated in both years considered (2015 = 11.2%; 2017 = 14.9%; P = 0.005). Almost all pathology reports included the number of regional lymph nodes that were excised (Indicator 4; 2015 = 99.6% and 2017 = 98.6%). In both years, SLNB was performed on around 85% of patients with a primary CMM more than 1 mm in thickness (approaching the 90% threshold; Indicator 5), but on less than 5% of patients with lesions smaller than 0.8 mm in thickness and no reported ulceration/mitoses (Indicator 6). The percentage of stages I–IIA CMM patients who underwent CT/MRI/NMRI scans (of the head, chest, or abdomen) within 180 days of their diagnosis (Indicator 7) was significantly higher in 2015 (17%) than in 2017 (3%; P < 0.001), and thus, the established threshold was not met until the latter year. The percentage of stage IV CMM patients screened for BRAF mutations in 2017 (Indicator 8) was also consistent with threshold.

Clinical phase B: indicators focusing on surgical treatments

The percentage of patients who waited more than 90 days between their initial biopsy and subsequent wider surgical excision (Indicator 9; recorded only for 2017) was marginally below the threshold (86% vs. 90%). With the exception of axillary lymphadenectomies performed in 2017 (85.3%), the proportion of patients having an adequate number of lymph nodes removed from the axillary (Indicator 14) or inguinal (Indicator 15) nodal stations was consistent with the 90% threshold (in both years). In both years, the percentage of cases found positive on SLNB (Indicator 12) met expectations. The following indicators failed to meet the thresholds: (a) percentage of CMM resections with adequate margins (Indicators 10 and 11); (b) percentage of patients undergoing SLNB at local centers (Indicator 16); and (c) percentage of patients completing their surgical treatment(s) at local healthcare facilities (Indicator 17).

Clinical phase D: indicators focusing on medical anticancer therapy

While the majority of stage IV CMM patients were promptly screened for BRAF (Diagnostic Indicator 8), the proportion of patients given targeted therapies based on their BRAF/mitogen-activated protein kinase (MEK) status (Indicator 18; only recorded for 2017) was just two-thirds of the expected level (61%, whereas the threshold was >90%).

Clinical phase E: indicators focusing on end-of-life management

The proportion of patients who had surgical treatments (Indicator 20) or radiotherapy or immunotherapy (Indicator 21) within 30 days before their death was recorded for the first time in 2017, and were largely consistent with expectations.

Discussion

Based on a series of quality of clinical care indicators, this study examined how well CMM patients in Veneto (Italy) were managed in three crucial phases of their clinical pathway: diagnosis, surgical and medical treatments, and end-of-life care. The quality indicators were selected by the regional oncology working group, which possessed interdisciplinary expertise.

Diagnostics and staging (including medical treatments related to molecular profiling)

In the 2 years considered, a significant decline in the incidence of early invasive CMM went hand in hand with a rising number of cases with advanced disease [20]. These disappointing findings suggest weaker compliance with effective CMM secondary prevention strategies. When compared with international data, the stage-specific CMM incidence identified here contrasts sharply with the trends observed in similar studies, which reported a successful stepwise increase in the prevalence of stage I-CMM from 1996 to 2015 [21]. Two important factors, however, may attenuate the alarming nature of these findings: (a) the results need to be confirmed by extending the monitoring period and (b) the study does not account for in-situ CMM, the prevalence of which might make the present findings appear significantly less disheartening. That said, promoting educational campaigns for primary prevention remains a priority, especially in the Veneto region, which in 2017 reported the highest age-standardized rate of CMM per 105 inhabitants in Italy (albeit with substantial intraregional differences) [22-25].

As consistently demonstrated by Indicators 2 and 3, the quality of pathology reporting was excellent, always meeting or exceeding expectations. Not only is this proof of optimal diagnostic performance, but also of effective interdisciplinary cooperation between surgeons (in the preanalytic phase) and pathologists (in the analytic and postanalytic phases) [26]. These results are comparable with those documented in the USA [6] and the National Comprehensive Cancer Network (NCCN) recommendations [27].

The proportion of patients with melanomas of 1–4 mm in thickness who underwent SLNB approached the desirable threshold (90%), in accordance with national and international guidelines based on the study by Morton et al. [28]. Considered quality indicators also confirmed an increasingly appropriate use of diagnostic imaging. This trend benefits patients (who avoid unnecessary and potentially harmful exposure to radiation) and improves efficiency in the healthcare system. The reduction in the use of imaging from 2015 to 2017 unequivocally demonstrates how quickly and consistently consensus guidelines can be adopted in clinical practice, positively improving both quality and cost of care [29,30]. Targeted BRAF and MEK-inhibiting therapies are first-line treatments for BRAF-mutated, advanced CMM. Indicator 8, a diagnostic indicator focusing on the assessment of a patient’s BRAF status, demonstrated that the standard molecular testing procedure was fully adhered to.

Surgical treatments

Monitoring of patients’ surgical management (Indicator 9) has revealed significant room for improvement in shortening the time between the initial biopsy-based CMM assessment and complete (wider) excision. A critical analysis of this indicator points to the need to review the surgical management pathway in light of the potential benefits of a timely, wider cancer excision [31].

The NCCN guidelines recommend lymphadenectomy with complete nodal dissection for CMM patients with metastatic sentinel nodes [27,32,33]. In this study, lymphadenectomy was performed on 86.6% (in 2015) and 81.1% (in 2017) of SLNB-positive patients (Indicator 13). The fact that similar (unsatisfactory) results were reported in previous national and international studies [i.e. 63% in Tuscany (Italy) in 2013 [34]; 50% in the USA in 2008] confirms how difficult it is to attain the desired clinical performance [35]. From among the patients in this study who had a lymphadenectomy, the number of inguinal nodes removed vastly exceeded expectations, whereas the number of axillary nodes removed fell just short of the threshold. Both these indicators showed an improvement over those reported in 2008 by the NCCN [27].

Only a small proportion of our two CMM cohorts had been managed exclusively at their local referral hospital. Patient migration within the region revealed an uneven distribution of surgical teams specializing in CMM. This structural flaw was rectified in 2018 [36].

Medical treatments

While molecular testing indicators exceeded expectations, the results for Indicator 20 showed that only 61% of CMM patients potentially eligible for BRAF/MEK inhibitors and immunological checkpoint inhibitors received these treatments. This necessitates an investigation into the reasons for such a significant gap between the expected percentage (90%) and the results obtained, although we need to consider that the use of targeted adjuvant systemic therapies in CMM patients with advanced disease remains controversial due to doubts regarding its clinical benefit and potential toxicity [27,37].

End-of-life management

Consistent with the expected threshold, only a negligible percentage of CMM patients underwent surgery, radiotherapy, or immunotherapy within 30 days before they died. The two quality indicators focusing on end-of-life care demonstrated that the approach to patient management was generally ethical when anticancer therapies ceased to be beneficial and their potential adverse effects rendered them pointless [38-43].

This study has its strengths. The fact that the study was population-based (rather than center-specific), which minimizes the risk of selection bias, is its main strength. Moreover, the use of the standardized algorithms by year reduced measurement variability, hence increasing the reliability of the values assumed by the indicators. The study’s main weakness is that it only covers 2 years (and for some indicators only one), making it impossible to conduct any robust trend assessments. Moreover, even if the monitoring of quality management indicators is a strategy driving the standardization of practices toward quality of care, this monitoring system did not permit an assessment of the reasons for deviation from quality, which could be several. Certain patients’ unwillingness to consent to recommended testing, management, and/or follow-up, or a patient’s poor general health unrelated to CMM, which can impede certain procedures, are just two examples.

Conclusion

A standardized approach to the clinical management of CMM is a prerequisite for assessing and monitoring the quality of clinical care pathways and promoting their improvement. Interdisciplinary clinical interactions, high-resolution cancer registration, and linkages with administrative big data collection [44] can be exploited consistently and effectively to drive action to ensure best care practices and the sustainability of public healthcare systems.

Acknowledgements

This study was funded by CARIPARO, Fondazione Cassa di Risparmio di Padova e Rovigo. The foundation played no part in the study’s design and the collection, analysis, or interpretation of data, manuscript writing, or decision to submit the paper for publication. This research received funds from “Ricerca Corrente 2022” to cover publication costs.

Ethical approval: the analysis was conducted on aggregate, anonymized data, with no possibility of identifying the individuals concerned. The Veneto Oncological Institute’s Ethics Committee granted its approval for the study (No. 52/2016).

Conflicts of interest

There are no conflicts of interest.