Method of Assessing Skin Cancerization and KeratosesTM (MASCK™): development and photographic validation in multiple anatomical sites of a novel assessment tool intended for clinical evaluation of patients with extensive skin field cancerization.

BACKGROUND: A range of 'field-directed' treatments is available for the management of extensive skin field cancerization (ESFC), but to date, the only validated objective quantitative tools are limited to assessment of actinic keratoses (AKs) affecting the head. AIMS: To develop a versatile quantitative instrument for objective clinical assessment of ESFC and perform initial internal validation across multiple anatomical zones.
METHODS: The study comprised instrument development, pilot testing and instrument refinement and two rounds of reliability and inter-rater validation testing. The study was noninterventional and used a convenience sample of de-identified patient photographs selected based on preset criteria. An expert panel developed the instrument and scoring system via a modified Delphi voting process. A sample of 16 healthcare professionals from multiple specialties undertook the pilot testing, and a panel of seven dermatologists were involved in validation testing. Validation was determined by assessment of overall inter-rater agreement using Gwet chance-corrected agreement coefficients (ACs).
RESULTS: The instrument produced, called the Method for Assessing Skin Cancer and Keratoses™ (MASCK™), comprises the Skin Field Cancerization Index (SFCIndex), derived from area of skin involvement and AKs (number and thickness), a global assessment score and a cancer-in-zone score, and uses Likert scales for quantitative scoring. The SFCIndex is a composite score comprising the number and thickness of AKs multiplied by area of skin involvement. ACs for the SFCIndex components, the overall SFCIndex score and the global assessment score were > 0.80 (rated 'almost perfect') while the AC for the cancer-in-zone metric was lower (0.33, rated 'fair'). Internal consistency was demonstrated via positive correlation between the overall SFCIndex score and the global assessment score.
CONCLUSIONS: Our study found near-perfect agreement in inter-rater reliability when using MASCK to assess the severity of ESFC in multiple anatomical sites. Further validation of this novel instrument is planned to specifically assess its reliability, utility and feasibility in clinical practice.

Introduction

Extensive skin field cancerization (ESFC) refers to an area or ‘field’ of sun‐damaged skin that contains multifocal clinical and subclinical disease arising from ultraviolet (UV) damage. This damage ranges from single UV‐damaged keratinocytes and subclinical lesions, through early clinical lesions such as actinic keratoses (AKs) and Bowen disease (intraepithelial carcinoma), to advanced clinical lesions, commonly invasive cutaneous squamous cell carcinoma (SCC) and basal cell carcinoma (BCC). , AKs are the most common lesions of the skin, and are the signature lesion in ESFC.

The Australian population is at significant risk of actinic damage and subsequent development of ESFC, due to the combination of fair skin phototypes and an outdoor lifestyle resulting in high levels of UV exposure. In the context of the ageing population, increasing numbers of people will require treatment for keratinocyte cancer (KC) and ESFC. ,

The overall understanding of AK has evolved over time. Previously considered a ‘premalignant’ lesion, AK may progress to invasive SCC (0.025%–16% per lesion, per year ). Some authors have advanced the concept that AK be viewed as carcinoma in situ, as such lesions demonstrate signature cytological features of SCC. AKs are a common manifestation of actinic damage, with prevalence estimates ranging between 40% and 60% in adults aged > 40 years in Australia. ,

While the risk of an individual AK transforming to in situ and invasive SCC is low, many patients have multiple AKs and AKs occurring in multiple areas of skin affected by ESFC, thus increasing the cumulative risk of malignant transformation. Given the current inability to predict how an individual lesion will behave or progress, it has been proposed that all AKs should be treated. Clinical practice should reflect our understanding of AK as an indicator of skin field cancerization, and patients with AK should be evaluated for the presence of field cancerization and treated accordingly.

A range of ‘field‐directed’ treatments are available for the management of ESFC. These include topical treatments such as 5‐fluorouracil, imiquimod, diclofenac gel, cryotherapy, photodynamic therapy, laser therapy and widefield radiation therapy (e.g. intensity‐modulated radiotherapy, volumetric‐modulated arc therapy). Field‐directed treatments represent a comprehensive therapeutic approach to the management of ESFC as they treat the entirety of the affected skin field, including clinical and subclinical disease. Until recently, the assessment of severity was limited to subjective clinical description and/or clinical assessment tools designed to assess individual AKs, such as the Olsen tool, with no assessment of the surrounding skin field. Attempts to overcome this limitation by combining the Olsen tool with ‘lesion counts’ have been plagued by poor inter‐rater reliability, even among experts. , Currently, there are two tools for assessment of AKs in the context of the surrounding skin field, the Actinic Keratosis Field Assessment Scale (AK‐FAS) and the Actinic Keratosis Area Severity Index (AKASI), both published in 2017. These tools, which are both validated for assessment of AK affecting the head, have demonstrated good reproducibility in their validation studies. , However, despite the many treatment options for ESFC, to our knowledge there currently are no validated clinical assessment tools that can be used across any anatomical or regional zone for ESFC. The development of an objective clinical assessment tool for ESFC would enable clinicians to grade a field in any anatomical site, encompass the broad range of clinical severity and assess treatment efficacy over time. To address this need, we have developed a novel and versatile quantitative instrument for the objective assessment of ESFC across multiple sites, which we have called the Method for Assessing Skin Cancerization and Keratoses™ (MASCK™), and present here the results of a photographic validation study.

Methods

The project was prospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12620000056998), received ethics approval from Bellberry Ltd (application number: 2018‐09‐793) and was conducted in accordance with ethical principles founded in the Declaration of Helsinki. All photographic images were originally collected as part of standard clinical care, during which informed consent for taking and use of the images was obtained. The research data are stored in an institutional repository and will be shared upon reasonable request and with permission of Cancer Care Research Pty Ltd (trading as GenesisCare; skinandbenign.research@genesiscare.com).

Study design

The study comprised three separate phases: (i) instrument development, (ii) pilot testing and instrument refinement, and (iii) reliability and inter‐rater validation. The primary endpoint was the development of an assessment instrument and associated scoring system, and the secondary endpoint was internal validation, as demonstrated by a measure of overall inter‐rater scoring consistency using Gwet change‐corrected agreement coefficient (AC). ,

This initiative was noninterventional and used a convenience sample of de‐identified patient photographs. The photographic images were retrieved from the clinical databases of patients who had previously been treated at the Queensland Institute of Dermatology (Brisbane, QLD, Australia) or St Vincent's Hospital (Melbourne, Vic., Australia) or by one of the investigators, and were selected retrospectively based on having met key inclusion/exclusion criteria (Table 1).

Table 1

Selection of photographic images for evaluation.

Inclusion criteria

Patient demographics: male/female, age > 18 years

Clinical characteristics: treatable region of Skin Field Cancerization in one of the following anatomical zones: scalp, forehead, nose, cheek, ear, forearm, back of hand, lower leg, top of foot, chest

Exclusion criteria

Poor‐quality photographs

Other dermatoses affecting assessment (e.g. inflammatory skin diseases) that may confuse scorers

Phase 1. Instrument development: components and scoring system

The initial development of the assessment instrument was clinically based and did not rely on photography. A panel, comprising seven dermatologists (CB, JC, PF, SS, LS, RS, WW) discussed their experiences of ESFC assessment to determine the features that they had used in their clinical and research practice. From this, they developed a primary set of core clinical features that were of most relevance for assessing field cancerization tumour burden. The panel then reviewed these criteria, ranked them in order of relevance and using a modified Delphi voting process, determined the most salient to include in the assessment instrument. The panel then discussed and agreed upon standard measurements, using Likert scales, to quantify severity. Having established the criteria and overall scale, the group then agreed on the overall scoring design and working definitions of each scoring item.

Phase 2. Pilot testing and refinement

Preliminary testing and refinement of the draft assessment instrument was conducted by a testing group comprising 16 healthcare professionals with a special interest in dermatoses [dermatologists (n = 8), radiation oncologists (n =5) and registered nurses (n = 4)], which convened in several workshops over a 12‐month period. During the workshops, participants were supplied with 50 de‐identified patient photographic images and instructed to use the draft assessment instrument to assess tumour burden. They then provided feedback on the usefulness of the instrument and on the practicality of the scoring system. The descriptive findings of the testing group were used to make refinements to the draft MASCK and its associated scoring system prior to validation testing.

Phase 3. Validation testing

Validation of the MASCK comprised evaluation of internal consistency and inter‐rater reliability. Validation was conducted in two rounds, undertaken 2 months apart. De‐identified photographic images from 30 patients with varying degrees of ESFC affecting a range of anatomical zones [scalp (n = 3), forehead (n = 4), cheek (n = 4), nose (n = 2), ear (n = 2), hand dorsum (n = 6), lower leg (n = 3) and forearm (n = 1)] were used. In each validation round, seven dermatologists were each provided with 30 de‐identified photographic images and individual scoring sheets to enable independent assessment. To reduce recall bias, photographs were re‐randomized for each round. The scores from both validation rounds were collated for formal statistical analysis. The working group used the statistical outputs and descriptive feedback obtained after Validation round 1 to further refine the MASCK prior to the commencement of Validation round 2.

Statistical considerations

Statistical analyses were completed using Stata MP for Mac (V15.1; StataCorp, College Station, TX, USA). Inter‐rater agreement is defined as the propensity for ≥ 2 raters to independently classify a given patient into the same predefined category, and was calculated using Gwet AC in Stata, with AC1 for nominal data, including cancer in zone and AC2 for ordinal data. , This procedure allows a weighting where ≥ 3 raters have been used and accounts for partial agreements. Benchmark intervals were based on the Landis and Koch 1977 scale and the method proposed by Gwet in 2014 (Table 2). In this analysis, the probabilistic method was implemented, which selects the benchmark interval associated with the smallest cumulative membership probability being > 95%. Outputs were presented as estimated inter‐rater agreement coefficients, within the range − 1 to 1, with two‐sided standard errors. Correlation coefficients were calculated for the three core components of the MASCK instrument and for the subcomponents of the Skin Field Cancerization Index (SFCIndex) component. For each validation round, correlation coefficients between the SFCIndex and the global score were calculated using pairwise correlations (pwcorr procedure in Stata).

Table 2

Benchmark scale for interpretation of correlation coefficients. ,

Coefficient value range	Interpretation of inter‐rater agreement
0.00	Poor
0.00–0.20	Slight
0.21–0.40	Fair
0.41–0.60	Moderate
0.61–0.80	Substantial
0.81–1.00	Almost perfect

Results

Preliminary MASCK components and scoring scales

In Phase 1 of the study, the investigators identified four clinical features as the most salient for field cancerization and incorporated these into the SFCIndex: (i) the number of AK, (ii) thickness of AK, (iii) number of atypical keratosis KCs and (iv) the percentage area of involvement in the skin field. These were each evaluated individually using Likert scales and then computed to provide a composite SFCIndex with a range of 0–45 (0 being no disease, 45 being extremely severe).

In addition, it was determined that a global assessment score would be useful. This was designed to provide an overall clinical assessment of the severity of ESFC in the field or the zone and was scored on a five‐point Likert scale. The initial scoring design therefore comprised two individual components: the composite SFCIndex (four subscales, score range 0–45) and the global assessment score (range 0–4).

Validation testing: preliminary MASCK instrument

Statistical outputs from Validation round 1 (Table 3) showed a spectrum of correlation coefficients. The inter‐rater agreement for the number of AK was ‘almost perfect’, and there was ‘substantial’ inter‐rater agreement for the thickness of AK, the percentage area of involvement in the skin field and the global assessment of field cancerization. However, there was only ‘moderate’ inter‐rater agreement for atypical keratosis. Removal of atypical keratosis from the SFCIndex score calculation resulted in an overall improvement in SFCIndex correlation coefficient from a mean ± standard error of 0.63 ± 0.37 to 0.70 ± 0.03. Owing to its poor reliability and its impact on the overall inter‐rater agreement of the SFCIndex score, atypical keratosis was removed from the SFCIndex and replaced by assessment of cancer in zone as a third and separate criteria prior to Validation round 2.

Table 3

Correlation coefficients for validation rounds.

	Gwet AC1 coefficient, mean ± SE	P int a	P cumul > 95% b	PBI	Agreement, %
Validation round 1
AK number	0.85 ± 0.02	0.99	0.99	0.8–1.0	92.9
AK thickness	0.76 ± 0.04	0.82	1.00	0.6–0.8	89.0
Atypical keratosis	0.53 ± 0.08	0.76	0.96	0.4–0.6	84.4
Area score	0.64 ± 0.04	0.19	1.00	0.4–0.6	88.4
SFCIndex (modified c )	0.70 ± 0.03	1.00	1.00	0.6–0.8	93.0
Global assessment score	0.69 ± 0.04	0.99	0.99	0.6–0.8	88.9
Validation round 2
AK number	0.91 ± 0.01	1.00	1.00	0.8–1.0	96.0
AK thickness	0.89 ± 0.02	1.00	1.00	0.8–1.0	94.6
Area score	0.86 ± 0.03	0.98	0.98	0.8–1.0	95.0
SFCIndex	0.81 ± 0.03	0.37	1.00	0.6–0.8	95.5
Global assessment score	0.85 ± 0.02	0.99	0.99	0.8–1.08	94.6
Cancer in zone	0.33 ± 0.09	0.07	1.00	0.00–0.20	66.0

AK, actinic keratosis; PBI, probabilistic benchmark interval; SE, standard error; SFCIndex, Skin Field Cancerization Index.

Statistical probability that the coefficient falls within a predefined probabilistic benchmark interval (where values closer to 1 indicate better agreement).

Cumulative interval membership probability.

Recalculation of the SFCIndex after removal of atypical keratosis from the composite score.

Final MASCK instrument and scoring system

The final MASCK instrument comprised three components: (i) modified SFCIndex, derived from area of skin involvement and AK (number and thickness), (ii) global assessment score and (iii) cancer‐in‐zone score (Fig. 1). The Likert scales for the SFCIndex and the global assessment were retained from the draft instrument with the addition of instructional descriptors for each scoring criteria to facilitate accuracy (Table 4). After removal of the atypical keratoses from the SFCIndex, the modified composite score (three subscales, score range 0–30) was calculated as follows: (number of AK + thickness of AK) × area of involvement.

Figure 1

Method of assessing skin cancer and keratoses™ (MASCK™). SFC, Skin Field Cancerization. [Colour figure can be viewed at wileyonlinelibrary.com]

Table 4

Method for assessing skin cancer and keratoses: clinical features and scoring criteria.

Clinical feature	Score	Scoring criteria
AK number	0–3	0: no sign of keratoses
		1: mild signs of keratoses, isolated keratoses, few in number (≤ 5)
		2: moderate signs of keratoses, isolated keratoses, moderate in number (6–15)
		3: severe keratoses, continuous keratoses, extensive in number (> 15)
AK thickness a	0–3	0: no thickness present
		1: thin, slightly palpable or just perceptible (≤ 1 mm)
		2: moderate, easily felt or seen (> 1–3 mm)
		3: very thick, including cutaneous horns (e.g. > 3 mm)
Area of involvement b	0–5	0: no area of involvement
		1: small area of involvement (1–5%)
		2: larger but less than one‐third involvement (6–33%)
		3: over one‐third but less than two‐thirds involvement (34–66%)
		4: over two‐thirds but less than complete involvement (67–95%)
		5: almost‐complete to complete involvement (96–100%)
SFCIndex	0–30	Score = (number of AK + thickness of AK) × area of involvement
Global assessment score c	0–4	0: skin in the zone is generally smooth with no keratoses evident
		1: mild: small area within the zone affected with few or thin keratoses
		2: moderate: patchy involvement in the zone and/or moderately thick keratoses
		3: severe: extensive involvement of the zone and/or numerous thicker keratoses
		4: very severe: (i) extensive involvement of the zone with (a) numerous thicker keratoses including cutaneous horn or (b) continuous keratoses; or (ii) near‐complete involvement of the zone with numerous thicker keratoses
Cancer in zone d	+ or –	+: cancer present or has occurred in zone
		–: no cancer present or has occurred in zone

AK, actinic keratosis; BCC, basal cell carcinoma; SFC Index, Skin Field Cancerization Index; SCC, squamous cell carcinoma. Throughout this table the 'zone' is the defined area of skin or anatomical region scored.

Assessed by the thickest keratoses present in the zone (it is not an average of all keratoses).

Percentage of affected skin in the zone that is abnormal and showing actinic damage; it is important that the area scored for an individual is recorded and defined and that the same area is scored at subsequent assessments for consistency.

Overall assessment of the field or zone; when applying this score, it is important to define the area being assessed and to use the same area for subsequent assessments.

Presence of cancer in the zone (in situ or invasive SCC or BCC ) based on clinical diagnosis at the time of assessment or proven histologically within the past 6 months; ≥ 1 lesion.

In situ SCC (Bowen disease or intraepithelial carcinoma) is defined clinically as a lesion showing 3 or more of the following features: size > 5 mm, base induration, irregular shape, hyperkeratosis or cutaneous horn, deep redness, erosion/crusting.

The third component, the cancer‐in‐zone score, referred to identification of cancer (including superficial and invasive KC such as Bowen disease, BCC or SCC) in the assessed zone. This assessment was based on clinical diagnosis at the time of the assessment or histological diagnosis within the past 6 months. A score of ‘+’ indicated that cancer was present or had occurred in the zone and a score of ‘–’ indicated that no cancer was present or had occurred in the zone in the past 6 months. Figure 2 provides examples graded using the final MASCK™ instrument.

Figure 2

Illustrative examples of skin cancer and keratoses assessments using the MASCK™ instrument. [Colour figure can be viewed at wileyonlinelibrary.com]

Validation testing: final instrument

Statistical outputs from Validation round 2 (Table 3) showed an overall improvement in each of the individual correlation coefficients for the SFCIndex components, the overall SFCIndex score and the global assessment score; these metrics all met the predefined probabilistic benchmark interval for ‘almost perfect’ inter‐rater correlation. The correlation coefficient for the newly added cancer‐in‐zone metric was low (mean ± SE 0.33 ± 0.09) and inter‐rater agreement was defined as ‘fair’.

Correlation between the overall Skin Field Cancerization Index and the global assessment score

In both validation rounds there was a statistically significant (P < 0.001) positive correlation between the overall SFCIndex score and the global assessment score (Fig. 3).

Figure 3

Concordance between the SFCIndex and global assessment scores during (a) Validation round 1 and (b) Validation round 2. [Colour figure can be viewed at wileyonlinelibrary.com]

Discussion

The primary objective of this research study was to develop a quantitative instrument and associated scoring system for the objective assessment of ESFC. This has been achieved with MASCK, which provides a composite score that objectively assesses severity of ESFC of any anatomical site or defined zone. It has not been designed to measure total tumour burden or aggregate score of multiple zones of SFC, and is therefore intended for use only in areas of solar damage.

The outcomes of the validation testing support the versatility of MASCK to objectively assess ESFC at various anatomical sites, with ‘almost perfect’ inter‐rater reliability among expert dermatologists demonstrated. In addition, the global assessment score was significantly positively correlated with the SFCIndex across both rounds of validation, providing further support for the validity of both the SFCIndex and global assessment score components of the MASCK. MASCK joins other tools that assess AK in the context of the surrounding skin field. , To our knowledge, it is the first tool to assess ESFC specifically and to do so across multiple anatomical sites.

The performance of MASCK in this validation study was comparable with that of the AKASI and superior to the AK‐FAS in their respective validation studies, both of which were restricted to assessment of the head. , In the MASCK score there was a strong correlation between the SFCIndex and global score (0.87); this was also reported with the AKASI with similarly strong correlation with the global disease severity (as measured by Physician Global Assessment score). While the AKASI pilot validation study did not specifically measure inter‐rater agreement, the authors reported the coefficient of variation for AKASI scores to be low, consistent with the results of this preliminary validation study. In Validation round 1, the MASCK achieved ‘substantial’ inter‐rater agreement, which is comparable with the results of the AK‐FAS validation study, where there was ‘substantial’ inter‐rater agreement for AK area and hyperkeratosis, and ‘moderate’ inter‐rater agreement for sun damage. However, after modification, the MASCK improved in the second round of validation to ‘almost perfect’ inter‐rater agreement for all criteria except cancer in zone. The stronger inter‐rater agreement for hyperkeratosis in MASCK could be attributed to the more easily applicable and broader scoring criteria compared with that of the AK‐FAS. The AK‐FAS has a requirement for AK lesions to be minimum Olsen grade II or III and hyperkeratosis to be > 5 mm in diameter to qualify for grading as hyperkeratosis present +/−'. In contrast to both the AK‐FAS and AKASI, a further strength of the MASCK is its validation across multiple anatomical sites.

There is no consensus regarding a standardized clinical definition of field cancerization. Most definitions require multiple AK lesions on a background of UV‐damaged skin. However, some definitions do not require the presence of AK and some do not require background UV damage. It is possible for a skin field that has undergone cancerization to be entirely affected by subclinical disease. As a clinical assessment tool, the MASCK requires the presence of AK and/or cancer in zone because these lesions are important markers of the activity and behaviour of the field. They correlate with the degree of UV damage and skin response to UV exposure, and to the risk of transformation to invasive malignancy and therefore, the need for treatment. The assessment of cancer in zone is a unique feature of the MASCK that is not present in the AK‐FAS or AKASI. Our expert panel replaced the atypical keratosis component of the original SFCIndex with the cancer‐in‐zone component after atypical keratosis achieved only ‘moderate’ inter‐rater agreement in Validation round 1. Disappointingly, cancer in zone had worse reproducibility, achieving only ‘fair’ inter‐rater agreement in Validation round 2. Nevertheless, the documentation of cancer in zone is considered to be of importance in demonstrating severity of SFC, and our opinion is that it should be retained. It is likely that the photographic nature of this study contributed to the relatively poor reproducibility of these criteria, and that clinical assessment would increase inter‐rater agreement for both criteria.

Although thickness of hyperkeratosis is independent of risk of malignant transformation in AK, , keratosis thickness was included in the MASCK because it affects treatment selection, with topical treatments generally not considered effective for more hyperkeratotic lesions. , , , , , , , , , The MASCK also incorporates a global assessment score, which acts as another, more holistic assessment of disease severity. This unique feature of the MASCK distinguishes it further from the AK‐FAS and AKASI. Moreover, in contrast to AK‐FAS and AKASI, the MASCK does not incorporate assessment of background erythema, pigmentation or other signs of sun damage. We noted the subjective nature of assessing these variables in our experience and the ‘moderate’ inter‐rater reliability of sun‐damage in the AK‐FAS validation study, and excluded these signs from the assessment criteria on that basis.

The results of the present study are promising and, given the trend towards recognizing AK as a manifestation of ESFC, the MASCK may have a role to play in routine clinical practice, particularly in monitoring the success of field‐directed treatments of ESFC. Objective clinical assessment tools for ESFC would allow the assessment of pretreatment disease severity, objective treatment monitoring and an assessment of treatment efficacy, including maintenance of benefit or relapse. Moreover, as data accumulate over time, scores could provide greater insight into risk of progression to invasive SCC, as has been achieved with the AKASI. Further validation of the MASCK is planned to specifically assess its reliability, utility and feasibility in clinical practice.

The number of assessors used in the final validation rounds is relatively small (n = 7), which is a limitation of this study; however, it is strengthened by the expert status of the panel and the breadth of experience of its members, who work in diverse locations across Australia. In addition, the cases scored included a broad range of severity of ESFC from mild to very severe involvement, and covered various anatomical regions. Studies have shown that standardized photographic training and lecture‐based training were able reduce inter‐rater variability in grading psoriasis with the Psoriasis Area and Severity Index, and it may be useful in future to determine the effect of such training on inter‐rater agreement for the MASCK.

Conclusion

The MASCK is an exciting, novel and versatile clinical tool that has the potential to objectively assess ESFC in multiple anatomical sites in a standardized manner with excellent reproducibility. The MASCK has a significant role to play in monitoring treatment of ESFC. The reproducibility and the utility of the MASCK will be explored further in clinical validation studies.

Conflict of interest

CB, PF, SS, RS and LS report receipt of personal fees for consultancy work from GenesisCare outside of the submitted work. LS reports being a paid advisor for the National Dermatology Radiation Oncology Register (NDROR) and being an unpaid member of an advisory panel that consults on the guidelines for the NDROR. CB reports the receipt of personal fees for other from GenesisCare outside of the submitted work. AJ, JC, WW and MS report no conflicts of interest.