Systematic development of quality indicators for skin cancer management in primary care: a mixed-methods study protocol.

INTRODUCTION: Australia has the highest incidence of skin cancer in the world, with two out of three Australians expected to be diagnosed with skin cancer in their lifetime. Such incidence necessitates large-scale, effective skin cancer management practices. General practitioners (in mainstream practice and in skin cancer clinics) play an important role in skin cancer care provision, making decisions based on relevant evidence-based guidelines, protocols, experience and training. Diversity in these decision-making practices can result in unwarranted variation. Quality indicators are frequently implemented in healthcare contexts to measure performance quality at the level of the clinician and healthcare practice and mitigate unwarranted variation. Such measurements can facilitate performance comparisons between peers and a standard benchmark, often resulting in improved processes and outcomes. A standardised set of quality indicators is yet to be developed in the context of primary care skin cancer management. AIMS: This research aims to identify, develop and generate expert consensus on a core set of quality indicators for skin cancer management in primary care.
METHODS: This mixed-methods study involves (1) a scoping review of the available evidence on quality indicators in skin cancer management in primary care, (2) identification and development of a core set of quality indicators through interviews/qualitative proforma surveys with participants, and (3) a focus group involving discussion of quality indicators according to Nominal Group Technique. Qualitative and quantitative data will be collected and analysed using thematic and descriptive statistical analytical methods. ETHICS AND DISSEMINATION: Approval was granted by the university's Research Ethics Committee (HREC no. 520211051532420). Results from this study will be widely disseminated in publications, study presentations, educational events and reports. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

First study to identify, develop and generate expert consensus on a core set of quality indicators for skin cancer management in primary care in Australia.

This study provides a greater understanding of perceived benefits of identifying, developing and implementing quality indicators in this context.

The mixed-methods approach will enable rich datasets on a topic that has received little attention.

While exploratory in nature, limiting generalisability of findings, this will lead to a quality indicator intervention for roll-out across Australia.

Introduction

Background

Skin cancer develops as a result of abnormal proliferation of the epidermis.1 The most prevalent skin cancers are the non-melanocytic skin cancers (NMSC), of which the most common are basal cell carcinoma and squamous cell carcinoma.1 These keratinocyte cancers make up the majority of skin cancer cases, however, the risk of metastases and death is low, and the wide range of treatment options results in high survival rates.2 Conversely, melanoma (the rarer form of skin cancer) has a much higher risk of metastasis and death.3 Due to significant global variation of NMSC data registration practices, NMSC incidence rates are difficult to determine definitively.4 5 However, in 2020, approximately 63 731 global deaths resulted from NMSC6 and 57 043 deaths from melanoma.6 Australia has the highest incidence of melanoma and NMSC globally,7 with two out of three Australians expected to be diagnosed with skin cancer within their lifetime.8 Australia’s incidence of melanoma is rising among the ageing population3 and 1405 deaths resulted from melanoma in 2019.9 As skin cancer constitutes the majority of all cancers diagnosed in Australia,10 this places a significant burden on the pubic healthcare system3 11 with Australia’s annual costs associated with NMSC reaching $700 million12 and melanoma $272 million.13

Australians’ elevated skin cancer risk is attributed to increased ultraviolet exposure due to equator proximity, lifestyle factors and preferences, extensive periods of time spent out of doors and a population made up of individuals from a predominantly European origin.14 In response to Australia’s high risk and increasing incidence of skin cancer,10 the Australian healthcare system has effectively evolved to meet this high demand for skin cancer management services, with specialist and generalist provider involvement.15

In the context of cancer management, general practitioners (GPs) typically act as an intermediary (providing support and referral) between patients and specialists.15 16 However, skin cancer is an exception to this rule. While specialist dermatologists manage a range of skin cancer cases in Australia, particularly more complex cases, GPs manage the majority of presentations17 and are routinely the first practitioners to encounter skin cancer.15 18 The overwhelming incidence of skin cancer in Australia necessitates the integral role of GPs in melanoma diagnosis, management and treatment.17 19

Reflecting the significant demand for skin cancer management, primary care skin cancer clinics have emerged over the past 20–30 years in Australia as treatment and care centres.15 17 These clinics are typically operated by GPs with a special interest in skin cancer, reflecting the evolution of special interests in primary care in Australia.15 20 Despite GPs’ management of most skin cancer cases in Australia, relatively little is known about the quality of care they provide and how quality of care should best be measured.15 17

Measurement of quality in healthcare

The Donabedian model is a conceptual model that suggests quality in healthcare is best measured according to a framework of domains: structure, process and outcome.21 Measuring quality in healthcare aims to improve performance and ultimately, patient care,22 with evidence citing reduced rates of hospitalisation23 and decreased mortality rates as a result of instated measurements.24 Quality indicators (QIs) provide healthcare professionals with a measure of their personal performance against their peers and act as a standard benchmark. Observing quality in this relative manner enables healthcare professionals to address any variation in their performance, which can be subsequently adjusted and improved.25 QIs have been developed and implemented effectively in multiple healthcare contexts, including general oncology.26

Despite Australia’s extensive and effective network of GPs in skin cancer management, no set of QIs appears to exist to benchmark quality in this rapidly expanding mode of service delivery. At present, GPs are able to refer to national protocols to guide their performance.22 For instance, specific protocols are available to guide practitioner performance around excision margins, infection rates, biopsy to treatment ratio and number needed to treat, which evidence-based guidelines suggest should fall within certain parameters.22 27 28 More commonly, however, the literature indicates that primary care clinicians rely on a combination of training, guidelines, experience, personal research and discussion with colleagues and patients.29 As a result, it currently appears that clinician decision-making practices are diverse30 and clinician and practice adherence to evidence-based guidelines and/or protocols is variable.31–33 While some variation in healthcare is necessary and warranted,34 unwarranted variation in healthcare (such as deviations in clinical decision-making, standards of care and adherence to evidence-based guidelines35) can have harmful consequences, for example, inadequate patient care.36 Unwarranted variation in healthcare can be identified and subsequently mitigated through the implementation of QIs.37 While discrete measures of quality may exist in skin cancer management at the level of clinic or practice, quality does not appear to be recorded in a uniform way, compared with a standard benchmark. Furthermore, there are no existing instruments and no clear structure for assessment of skin cancer care in primary practice, directly linked to QI development and assessment in Australia. Hence, the need for this study and its importance within the Australian context. As such, while we recognise that there are established international skin cancer guidelines28 38 and reporting standards (such as Guideline International Network QIs39), we are not focusing exclusively on those, but wish to draw on expert opinion to clarify existing practices. As little is known about how skin cancer guidelines are implemented in primary care, we believe this approach will generate a more practical set of QIs, framing our study around the development of appropriate measures of quality that may be of benefit to patients, clinicians and clinics.

Aims

This research aims to identify, develop and generate expert consensus on a core set of QIs for skin cancer management in primary care.

Objective

The primary objectives of this study are to:

Identify QIs from the literature on skin cancer management in primary care.

Develop a core set of QIs and reach consensus with key informants in skin cancer management on their relevance to skin cancer management in primary care.

Methods and analysis

Study design

A mixed-methods study will involve (1) a scoping review of the available evidence on QIs in skin cancer management in primary care, (2) semistructured interviews or qualitative proforma surveys (including demographic questions) with participants representing skin cancer clinical professionals and clinic managers to identify key informants’ perspectives on QIs in primary care skin cancer management, and (3) a facilitated focus group with a select number of key informants from step 2 employing Nominal Group Technique (NGT). Similar to semistructured interviews, qualitative proforma surveys, which contain free-text boxes for participant answers to set interview schedule questions, provide participants with an opportunity to present opinions and experiences in considerable detail.40 The NGT will involve the expert examination, appraisal and ranking of QIs drawn from the literature and from interviews or qualitative proforma survey completion, through a researcher-facilitated focus group and a subsequent scoring survey. The research study will take place from September 2021 to October 2022.

Due to the complex nature of the study, each step of the research design (see figure 1) is clarified in relation to the study setting, recruitment and sampling approaches and data collection and analysis methods.

Figure 1

Research study process.

Setting (steps 2–3)

Participants taking part in interviews, qualitative proforma surveys and focus groups (see the Data collection section) will be recruited from a wide range of healthcare settings across Australia and internationally, including clinics and public or private GP practices, hospitals, universities and other relevant community settings in order to ensure a breadth of perspectives.

Recruitment (steps 2–3)

Participants will be identified through research team members’ knowledge of experts working in the field of skin cancer and patient management in primary care and skin cancer clinics in Australia and internationally. Participants will be chosen purposively (contacted via email), to include senior clinicians, managers, administrators, policymakers, allied healthcare professionals, healthcare administrators, GPs, and clinical and non-clinical academics.

Sample (steps 2–3)

Approximately 20 participants will be recruited for step 2 (or until data saturation, where no new themes are evident in the data collected). The sample size rationalisation included literature on the optimal number of participants in research of this nature,41 the complexity of the topic,42 the open-ended, semistructured nature of discussion and heterogeneity of participant sample43 enabling a detailed examination of expert opinion and experience of skin cancer management and care. Purposive sampling42 will encourage that a multiplicity of perspectives are obtained from professionals across a wide range of disciplines. Participants will be encouraged to nominate others who are experts in the field (snowball sampling)44 to ensure sample characteristics are defined by those with the greatest knowledge of the field.44 Such diversity is expected to increase the uptake and acceptability of the core set of QIs. Through purposive sampling, the research team aims to ensure the sample includes dermatologists, primary care physicians, epidemiologists, health economists and academics with direct experience in QI development and/or implementation. Approximately 10–16 participants will take part in a subsequent focus group (step 3). The focus group sample size has been calculated in recognition of sampling attrition rates (20%).45 46 Focus group samples of this size have been stated as optimising participation and enabling strong group dynamics.46 47 The cohort will include a mix of gender, professional seniority, location (urban, rural and remote work locations) and work settings.

Patient and public involvement

Patients or the public were not involved in the initial study design. Participants will be involved in the dissemination of and development of the intervention.

Data collection

Scoping review (step 1)

A scoping review will be carried out to map the research that exists around QIs in skin cancer management in primary healthcare, and identify the types of available evidence while clarifying key concepts and definitions being used. We will be interested in research relating to structure, process or outcome measures of quality, according to the Donabedian model.21 These measures may be existing or newly implemented (and not yet standardised or termed ‘quality indicators’), for example, audit measures, patient satisfaction survey results, pathology reports, etc. We are purposefully keeping the scope of the review broad at this stage to understand the range and breadth of QIs reported in the literature.

Following the development of an appropriate search strategy (developed in association with an information expert), relevant studies will be identified from the following databases: MEDLINE, PSYCinfo, Embase (through Ovid), Scopus, Cochrane Library and CINAHL Complete. Two researchers will independently screen the results of an initial search to determine, based on title and abstract, whether studies are eligible for inclusion. If there is any disagreement, a third researcher will be involved in the decision. This process will be mediated by reference to the inclusion and exclusion criteria (table 1). Following agreement, papers will be read in full by two researchers to determine if they should be included in the study. QIs (which may be described otherwise, that is, standards, guidance, parameters, etc) will be identified and catalogued (according to the Donabedian model21) in data extraction tables.

Table 1

Scoping review inclusion and exclusion criteria

Inclusion criteria

Exclusion criteria

Articles reporting on skin lesions/neoplasms (benign or malignant) (skin cancer/skin lesions) Articles reporting on skin lesions/neoplasms in the context of primary care (general practice/skin cancer clinics) (primary care) Articles reporting on specific quality indicators (not general outcomes) or the use of outcomes as a measure of quality (quality indicators) Published in English (English) Qualitative, quantitative or mixed-methods methodologies (methodology) Peer-reviewed articles; conference abstracts; commentaries or editorials; guidelines; models of care; clinical pathways (any research type) No date restrictions have been applied (any date range) No location restrictions have been applied (international)

Papers exploring quality indicators relating to education or training in primary care skin cancer management (education/training) Papers exploring general outcomes, or non-specific measures of quality will be excluded, that is, reference to ‘quality’ generally (general/non-specific quality indicators) Full text that is unattainable

Demographic questionnaire (step 2)

A demographic questionnaire will be designed by the research team to clarify participants’ personal and work characteristics. Demographic questions will include details about: gender, country of residence, profession, time since qualification, qualification obtained, population served and years of experience working in the field of skin cancer. The demographic questionnaire will be sent to all participants prior to further data collection.

Interviews and qualitative proforma surveys (step 2)

Semistructured interviews will be conducted over the telephone or by video conference by the interviewer (BIL) who is a trained researcher with relevant qualitative research experience. All interviews will be audio recorded while any video files obtained during zoom calls will be immediately deleted leaving only audio files for analysis purposes. This will ensure confidentiality and de-identification of all data ongoing during data analysis. The researcher (BIL) will have no prior relationship with participants. She will make participants aware that she is a researcher and not a clinician, thus encouraging participants to define QIs and other technical terms where possible. The Participant Information and Consent Form (PICF) will ensure participants are made aware of the study aims and thus understand the researcher’s reasons for doing the research. The researcher (BIL) will take field notes throughout the interviews, recording details of participant responses, reactions, etc. Alternatively, participants may choose to complete a qualitative proforma survey (containing the same questions as the interview schedule contains)48 if more convenient. Participants will be asked if they would like to participate in step 3 (facilitated focus group) at the end of the PICF. The PICF will also advise participants of the time commitment required for participation: 45 min for interviews/qualitative proforma surveys and 90 min for the subsequent focus group (optional). A stratified random sample of interested participants will be invited to attend the focus groups (10–16 of the total cohort from step 2). The stratified sample has been chosen to ensure focus group participants represent a wide population mix.

Semistructured interviews will provide insights about participants’ understanding of QIs suitable for use in primary care, and their potential to improve patient care and clinical practice. Semistructured interviews will allow for a rich understanding of healthcare contexts, participant opinion and experience.49 Interviewing individuals working in the field of skin cancer management and care is likely to provide invaluable insights into not only the effectiveness of introducing a new QI and quality assessment tool, but also the feasibility of so doing in primary care skin cancer management settings. Questions included in the interview schedule (also forming the qualitative proforma survey) will be developed through team discussion and with reference to the relevant literature, the study aims and study objectives and will include:

Characteristics of QIs (structure, process and outcome measures of quality reported by individuals to be of benefit to patients, clinicians and/or other allied healthcare professionals).

Care, processes and management of skin cancer.

Appropriateness, acceptability and effectiveness of QIs.

Factors that might support or impede implementation of QIs in primary care.

The interview schedule will be piloted with two participants to test it for clarity and relevance. If there are no issues arising, pilot data will be incorporated into the full dataset and the schedule will be rolled out in the study proper.

Facilitated focus group and scoring survey (adapted Nominal Group Technique) (step 3)

Step 3 aims to provide context and rationale for the QIs identified and developed in the interviews and qualitative proforma surveys and to reach consensus among participants on key QIs for use in primary care contexts.50 Consensus methods such as the NGT are useful for removing outliers from data, resolving issues of misunderstanding and developing unity among participants in important discussions.46 Commonly, the NGT develops a consensus over four distinct phases: (1) the population considers the ideas and questions under review, (2) these considerations are then openly shared with the group involved in the consensus building activity, (3) ideas are discussed among the group, and finally, (4) a voting phase takes place, whereby individuals each score and rank ideas privately to ensure the perspectives of each individual have been fairly captured and addressed.50 Members of the research team have significant experience using this technique50–52 which will be adapted for use in the present study. Detailed steps of the adapted NGT are elucidated in figure 2, where consensus will be developed through a facilitated focus group accompanied by two scoring surveys. The focus group will be conducted by trained researchers and conducted and recorded online using video conferencing software.53

Figure 2

Individual steps in the adapted Nominal Group Technique. GP, general practitioner.

Data analysis (steps 1–3)

Data from the scoping review will be synthesised to derive key QIs (which will be catalogued according to the Donabedian model21). Data from the demographic questionnaire will be reported using descriptive statistics. Qualitative data from interviews, the qualitative proforma surveys and the focus groups will be analysed using thematic analytical methods54 supported by NVivo software (released in March 2020).55 Following data collection, data will be collaboratively coded by members of the research team. Themes will be identified, reviewed and defined.54 Data from steps 1 and 2 will be presented in a summary form and used to facilitate the focus group discussion. The list of QIs developed during focus group discussion will be distributed to individual participants via REDCap56 software for private scoring and ranking, ensuring the rigour application of the NGT. The final list will be refined to produce a definitive list of 12 key QIs (10–12 indicators have been a proven optimal number around which consensus can be derived).51 52 57

Ethics and dissemination

Ethical considerations

Ethics approval for this study has been obtained and research will only be conducted following signed consent from all participants. Approval was granted by the university’s Research Ethics Committee (HREC no. 520211051532420).

Data storage and retention

All electronic data will be stored on the main university’s secure server (only accessible by members of the research team). Any hard copies will be stored in a locked filing cabinet at the main university campus and identifying details of participants will be kept separate from the transcripts and surveys themselves.

Dissemination

Study outcomes will be widely disseminated through peer-reviewed publication, national and international conference presentations, educational events (teaching the methodological approach used in this study) and funding reports. Any verbatim quotations that are included in publications, presentations, educational events or reports will remain de-identified. A final executive summary will be prepared for all interested clinical and non-clinical parties.

Significance and impact of the study

While QIs have been implemented in various healthcare contexts, including oncology,26 a standardised set of QIs has yet to be developed in the context of primary care skin cancer management. This research is expected to result in:

The development of appropriate, acceptable and effective QIs in skin cancer management in primary care.

Increased understanding and knowledge of QIs in skin cancer management.

Conducting a scoping review alongside interviews and qualitative proforma surveys will drive the development and delivery of a core set of QIs in skin cancer management nationally. Through further discussion, a unified set of QIs will be developed for those working in primary care nationwide. These QIs are expected to be appropriate, acceptable and effective from the perspective of primary healthcare professionals and GPs more widely. Planned future research includes development of a QI intervention that will be rolled out across Australia.