Pragmatic, consensus-based minimum standards and structured interview to guide the selection and development of cancer support group leaders: a protocol paper.

INTRODUCTION: Across the globe, peer support groups have emerged as a community-led approach to accessing support and connecting with others with cancer experiences. Little is known about qualities required to lead a peer support group or how to determine suitability for the role. Organisations providing assistance to cancer support groups and their leaders are currently operating independently, without a standard national framework or published guidelines. This protocol describes the methods that will be used to generate pragmatic consensus-based minimum standards and an accessible structured interview with user manual to guide the selection and development of cancer support group leaders. METHODS AND ANALYSIS: We will: (A) identify and collate peer-reviewed literature that describes qualities of support group leaders through a systematic review; (B) content analyse eligible documents for information relevant to requisite knowledge, skills and attributes of group leaders generally and specifically to cancer support groups; (C) use an online reactive Delphi method with an interdisciplinary panel of experts to produce a clear, suitable, relevant and appropriate structured interview comprising a set of agreed questions with behaviourally anchored rating scales; (D) produce a user manual to facilitate standard delivery of the structured interview; (E) pilot the structured interview to improve clinical utility; and (F) field test the structured interview to develop a rational scoring model and provide a summary of existing group leader qualities. ETHICS AND DISSEMINATION: The study is approved by the Department Human Ethics Advisory Group of The University of Melbourne. The study is based on voluntary participation and informed written consent, with participants able to withdraw at any time. The results will be disseminated at research conferences and peer review journals. Presentations and free access to the developed structured interview and user manual will be available to cancer agencies. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Novel and robust method for developing a structured interview using an interdisciplinary panel of experts.

Protocol designed to be feasible, acceptable and valid in a community setting.

Development of the first pragmatic and consensus-based minimum standards for the selection and development of cancer support group leaders.

Studies described in the protocol will not ascertain competency level of the support group leader once in the role nor address cross-cultural adaptation of the structured interview.

Self-reporting of knowledge, skills and attributes by potential group leaders is subjective and may be incorrect, incomplete or biased.

Introduction

The number of cancer cases across the globe has grown rapidly, along with improved survival due to increased rates of early detection and better access to effective treatment in developed countries.1 With no centralised registry, the exact number of peer support groups for cancer survivors led by peers is unknown but thought to be considerable. Peak cancer agencies have established relationships with support groups in an effort to sustain and strengthen delivery of peer support. Agencies across the globe have extended funding, training, resources and support staff to independently run groups, with leaders being the primary point of contact. For those who either access support groups or recommend them as a low-cost psychosocial support, having a trained group leader is thought to be an important component to a group’s effectiveness.2 However, challenges have been reported in maintaining quality of life and burn out in group leaders who are mainly volunteers, often with a diagnosis of cancer themselves. The ability of the individual to function within the role and maintain this role over a period of time is important for group sustainability.3 However, little is known about the essential qualities of group leaders or how to determine a person’s suitability for the role.4

Initial scoping revealed the lack of a relevant role analysis or, indeed, any detailed synopsis of the knowledge, skills and attributes required for the cancer support group leader role. It also failed to uncover published guidelines, standards or tools to guide the selection and development of leaders of cancer support groups, yet these are needed to inform policy and practice within and across organisations involved with these groups.4 Given the very specific nature of the cancer support group leader role, a rigorous, robust and systematic approach to the development of minimum standards and a tool to assess suitability and readiness to undertake the support group leader role based on these standards is warranted. As a first step, minimum standards, rather than best practice standards, are needed to enrich the quality of support being delivered in the community by existing group leaders. Additionally, any tool to assess suitability and readiness will need to balance contextual demands (ie, the reality that most peers are volunteers and the fact that resourcing for selection and development are limited) against psychometric demands (ie, the validity and reliability of the interview). This protocol seeks to describe and justify the methods that will be used to develop pragmatic, consensus-based minimum standards and a structured interview with user manual to guide the selection and development of cancer support group leaders. Here, the intended aim of better selection and development is to enhance the experience of both group leaders and members and to maximise the sustainability of cancer support groups in the community.

In the absence of a single agreed approach to developing minimum standards, this study drew on methods used by the International Society for Quality of Life Research (ISOQOL) to develop minimum standards for the design and selection of patient-reported outcome measures for use in patient-centred outcomes and comparative effectiveness research.5 These methods were considered appropriate for at least three reasons. First, the authors employed a compatible definition of a minimum standard, with a focus on the identification of critical attributes and judgements of suitability. Second, the approach described facilitated identification of best practice standards in addition to minimum standards. Third, many of the identified standards for patient-reported outcome measures are relevant to the design of structured interviews.6 As such, these standards will be given consideration when developing the structured interview, for example, the knowledge, skills and attributes to be revealed by interview questions (content validity), the interpretability of scores and interviewer–interviewee burden.

A structured interview assessing role-related dimensions was considered the selection technique of choice for at least three reasons. First, interviews are a popular selection technique,7 so most organisations likely to use this tool will be familiar with the interview process. Second, compared with other selection techniques, interviewees perceive interviews as fair.8 Interviews are also seen as an expected part of the selection process.9 Third, compared with ‘unstructured’ interviews, judgements based on more highly structured interviews are more predictive of job performance,10 where structure refers to any enhancement that increases standardisation of the interview content and evaluation. According to Campion et al,6 there are 15 components of structure that can be manipulated to increase the validity of interviewer evaluations. More recently, however, Dipboye et al in 200411 described a tighter conceptual framework corresponding to the life cycle of an interview, which includes interview development, conduct and evaluation. Validities can be maximised by enhancing: job-relatedness (or role-relatedness) in the development of the interview; standardisation of process in the conduct of the interview; and increasing structure in the use of the data for interviewee evaluation and decision making. Increasing structure in the use of data can be achieved by utilising behaviourally anchored rating scales, formal (or statistical) methods for combining ratings and consistently applied decision rules. In this study, we will aim to optimise all three dimensions in our structured interview.

Finally, the interview will incorporate assessment of both suitability and readiness; prior experience as a group leader is likely to be the exception rather than the rule, so it seems unreasonable to expect those who seek to undertake the group leader role to be ready at the outset (ie, have all requisite knowledge, skills and attributes).

To enable broad uptake and integration into routine practice, the minimum standards and structured interview need to be readily understood, appropriate and acceptable to end-users.12 13 Aspects of clinical utility—including appropriateness, accessibility, practicability and acceptability14 —will be considered and appraised at various stages throughout the project.

Study objectives

This study aims to:

identify and summarise literature describing qualities of cancer support group leaders

identify minimum and best-practice standards for the role of a cancer support group leader

produce, in draft form, a structured interview designed to assess the knowledge, skills and attributes of individuals who seek to undertake the cancer support group leader role

produce, in draft form, a user manual to facilitate standard delivery of the structured interview

pilot test the structured interview to appraise aspects of clinical utility including usability and acceptability to end-users

field test the structured interview and use results to establish a rational scoring model and produce preliminary data on the knowledge, skills and attributes of current cancer support group leaders

disseminate guidelines and minimum standards to audiences in academia and cancer agencies for uptake.

have an accessible study protocol to facilitate knowledge transfer and assist others to further develop the structured interview.

Study outputs

We aim to generate three main outputs:

pragmatic, consensus-based minimum standards for the role of a cancer support group leader

a structured interview to guide cancer agencies involved in the selection and development of support group leaders

a user manual for cancer agency workers conducting the structured interview.

Methods and analysis

Study design

Systematic literature review, online reactive Delphi study, as well as a pilot and field test of the structured interview undertaken between 2014 and 2017 (figure 1).

Figure 1

Overview of protocol study aim, design, methods and outputs. A flow chart outlining the four mixed-method study steps, to be undertaken from 2015 to 2017, to develop pragmatic consensus-based minimum standards and a structured interview to guide the selection and development of cancer support group leaders. Boxes coloured in dark steel blue represent these study outputs.

Systematic literature review

Systematic reviews are routinely used in healthcare to ensure justification for further research and as a starting point for developing clinical practice guidelines.15 In this study, we will undertake a systematic review as part of a job (or role) analysis, that is, ‘a thorough and systematic analysis of the job for which the candidate is being considered’ (ref 11, p. 300). Possible task dimensions and the knowledge, skills and attributes (or qualities) required to successfully undertake the role will be the focus of this review. A role analysis is crucial to the design of a structured interview, including its questions and rating scales. It provides an ‘analysis of the fundamental behavioural dimensions underlying this content’ (ref 11, p. 300) and, as stated above, the predictive validity of interviewer evaluations may be enhanced by ensuring the role-relatedness of interview content.16 17

Consultation will occur with a specialist librarian to identify appropriate electronic databases and publication dates and to generate combined subject heading and text word searches to maximise scope and increase relevancy. The PRISMA statement, checklist and flow diagram will be used to optimise the review. PROSPERO format will be followed for the systematic review; however, as the study is not intervention focused, it does not meet eligibility criteria and therefore will not be registered.

Inclusion and exclusion criteria will be set by the research team to ensure the content derived from the literature is relevant to adult peer support groups in health settings. All citations identified through database searches and reference lists will be reviewed by an author and screened for eligibility. Any uncertainty regarding eligibility will require review and discussion with a second coauthor. At a minimum, data extracted from eligible documents will include year of publication, country, study design, method, group type, sample description, group leadership and group leader qualities. Summarising content analysis will be used to analyse eligible documents for content relevant to qualities of support group leaders. All extracted data will be entered into an Excel spreadsheet, then imported into R (reference index V.3.1.3 or higher) for analysis and graphing; the R package ‘ggplot2’ will be used to prepare graphs.18 Descriptive statistics will be used to summarise data from all eligible documents and by group type (cancer or non-cancer and mixed). The output of this phase will be a provisional list of requisite knowledge, skills and attributes that will feed into the next stage of the project—the online Delphi study.

To ensure the breadth of content relevant to and representative of knowledge, skills and attributes required by cancer support group leaders, the review will include a wide range of research studies (ie, qualitative, quantitative and mixed methods) and then be synthesised qualitatively. Thematic synthesis will be used to formalise the identification and development of themes. This method can be applied to systematic reviews that address questions about people’s experiences and perspectives.13

Online Delphi study

The purpose of the Delphi technique is to enable reflection and discussion among a panel of experts with a view to getting as close as possible to consensus and documenting both the agreements reached and the nature and extent of alternative opinions.19 In this study, an online reactive Delphi method will be used to obtain expert agreement on the minimum standards (or qualities considered essential to the role) and the content and structure of the structured interview. In the development of highly structured interviews, subject-matter experts are usually engaged to provide input into the analysis of the role for which candidates are being considered,11 in this case, to judge the importance of putative task dimensions and the knowledge, skills and attributes required in the support group leader role. Their opinion is sought on the boundaries of the behavioural dimensions as well as the knowledge, skills and attributes crucial to performing well on each dimension.

Influential factors on the quality of the Delphi process include: composition (expertise, diversity) of the expert panel; selection of background literature and evidence to be discussed by the panel (validity, representativeness and completeness); adequacy of opportunities to read and reflect (balance between accommodating experts’ time limitations and keeping the study to a timeframe); qualitative analysis of responses (depth of reflection and articulation of key issues); quantitative analysis of responses (accuracy and appropriateness of statistical analysis and clarity in feedback); and how difference and ambiguity are treated (avoidance of ‘group thinking’).19 20

Evidence suggests that an online medium is more likely to improve quality of the consensus development process.20 In addition, online communication has well-established benefits in promoting construction of knowledge and reflection.21 There are also examples of successful online Delphi studies conducted across geographical locations of participants.19 22 This is of particular importance for this study, as participants are likely to be dispersed across the country. Reported benefits also include no cost and flexibility for participants with scheduling completion of responses.23

Given the broad range of participant perspectives and likely large number of knowledge, skills and attributes to be considered in Delphi rounds, an acceptable range of consensus will be required. The definition of consensus for this study is 75% agreement.24–26

During the Delphi study, based on content from the systematic review and previous Delphi rounds, the research team will determine the appropriate presentation of data, estimate time needed to collect data, analyse and feedback the results to participants and how to enhance response rates.19 27

Panel participants

A key component of a successful Delphi process will be to include different perspectives by recruiting a wide range of experts involved in the research, referral, support and delivery of cancer support groups. Expert panel participants will be from Australia and purposely selected by the research team through professional networks and snowballing,28 when participants suggest other potential participants. The Delphi panel will include experienced academics, health professionals, cancer agency workers and cancer support group leaders. Individuals also will be identified during the initial consultation phase of the project with various stakeholders relevant to the cancer support groups. Experienced support group leaders will be identified from current support group listings of three peak cancer agencies. A minimum of 10 is considered acceptable for a Delphi study.29 To ensure equal representation across expert groups, this study aims to recruit around 30 participants.

Delphi rounds

The Delphi panel will be conducted anonymously and entirely via email. Sequential rounds of questionnaires will be developed, with set completion periods in order to allow for feedback integration and progression to the next round. Consent will be assumed if participants email completed questionnaires back, with responses to be saved electronically and coded and deidentified. In order to create a structured interview applicable across all cancer types, questions will not be specific to a particular cancer type. As required, participants’ responses to each Delphi round will be entered into an Excel spreadsheet, then imported into R (reference index V.3.1.3 or higher) for analysis and graphing. Descriptive statistics will be used to summarise participants’ responses. Summarised results will be returned to participants in the form of another questionnaire.

Round 1

Round 1 will consist of a questionnaire with an initial pool of requisite knowledge, skills and attributes of support group leaders deduced through the systematic literature review. Panel members will be asked to give their opinion about each quality, expand on the content and identify additional knowledge, skills and attributes to ensure the pool is relevant to and representative of qualities required in the cancer support group leader role. Second, experts will be asked to identify attributes or qualities that would automatically exclude someone from being a group leader. Responses will be summarised in a set of provisional statements, listed in a table and sent to participants for ranking (round 2).

Round 2

The second round will canvass opinions and reach consensus on key cancer support group leader qualities. This set of qualities will form the minimum standards; these will be used to develop the structured interview (described as part of round 3). Participants will be asked to confirm the relevance of listed knowledge, skills and attributes for the support group leader role. The purpose of this round will be for experts to determine what knowledge, skills and attributes of support group leaders are required or considered essential to undertake the role, compared with what qualities are ideal. An acceptable range of consensus will be based on 75% or more agreement by experts for each attribute (eg, individual knowledge, skills and attributes).

Round 3

A structured interview will be drafted by the research team with the aim of optimising the predictive validity and reliability of interviewer evaluations. We will do this by: ensuring good coverage of consensus qualities (or role-relatedness of the interview); ensuring a mix of questions (ie, situational, behavioural and experience), constraining phrasing of questions and limiting the use of follow-ups and probes (or standardisation of the interview process); and using behaviourally based rating scales (or structured use of data to evaluate candidates).11 The draft structured interview will be distributed to the expert panel and will form the basis of a third Delphi round. This will include questions assessing: the technical quality of structured interview questions; the suitability of limited probes; proposed ratings for each answer; and the technical quality and appropriateness of behaviourally anchored rating scales. The panel will be asked to assess whether the interview is clear, suitable, relevant and appropriate. Separation and categorisation of knowledge, skills and attributes required for selection purposes compared with development needs will also be confirmed in this round.

Development of the user guide

Usability has the potential to impact on the usefulness, effectiveness, efficiency, learnability and satisfaction users can achieve with a particular product or service.30 One objective of this study is to produce a publicly and freely accessible user manual to support the uptake and delivery of the interview by cancer agencies. Taking into account the format and orientation of other comparable materials, we will develop a user guide to increase ease of use and standardisation of the interview process. Development of a user guide is also intended to be a pragmatic way of providing interviewer training.

It is suggested that the rating scale used be as simple as possible, well defined and with the ability to identify development areas.30 For example, a rating scale could be an ordinal-level scale (eg, experienced, intermediate and not suitable) or as simple as acceptable or unacceptable. Therefore, an orientation or training for interviewers is highly recommended. Given the practical and time constraints on potential users to access training or support to learn and understand the structured interview, the research team decided to develop a user manual to accompany the structured interview employing usability methods to ensure optimum usability.30

A set of instructions will be developed on the use of the structured interview—its questions, probes and behaviourally anchored rating scales—consistent with Campion et al to reduce subjectivity and inconsistency. The background and purpose of the interview will be outlined, along with rapport-building techniques. How to ask questions and how to probe further will be explained. Instructions on how to record and evaluate answers will be given as well as how to use the rating scale. Interviewers will be directed to focus on descriptions rather than judgements and facts rather than opinions. The importance of note taking will be stressed to provide documented evidence of the interview and objective rating of responses. How to avoid potential rating errors will be outlined such as: first impressions, contrast effect and personal bias.

Development of effective instructions will increase consistent application of the interview and allow interviewers to evaluate potential group leaders from a common reference point. The structured interview and accompanying user guide have been developed as a stand-alone measure for selecting and developing group leaders, based on agreed minimum standards. However, we anticipate that these resources may need to be adapted and perhaps supplemented by cancer agencies based on their own organisational needs and requirements.

Pilot testing

A small-scale pilot study will be conducted to appraise aspects of clinical utility including usability and acceptability to end-users. Three cancer agency workers who have direct contact with cancer support groups will be recruited to conduct the interviews. Workers will be selected from different cancer agencies. Workers will be asked to read the user manual and familiarise themselves with the structured interview schedule and standard form for documenting interview responses. Cancer agency workers will record and rate support group leaders’ responses using the standard form, with interviews audio-taped. A total of 12 current support group leaders will be recruited via three peak cancer agencies. Leaders will also be asked to take part in a telephone-based interview.

After conducting the newly developed structured interview, cancer agency workers will be asked to provide feedback on their experience through semistructured interviews with a member of the study team (AP). Feedback will be solicited on the ease of use, time involved, selection process, potential barriers to implementation and likelihood of using the structured interview in their current practice. Interviews will not be transcribed, but notes will be taken by the researcher during the interview. Responses will be synthesised and then used to review the tool to determine what components worked well and what should be further improved. Results obtained by the cancer agency worker regarding the participants’ suitability and readiness for the group leader role will be cross-checked by the researcher. This will involve the researcher (AP) listening to the audio recording of each pilot interview and comparing scores with that of the pilot interviewer. Format, questions and instructions will be revised as required.

Field testing

A large-scale field test will be undertaken for two main purposes: to provide a summary of existing group leader qualities and to develop a rational scoring model. In more structured behavioural interviews, the interviewer provides numerical ratings on each of several dimensions and the interview is ‘scored’ by statistically combining the interviewer’s ratings. Therefore, the use of statistical combinations of data, rather than clinical predications, to form judgements yield better results.11 In this case, the knowledge, skills and attributes of current support group leaders will be used as a benchmark to appraise the reasonableness of behaviourally anchored rating scales to interview questions. They will also be used to establish appropriate and acceptable cut scores for suitability and readiness.

Current cancer support group leaders will be recruited through three peak cancer agencies that support cancer support groups and cancer support group leaders. A network of leaders from 170 prostate cancer support groups and over 300 breast cancer support groups will be invited to participate in the field testing. Structured interviews will be conducted over the phone by cancer agency workers from collaborating cancer agencies. Participation will be voluntary and anonymous. Interviewers will be asked to complete interviews, approximately 10 to 20 interviews each. Support group leaders will be asked a small number of questions for the purposes of characterising the study sample before taking part in the structured interview (eg, age, gender, support group type (breast, prostate) and time as support group leader).

Support group leaders’ responses to demographic questions and interviewers’ ratings to interview questions will be entered into an Excel spreadsheet, then imported into R (reference index V.3.1.3 or higher) for analysis and graphing. Descriptive statistics will be used to summarise sample characteristics and participants’ responses to the structured interview questions. This will be done for the full sample and by support group type (breast and prostate). Interviewer ratings to interview questions along with interviewer ratings of suitability and readiness will be used to determine appropriate and acceptable cut scores.

Discussion

Despite substantial numbers of peer-based cancer support groups being in operation, there are currently no existing guidelines or minimum standards relevant to the selection and development of group leaders. A pragmatic, consensus-based structured interview with user manual may help organisations rationalise the provision of support and assistance to cancer support group leaders. In addition, establishment of consensus-based minimum standards may help reduce concerns of clinicians and potential barriers in referral pathways.

The proposed study will use accepted qualitative and quantitative methodologies—a systematic review and qualitative synthesis, a Delphi study with an interdisciplinary panel (three rounds) along with pilot and field testing—to develop clinically relevant and acceptable minimum standards and a means to implement these standards in the selection and development of cancer support group leaders. We hope the use of these outputs will lead to greater consistency, equality and targeted use of limited cancer agency resources available to support cancer support groups. We also believe our approach and outputs (minimum standards and structured interview) could be used or adapted for other healthcare or community settings where peer support groups are in operation.

Conclusions

The development of pragmatic and consensus-based minimum standards is an important first step in building a framework for support group leader selection and development. The aim of this study is to assist cancer agencies in their selection and development of support group leaders and lead to greater consistency and equality across agencies. It is recognised that due to the varying types of support groups, along with different relationships and supports provided by cancer agencies to support groups, it would be detrimental to be overly prescriptive about what must be covered in the application of the standards. Instead, these standards are intended as a starting point with the need for ongoing review and development. It is also hoped that following field testing, further research is undertaken to determine the appropriateness of the content and structure in other countries. By contributing to the model of peer support in this way, it is hoped that we can optimise the value of the cancer support group experience for leaders and group members.

Ethics and dissemination

All procedures proposed in this study involving human participants are approved by the Department Human Ethics Advisory Group, Melbourne School of Psychological Sciences, The University of Melbourne, Victoria, Australia (Minimal Risk application 1443027.1). Informed consent to participate in the study to be obtained from participants as outlined in approved application. The three main study outputs will be reported in publications and conference presentations, with final versions of the structured interview and user manual freely accessible to cancer agencies. Consideration will be given to transferability of outputs to different cancer groups (eg, different disease types and stages, and different genders).