'Difficult Conversations': evaluation of multiprofessional training.

OBJECTIVES: Evidence-based communication skills training for health and social care professionals is essential to improve the care of seriously ill patients and their families. We aimed to evaluate the self-reported impact of 'Difficult Conversations', a multidisciplinary half-day interactive workshop, and gain feedback to inform future development and evaluation.
METHODS: Service evaluation using questionnaire data collected before and immediately after workshops from February 2015 to August 2016 regarding participant self-assessed confidence, knowledge and skills. Qualitative free-text comments provided feedback about the workshop and were subjected to content analysis.
RESULTS: Of 886 workshop participants, 655 completed baseline questionnaires and 714 postworkshop questionnaires; 550 were matched pairs. Participants were qualified or trainee general practitioners (34%), community nurses and care coordinators (32%), social care professionals (7%), care home staff (6%), advanced practice/specialist nurses (5%), care workers (5%) and allied health professionals (3%). All groups demonstrated significant increases in mean self-assessed confidence (2.46, 95% CI 2.41 to 2.51; to 3.20, 95% CI 3.17 to 3.24; P<0.001), knowledge (2.22, 95% CI 2.17 to 2.27; to 3.18, 95% CI 3.14 to 3.22; P<0.001) and skills (2.37, 95% CI 2.32 to 2.42; to 3.09, 95% CI 3.05 to 3.12; P<0.001). Qualitative findings showed participants valued role play, the communication framework acronym and opportunities for discussion. They commended workshop facilitators' skills, the safe atmosphere and interprofessional learning. Suggested improvements included more prepared role play and greater coverage of the taught topics.
CONCLUSIONS: 'Difficult Conversations' workshops were associated with improvements in participants' self-assessed confidence, knowledge, and skills. Our findings identify workshop characteristics that are acceptable to multidisciplinary trainees. Further testing is warranted to determine effectiveness and accurately identify workshop components leading to change. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Introduction

Clear, compassionate communication is important to patients with life-limiting disease and their families.1 This is reflected in policy commitments to improve this area of care.2 However, poor experiences of communication remain a principle area of complaint within the British National Health Service (NHS)3 and are associated with patient4 and caregiver5 distress and poor staff outcomes.6 Therefore, providing effective, evidence-based communication skills training remains a priority.

‘Difficult Conversations’ is a multidisciplinary, half-day interactive workshop developed by experienced palliative and end of life care (EoLC) clinicians (CM-S and JN). It aims to equip health and social care professionals with the knowledge, confidence and skills required to have potentially difficult conversations with patients with serious and life-threatening illness and their families. This may include, but is not limited to, conversations around diagnosis, deterioration of health, and advance care planning. In brief, the workshop commences with an interactive seminar, video examples and group work covering the principles of breaking bad news, and introducing the ‘SCARS’ communication framework. The ‘SCARS’ acronym presents an ‘aide memoire’ to help navigate difficult conversations: Setting, Communicate with kindness, Ask, Respond and reflect and Summary and plan. Following agreement of ground rules, participants engage in role play sessions. Finally, participants discuss their roles and responsibilities and learn about mental capacity and advance care planning, including relevant legal information. Please see online supplementary 1 for further details. This format reflects other communication skills training7–9 but uses a unique communication framework and targets multidisciplinary groups.10

‘Difficult Conversations’ has trained over 1600 professionals and is endorsed by the British General Medical Council and Royal College of General Practitioners South London Faculty. However, effectiveness has not been externally evaluated. Here we report a service evaluation examining the participant-assessed impact of ‘Difficult Conversations’ and summarising their feedback to inform future workshop refinement and evaluation.

Methods

Design, recruitment and data collection

This service evaluation uses a pre-post test design. Data were collected from ‘Difficult Conversations’ workshops from February 2015 to August 2016, across community settings in South and North West London serving a population of 4.7 million.11

Participants completed a questionnaire before the workshop (baseline) and immediately after. The precourse questionnaire asked participants to rate their confidence, knowledge and skills in conducting difficult conversations (eg, breaking bad news or about EoLC) with patients and carers on a 4-point Likert scale (eg, from ‘not confident’ to ‘very confident’). The postcourse questionnaire reassessed these three domains, and asked additional questions regarding the likelihood the workshop would improve their practice, and whether they would recommend the workshop to a colleague. Free-text questions asked participants to provide feedback regarding what they enjoyed about the workshop and how it could be improved.

Analysis

Questionnaires that included participants’ names were matched and assigned an anonymous paired ID prior to analysis. Differences between matched baseline and postworkshop scores were analysed using paired t-tests. Independent t-tests were used to compare scores between matched and unmatched questionnaires at baseline and postworkshop. Due to concerns regarding treating the 4-point Likert scale data as parametric interval data, non-parametric alternatives were also used to check results with a more conservative method. Due to multiple testing, Bonferroni corrections were applied.

Free-text data from all postworkshop questionnaires were summarised using content analysis.12 Themes were inductively identified for each question, and each answer was coded at one or more themes as appropriate.

Results

Self-assessed confidence, knowledge and skills

Of 886 workshop participants, 655 (74%) completed baseline questionnaires and 714 (81%) postworkshop questionnaires; 550 represented matched pairs. The majority of participants were general practitioners (qualified GPs: n=159, 29%; trainee GPs: n=29, 5%), community nurses and care coordinators (n=175, 32%), social care professionals (n=39, 7%), care home staff (n=31, 6%), advanced practice and specialist nurses (n=30, 5%), care workers (n=30, 5%) and allied health professionals (n=18, 3%). Others (n=39, 7%) included a range of professions including hospital doctors, managers and receptionists. For more information on these staff groups, please see  box S1 in online supplementary 1.

Self-rated confidence, knowledge and skills in conducting difficult conversations with patients and their families all increased significantly from baseline (table 1). This remained true when data for each staff group were analysed separately. The largest improvements in confidence, knowledge and skills were observed for care workers (baseline means: 2.27, 1.97 and 2.07; mean change scores: 1.07, 1.33 and 1.07, respectively), while the smallest were for qualified GPs’ confidence and skills (baseline means: 2.63, 2.58; mean change scores: 0.61 and 0.53, respectively) and trainee GPs’ knowledge (baseline mean: 2.24; mean change score 0.72).

Table 1

Self-assessed confidence, knowledge and skills

Item	n	Baseline		Postworkshop		Paired t-test*
		Mean	95% CI	Mean	95% CI	t	P
Confidence	547	2.46	(2.41 to 2.51)	3.20	(3.17 to 3.24)	−29.5	<0.001
Knowledge	550	2.22	(2.17 to 2.27)	3.18	(3.14 to 3.22)	−37.6	<0.001
Skills	548	2.37	(2.32 to 2.42)	3.09	(3.05 to 3.12)	−27.6	<0.001

*Results were also significant (Ps<0.001) with non-parametric Wilcoxon signed-rank tests.

Scores for matched questionnaires (included in the paired analysis) and unmatched questionnaires (that could not be included in the paired analysis) did not differ significantly at baseline or postworkshop on confidence, knowledge and skills, or postcourse perceived impact on practice. However, scores for unmatched questionnaires were significantly lower than matched questionnaires as to whether participants would recommend the workshop to colleagues (mean 3.74, 95% CI 3.67 to 3.81 vs mean 3.85, 95% CI 3.82 to 3.88; t=2.872, P=0.004).

Participant feedback

Of 714 postworkshop questionnaires, 666 included views on what participants valued about the workshop. Many positive comments were about the interactive teaching methods (n=102) and enjoying the workshops (n=88). Participants particularly valued role play (n=168), the ‘SCARS’ communication framework (n=104), opportunities for discussion and group work (n=80) and the videos (n=43). Many commented on the relevance of the topics discussed (n=79), highlighting the following as particularly useful: Do Not Attempt – Cardio-Pulmonary Resuscitation (DNA-CPR) orders, mental capacity, and advance care planning. Participants stated the workshop provided knowledge (n=112), confidence (n=27) and skills (n=17). They praised the expertise of the workshop facilitators (n=64), noted the benefits of interprofessional learning (n=36) and spoke positively of the safe and informal atmosphere (n=36).

There were 227 suggestions on how the workshop could be improved. The most related to improvements in the role play (n=59): requesting more time spent on this activity (n=21; n=5 requested less) and 26 suggesting the scenarios be preprepared and specific, rather than generated with participants during the workshop. There were also requests for longer workshops (n=45) and to focus on specific topics in greater detail (n=43), including advance care planning, mental capacity and DNA-CPR orders.

Almost all participants completing the postworkshop questionnaire said the course was likely (33.1%) or highly likely (66.1%) to improve their practice and that they were likely (17.1%) or highly likely (82.6%) to recommend the workshop to colleagues.

Discussion

We found increases in the self-reported confidence, knowledge and skills of those attending the ‘Difficult Conversations’ workshops. These increases were most prominent among care workers who, in our ageing population, will likely play an increasing role in generalist palliative and EoLC for the frail elderly.13 This may reflect less previous training in this area, compared with those working in, for example, general practice. Alternatively, this may relate to differing training needs and expectations across disciplines. Further work is needed to understand the impact of these differences on training outcomes. However, regardless of professional group, all participants reported significant benefits of the workshop.

Participants particularly valued the role play, the ‘SCARS’ communication framework and opportunities for discussion and group work. They also noted the skills and sensitivity of the facilitators, the safe learning atmosphere and the opportunity for interprofessional learning. These findings match evaluations of other similar training courses (ie, short duration workshops using role play and a communication framework), which also report self-assessed improvements in participants’ abilities.7–9 The value participants consistently place on role play and discussion,9 communication frameworks8 and providing a safe atmosphere to learn and practice7 suggests these could be key elements of acceptability. To what extent they represent ‘active ingredients’ of the intervention warrants further exploration. Although interprofessional learning is rare in EoLC communication skills trainings,10 it was viewed favourably by participants attending the ‘Difficult Conversations’ workshop. How this relates to the learning experience should be tested.

This evaluation has limitations. First, there might be unknown differences between the majority (80%) of participants who shared names on their questionnaires (facilitating matching) and those who did not. Second, the uncontrolled nature of the evaluation means there might be alternative explanations for the increase in self-assessed abilities. Third, the questions asked of participants have not been psychometrically tested. Moreover, we do not know whether there is a lasting impact beyond the immediate postworkshop measurement, nor whether the perceived increase in confidence, knowledge and skills has beneficial effects on patients and their families. Future evaluation would therefore benefit from a controlled design, longer term follow-up and use of validated staff-reported and patient/carer-reported outcome measurement.10 14 Such work should be implemented in line with the MORECare statement on evaluating complex interventions in EoLC.15

This service evaluation of the ‘Difficult Conversations’ workshop indicates a self-reported improvement in participants’ confidence, knowledge and skills. This favourable finding encourages continuation and development of the course alongside rigorous evaluation, in addition to exploring potential mechanisms of action. Based on our initial findings, we recommend other communication skills courses consider their acceptability in terms of: the value of interprofessional learning, the use of communication frameworks and role play, opportunities for open discussion and a safe learning environment.