Ability of observer and self-report measures to capture shared decision-making in clinical practice in the UK: a mixed-methods study.

OBJECTIVES: To examine how observer and self-report measures of shared decision-making (SDM) evaluate the decision-making activities that patients and clinicians undertake in routine consultations.
DESIGN: Multi-method study using observational and self-reported measures of SDM and qualitative analysis.
SETTING: Breast care and predialysis teams who had already implemented SDM. PARTICIPANTS: Breast care consultants, clinical nurse specialists and patients who were making decisions about treatment for early-stage breast cancer. Predialysis clinical nurse specialists and patients who needed to make dialysis treatment decisions.
METHODS: Consultations were audio recorded, transcribed and thematically analysed. SDM was measured using Observer OPTION-5 and a dyadic SureScore self-reported measure.
RESULTS: Twenty-two breast and 21 renal consultations were analysed. SureScore indicated that clinicians and patients felt SDM was occurring, but scores showed ceiling effects for most participants, making differentiation difficult. There was mismatch between SureScore and OPTION-5 score data, the latter showing that each consultation lacked at least some elements of SDM. Highest scoring items using OPTION-5 were 'incorporating patient preferences into decisions' for the breast team (mean 18.5, range 12.5-20, SD 2.39) and 'eliciting patient preferences to options' for the renal team (mean 16.15, range 10-20, SD 3.48). Thematic analysis identified that the SDM encounter is difficult to measure because decision-making is often distributed across encounters and time, with multiple people, it is contextually adapted and can involve multiple decisions.
CONCLUSIONS: Self-reported measures can broadly indicate satisfaction with SDM, but do not tell us about the quality of the interaction and are unlikely to capture the multi-staged nature of the SDM process. Observational measures provide an indication of the extent to which elements of SDM are present in the observed consultation, but cannot explain why some elements might not be present or scored lower. Findings are important when considering measuring SDM in practice. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A key strength of this study was measurement of shared decision-making (SDM) in clinical practice in teams that had embedded SDM in the National Health Service.

The study used mixed methods to measure and explore SDM in clinical practice using observer, self-reported and qualitative assessments providing a multidimensional understanding.

A limitation of this study was the use of the SureScore self-reported measure with discrete scoring options that resulted in ceiling effects.

Background

There is strong policy support for the routine use of shared decision-making (SDM) in clinical practice,1 2 accompanied with an increasing need to identify if and how SDM is taking place. Theoretically during SDM, patients are encouraged to think about the available treatment options, and the associated pros and cons in relation to their own preferences. Patients and clinicians are expected to collaborate and make decisions together that are informed by the best available evidence and truly aligned with patient preferences.3 Incentives for doing SDM with patients are being considered to help overcome some of the barriers associated with implementation.1 2 Therefore, for policy, there is a need to accurately measure SDM to identify whether it is happening and how it can be improved.

The National Institute for Health and Care Excellence’s ‘Shared Decision Making Collaborative’ recently published a comprehensive action plan for incorporating SDM into everyday care in the UK. This includes identifying strategies to successfully measure SDM in practice.2 The General Medical Council in the UK has also included SDM as a core professional competency in their updated generic professional capabilities framework.3 Assessing SDM capabilities places further value on the skill of sharing decisions with patients, and could encourage organisational support and wider implementation of the approach.

If SDM is to become part of routine care, we need appropriate summative measurement methods; policymakers and healthcare managers will need to know if and how well clinicians are doing SDM in practice, its effects on clinical practice and the difference it makes to patients.2 However, there are risks of measurement tools not capturing SDM in practice correctly, not reflecting what SDM looks like in routine clinical practice. Theoretical models do provide a useful framework for teaching and learning SDM skills, but there are a large number of clinician, patient and contextual factors, which will mean the SDM process will vary considerably between encounters. Further, SDM research and theory has traditionally conceptualised the process as occurring in dyadic encounters during a specific consultation.4 However, healthcare decisions are often made in a distributed manner,4 across numerous consultations and/or with different clinicians and significant others (eg, family members). The influence of contextual factors and the distributed nature of decision-making is especially important if SDM measurement in clinical practice is to be implemented for inspection or incentive schemes. Many measurement tools are available,5–8 but there is currently no ‘gold standard’ for measuring SDM.9 10 If summative, policy-related measurement tools are to be routinely implemented, we need to be sure that the theoretically developed measurement instruments, and the theories on which they are based, can capture what happens in routine clinical care. The same also applies to measurement for formative purposes; for example, ensuring that continuing professional development reviews or personal learning for clinicians is based on what is achievable and practical in routine care.

The Making Good Decisions in Collaboration (MAGIC) learning programme9 was commissioned by The Health Foundation to examine how best to embed SDM in routine clinical care. The legacy of this programme was clinical teams who had routinely embedded this approach and SDM interventions. This current study aimed to examine if and how observer and self-report measures of SDM evaluate the decision-making activities that patients and clinicians undertake in routine consultations. Key objectives included observing SDM in routine clinical settings with teams who have embedded the approach; examine if and how observer and self-report measures capture the SDM process; and compared and contrasted data from observer and self-report measures. The learning from this paper will be able to inform strategies for measuring SDM in routine clinical practice

Methods

We conducted a multi-methods study to measure SDM in clinical practice in two routine secondary care settings in the UK that had already implemented SDM. Consultations audio recorded by clinicians over a period of time were passed to the research team and subsequently transcribed verbatim. They were scored using OPTION-511 and thematically analysed. Clinicians and patients were also asked to complete the SureScore (box 1) measure at the end of their consultations.

Dyadic SureScore

Completed individually after consultation by the patient and clinician.

Item description

Three questions in the patient version:

Did you discuss treatment options?

Did you discuss risks and benefits of different options?

Did you discuss what is important to you in your treatment?

Clinician version is identical, but questions are framed from the clinician’s perspective.

Response categories

Items were scored based on three discreet response categories to each item. Patients/clinicians were asked to indicate their answer by selecting either: Yes, Unsure, No.

Participant recruitment

Clinical team recruitment

Two clinical teams where patients needed to make a key treatment decision were recruited within the Cardiff and Vale University Health Board (Wales, UK)—the Breast Care Centre and the Chronic Kidney Disease Predialysis team.

The Breast Care Centre key decision point was the decision between mastectomy or wide local excision with follow-up radiotherapy. The Chronic Kidney Disease Predialysis decision point concerned different types of dialysis and transplantation.

Both teams were previously trained in SDM and had embedded SDM as part of the 2010–2015 ‘Making Good Decisions in Collaboration’ (MAGIC) implementation programme.9 Most team members had attended theoretical and practical SDM skills training workshops, based on the ‘3 Talk’ theoretical model SDM.12 Each team had also developed and implemented a brief in-consultation decision support tool. All relevant clinical team members (eg, consultants, specialist nurses) who would consult with patients regarding treatment decisions were sent study information packs containing a cover letter, participant information sheet for clinicians and a consent form. Participation involved clinicians audio recording their consultations with patients and completing a postconsultation questionnaire. Only consultations where the clinician and the patient had agreed to participate were recorded.

Patient recruitment

Eligible patients were recruited consecutively by clinicians working in the predialysis team and Breast Care Centre between April 2014 and September 2015. Inclusion criteria stipulated that patients needed to be over 18 years old. Patients who were unable to communicate in English or who were deemed unsuitable due to other vulnerabilities identified by the clinical team were excluded. All patients were provided with a study pack which included a cover letter, patient information sheet and consent form before their consultation.

Data collection

All consultations were audio recorded and transcribed verbatim. Observer OPTION-5 and the dyadic SureScore measures were used to measure SDM in clinical practice during consultations. SureScore is a three-item self-reported questionnaire (Box 1), which captures both patients’ and clinicians’ perceptions of SDM during the consultation.11 The measure was adopted by the Cardiff and Vale University Health Board during the MAGIC implementation programme13 and integrated into routinely collected outpatient questionnaires. The observer measure of SDM (Observer OPTION-5) (Box 2) is a five-item measure based on the Model of Collaborative Deliberation,14 and is an update to the original OPTION-12 instrument.11 Observer OPTION-5 was used to independently assess the consultation audio recordings for elements of SDM.15 We also conducted thematic analysis of audio-recorded consultations to understand what the SDM process looked like in routine clinical settings, and to explore whether the measurement tools were able to adequately measure SDM in practice.16

We then compared all data to explore the ability of the different measures to measure the shared decision-making (SDM) process in routine clinical settings.

Observer OPTION-5

Completed by trained raters using audio recordings and verbatim transcripts of the consultations

Item description of the Observer OPTION-5 measure:

1. Alternative options

Clinician draws attention to or confirms that alternative options exist/need for a decision exists. Clinician responds by agreeing that options need deliberation if patient draws attention to availability of options.

2. Support deliberation

Clinician reassures/reaffirms the patient that they will support them to become informed/deliberate about options. If patient states they have sought information prior to consultation, the clinician supports the deliberative process.

3. Information and options

Clinician gives information and checks understanding about the options (including ‘no action’) to support the patient in comparing alternatives. Clinician supports any clarification needed by the patient.

4. Eliciting preferences

Clinician makes effort to elicit patient preferences in response to options presented. Clinician is supportive if patient declares their preferences.

5. Integrating preferences

Clinician makes effort to integrate patient preferences as decisions are made. If patient indicates how best to integrate their preferences into the decision, the clinician makes an effort to do so.

Response categories

Each item was scored on a Likert scale from 0 to 4. Items were scored according to the OPTION-5 manual.28

0=No effort—not observed/heard.

1=Minimal effort—the use of short phrases that indicate the issue is being raised.

2=Moderate effort—more substantive talk used to discuss issues and ideas.

3=Skilled effort—substantive talk is used to convey the ideas and issues, and understanding is checked.

4=Exemplary effort—excellent and careful attention to ideas and issues, and understanding is checked.

Patient and public involvement

Patient and public representatives were involved in reviewing the study documents at the start of the MAGIC project. GH is a coauthor in this manuscript and the current patient and public representative. GH worked collaboratively with the research team to contribute to the content of the manuscript. GH shared her experiences of SDM in healthcare, commented on the manuscript and agreed on the final version.

Analysis

Quantitative analysis

OPTION-5: Each consultation audio recording and verbatim transcript was analysed independently by two trained raters (DW and NJW). The raters had undergone training (using the OPTION-5 manual) and a scoring and standardisation process where they read the OPTION-5 training manual (and contacted the measure’s developers with any queries). To ensure standardisation, raters scored 10 consultations each (five for each clinical area), met to compare agreement levels and resolved any discrepancies through discussion. DW is a psychologist with expertise in healthcare communication in women’s health and long-term conditions. NJW is a psychologist with expertise in SDM and patient-centred care.

Each item was scored on a 5-point Likert-like scale (0=no effort, 4=exemplary effort). The total OPTION-5 score was calculated by adding up the scores of each item (score range 0–20) and it was then rescaled to range from 0 to 100. Two raters scored all consultations, therefore the mean of the two scores was taken for each consultation (out of a possible 100). Higher scores indicated that a greater number of SDM elements outlined by the measure were observed. Descriptive statistics were used to compare the mean scores between items and teams.

Items within the SureScore measure were analysed using SPSS for Windows V.22.17 Descriptive statistics were used to characterise the sample, and to observe the distribution of responses to SureScore items and the pattern of missing data as an indicator of their acceptability to patients and clinicians. Cross-tabulations of patient and clinician responses were used to identify patients and clinicians with discordant scores.

Qualitative analysis

Consultation transcripts were transcribed verbatim and thematically analysed. The breast cancer (BrCa) and predialysis consultations were analysed separately. The data were analysed using Braun and Clarke’s approach to thematic analysis,18 which included familiarising with the data, generating initial codes, searching for themes, reviewing and refining themes, and defining and naming themes. DW conducted the data coding. Regular qualitative research team meetings were used to discuss the development of the coding framework and data analysis, with each member of the qualitative group (DW, NJW, FW, AE) adding their own unique perspective to the analysis19 and resolving any discrepancies in coding through discussion. An inductive approach was used in analysis to allow themes to emerge from the data in order to understand SDM. Data were not double coded, instead themes were discussed in team meetings to discuss data production, the development of the coding framework and data analysis. This approach has been identified as appropriate in qualitative research.19 We were guided by the concept of ‘information power’20 rather than ‘saturation’.20 Information power is determined iteratively by the aims, sample specificity, quality of dialogue and analytic strategy.20 Sample size was iteratively determined reflexively based on the themes that were emerging through the data, and continued until no major new themes were being identified. Clinicians were informed when to stop collecting data based on the researchers’ assessment of information power. NVivo V.11 software was used to organise the data.

Results

Eleven members of the breast care team consented to participate in the study: four consultants/surgeons, one registrar, one consultant nurse and five clinical nurse specialists. All three clinical nurse specialists in the predialysis team consented to participate in the study.

SDM in clinical practice was assessed during healthcare consultations in clinic or at patients’ home where treatment options were discussed: this included one consultation per patient with chronic kidney disease (CKD; predialysis consultation with the clinical nurse specialist), and two consultations per patient with BrCa (diagnostic consultation and follow-up visit, involving different team members). Twenty-six patients who had been referred to the predialysis team (16 males, 10 females) and 25 female patients with BrCa were recruited and had their consultations audio recorded. Family members/friends were frequently present. The SureScore measure was returned by both the patient and clinician in 22 of the BrCa and 21 of the predialysis consultations.

The self-reported SureScore questionnaire data showed that both patients and clinicians thought SDM was happening during their consultations (see table 1). Missing data represented five CKD and three follow-up patients with BrCa who had not returned the measure (therefore we had 47 surveys that were returned from both diagnostic and follow-up BrCa appointments and 21 CKD surveys returned by both patients and clinicians).

Table 1

Clinician and patient scores on the SureScore survey for breast cancer and predialysis teams

		Yes (Clinician)	Yes (Patient)	Unsure(Clinician)	Unsure (Patient)	No(Clinician)	No(Patient)	Missing
Item 1: Did you discuss the treatment option?	BrCa diagnostic consultation	25	25	0	0	0	0	0
	BrCa follow-up consultation	22	22	0	0	0	0	3
	Predialysis consultation	26	21	0	0	0	0	5
Item 2: Did you discuss the risks and benefits of treatment option?	BrCa diagnostic consultation	22	23	1	0	2	2	0
	BrCa follow-up consultation	24	22	0	0	1	0	3
	Predialysis consultation	25	20	1	1	0	0	5
Item 3: Did you discuss what is important in your treatment/to the patient in their treatment?	BrCa diagnostic consultation	22	22	0	2	3	1	0
	BrCa follow-up consultation	25	22	0	0	0	0	3
	Predialysis consultation	24	20	2	0	0	1	5

BrCa, breast cancer.

A high proportion of respondents gave the maximum score, thus the instrument showed ceiling effects with limited variability in the data. There was little disagreement in the responses between the clinicians and patients. All patients and clinicians felt that treatment options were discussed during all consultations. Both clinicians and patients responded ‘Yes’ on ‘Item 1: Did you discuss treatment options?’ for both diagnostic and follow-up breast care consultations and predialysis consultations.

Most patients (45/50 BrCa and 20/26 CKD) and clinicians (46/50 BrCa and 25/26 CKD) felt that the risks and benefits of treatment options were discussed during consultations, scoring ‘Yes’ on item 2: ‘Did you discuss the risks and benefits of treatment options?’ On three occasions, it was felt that risks and benefits were not discussed in the breast care sample, where three clinicians and two patients scored ‘No’.

The vast majority of patients and clinicians in the predialysis team scored that the risks and benefits of the treatment options were discussed (20/21 and 25/26, respectively).

The minority of consultations scored as ‘No’ (BrCa clinician n=3, patient n=1) or ‘Unsure’ (patient with CKD n=1) by the clinicians or patients were inspected in more detail using the OPTION-5 data and thematic analysis. It was found that there was a lack of in-depth discussion of risks and benefits in these consultations, usually due to a number of factors such as distress of the patient during the consultation necessitating alternative discussions, or an early and strong indication of treatment preference by a well-informed patient.

Most patients (BrCa n=44, CKD n=20) and clinicians (BrCa n=47, CKD n=24) felt they had discussed what was important to the patient regarding their treatment in both the BrCa and predialysis consultations (item 3). Three clinicians and one patient felt they had not discussed what was important to the individual during the diagnostic breast consultation as well as one patient in the predialysis consultation.

The Observer OPTION-5 data provided more details. Predialysis consultations ranged from 50 min to 2 hours and 25 min (mean 1 hour and 35 min). Breast diagnostic consultations ranged from 19 min to 1 hour and 4 min (mean 33 min), follow-up consultations ranged from 14 min to 1 hour and 38 min (mean 51 min).

Elements of theoretical SDM were present in all consultations. Standardised global OPTION-5 scores ranged from 65.0 to 95.0 with a mean of 82.82 (SD 8.54) for the breast care consultations, and 47.50–80.0 with a mean score of 66.53 (SD 9.16) for the predialysis consultations.

Standardised scores for each item and team are presented in table 2. Differences in mean scores between items were observed with option presentation (item 3), preference elicitation (item 4) and preference integration (item 5) scoring consistently highest across both teams. Item 2, which looks at the extent to which the clinician reassured the patient that they were there to help support them to become informed and to deliberate over their treatment options, was consistently given a low score. Introduction of choice (item 1) also tended to score lower, suggesting it was not completed as comprehensively as the other SDM skills.

Table 2

Standardised (0–20) Observer OPTION-5 domain mean scores, SD, minimum and maximum scores for each item (n=25 breast cancer and n=26 predialysis consultations)

OPTION-5 item	Mean score, SD		Minimum score		Maximum score
	Breast	Renal	Breast	Renal	Breast	Renal
1. Alternative options	15, 3.60	11.05, 2.66	10	7.5	20	15
2. Support deliberation	12.5, 3.06	8.65, 2.79	7.5	5.0	20	15
3. Information about options	17.6, 2.22	16.05, 2.14	12.5	10	20	20
4. Eliciting preferences	17.7, 2.87	16.15, 3.48	10	10	20	20
5. Integrating preferences	18.5, 2.39	15.38, 3.65	12.5	7.5	20	20

By listening to the consultation audio recordings, it was clear that the implicit and unspoken understandings that occur across consultations are difficult for an assessor to score. For example, when clinicians had previously spoken to patients over the telephone when making an appointment and had outlined the rationale for offering a choice. Supportive SDM tasks in clinical practice were often implicit and broader than the instrument assessed. Lower scores on item 2 were likely to have resulted from the instrument’s inability to capture embedded and unspoken understandings, and did not reflect the actual support offered. The fact that the clinician was spending a long time outlining treatment options, discussing pros and cons as well as eliciting preferences implied that they were supportive of the patient. This is implicit support. Item 2 specifically states that a high score can only be given if support is offered explicitly, ‘The clinician reassures the patient or re-affirms that the clinician will support the patient to become informed or deliberate about the options.’

A summary of the ability of the SureScore and OPTION-5 measures to capture SDM in clinical practice is shown in table 3.

Table 3

Measurement of SDM in clinical practice using self-report and observational tools

Measurement tool type	What it does	What it does not do	Potential problems
Self-report questionnaire	Broadly inform us if patients/clinicians felt that SDM was occurring.Relatively easy to implement.	It is unable to tell us about the quality of the interaction.It is unlikely to provide an accurate representation of the ‘distributed’ SDM process.	Susceptible to the ‘halo effect’, social desirability and response bias.
Observational measure	Provides an indication of the extent to which elements of theoretical SDM are present in a consultation.	It does not account for why some elements of SDM are not observed or scored lower.It can only capture what is observed at the time of measurement, and does not account for the ‘distributed’ process of SDM.	Time and resource intensive.Needs specialist equipment, for example, audio/visual recorder.Can only be completed by trained raters.

SDM, shared decision-making.

Thematic analysis

Thematic analysis was performed on 25 BrCa consultations (diagnosis and follow-up) and 26 predialysis consultations. The aim of the thematic analysis was to explore SDM in clinical practice and to reflect this on the measurement tools. BrCa and predialysis consultations were analysed separately but due to similarities in themes emerging, results for both are presented below. An increasingly complex picture of the processes during SDM in clinical practice emerged when we examined the audio recordings of healthcare consultations—a picture that was not fully captured by either the self-reported or observer measures of theoretical SDM that we used. Three broad themes were identified which measurement tools were not fully responsive to: distributed SDM, multiple and interacting decisions and the contextually adapted process of SDM. Each theme will be discussed below.

Distributed SDM

SDM in clinical practice was ‘distributed’ over people and time across a number of consultations, involving multiple clinicians, the patient and their family members, and sometimes over a prolonged period of time. Quite often, a decision did not need to be made within one specific consultation. In the breast care team, treatment options were presented by the consultant during the diagnostic consultation. They were then discussed further during a follow-up consultation 1 week later with the specialist nurse, which was when a treatment decision was typically made.

Okay. Well remember when you saw [consultant name] in the clinic last week, he said to you it’s perfectly simple, there's an option of surgery. The first option being that we can remove the lump with the margin of tissue around it and then give you a course of radiotherapy and it’s that course of radiotherapy with just removing the lump that makes that as safe as having the whole breast removed the mastectomy. (Clinical nurse specialist, breast cancer follow-up, ID 2.14)

The ‘preference elicitation’ process was initiated during the diagnostic consultation, when patients were signposted to think about what mattered most to them, but the discussion of preferences typically occurred during the follow-up appointment.

Predialysis treatment discussions were distributed between consultants and predialysis nurses, and between patients and their family. Discussions around the initiation of treatment decision processes had often started before the predialysis appointment which we measured, and the final decision was made some time later. Unlike the BrCa decisions, these decisions were also not definitive as patients could review and revise their decision, making it particularly difficult to capture the SDM process that clinicians and patients were doing.

Okay, so just to let you know we'll see you back in clinic in a month's time (…) Please ring me if you've got any questions, don’t sit at home and think mmm, but I am quite happy at the moment to just keep an eye on you. We don’t need to do anything. The only thing I’ll urge you, think about transplantation and have a read of the information. Don’t have to do it obviously in the next week or two but before you come back in a month’s time, have a think about transplant because I will be asking you. Ordinarily I refer patients straight away for transplant but I think you just need to have a little bit of time, because it is a new concept to you isn’t it? (Clinical nurse specialist, predialysis consultation, ID 1.17)

Multiple and interacting decisions

Patients were often required to make multiple decisions in relation to one condition. For example, although the focus of the consultation for patients with BrCa was the type of breast surgery, several reconstructive or prosthesis options available were also discussed. Patients with CKD were considering dialysis as well as transplantation options. Patients were typically making multiple and interacting decisions; the outcomes of a later treatment option quite often influenced their decisions about initial treatment options (eg, outcomes of breast reconstructive options and choice of surgery). This complicated the preference elicitation and consolidation processes.

Patient: So if I had a mastectomy would I be able to have reconstruction?

Consultant: Yes, we do reconstruction, yeah. We do.

Patient: Do you do that straight after the operation?

Consultant: Yeah, we normally do. We normally do, yes. Um, the only thing that might affect that is if um, ah, this radiation issue again, if the um, for instance if there are lymph gland problems or anything like that there might be a recommendation that you have radiation. So, um, if you do have radiation as well as a mastectomy, which is, you don’t often have to have that, we then have to think about would that fit in well with immediate reconstruction?

Patient: I see.

Consultant: So often in that situation, if we think you’re going to have radiation then, um, we might say well, um, let’s leave the reconstruction till later.

Contextually adapted process of SDM

Patients’ previous knowledge about the condition and available treatment options also influenced the decision-making process. Sometimes previous knowledge and experience informed the patients’ prior preferences, where the patients expressed preferences before the treatment options had been presented for their current diagnosis.

Nurse: Okay, right, if I go through the operation, I know you said right from the start you know exactly which operation you want.

Patient: I do.

Nurse: And I have no wish to change your mind on that but I just want to make sure that I know you know all the pros and cons so that afterwards you don’t turn ‘round and think oh if I’d known that I wouldn’t have made that decision. (Breast cancer follow-up visit, ID 2.7)

The amount of information about options that was presented by clinicians varied based on the patient’s previous knowledge about the options, as well as their emotional response. For example, some patients were understandably emotionally distressed following their diagnosis of BrCa. In such cases, clinicians would sometimes limit the depth of information offered at that appointment and would defer in-depth explanations to the follow-up appointment.

Patients sometimes recalled information regarding treatment options based on their previous experiences (eg, recurring BrCa), as well as experiences of significant others who had received treatment and used these to contextualise perceived pros and cons.

My sister had two breasts removed at the same time, so it was a double…and my friend at the end of the street she’s got this [breast cancer]. I was talking to her on Friday…but the only thing is, she’s got it obviously on the one side, and it was really noticeable [the mastectomy]…that’s why I thought, obviously with the two [breasts], am I going to match? (Patient, breast cancer diagnostic consultation, ID 2.3)

In predialysis consultations, some patients had extensive knowledge and experiences of treatment options because of their family members’ experiences of dialysis. In these circumstances, the clinicians often altered the amount of information presented to patients, based on their previous knowledge.

Furthermore, the SDM process was also related to the context of the consultation. During the BrCa consultations, patients needed to make a decision relating to their imminent surgery. However, especially in the predialysis consultations, there was often no set time within which a decision needed to be made, because initiation of dialysis would be dependent on the patient’s gradual renal deterioration. Therefore, during the predialysis consultations there was sometimes no evidence of incorporating patients’ preferences into the decision, as no decision was being made at this time.

Patient: Can I just ask you, you’re putting this across as this is imminent, that I’m going to have to go through this?

Nurse: Yes

Patient: Or are you just advising me and…

Nurse: Usually if I come to do a home visit, at some stage you will start on this treatment.

Patient: Really?

Nurse: It could be six months, it could be twelve months, it could be eighteen months. But what we want to do is … prepare you for it.

(Predialysis consultation, ID 1.13)

Overall, we found that the SDM in clinical practice process was not standardised or consistent—it was continually adapted to contextual factors and was therefore very difficult to capture using self-reported or observer measures. This has raised significant challenges that must be considered when trying to measure SDM. A summary of the key measurement challenges in clinical practice is presented in table 4.

Table 4

Shared decision-making in clinical practice and associated measurement challenges

Observations of SDM in clinical practice		Measurement challenges
SDM is distributed over people and time.	SDM often involves different healthcare professionals and multiple consultations, especially in long-term conditions, for example, how many interactions (appointments/telephone calls) and with which clinicians?	SDM measurement tools are often based on theoretical models of SDM and require that all SDM stages are covered within one encounter. However, clinicians continually adapt their SDM approach to match the patient journey and a consultation which does not cover all aspects of SDM is not necessarily reflective of a poor SDM encounter.Individuals using scores from an SDM encounter need to be aware of the patient’s journey through the health system.
SDM involves multi-staged decisions.	Patients are often required to make multiple decisions in relation to one condition, resulting in the decision-making process not being discrete or orderly.	SDM measurement tools generally focus on the process of reaching one particular decision. In the absence of ‘discrete’ and ‘orderly’ decisions, together with the increasing incidence of patients with comorbidities, it raises the question of ‘which decision are we measuring?’
SDM is adapted to context.	The content of SDM consultations will vary based on the nature of the decision, for example, short term versus long term, the patient’s emotional response, previous knowledge and experiences of the treatment options.	Careful interpretation of measures is needed because a low item score does not necessarily mean a poor SDM consultation occurred; it might reflect the modification of content based on the context.

SDM, shared decision-making.

Discussion

In our observations we found that the observational and self-report measures we used were unable to fully capture the SDM process in clinical practice. This was because we observed SDM in clinical practice which was defined by multiple and staged decisions. This is similar to other studies4 who have described SDM as ‘distributed’. We found that SDM discussions were frequently distributed between consultations and healthcare professionals.

The observed mean OPTION-5 scores in this study were high compared with other studies that have used the tool to measure SDM in trained clinicians21 and the difference was higher than in clinicians who had not been trained in SDM.22 Furthermore, the OPTION-5 data showed that the BrCa team had higher mean scores than the CKD team. Based on theoretical models of SDM, these data might be interpreted that the CKD team was not as skilled in SDM as the BrCa team, but qualitative analysis suggested this was not the case. The Observer OPTION-5 scores did not provide an accurate representation of the quality of SDM in the consultations, as some of the SDM tasks scored by the instrument were not appropriate for that specific consultation. If achieving SDM in clinical practice is not a standardised process, and it is continually adapted for contextual factors for that specific patient at that time, how can we use standardised instruments that are not responsive to these processes? Overall, OPTION-5 provided a better indication of which parts of the process happened, and how well they were done, compared with SureScore, and it also highlighted what was not done so well or was missing. Observer measures of SDM are also less affected by ceiling effects or social desirability bias because they are completed by a third party.23

The data also suggested that self-report measures might be susceptible to the ‘halo effect’,23 where patients were reporting a good overall impression of the clinician because they liked them. Social desirability bias might have also influenced responses, reflecting key barriers to effective measurement that have been reported elsewhere.24 Clinicians may also have been susceptible to response bias or were limited by the response categories present in the measure. Other self-report measures such as CollaboRATE25 use ordinal rating scales instead to score SDM during consultations, with responses ranging from ‘no effort was made’ to ‘every effort was made’. Although scales have also been criticised for ceiling effects in general,26 such measures do provide a larger range of response categories and therefore might be more sensitive to reporting the full range of patient experience.

SureScore provided a broad indication of whether SDM in practice was thought to have happened or not, but it did not indicate the ‘quality’ of the SDM discussion (ie, Were the options discussed in a way that the patient fully understood?), or how the SDM process unfolded (ie, Did the patient or the clinician initiate the ‘shared’ discussion?). Self-reported measures also ask the patient to score a consultation based on the assumption that patients know what an SDM consultation might or should look like (although they may have never experienced one).

The detailed thematic analysis provided a richer understanding of what actually happens during SDM encounters between clinicians trained in SDM skills and their patients. We found that the Observer OPTION-5 instrument was somewhat insensitive to contextual factors, such as patients’ prior knowledge or experience of treatment options. We also observed that predialysis consultations rarely covered the core SDM skill of ‘decision talk’19—they were better characterised by ‘planning talk’, as a decision was rarely made at this point.

Detailed thematic analysis showed that the SDM encounter in clinical practice was distributed, and it identified it was contextually adapted and that individuals would often need to make multiple and interacting decisions (as explained in table 4). We identified that some patients already had predetermined treatment preferences. In such situations it is important to acknowledge that for SDM to occur, the clinician needs to understand the reasoning behind the treatment preference. Clinicians should be encouraged to explore how the treatment preference meets with patient goals, as well as their understanding of associated pros and cons. If the treatment preference meets the goals of the patient and if their understanding of the pros and cons is accurate, then the consultation should be seen as a process of SDM.

We observed that the process of SDM is distributed and adapted across consultations and clinicians and this poses important questions about when we should be measuring whether SDM took place, and whom we are assessing. Although the OPTION-5 measure could be used to capture SDM over time, by recording multiple visits, and scores, collecting these data might be unfeasible within the context of clinical practice. Alternatively, choosing one of the consultations to measure could lead to an unfair representation of the whole process (eg, incorporation of the patient’s preferences into the decision could happen at a later consultation), but measuring every consultation is perhaps not feasible and incurs more work for the patient if relying on self-report.27 If different clinicians are involved in the process, it might be difficult for patients to respond using the self-report measures, particularly if the ‘quality’ or ‘level’ of SDM differs between the clinicians (who should they be scoring?).

If we want to introduce routine measurement of SDM in clinical practice for inspection, incentive or formative purposes, we need to develop and validate new tools, or modify existing measures, to account for the distributed and personalised nature of SDM discussions within everyday clinical practice. The challenge will be to develop a measure that is responsive to the non-standardised nature of the SDM process, while also being generic enough to be easily administered and incorporated into routine healthcare practice and evaluation. It might be useful to have two types of SDM measurement tools, one type focusing on more discrete SDM discussions (eg, contraception choice within general practice, or surgical choices), and another for long-term conditions, which is particularly sensitive to the distributed nature of SDM and context.

It is often said that ‘what gets measured gets done,’ but when trying to monitor whether every patient is involved in their healthcare decisions, we must be cautious. Attempting to standardise the measurement of SDM, which is a highly contextualised process, is problematic. We must reflect on the purpose of measurement and what needs to be standardised. Theoretically, SDM requires that patients be aware of the options available to them, the associated benefits and barriers and that their personal preferences are elicited and incorporated within clinical practice. However, the process of achieving SDM in practice is adapted and contextually specific. Measuring SDM in routine practice is challenging, and caution is needed when using instruments that only capture a ‘snapshot’ of the entire process. We may instead focus on standardising the process of measurement for the purpose of improving SDM in clinical practice—that is, where and how often it is integrated into practice and how the data are made available for feedback and reflection. We may also consider measuring SDM at the endpoint of any given treatment decision or pathway, when the SDM process should have been completed, although caution is required as to what and whom patients are evaluating then. It is unlikely that one measure will ever be able to cover all clinical encounters which are very diverse in nature. SDM is often distributed over time, multistaged and context specific. The complex nature of SDM in clinical practice poses significant measurement challenges, and it is unlikely that one instrument will ever be able to account for all these complexities. Instead, we need to use an appropriate combination of instruments suitable for the clinical context in question. This might include observational measurement, self-report and/or qualitative analysis.

Limitations

As well as the limitations relating to the measurement tools used in this study which were discussed above, a number of additional limitations need to be considered. This study did not explore the ability of SDM measurement tools to capture SDM in consultations where discrete decisions are made, such as consultations focusing on contraception where a decision would likely be made at the end of the consultation. The Observer OPTION-5 and SureScore measures might have been able to capture the SDM process in the consultation more effectively. Further, we did not observe clinicians who had either not been trained in SDM or those who have recently been trained, we focused on clinicians who had been trained in SDM through an SDM implementation research study.

This study was conducted as an implementation study in a clinical setting and did not have control measures associated with typical research studies (eg, as in randomised controlled trials). Therefore, because the clinical team members recruited patients into the study sequentially and directly sent the consultation recordings to the research team, it cannot be guaranteed that this was done in a completely unbiased way. Nevertheless, we observed consultations that did not score highly on the Observer OPTION-5 measure, which gives us confidence that we were not sent a biased set of consultations. We were also unable to provide a response rate relating to how many patients were approached to take part and how many declined.

Conclusion

SDM in clinical practice is a distributed, multistaged and context-specific process and current measures fail to account for the whole process, especially in long-term conditions. Current measures can provide some useful information on whether SDM is taking place, and how well it is done (eg, for formative evaluations used in continuing professional development). However, given the limitations identified by this study, further work is needed before summative assessments are routinely implemented in clinical practice, to make sure they account for the complex, adaptive and distributed nature of the process. This is especially important if summative tools are used for the purposes of incentivisation.