Role of patient preferences in clinical practice guidelines: a multiple methods study using guidelines from oncology as a case.

OBJECTIVE: Many treatment decisions are preference-sensitive and call for shared decision-making, notably when benefits are limited or uncertain, and harms impact quality of life. We explored if clinical practice guidelines (CPGs) acknowledge preference-sensitive decisions in how they motivate and phrase their recommendations.
DESIGN: We performed a qualitative analysis of the content of CPGs and verified the results in semistructured interviews with CPG panel members.
SETTING: Dutch oncology CPGs issued in 2010 or later, concerning primary treatment with curative intent. PARTICIPANTS: 14 CPG panel members. MAIN OUTCOMES: For treatment recommendations from six CPG modules, two researchers extracted the following: strength of recommendation in terms of the Grading of Recommendations Assessment, Development and Evaluation and its consistency with the CPG text; completeness of presentation of benefits and harms; incorporation of patient preferences; statements on the panel's benefits-harm trade-off underlying recommendation; and advice on patient involvement in decision-making.
RESULTS: We identified 32 recommendations, 18 were acknowledged preference-sensitive decisions. Three of 14 strong recommendations should have been weak based on the module text. The reporting of benefits and harms, and their probabilities, was sufficiently complete and clear to inform the strength of the recommendation in one of the six modules only. Numerical probabilities were seldom presented. None of the modules presented information on patient preferences. CPG panel's preferences were not made explicit, but appeared to have impacted 15 of 32 recommendations. Advice to involve patients and their preferences in decision-making was given for 20 recommendations (14 weak). Interviewees confirmed these findings. Explanations for lack of information were, for example, that clinicians know the information and that CPGs must be short. Explanations for trade-offs made were cultural-historical preferences, compliance with daily care, presumed role of CPGs and lack of time.
CONCLUSIONS: The motivation and phrasing of CPG recommendations do not stimulate choice awareness and a neutral presentation of options, thus hindering shared decision-making. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A strength of the study is that we used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) for the qualitative analysis of the guidelines, as weak recommendations in GRADE reflect preference-sensitive decisions.

Another strength of the study is the validation of the qualitative analysis of the guidelines in indepth interviews with the guideline developers.

A limitation of the study is that we studied oncology guidelines from one country only.

Introduction

Many decisions in healthcare are preference-sensitive, in particular when treatments are burdensome, benefits are limited or uncertain, and harms may impact quality of life.1 Examples are decisions about adjuvant treatment in oncology2–4 or about hip or knee arthroplasty for osteoarthritis.5–7 Research shows that patients as well as clinicians often vary considerably in their evaluation of the balance of benefits and harms. Further, clinicians are not always able to predict their individual patients’ preferences for treatments or outcomes of treatment.8 9 Shared decision-making (SDM) is therefore advocated particularly in preference-sensitive decisions, but is not yet common practice.10 11 Clinicians are not prone to fostering choice awareness in their patients,12 13 often present treatment options in unbalanced ways, for example, by overestimating benefits and minimising harms,14 or steer in other ways, consciously or unconsciously.15 Further, numerical probabilities needed to make a trade-off are seldom discussed16 and patient preferences infrequently elicited.17 18 This raises the question if clinicians perceive these decisions as preference-sensitive. Clinical practice guidelines (CPGs) could play a role in this perception, given the impact they have on what treatment options clinicians present to their patients. While CPGs may use wording that suggests that a decision is preference-sensitive, such as ‘we suggest’ or ‘clinicians might’, rather than ‘we recommend’ or ‘clinicians should’, clinicians may still not fully appreciate the importance of offering more than one option to their patients.

It is unknown if recommendations in current CPGs identify preference-sensitive decisions and demand a role for patient preferences in decision-making. Two older studies showed that the relevance of preferences of individual patients was not acknowledged in many CPGs.19 20 CPG developers often assume ‘generally accepted’ values in developing recommendations, but do not acknowledge this in the phrasing of the recommendation.21 A request for a more systematic incorporation of patient preferences in CPGs has been expressed repeatedly in high-impact journals since the publications of these studies.22–25 The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group—whose approach is nowadays considered the standard in CPG development—has published a framework that acknowledges the integration of patients’ values and preferences in the development of CPG recommendations.26–31 In the GRADE approach, preference-sensitive decisions are reflected in the so-called ‘weak’ or ‘conditional’ recommendations. These arise when benefits and harms are closely balanced, evidence is lacking or of uncertain quality, when patients’ preferences are expected to vary substantially, but also when no evidence on patient preferences is available, even with moderate or strong evidence of high quality on the benefits of an option.28 In such situations, GRADE still leads to weak recommendations, assuming that most informed patients would choose the recommended treatment, but a substantial number would not28 29 31 (see box 1 for a summary of the role that GRADE proposes for patient values and preferences in CPG development).

GRADE offers an approach to rate the certainty in the evidence and strength of recommendations, in which strong and weak (also known as conditional) recommendations are distinguished. Consideration of patient preferences is a crucial step in deciding on the strength of the recommendation. According to the GRADE approach, first, the best estimates of effect for the interventions and the certainty in this evidence (quality of the evidence) are assessed, using up-to-date systematic reviews. Further, the CPG panel should consider a number of criteria that influence the strength of recommendations, such as variability or uncertainty in how patients value the main outcomes (both benefits and harms), the balance between benefits and harms, and considerations of resource use, health equity, feasibility and acceptability (from both the stakeholder’s and the patient’s perspectives) of an intervention.26–30 Based on an overall assessment across these criteria, CPG panels reach a conclusion about the direction of their recommendation (for or against the intervention) and the strength of their recommendation: strong or weak.26 A high level of certainty across the criteria (such as high-quality evidence, clear balance between benefits and harms, no uncertainty in patient preferences) allows for strong recommendations. A high level of uncertainty, that is, preference-sensitive decisions, leads to weak recommendations: there is more than one single best option available, there is important uncertainty or variability in patient preferences, or the benefits and harms are closely balanced. Tension has been shown to occur between adherence to GRADE and the wish to make a strong recommendation out of conviction that a treatment is beneficial, despite the evidence quality or certainty being (very) low.42

To guarantee the acknowledgement of patient preferences in the development of recommendations, the GRADE strategy asks to clearly present (1) how substantial benefits and harms are, what their balance is, and what the overall certainty of the evidence on these outcomes is; and (2) if there is uncertainty about or variability in how much patients value the important outcomes.26 27 43 In other papers, GRADE recommends guideline developers to make transparent and explicit statements (3) about the (variability in) patient values and preferences, as well as CPG panel assumptions of these values and preferences on which decisions on the strength of recommendations are based, in order to be able to judge the applicability of recommendations for decision-making with the individual patient.28 29

CPG, clinical practice guideline; GRADE, Grading of Recommendations Assessment, Development and Evaluation

Therefore, a key ingredient for the identification of preference-sensitive decisions is the acknowledgement of values and preferences in the rationale for CPG recommendations. The aim of our study was therefore to explore to what extent CPGs acknowledge preference-sensitive decisions in the way they support and phrase their recommendations. We further wished to assess if the CPGs facilitate the communication of the preference-sensitive nature of these decisions to patients.

Methods

We performed a qualitative analysis of Dutch oncological CPGs, which we next verified and refined in semistructured interviews with members of CPG development panels.

Qualitative analysis of CPGs

Selected CPG modules

We used Dutch oncological CPGs as a case because oncology is strongly guideline-driven, decisions are often preference-sensitive, the guideline development process is organised nationally, and the CPGs are open access. The Netherlands Comprehensive Cancer Organisation (IKNL) develops guidelines ‘under responsibility of the most relevant professional or scientific society, usually following evidence-based methodology’ (www.oncoline.nl). We selected three tumour-specific CPGs, and of each we selected all content of two modules to include in our analysis (ie, the sections of the CPGs that address specific treatments or patient groups). We selected a convenience sample of modules for prevalent cancers that we expected to contain at least one preference-sensitive decision, calling for a weak recommendation. This expectation was based on earlier research from our group (eg, refs 11 13 15), views from the oncology experts on our research team, and/or on the availability of literature on SDM and decision aids for the treatment in that module. Each of the modules included more than one recommendation.

Further criteria for selection of the CPGs and the modules were: published on www.oncoline.nl, issued in 2010 or later, and concerning primary treatment with curative intent. Table 1 presents the CPGs and modules we selected. For the breast cancer CPG, our contact person at the IKNL provided us confidentially with the most recent revision of the two selected modules, which were not yet published at the time of our analysis. In none of the modules explicit reference was made to GRADE.

Table 1

Overview of CPGs and modules analysed, number of interviews, and the role and specialty of interviewees

Localisation	Module	Publication date	Approach	Options discussed in recommendations (n)	Strength of recommendations					Role and specialty of interviewees
					In favour		Neutral	Against
					Strong	Weak		Weak	Strong
Breast cancer	Ductal Carcinoma In Situ (DCIS)	IKNL, unpublished concept (27 February 2017)	GRADE	5	1	0	3	0	1	Surgeon (n=2)Radiotherapist (n=2)
	Endocrine therapy	IKNL, unpublished concept (27 March 2017)	GRADE	11	4	7	0	0	0	None
										IKNL supervisor (n=1)*Patient representative (n=1)
Colorectal cancer	T1 carcinoma in polyp	16 April 2014(version 3)	Evidence-based	3	1	2	0	0	0	Surgeon (n=1)Gastroenterologist (n=1)
	Adjuvant chemotherapy	16 April 2014(version 3)	Evidence-based	6	4	2	0	0	0	Oncologist (n=1)
										IKNL supervisor (n=1)*Patient representative (n=1)
Resectable non-small cell lung cancer (NSCLC)	Stereotactic radiotherapy	16 April 2014(version 3)	Evidence-based (2011) and consensus-based (2013)	3	1	1	0	1	0	Radiotherapist (n=3)†
	(Neo)adjuvant radiotherapy	16 April 2014(version 3)	Evidence-based (2011) and consensus-based (2013)	4	1	2	0	0	1	Radiotherapist (n=1)†
										IKNL supervisor (n=1)
n=3	n=6			n=32	n=12	n=14	n=3	n=1	n=2	IKNL supervisor (n=2)Patient representative (n=2)Radiotherapist (n=5)Surgeon (n=3)Oncologist (n=1)Gastroenterologist (n=1)

*Interviewed once, both about the breast cancer and the colorectal carcinoma guidelines.

†One radiotherapist was interviewed once about two modules of the NSCLC CPG.

CPG, clinical practice guideline; GRADE, Grading of Recommendations Assessment, Development and Evaluation; IKNL, Netherlands Comprehensive Cancer Organisation.

Data extraction and analysis

We assessed if the CPG acknowledges preference-sensitive decisions and whether the user is to understand the strength of a recommendation, based on the information presented. To this aim we developed a coding scheme that consisted of the following five themes, based on the GRADE framework.28

Strength of recommendations

First, we scored the strength of the recommendation (strongly in favour/weakly in favour/neutral/weakly against/strongly against a specific option) for each treatment option described in the recommendation section of the CPG. Scoring was solely based on the phrasing used in that section. The categories strong and weak that we used are in line with GRADE. We added the ‘neutral’ category if a weak recommendation for more than one option was given.

Next, we assessed whether this strength of recommendation was supported by information elsewhere in the guideline, including information about the certainty of the evidence, the balance between benefits and harms and their probabilities, the variability or uncertainty in how patients value the benefits and harms, or the absence of evidence on patient preferences, even with moderate or strong evidence of high quality on the benefits of an option. If other criteria were provided, we coded these as well. We extracted all information that indicated a discrepancy with the strength of recommendation, and scored whether or not textual discrepancies were identified (yes/no). We based this on the CPG text and did not resort to the supporting literature.

Balance of benefits and harms (trade-offs)

We defined a trade-off as a statement presenting the balance of benefits and harms in the treatment decision, ideally based on the probability of benefits and harms, the quality of the evidence, and on how much patients value the outcomes. We extracted statements about the trade-offs made in the CPG or about the trade-offs to be made in the clinical encounter with the individual patient (trade-offs made explicit/trade-offs not made explicit). We also judged whether the presentation of outcomes was sufficiently complete and clear to inform the trade-off (sufficient/insufficient).

Patient preferences

We assessed if patient preferences had been incorporated (yes/no), and if so, how (literature search/data collection by CPG panel/other). Also, we extracted whether explicit assumptions were made regarding patient preferences (yes/no).

CPG panel’s values and preferences

We extracted information about the preferences that supported the CPG panel’s weighing of benefits and harms, and summarised per treatment recommendation if these preferences were explicitly mentioned (yes/no). This theme does not directly originate from the GRADE recommendations. We added it as we encountered statements suggesting that CPG panel’s values and preferences had influenced the development of recommendations. Finally, we assessed if the CPGs facilitated discussion of patient preferences for weak recommendations, as for the latter ‘clinicians and other health care providers need to devote more time to the process of shared decision-making by which they ensure that the informed choice reflects individual values’.28

Advice on how to involve the patient

We extracted statements that described how to involve an individual patient or his/her preferences in the decision-making process, and summarised per recommendation if such statement was given (yes, actively involving the patient or patient preferences in the decision-making/yes, informing the patient/no advice about patient involvement).

Two coders (FRG and AMS) independently applied a first draft of the coding scheme to a CPG module that would not be included in the final selection. They subsequently discussed the coding process and any inconsistencies, and updated the coding scheme. They had not been involved in the development of any CPG in oncology nor GRADE, and had no existing working relationship with the members of the respective CPG panels. The coders independently applied the coding scheme to one of the selected modules and resolved any discrepancies by consensus. Based on this discussion no further changes were made to the scheme. One researcher (FRG) then coded the remaining modules, and the second checked the extraction and scoring. They discussed any inconsistency between them until agreement was reached. Data extracted were analysed descriptively.

Semistructured interviews with CPG developers

Sampling

We or our IKNL contact person invited the panel members involved in the development of the selected modules for participation. Membership and size of the different CPG panels varied, not all were multidisciplinary, and not all included a patient representative. We aimed to interview at least one member of each specialty involved in the development of a module, the patient representative, and the IKNL supervisor of the CPG. As patient representatives did not participate in this study based on a paid position, the respective patient organisations received an incentive of €100.

Data collection

In semistructured interviews, we first checked whether the interviewee agreed with our interpretation of the strength of recommendations, our extraction of the discrepancies found in the CPG text, of the trade-offs, and the completeness and clarity of the presentation of the benefits and harms, of the role of patient preferences, and of the preferences of the CPG panels that supported the recommendations. For the benefits, harms and trade-offs, we asked them how the developers selected which ones to present, and whether the presentation of benefits and harms aimed to facilitate communication in the clinical encounter. Finally, we discussed the function of statements concerning the involvement of patients and their preferences in decision-making for the individual patient.

We adapted the questions to the specific content of the module to be discussed. For each subsequent interview we added or adapted questions based on earlier interviews. Interviews lasted 30–60 min, were audiotaped and transcribed verbatim. One interviewer (FRG) trained in qualitative research methods and highly experienced in interviewing carried out all interviews.

Coding and analysis

We adhered to the Framework Approach to code and analyse the interviews.32 33 The coding scheme was based on the five themes of the CPG analysis described above. First, two researchers (FRG and AMS) independently familiarised themselves with the data, and coded three interviews deductively, to supplement our coding scheme with any additional emerging themes. Dissimilarities in coding were discussed and codes were adapted based on consensus. Second, one researcher applied deductive coding to all other interviews and refined and reduced the codes in a process of rereading and constant comparison of codes. Third, categories of codes were clustered to generate (sub)themes. Steps 2 and 3 were performed by one researcher and checked by the second. Inconsistencies in interpretation of the data and formulation of codes and themes were discussed until consensus was reached. Coding was performed using Atlas.ti software.34

Patient involvement

The CPG committee involved patient representatives for two modules, and we interviewed these patients. One patient (DH) took part in the writing of the manuscript. The article will be shared with the Netherlands Federation of Cancer Patient Societies NFK.

Results

We present the results of the content analysis and the interviews together, structured around the five themes mentioned above. We interviewed 14 CPG panel members: 10 clinicians, 2 patient representatives and 2 IKNL supervisors (table 1). For one module (adjuvant endocrine therapy in breast cancer), only one of the clinician panel members indicated to have time to participate. After an interruption due to a clinical urgency, she did not want to resume the interview because she found the questions too critical. Therefore only the IKNL supervisor and the patient panel member were interviewed. Patients were not part of the CPG panel for the Non-Small Cell Lung Cancer (NSCLC) modules. To illustrate our analyses we add examples of the extractions of the CPG modules in boxes 2–4 and table 2.

Table 2

Examples of CPG panels’ values or preferences reflected in the CPG modules

CPG statement on which the interpretation of the panel’s preference is based	Description of the identified CPG panel’s preference	Concerning what type of recommendation
If breast-conserving surgery is not feasible or desirable, there is an indication for mastectomy. ( S ection : literature review, module 1)	The panel appears to prefer breast-sparing surgery to mastectomy; mastectomy is considered only when breast-sparing surgery is not feasible or desirable.	2 weak.
Ductal Carcinoma In Situ (DCIS) is often discovered based on calcifications on the mammogram that, when biopsied, turn out to be associated with this DCIS. DCIS does not metastasise, and patients with DCIS hence have an excellent prognosis with adequate local treatment. (Section: introduction, module 1)	The panel prefers local treatment and therefore has a more positive attitude about radiotherapy and a less positive attitude about endocrine therapy for DCIS from the outset.(Supplemental note: No survival benefit has been demonstrated for either radiotherapy or endocrine therapy. It is, however, suspected that a subgroup of the radiotherapy group does indeed have improved survival. Radiotherapy also has an effect on the risk reduction of an invasive recurrence, which appears to be more limited with endocrine therapy. This could be a reason for the more positive attitude towards radiotherapy compared with endocrine therapy.)	1 strong.
The risk of radiation pneumonitis seems to be acceptable. (Section: conclusions, module 5)	The panel finds the risk of radiation pneumonitis acceptable. In the literature, this risk is only represented in chance words: the risk is ‘very low’ and ‘generally low’. The reader is shown neither the absolute risk nor patient preferences relevant to this trade-off.	1 strong and 1 weak.
Radiotherapy hence appears to be effective, considering that without adjuvant radiotherapy the risk of recurrence is expected to be higher and the chance of cure to be lower. (Section: literature review, module 6)	In case of positive surgical margins, there is a strong recommendation in favour of adjuvant radiotherapy, arising from the assumption that the benefits outweigh the disadvantages. The phrase ‘appears to be effective’ is used, but the guideline does not state the absolute survival gain and does not address side effects, short term or long term. Furthermore, we do not know if patients differ in how they weigh these considerations.	1 strong.

CPG, clinical practice guideline.

Strongly phrased recommendation for adjuvant radiotherapy after lumpectomy in DCIS patients, combined with a statement about the relevance of patient involvement in the decision:

Recommendation

‘After complete excision of Ductal Carcinoma in Situ (DCIS), radiotherapy of the whole chest wall (with or without boost) is recommended’. (Section: recommendations, module 1)

Statement about patient involvement

‘Individual risk assessment and good deliberation with the informed patient determine whether radiotherapy is applied, with or without boost’. (Section: recommendations, module 1)

Strongly phrased recommendation for adjuvant chemotherapy for patients with an MSI (microsatellite instability) colon carcinoma, combined with a statement about very low-quality evidence.

Recommendation

‘It is recommended that patients with an MSI carcinoma are offered only fluoropyrimidine-oxaliplatin-based chemotherapy’. (Section: recommendations, module 4)

Statement about the evidence

‘The limited evidence concerning the value of oxaliplatin-based chemotherapy in this group shows no difference compared to patients with microsatellite stable (MSS) tumours, so for patients with stage III MSI tumours, oxaliplatin-based chemotherapy remains recommended for now’. (Section: literature review, module 4)

Trade-off statement for a strong recommendation.

It is generally agreed upon that a dose of 45–60 Gy in 3 fractions is safe and can achieve good (>80%) local tumour control. The risk of radiation pneumonitis appears to be acceptable. However, long-term data on the late toxicity of Stereotactic body radiation therapy (SBRT) are lacking, especially for T2 tumours. Evidence pertaining to quality of life is likewise sparse. (Section: conclusions, module 3a)

Trade-off statements for weak recommendations.

Additional surgical resection after endoscopic removal of a malignant polyp should always be a balanced decision because of the relative high number needed to treat, for which the patient should always be fully informed about the potential oncological benefit on the one hand and the risk of complications on the other. (Section: recommendations, module 2a, used for weak recommendations)

In various case series, the incidence of local lymph node metastases in T1 colorectal carcinoma varies from 8% to 14%. There is also a large chance that surgical (segmental) resection of the colon has no therapeutic benefits, while being associated with morbidity and even mortality. Hence, it is important to make a well-considered choice for the treatment of malignant polyps. (Section: literature review, module 2a)

For high-risk malignant colon polyps, the oncological benefit of additional resection should be balanced against the risk of morbidity and possibly even mortality. In this trade-off, the age, tumour location, comorbidity of the patient and the preference of the patient should be taken into account. All patients should be discussed in the multidisciplinary team. (Section: considerations, module 2a, used for weak recommendations)

A retrospective subgroup analysis of the MOSAIC (Multicenter International Study of Oxaliplatin/Fluorouracil/Leucovorinin the Adjuvant Treatment of Colon Cancer) study in patients with stage II colon carcinoma has shown that adding oxaliplatin to a fluoropyrimidine does not convey significant gain in Disease Free Survival and Overall Survival. It seems useful to educate patients with high-risk stage II colon carcinoma about the possible advantages of adjuvant chemotherapy and the concomitant side effects. (Section: literature review, module 2b, used for weak recommendation)

Treatment of centrally located tumours is still under debate, given its high toxicity. (Section: conclusions, module 3a, used for weak recommendation)

Statements that propose to inform the patient.

Additional surgical resection after endoscopic removal of a malignant polyp should always be a considered decision, given the relatively high ‘number needed to treat’, in which the patient must be fully informed about the possible oncological benefit on the one hand and the risk of complications on the other. (Section: recommendations, module 3)

It appears worthwhile to inform patients with a high-risk stage II colorectal carcinoma about the possible advantages of adjuvant chemotherapy and the associated side effects. (Section: literature review, module 4)

Statements that propose to include the patient’s preferences in making the decision.

Side effects and effectiveness of both endocrine therapy and radiotherapy should be weighed together with the patient. (Section: recommendations, module 1)

For high-risk malignant colon polyps, the oncological benefit of additional colon resection should always be weighed against the risk of morbidity and even mortality. Age, tumour location, comorbidity and the patient’s preference should be included in this trade-off. (Section: considerations, module 3)

Strength of CPG recommendations

In the six modules we identified 32 recommendations, of which 14 were phrased as strong and 18 as weak or neutral. The proportion of weak or neutral recommendations was just over half for all modules, except for that on adjuvant chemotherapy for colorectal carcinoma, which had fewer weak recommendations (33%). For five of the recommendations, both strong (three) and weak (one) or neutral (one), we found discrepancies between the strength of recommendation and extracted sentences from the module text. Box 2 shows examples of such discrepancies. In two of the strong recommendations, the discrepancy concerned evidence that was limited or of (very) low quality.

The CPG panel members confirmed our interpretation of the strength of recommendations. They explained that the three strong recommendations in the case of limited evidence were based on a valuation of the outcomes by the CPG panel (see further in the ‘CPG panels’ values and preferences’ section). One explanation for the discrepancies between the strength of recommendation and the extracted were the differences in the handling of low-quality evidence between methodologists and clinicians. One clinician described methodologists as being more careful in drawing conclusions, while clinicians incorporate current standards of practice in the formulation of recommendations.

Panel member: I think that it is inherent to making recommendations, where clinicians and methodologists clash. I am currently preparing the revision of the guideline, and what one sees is that we simply clash immediately with the methodologists in the preparation of the revision. Those are very dogmatic in their methodologic thinking. And the problem is, that that does not work, particularly not for the medical literature, so to say. And that is why the GRADE methodology explicitly discusses that in their approach, that one can upgrade the recommendation if one agrees as professional group that something should or should not be done. (Interview 10, about T1 carcinoma in polyp)

Information supporting the balance of benefits and harms

Three of the modules (T1 carcinoma in polyp and adjuvant chemotherapy in colorectal cancer, stereotactic radiotherapy in Non-Small Cell Lung Cancer) included explicit trade-off statements (see box 3). Probabilities of outcomes were mentioned in one of these, but for the benefits only. One trade-off statement substantiating a strong recommendation included the presentation of a value judgement, but it was unclear whose values it presented: ‘ it is agreed upon that it is safe …’ and ‘ the risk of radiation pneumonitis seems acceptable’.

For one of the six modules, adjuvant chemotherapy for colorectal cancer, we rated the report of benefits and harms and their probabilities as sufficiently complete and clear to inform the strength of recommendation. In three modules information was lacking about benefits, in four about harms, and harms were often only presented generically (eg, ‘complications’, ‘psychological impact’). Relative rather than absolute risk reduction was often presented, and verbal labels rather than numbers were used to convey risk, for example: ‘ The chance of eventually preserving the breast is higher if radiation of the breast already takes place after the first excision’.

Some interviewees found that transparency about the trade-offs in the CPG text could be improved, while others found an explicit mention, including details about benefits and harms and their probabilities, unnecessary. Reasons for the latter were time constraints, the aim to keep the CPG short, the assumption that CPG users know the balance of benefits and harms, or that the weighing of benefits and harms was acceptable to everyone. One interviewee, for example, stated that not recommending endocrine treatment in Ductal Carcinoma In Situ was “common knowledge” and that “we also could have chosen to just leave out the whole paragraph about this adjuvant therapy, to just not mention it at all ” (Interview 15).

The interviewees indicated that in none of the modules patients had been involved in the selection of the outcomes described. Some acknowledged that outcomes might be missing, but a substantial number did not regard a complete presentation of outcomes and their probabilities as necessary, using the following arguments: guidelines should be short, harms are assumed to be common knowledge for clinicians or might be presented in other modules, evidence for long-term harms is lacking, and probabilities from the literature are not applicable to the Dutch setting or would only be representative at the hospital level, not at that of the individual clinician (ie, for mortality due to surgery). Several interviewees were especially reticent to present probabilities in terms of absolute risk reduction, as those percentages would soon be dated, differed between patient groups, or would be too time-consuming to calculate. One stated to have argued to include numbers needed to treat in the CPG, to no avail.

Interviewer: and for what reason is the other side of the coin not mentioned in the CPG? You indicated, already, that actually…

Panel member: the CPG is mostly written to, what we provide as recommendation towards the patient, for the outcome of treatment. I don’t know if the CPG is written at least, I have never interpreted it as such, but I don’t know if one should put in the CPG, let’s say, what’s it called, all risks of treatment. That differs per agent, have different risks. And then the CPG becomes much more extensive. But that is also the baseline knowledge that every oncologist should have. (Interview 7, about adjuvant chemotherapy for colon carcinoma)

Patient preferences

None of the modules stated that evidence about patient preferences had been searched for or elicited. No information was presented about generic patient preferences or about variation in patient preferences, either from the literature or assumed by the panel.

Some interviewees acknowledged that patient preferences may vary and may differ from clinician preferences, and they stressed that the awareness of such variation sometimes motivated a weak recommendation. Reasons not to include information about patient preferences were time and capacity constraints, the assumption that no evidence exists, or lack of awareness that this information is to be included. Others were reluctant to include information about preference variation because it could threaten the relationship between specialties (if this information would lead to patients choosing against the generally accepted treatment modality). Numerous assumptions about patient preferences were voiced, such as that patients prefer lumpectomy to mastectomy, length of life to quality of life, and active treatment to refraining from treatment. Interviewees also stressed that if patients have a strong preference, they will express it anyway.

CPG panels’ values and preferences

None of the modules explicitly labelled statements as presenting the CPG panel’s values and preferences that underlie their weighing of the benefits and harms. We found implicit reference to CPG panels’ preferences having influenced the development of the recommendation in 15 of 32 recommendations (see table 2). These preferences concerned 9 of 14 strong recommendations and 6 of 18 weak recommendations (see table 3).

Table 3

Quantitative overview of the results of the CPG analysis

		Strength of recommendation
		Strongn (%)	Weak or neutraln (%)	Totaln (%)
		14 (44)	18 (56)	32
Trade-offs mentioned	Yes	7 (50)	11 (61)	18 (56)
	No	7 (50)	7 (39)	14 (44)
Patient preferences assessed	Yes	0	0	0
CPG panel’s preferences mentioned	Yes, explicitly	0	0	0
	Yes, implicitly	10 (71)	7 (39)	17 (53)
	No	4 (29)	11 (61)	15 (47)
Statements about patient involvement included	Yes, to actively involve the patient	6 (43)	12 (67)	18 (56)
	Yes, to inform the patient	0	2 (11)	2 (6)
	No	8 (57)	4 (22)	12 (38)

CPG, clinical practice guideline.

As described in the ‘Strength of CPG recommendations’ section, the interviewees sometimes explained discrepancies between the strength of recommendation and the extracted information by the CPG panel’s valuation of the outcomes. Explanations for the panel members’ preferences beyond the evidence were compliance with daily practice, the organisation of care, culture (a preference for radiotherapy seemed more culturally and historically determined than evidence-based), and concerns about keeping a good relationship between specialties when their treatments compete.

Some interviewees found that CPG panels’ preferences underlying the weighing of benefits and harms should be made explicit. One interviewee stated that having an external party critically reviewing the CPGs before publication would foster this. The panel members often expressed their own preference for active treatment versus refraining from (further) treatment or active surveillance, even at the expense of overtreating a substantial part of the patient population.

Panel member: That is watertight, radiotherapy does have an effect. Not for everyone, far from it, but for some. And we cannot sufficiently select for whom it does, so we say, give radiation to all. (Interview 4, about radiotherapy for DCIS patients)

Their motivation was mostly a strong belief in survival gain for a subgroup that cannot be identified as of yet. In these instances, panel preferences for active treatment had influenced the balancing of benefits and harms, such that a recommendation for active treatment would not be a weak one. This was argued, for example, for treatment aimed at reducing local recurrence rates without concomitant survival gain. Concerning this example, an interviewee argued in one instance that it was preferable simply to not include survival as an outcome, as no survival gain was possible given the already high survival (Interview 2, about radiotherapy for DCIS).

Panel member: … but I find it a bit of a bromide to say that DCIS, or rather that radiotherapy for DCIS yields no survival benefit and therefore we shouldn’t do it. Because one cannot improve upon 99% survival benefit. The important thing is, in which sub-groups those recurrences occur that might not be such nice recurrences, that call for a lot more treatment and the like.… (Interview 2, about radiotherapy for DCIS patients)

At the same time, others voiced opinions against overtreatment and pointed out that the paradigm in favour of overtreatment to avoid undertreatment is shifting, particularly in patients diagnosed by population screening (DCIS, T1 carcinoma in polyp).

Advice about patient involvement in decision-making

Five modules included in total 20 statements about the patient’s role in decision-making (see box 4). Relatively more statements14 were seen for the weak than for the strong6 recommendations. All statements recommended to include patients’ preferences in making the decision except for two, relating to weak recommendations, which recommended to inform the patient about the trade-off. One of the three CPGs included a separate chapter about decision-making, in which it was recommended to elicit the preferences of the patient in an SDM process.

Interviewees disagreed on the necessity of recommendations about patient involvement in decision-making. Several stressed that these statements were included only because the patient representative asked for it. Others mentioned that the inclusion was based on the opinion of individual panel members.

Discussion

Healthcare is increasingly guideline-driven, which promotes quality of care and reduces unwarranted practice variation. But guidelines may be a barrier to SDM if they do not acknowledge the preference-sensitive nature of many treatment decisions.1 30 The aim of this study was to explore to what extent CPGs acknowledge preference-sensitive decisions in their recommendations. Our analysis showed that the guidelines involved incomplete and unclear presentation of benefits, harms and the probabilities thereof. This makes it difficult for the users to judge the appropriateness of the strength of the recommendation. Further, it may hinder patient engagement in decision-making, which requires that patients are fully informed about the trade-offs. Moreover, patients may be directly accessing the guidelines, and inclusion of this information makes guidelines also more useful to them. Whether or not clinicians have complete knowledge about all benefits and harms and their probabilities is questionable, and from an earlier study we know that at least many clinicians do not share this information with their patients during the decision-making process.14 15 Complete and clear presentation in CPGs of the benefits and harms helps to fill knowledge gaps in CPG users, and acknowledges the importance of the information for the trade-offs to be made with the individual patient in preference-sensitive decisions.

Furthermore, information on patient preferences or the variation therein was not included in any of the six modules analysed. If GRADE were to be followed, this lack of evidence on patient preferences should have led to more weak recommendations than seen. Additionally, we found indications that panel members’ assumptions about patient preferences as well as their own preferences determined the recommendations. This corroborates findings of de Kort et al 21 on the role of value judgements in guideline formulation in palliative oncology. They found that preferences, such as those for intervening and prolonging life, were not mentioned in the guidelines but had played an important role in determining final recommendations. In line with a study by Alexander et al,35 it appeared that panel members find it difficult to refrain from providing a clear recommendation in a case of limited or conflicting evidence. CPG panel preferences for active treatment had influenced the way the panel had balanced benefits and harms, such that a recommendation for active treatment would be strong and overtreatment likely. The strong belief in survival gain for a subgroup that cannot be identified as of yet fosters the so-called therapeutic illusion, in which both physicians and patients overestimate the benefits of treatment, since patients are seemingly cured by treatment while they might have had the same outcome without treatment.36 Rather than routinely resort to active treatment in these instances, the discussion should be opened on how to deal with such uncertainties. Little research is available yet on how best to communicate uncertainty,37 but this does not relieve us from the obligation to discuss matters honestly with patients. Such openness would contribute to reducing unnecessary treatment, addressing unacceptable variation and delivering more appropriate, personalised care.38 Guidelines can facilitate this discussion by acknowledging preference-sensitive decisions, and encouraging users to become more aware of choice and presenting multiple options to patients.

A limitation of the format of GRADE is that it asks for a dichotomous categorisation (weak vs strong) and a recommendation either for or against. This categorisation makes it difficult to explicitly state that multiple options are medically reasonable. Furthermore, information on patient preferences should be more often sought in guideline development. Oncologist experts are invited in guideline panels because of their content expertise, but this involves a risk when more evidence is available for benefits than for harms, and when there is no evidence on patient preferences. Then chances increase that panel members resort to their own preferences, often favouring active treatment and neglecting harms.39 The guideline development process, while aiming at achieving Evidence-Based Medicine, may threaten it by its reliance on expert judgement at the expense of involving patient preferences. GRADE publications accede that panels’ judgements of patient preferences often rely on their interactions with patients, but how well such judgements correspond to typical values and preferences is uncertain.

We do not know to what extent our analysis will hold for CPGs from other countries than the Netherlands. Dutch healthcare is likely less paternalistic than that in many other countries, and the Netherlands is leading in the implementation of SDM.40 We therefore expect more discrepancies between evidence and recommendations to arise elsewhere. de Kort et al 21 analysed a sample of evidence-based oncology guidelines from other countries, and found that recommendations were rarely explained and value judgements were not made explicit either. Further, we do not know if, but have no reason to expect that, our findings will be different for other specialties. We urge researchers in other countries and other fields to evaluate their guidelines with preference sensitivity in mind as well.

An analysis like the one performed runs the risk of subjectivity, as the data extraction and coding require interpretation. We therefore checked our results with the developers of the guidelines we studied. This provided a validation of our analysis. The aim of this endeavour was to highlight an issue that is a major barrier to patient-centred care and SDM in particular.41 With the strong current call for patient involvement, worldwide, it is important to establish to what extent guidelines potentially hinder such involvement, and our study may be seen as a first step in that direction.

In sum, our analysis points to a lack of transparency in the CPG development process about benefits and harms and their probabilities, the preferences of the guideline panel members, and their assumptions about patient preferences. Awareness needs to be created among CPG developers that their judgements of the balance of benefits and harms are value-laden, and that variation exists in these judgements, among both clinicians and patients. Clear instructions and training to enhance knowledge and implementation of GRADE might improve the acknowledgement of preference-sensitive decisions in guidelines and support SDM. This will help avoid what McCartney et al 23 feared in their 2016 analysis in the BMJ: ‘there is the danger of guideline recommendations being applied to people who do not place the same values on those recommendations as their clinician (…)’.23