When evidence is lacking: a mixed-methods approach for the development of practice guidance in liver transplantation.

BACKGROUND: Most interventions for conditions with a small cohort size, such as transplantation, are unlikely to be part of a clinical trial. When condition-specific evidence is lacking, expert consensus can offer more precise guidance to improve care. Management of cardiovascular risk in liver-transplant recipients is one example for which clinicians have, to date, adapted evidence-based guidelines from studies in the general population. However, even when consensus is achieved, implementation of practice guidance is often inadequate and protracted. We report on a novel mixed-methods approach, the Northwestern Method©, for the development of clinical-practice guidance when condition-specific evidence is lacking. We illustrate the method through the development of practice guidance for managing cardiovascular risk in liver-transplant recipients.
METHODS: The Northwestern Method© consists of (i) adaptation of relevant, existing, evidence-based clinical-practice guidelines for the target population; (ii) consensus by experts of the proposed practice guidance; (iii) identification of barriers to guidance adherence in current practice; and (iv) recommendation for implementation and dissemination of the practice guidance. The method is based on an iterative, user-centered approach in which the needs, wants, and limitations of all end users, including patients, are attended to at each stage of the design and development process.
CONCLUSIONS: The Northwestern Method© for clinical-practice-guidance development uses a mixed-methods approach to bring together broad representation from multiple disciplines and practice settings to develop consensus considering the unique needs and preferences of patients, caregivers, and practitioners who are directly impacted by clinical-practice-guidance recommendations. We hypothesize that a priori involvement of end users in the guidance-development process will lead to sustainable implementation of guidance statements into clinical practice.

Introduction

Liver transplantation is a high-risk, high-reward treatment for end-organ failure. Approximately 8,000 liver transplants are performed in the USA each year [1] and represent a very modest cohort of patients with substantial clinical differences (e.g. etiology of liver disease and co-morbidities). Most clinical interventions for rare disease conditions, because of the small cohort size, have not been and are unlikely to be part of any clinical trial. Management of cardiovascular risk in liver-transplant recipients is one such example for which most clinicians have, to date, adapted evidence-based guidelines from studies conducted in the general population. Our prior work showed that cardiovascular disease is a leading cause of morbidity and mortality in liver-transplant patients [2, 3]. Further, we recently showed that liver-transplant recipients receive <50% of the American Heart Association/American College of Cardiology clinical-practice recommendations for the detection and management of potentially modifiable cardiovascular-disease risk factors (e.g. high blood pressure), suggesting that adaptations to the guidelines may be needed for liver-transplant recipients [4].

Clinical-practice guidelines offer recommendations for the management of typical patients. In order to help clinicians to make evidence-based medical decisions, developers of practice guidelines usually grade the strength of their recommendations and rate the quality of the evidence informing the recommendations by conducting a systematic review of the available evidence and assessing of the benefits and harms of alternative options [5]. Consensus statements (also called practice-guidance documents) typically address topics in which the evidence base is less extensive compared to clinical-practice guidelines. Consensus statements are developed by experts, usually multidisciplinary, convened to review the research literature in an evidence-based manner for the purpose of advancing the understanding of a topic [6].

In liver transplantation, both clinical-practice guidelines and consensus documents have been developed using the best available evidence in order to provide recommendations for optimizing patient care. However, the quality of clinical-practice guidelines and consensus documents for transplant recipients varies widely [7, 8]. Recommendations for guideline-directed management and therapy, which encompass clinical evaluation, diagnostic testing, and treatments, are effective only when adopted by both practitioners and patients. Indeed, shared decision-making between clinicians and patients increases adherence to recommendations and patients should be engaged to make decisions based on their individual values, preferences, and associated conditions and co-morbidities [9, 10]. However, guidance documents typically do not consider the attitudes and preferences of end users (e.g. practitioners, patients, and caregivers) nor do these documents consider system factors that may influence implementation. We conducted a literature review of recommendations for liver-transplant patients, published between 2009 and 2019, and found that less than half of the documents included any recommendations about the implementation of the guidance statements into clinical practice (Supplemental Table 1). The topics of these documents are shown in Figure 1. Ultimately, concerns regarding the quality of the guideline/consensus documents, lack of robust evidence supporting recommendations, and unidentified patient, provider, or system factors that influence both engagement and implementation may contribute to poor uptake of guidance documents [11].

Figure 1.

Distribution of topic areas of focus among published clinical-practice guidelines and consensus topics in solid-organ-transplant recipients, 2009–2019

To address these gaps, we developed a novel mixed-methods approach, The Northwestern Method©, to achieve both consensus on clinical-practice guidance and identification of essential elements for the successful implementation and dissemination of the clinical-practice guidance. The Northwestern Method© harnesses the perspectives of multi-professional, multidisciplinary experts and considers the goals of end users, including patients and caregivers, in order to facilitate the future integration of clinical-practice guidance into clinical care. The method was used to develop clinical-practice guidance for the management of cardiovascular risk in liver-transplant recipients.

The Northwestern Method©

The Northwestern Method© consists of (i) adaptation of existing, evidence-based clinical-practice guidelines for the relevant target population; (ii) consensus by expert clinicians of the proposed clinical-practice guidance; (iii) identification of barriers and facilitators to the integration of the guidance into clinical practice; and (iv) recommendations for the implementation and dissemination of clinical-practice guidance by a learning collaborative of key end users (Figure 2). The method is based on an iterative, user-centered approach or framework in which the needs, wants, and limitations of all end users, including patients and caregivers, are attended to at each stage of the design and development process [12]. All methods described were conducted in accordance with the Helsinki Declaration of 1975 and approved by the Institutional Review Board at Northwestern University in Chicago, IL.

Figure 2.

Schematic diagram of The Northwestern Method©

Adaptation of evidence-based clinical-practice guidelines

The adaptation of evidence-based clinical-practice guidelines began with the assembly of a multidisciplinary (e.g. hepatology, surgery, cardiology, endocrinology, nephrology, primary care), multi-professional (e.g. physician, nursing, pharmacy) team of experts in liver transplantation and cardiovascular-disease risk. During a series of in-person sessions, the team of experts reviewed existing evidence and recommendations. Guidelines for each of the six clinical-practice cardiovascular-disease risk-factor domains (lipids, renal function, blood pressure, glucose control, weight management, and tobacco use [13]) were reviewed. Clinical-practice guidelines and consensus documents published from major US and international societies pertaining to cardiovascular-disease risk and care of solid-organ-transplant recipients or the general population served as the key source documents [14-33]. The team was asked to propose adaptations for each guideline/consensus statement by considering whether the clinical-evaluation, diagnostic-testing, or treatment recommendations required adaptation when applied to a liver-transplant recipient. Table 1 provides an example of the key elements (measurement, frequency, treatment targets, secondary prevention, and primary prevention) that were adapted for liver-transplant recipients for one domain (lipids).

Table 1.

Example cardiovascular disease (CVD) lipid clinical-practice-guideline assessment measures

Lipids	2012 AASLD/AST recommendation for liver-transplant recipients	2013 AHA/ACC recommendation for the general population	2009 KDIGO guidelines for kidney-transplant recipients	Potential adaptation of current guideline elements
Measurement frequency	The measurement of blood lipids after a 14-hour fast is recommended annually	For adults aged 20–79 years perform risk assessment and lipid panel every 4–6 years if free of ASCVD	Measure a complete lipid profile in all adult (>18 years old): 2–3 months after transplant 2–3 months after a change in treatment or other conditions known to cause dyslipidemia at least annually, thereafter	Standard measurement of lipid panels based on time of transplant
Treatment targets	An elevated LDL-C >100 mg/dL, with or without hypertriglyceridemia, requires therapy   No definition of hypertriglyceridemia given	The Expert Panel makes no recommendations for or against specific LDL-C or non–HDL-C targets for the primary or secondary prevention of ASCVD   For individuals with diabetes:   evaluate and treat patients with fasting triglycerides >500 mg/dL	If LDL-C ≥100 mg/dL, treat   All kidney-transplant recipients: evaluate and treat patients with fasting triglycerides >500 mg/dL	Incorporation of LDL-C targets based on risk and comment on triglyceride targets particularly as it relates to immunosuppression side effects (e.g. mTOR inhibitors)
Secondary prevention in clinical ASCVDa	No specific guideline	For adults ≤75 years old:   high-intensity statin therapyb when high-intensity statin therapy is contraindicated, moderate-intensity statin should be used as the second option if tolerated   For adults >75 years of age:   it is reasonable to evaluate the potential for ASCVD risk-reduction benefits and for adverse effects and drug–drug interactions and to consider patient preferences when initiating a moderate- or high-intensity statin. It is reasonable to continue statin therapy in those who are tolerating it	No specific guideline	Provide a guideline for statin therapy and dosing for those with clinical ASCVD   Consider drug–drug interactions (e.g. calcineurin inhibitors)
Primary prevention	Statin therapy should be introduced   No recommendation on statin dosing   Suboptimal control with statins can be improved by the addition of ezetimibe   Isolated hypertriglyceridemia is first treated with omega-3 fatty acids (up to 4 g daily if tolerated)   If omega3s not sufficient for control, gemfibrozil or fenofibrate can be added, although patients must be followed carefully for side effects, especially with the concomitant use of statins and calcineurin inhibitors	For individuals with LDL-C ≥190 mg/dL:   use high-intensity statin therapy unless contraindicated   For individuals unable to tolerate high-intensity statin therapy:   use the maximum tolerated statin intensity   Individuals with diabetes and LDL-C 70–189 mg/dL   For individuals aged 40–75 years:   use moderate-intensity statin therapy   For individuals aged 40–75 years with 10-year ASCVD riskc ≥7.5%:   initiate high-intensity statin therapy unless contraindicated   For individuals aged <40 or >75 years, or with LDL-C <70 mg/dL:   evaluate the potential for ASCVD benefits and for adverse effects and drug–drug interactions and consider patient preferences when deciding to initiate, continue, or intensify statin therapy   For individuals without diabetes and with LDL-C 70–189 mg/dL:   estimate 10-year ASCVD risk to guide initiation of statin therapy for the primary prevention of ASCVD   For individuals aged 40–75 years with 10-year ASCVD risk ≥7.5%:   use moderate- to high-intensity statin therapy   For individuals aged 40–75 years with 10-year ASCVD risk of 5% to <7.5%:   offer treatment with a moderate-intensity statind	Individuals with LDL-C ≥100 mg/dL:   treat to reduce LDL-C to <100 mg/dL   For individuals with LDL-C <100 mg/dL, triglycerides ≥200 mg/dL, and non-HDL-C ≥130 mg/dL:   treat to reduce non-HDL-C to <130 mg/dL   No recommendation on drug type or dosing	Risk-based treatment including statin-dose recommendations with consideration for drug–drug interaction (e.g. calcineurin inhibitors)

AASLD, American Association for the Study of Liver Diseases; ACC, American College of Cardiology; AHA, American Heart Association; ASCVD, atherosclerotic cardiovascular disease; AST, American Society for Transplantation; LDL-C, low-density lipoprotein cholesterol; KDIGO, Kidney Disease Improving Global Outcomes; non-HDL-C, non-high density lipoprotein cholesterol.

Clinical ASCVD includes acute coronary syndromes, history of myocardial infarction, stable or unstable angina, coronary or other arterial revascularization, stroke, transient ischemic attack, or peripheral arterial disease presumed to be of atherosclerotic origin.

High-intensity statin therapy (lowers LDL-C by ≥50%): Atorvastatin 80 (40) mg, Rosuvastatin 20 (40) mg.

Moderate-intensity statin therapy (lowers LDL-C by 30% to <50%): Atorvastatin 10 (20) mg, Rosuvastatin10 (5) mg, Simvastatin 20–40 mg, Pravastatin 40 (80) mg, Lovastatin 40 mg, Fluvastatin XL 80 mg, Fluvastatin 40 mg twice per day, Pitavastatin 2–4 mg.

Low-intensity statin therapy (lowers LDL-C by <30%): Simvastatin 10 mg, Pravastatin 10–20 mg, Lovastatin 20 mg, Fluvastatin 20–40 mg, Pitavastatin 1 mg.

Bold: evaluated in randomized clinical trials (RCTs) included in CTT (Cholesterol Treatment Trialists Collaboration) 2010 meta-analysis that demonstrated a decrease in CVD-event rates. Italics: FDA-approved but no RCTs have studied.

Estimated 10-year or ‘hard’ ASCVD risk includes first occurrence of nonfatal myocardial infarction, coronary-heart-disease death, and nonfatal and fatal stroke as used by the Risk Assessment Work Group in developing the Pooled Cohort Equations (http://my.americanheart.org/cvriskcalculator and http://www.cardiosource.org/en/Science-And-Quality/Practice-Guidelines-and-Quality- Standards/2013-Prevention-Guideline-Tools.aspx).

Before initiation of statin therapy for the primary prevention of ASCVD in adults with LDL-C 70–189 mg/dL without clinical ASCVD or diabetes, it is reasonable for clinicians and patients to engage in a discussion that considers the potential for ASCVD risk-reduction benefits and for adverse effects and drug–drug interactions, as well as patient preferences for treatment.

high-intensity statin therapy

offer treatment with a moderate-intensity statin

Consensus by expert clinicians of the clinical-practice guidance

To reach initial consensus, the proposed guidance statements were incorporated into six surveys (corresponding to each of the six cardiovascular-disease risk-factor domains) for administration to a group of multidisciplinary, multi-professional clinicians across the USA with clinical expertise in liver-transplant recipients and cardiovascular-disease care. In the survey, each practice-guidance statement was assessed across four domains: accuracy, importance, implementation, and clinical provider responsible for its implementation. The surveys were administered using the research electronic data capture (REDCap) platform. If a proposed guidance statement was rated as inaccurate, a respondent was asked to provide the rationale for their assessment. Respondents ranked the importance and feasibility of implementation of each guidance statement using a Likert scale. Respondents were asked to indicate which clinical provider (e.g. primary care, hepatologist, endocrinologist) should be designated as the provider primarily responsible for provision of the proposed guidance statement. A statement with >80% agreement was considered to have reached consensus. Any statement with ≤80% agreement was selected for further discussion. A modified Delphi process [34] was used for the discussion by convening a group of national experts in liver transplantation and cardiovascular-disease risk for an online panel focus group. Panel participants first partook in an initial anonymous vote of each statement selected for further discussion, followed by open discussion of proposed revisions to the guidance statement and anonymous vote, iteratively, until >80% consensus was achieved.

Identification of barriers and facilitators in current clinical practice

Current barriers and facilitators to the use of clinical-practice guidelines about cardiovascular risk for liver-transplant recipients were assessed by conducting a series of focus groups that included all ‘end users.’ The aim of the focus groups was to identify, from the perspective of all end users, perceptions of failures, inefficiencies, and barriers, as well as any facilitators in the processes and systems of using clinical-practice guidelines/guidance [35]. We conducted seven separate focus groups with key end users who provide the cardiovascular-disease care of liver-transplant recipients, as well as liver-transplant recipients and their caregivers. Clinician focus groups were divided by medical specialty and separate groups were conducted with liver-transplant patients and with caregivers, respectively [36]. Each focus group included 8–10 participants in a 90-minute audio-recorded discussion. A trained facilitator moderated each focus group, using a standardized guide that assessed (i) perceptions of barriers to guideline-directed cardiovascular care after liver transplant (e.g. ‘What problems do you have with providing/getting cardiovascular care?’); (ii) cardiovascular-disease healthcare-delivery experience after liver transplant (e.g. ‘What works/doesn’t work with delivering/receiving cardiovascular disease care?’); and (iii) unmet information needs (e.g. ‘What additional information about cardiovascular disease risk/care after liver transplant would you like/do you feel is needed?’).

Strategy for the implementation and dissemination of clinical-practice guidance

The final step of the Northwestern Method© involves developing a strategy for the implementation and dissemination of clinical-practice guidance. A learning collaborative (LC) model was used for this step. The LC model, developed by the Institute for Healthcare Improvement, has five key components: (i) quality-improvement leadership; (ii) content or domain experts; (iii) multidisciplinary team; (iv) targeted goals; and (v) face-to-face collaboration and frequent sessions with the sharing of data, best practices, and experiential learning [37]. We created an LC of eight multispecialty, multi-professional providers who provide cardiovascular-disease care to liver-transplant recipients. The goal of the LC was to develop a set of recommendations for the implementation and dissemination of cardiovascular-disease clinical-practice guidance in liver-transplant recipients. The recommendations were derived from solutions elucidated during the focus groups in response to questions about barriers and facilitators to current clinical practice. After the initial development of a recommendation, the LC members conducted semi-structured interviews with end users, including patients, caregivers, and a range of multispecialty, multi-professional providers to assess the generalizability and acceptance of the recommendation for the implementation of a statement of the clinical-practice guidance. Target recommendations were redesigned until the recommendations were determined to be feasible and likely to be successful based on the expected change in process and outcome measures. A final implementation plan was created that describes the recommendations or ‘best practices’ are described for the implementation of each statement of the clinical-practice guidance for cardiovascular-disease-risk care in liver-transplant recipients, including recommendations for which the provider should be primarily responsible for providing each guidance statement.

Discussion

We propose a novel mixed-methods approach for the development of clinical-practice guidance for rare disease conditions for which evidence-based guidelines are unlikely to ever be created. The approach was used to develop clinical-practice guidance for cardiovascular-disease risk in liver-transplant recipients, which we, and others, have demonstrated is the leading cause of early (<1-year) mortality and the third leading cause of late (>1-year) mortality after liver transplantation [2, 38–40]. This approach is a highly iterative, end-user-centered approach that attends to the needs, wants, and limitations of end users at each stage of the design and development process with the goal of not only developing the clinical-practice guidance, but also facilitating effective and sustainable implementation and dissemination. The final phase of this project will be to monitor the implementation and apply continuous quality improvement to assure a sustainable and effective reduction in cardiovascular events among liver-transplant recipients.

The three best-known consensus methods are the Delphi process, the expert panel, and the consensus-development conference [41]. All of these methods involve measuring consensus, while the last two methods also focus on developing consensus. The overall aim of any consensus method is to determine the extent to which experts agree (or disagree) about a given issue. The Northwestern Method© is similar in scope, but the process draws from each of these established methods, adding qualitative research methodology in the form of focus groups to obtain key stakeholder perspectives not otherwise captured in the guidance-development process.

Many studies demonstrate the efficacy of clinical-practice-guideline recommendations (e.g. aldosterone antagonist after acute myocardial infarction in selected patients with ejection fraction <40% and diabetes) for improving the processes and outcomes of care (e.g. reduction in death) [42]. However, effective application of clinical-practice guidelines is difficult to achieve (e.g. <15% of acute-myocardial-infarction survivors with ejection fraction <40% and diabetes receive an aldosterone antagonist) [43]. Practitioners do not follow guidelines for many reasons, including agreement with the evidence, inertia of changing practice, lack of awareness, self-efficacy (e.g. lack of confidence in providing effective smoking-cessation counseling) or outcome expectancy (e.g. smoking-cessation counseling is unlikely to lead to smoking cessation), and external barriers (e.g. time limitations) [44]. Thus, mere dissemination of guidelines is insufficient. User-centered approaches in which patients and clinicians themselves identify potential barriers to guideline adoption, evaluate and redesign clinical workflows, and create and disseminate practical recommendations and educational material are proven strategies to enhance guideline uptake and adherence [45]. The Northwestern Method© seeks to adapt this type of user-centered approach to the process of clinical-practice-guidance development. The approach also sets the stage for future pragmatic trials that assess both the effectiveness and implementation of clinical-practice-guidance interventions in real clinical settings.

Importantly, The Northwestern Method© is not meant to replace the rigorous standards that have been developed to summarize, rate, and grade the best available evidence [5]. For example, the approach proposed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) Working Group offers a transparent and structured process for developing and presenting evidence summaries and for carrying out the steps involved in developing recommendations. The GRADE handbook states that GRADE ‘seeks to bring together the evidence with considerations of values and preferences of patients and society to arrive at recommendations’ [5]. However, there is no transparent strategy outlined in GRADE for evaluating the values and preferences of patients. The Northwestern Method© offers one potential solution to this limitation.

In conclusion, The Northwestern Method© for clinical-practice-guidance development uses a mixed-methods approach to bring together broad representation from multiple disciplines and practice settings to develop consensus considering the unique needs and preferences of patients, caregivers, and practitioners who are directly impacted by clinical-practice-guidance recommendations. Future studies are needed in which this methodology is applied and evaluated to determine whether a priori involvement of end users in the guidance-development process leads to sustainable implementation of guidance statements into clinical practice.

Supplementary data

Supplementary data is available at Gastroenterology Report online.

Authors’ contributions

L.B.V., D.M.L.-J., and J.L.H. conceived and designed the project. L.B.V., M.K., P.C., S.P., A.D., D.J.F., D.M.L.-J., and J.L.H. collected the data. L.B.V., M.K., P.C., S.P., A.D., D.J.F., D.M.L.-J., and J.L.H. analysed and interpreted the data. L.B.V. and J.L.H. drafted the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by the National Heart, Lung and Blood Institute at the National Institutes of Health to L.B.V. [grant number, K23 HL136891].

Click here for additional data file.