Standardized Outcomes in Nephrology-Transplantation: A Global Initiative to Develop a Core Outcome Set for Trials in Kidney Transplantation.

BACKGROUND: Although advances in treatment have dramatically improved short-term graft survival and acute rejection in kidney transplant recipients, long-term graft outcomes have not substantially improved. Transplant recipients also have a considerably increased risk of cancer, cardiovascular disease, diabetes, and infection, which all contribute to appreciable morbidity and premature mortality. Many trials in kidney transplantation are short-term, frequently use unvalidated surrogate endpoints, outcomes of uncertain relevance to patients and clinicians, and do not consistently measure and report key outcomes like death, graft loss, graft function, and adverse effects of therapy. This diminishes the value of trials in supporting treatment decisions that require individual-level multiple tradeoffs between graft survival and the risk of side effects, adverse events, and mortality. The Standardized Outcomes in Nephrology-Transplantation initiative aims to develop a core outcome set for trials in kidney transplantation that is based on the shared priorities of all stakeholders.
METHODS: This will include a systematic review to identify outcomes reported in randomized trials, a Delphi survey with an international multistakeholder panel (patients, caregivers, clinicians, researchers, policy makers, members from industry) to develop a consensus-based prioritized list of outcome domains and a consensus workshop to review and finalize the core outcome set for trials in kidney transplantation.
CONCLUSIONS: Developing and implementing a core outcome set to be reported, at a minimum, in all kidney transplantation trials will improve the transparency, quality, and relevance of research; to enable kidney transplant recipients and their clinicians to make better-informed treatment decisions for improved patient outcomes.

Advances in transplantation medicine have led to substantial improvement in short and intermediate outcomes in kidney transplant recipients worldwide with a 1-year graft survival rate exceeding 95% in some countries.[1-5] However, major hurdles to the success of kidney transplantation persist. Long-term graft survival has remained largely unchanged over the past few decades.[2,6-11] For recipients with a functioning graft 1 year after transplant, their 10-year graft survival is estimated to be 50% to 70%.[6,10,12,13] Furthermore, long-term immunosuppression leads to adverse outcomes including cancer, cardiovascular disease, diabetes, and infection—all major causes of premature death in recipients.[8,11,14-18] In this complex setting, decision making should be informed by clinical trials that measure and report outcomes of relevance to patients and clinicians.

Unfortunately, many trials in kidney transplantation are short-term, focused on immunosuppression only, and frequently use surrogate endpoints, such as serum creatinine and glomerular filtration rate, because they require a smaller sample size and less time and resources to determine treatment efficacy.[19] However, these outcomes are largely un-validated, ill defined, vary widely across trials, and may not be meaningful or directly relevant to patients.[19-23] Such problems have important implications because patients and clinicians make treatment decisions that require individual-level and multiple tradeoffs between graft survival and the risk of side effects, adverse events, and mortality.[20,24] Regulatory agencies may also be required to approve products on the basis of unvalidated and/or short-term surrogate endpoints.[19] In recent years, there has been a surge in the number of biomarkers used to define acute rejection, delayed graft function, progression of native kidney disease, and late allograft injury,[25-30] but the problem of validation has largely remained.[6,29]

There is substantial heterogeneity in the myriad of outcomes in trials in kidney transplantation, and incomplete reporting of outcomes can render estimates of treatment effect unreliable. Findings from a recent systematic review of trials in kidney transplantation indicated that only 79% provided complete reports of death, less than half reported time to death, and complete reports for graft function were infrequent with 35% and 28% trials reporting estimated glomerular filtration rate and creatinine, respectively. Similarly, studies have shown that kidney function has been assessed using different outcome measures and at varying time points.[31,32]

As in most research, there is a tendency for trialists in kidney transplantation to measure and report outcomes for reasons of feasibility and efficiency rather than relevance to patients and clinicians. The mismatch between the outcomes of relevance to stakeholders, such as patients and clinicians, compared with what is routinely measured and reported is apparent across medical specialties including kidney transplantation. Although graft loss, mortality, cancer, diabetes, cardiovascular disease, and infection have been shown to be of greatest importance to patients,[20,24] long-term studies in kidney transplantation still primarily measure and report surrogate endpoints, such as kidney function (serum creatinine, creatinine clearance) and graft rejection. These surrogate endpoints may not be associated with clinical outcomes.

Immunosuppressive medications are potentially harmful, increase the risk of comorbidities and may cause severe acute side effects that impair quality of life (QoL).[24] These adverse outcomes amplify the treatment burden for patients and can potentially lead to nonadherence, which is associated with a 7-fold increase in the risk of graft failure.[33,34] Yet, only 2% of trials of immunosuppression in kidney transplantation report QoL outcomes, and 95% of these QoL outcomes favored the intervention, strongly suggestive of a major outcome reporting bias.[21] Reliable evidence for treatment decision-making based on QoL outcomes is lacking in kidney transplantation, and the inability to address patient priorities and concerns can contribute to nonadherence.

The problems with outcome selection and reporting are prevalent and increasingly recognized as a major cause of inefficiency and waste in biomedical research across all medical disciplines.[35] Many initiatives have been established worldwide to develop core outcome sets—an agreed minimum set of standardized outcomes that should be measured and reported in all trials for a specific clinical area (Figure 1).[36-39] Core outcome sets can minimize reporting bias, promote consistency across trials to enable direct comparisons of the effect of different intervention, and ensure outcomes are relevant and important to patients and health professionals who make health care decisions.[36,37,39,40] The Outcome Measures in Rheumatology (OMERACT) consensus initiative commenced in 1992 and has been at the forefront in engaging patients, providers, and policy makers to develop core outcome sets for clinical trials in rheumatology that has helped to improve the completeness of outcome reporting in trials.[41-43] Then, in 2010, the Core Outcome Measures for Effectiveness Trials was launched to foster the development and implementation of core outcome sets.[37,40]

FIGURE 1

Conceptual schema of core outcomes.

In kidney transplantation, there is increasing recognition of the need to develop core outcomes spurring some efforts to review outcomes used in trials in kidney transplantation.[22,44-46] The Standardized Outcomes in Nephrology (SONG) initiative was recently formed to establish core outcomes across the full spectrum of chronic kidney disease with an initial focus on hemodialysis.[47] The SONG-Tx project aims to establish a core outcome set for trials in kidney transplantation and other types of research including observational studies (eg, registries, quality indicators for clinical care).

MATERIALS AND METHODS

The SONG-Tx project involves 3 phases: systematic review, international online Delphi survey, and a consensus workshop (Figure 2). The methods detailed below are adapted from the processes used in SONG-Hemodialysis, and the OMERACT[48] methodological framework that has been recognized by the World Health Organization[49] as a valid approach for developing core outcomes.

FIGURE 2

SONG-Tx process.

Phase 1: Systematic Review of Outcome Domains Reported in Randomized Controlled Trials of Interventions for Kidney Transplant Recipients

A systematic review will identify and compare outcome domains and measures reported in randomized controlled trials (RCTs) of interventions for adult kidney transplant recipients.

Search Strategy

We will search the Cochrane Database of Systematic reviews to identify all systematic reviews that included at least 1 RCT involving adult kidney transplant recipients. We will retrieve the full text article of all RCTs included in the systematic review, and also search for all additional articles published from the same trial using key word searches in electronic databases (MEDLINE, Embase) and trial registries. No date or language restrictions will be applied.

Types of Studies

For feasibility, the Cochrane Database of Systematic reviews will be used as a sampling frame to identify RCTs in kidney transplantation, as has been done in previous analysis of outcomes reporting in RCTs.[22,50] Based on a preliminary search, we expect that more than 300 RCTs will be included.

Types of Interventions

Any intervention (including but not limited to surgical, pharmacological [immunosuppression, antibiotics, antifungal, antivirals, diuretics, statins, vaccines], psychosocial and lifestyle) for kidney transplant recipients will be eligible.

Types of Participants

Adult kidney transplant recipients (aged ≥ 18 years).

Exclusion Criteria

Randomized controlled trials that included other solid organ or tissue transplant recipients (eg, lung, liver, heart, pancreas, bone marrow) will be excluded as our primary focus is on kidney transplantation.

Eligibility of Studies

Two reviewers (B.S., N.E.) will independently assess all records obtained from the searches. Full texts of all potentially relevant systematic reviews and RCTs will be assessed independently by the 2 reviewers, and any disagreement on the eligibility of included studies will be resolved through consultation with a third reviewer (A.T.).

Data Extraction

One reviewer (B.S.) will extract characteristics of all the trials including: first author, date of publication, country/ies in which the trial was conducted, sample size, participant characteristics (range and mean age, time since transplant, sex), trial duration, name and type of intervention (eg, surgical, pharmacological, psychosocial, lifestyle), and all outcomes as reported in the trial (including definitions, tools for measurement, thresholds, time points or time frames for measurement, change in level or percentage, scores). Two reviewers (N.E., A.T.) will cross check the data extraction.

Data Analysis and Presentation

The data will be entered into Microsoft Excel to assist with data management, tabulation, and analysis. Author B.S. will group all similar outcomes into appropriate outcome domains, which will be reviewed and discussed by the SONG-Tx Steering Group (J.R.C. [Chair], K.B., J.G., M.A.J., L.M., T.P.P., D.R., L.R., A.W., G.W.] and SONG Executive Committee (J.C.C., S.C., J.G., T.H., B.H., B.M., P.T., W.V.B., D.C.W., W.C.W., A.T.). The outcome domains will be broadly classified as surrogate, clinical, or patient-reported outcomes. We will identify the number of trials that reported each outcome domain. For each outcome domain, we will assess the number of different outcomes (including measures) and the number of trials that assessed each specific outcome and perform statistical analyses using the software package R (version 3.2.3).

Phase 2: International Online Delphi Consensus Survey

An international online Delphi survey will be conducted to generate consensus on the outcome domains that are most important to all stakeholder groups. The Delphi has been used to obtain reliable consensus on core outcome sets across a range of health conditions.[51-54] This technique involves 2 or 3 rounds of surveys completed sequentially and anonymously by a panel of experts with experience or expertise on the topic[38,55] and allows feedback of individual contributions of knowledge, information, and perspectives; assessment of the group view; and opportunities for respondents to revise their opinions.[56] The controlled communication in this process minimizes direct confrontation and allows individual respondents to express independent thought and enables equitable contribution from all participants. This study has been approved by the Human Research Ethics Committee of The University of Sydney (2015-228).

Participants and Recruitment

There is no universal agreement on the “minimum” or appropriate sample size for a Delphi panel. The majority Delphi of studies in core outcome development have included less than 200 respondents. Given the large health professional and consumer networks in kidney transplantation and to achieve broad engagement internationally, our minimum target sample size will be 500 respondents with at least 250 patients with chronic kidney disease (either with a functioning graft, on dialysis having lost a kidney transplant, or waiting for a kidney transplant)/family members/living kidney donors) to ensure a sample size balance between patients/family members/donors and health professionals. We will aim to recruit nephrologists and surgeons (minimum n = 150); nursing and allied health professionals (n = 50); and policy makers (including regulators), researchers, and representatives from industry (n = 50); who have experience, expertise, or interest in kidney transplant recipient outcomes.

Purposive sampling to obtain a maximum variation in the demographics, clinical characteristics (patients), and professional experiences and roles (health professionals), and snowballing strategies (where participants can nominate or extend an invitation to other relevant stakeholder members to participate) will be used. Patients/family members will be recruited through participating hospital/university institutions of the SONG Executive, SONG-Tx Steering Group and investigators, patient/consumer organisations, and the SONG Initiative database. Health professionals will be recruited via the collegial networks of the investigators and professional transplantation and nephrology societies.

Data Collection

Generating the List of Outcomes

The Delphi survey will include outcome domains from the systematic review of outcomes reported in RCTs (phase 1). To ensure that patient-important outcomes are considered, we will also identify outcomes from studies that involved patients in prioritizing or reporting outcomes in kidney transplantation. The key studies that will be used to directly inform the Delphi are outlined in the following.

A recent nominal group technique study conducted with 57 kidney transplant recipients who participated in 8 nominal groups in Australia to identify and rank outcomes relevant to immunosuppressant medications.[20] All outcomes identified will be considered for inclusion in the Delphi. The top 10 ranked outcomes overall were: kidney rejection leading to graft loss, kidney function, damage to other organs, cancer (all), diabetes, skin cancer, cardiovascular disease, infection, weight gain and excessive appetite, and in addition the top 5 psychosocial or physical symptoms were: impact on family, depression, impact on work, gastrointestinal problem, and concentration and memory.[20] Potential outcomes for the Delphi survey will also be extrapolated from a systematic review of qualitative studies on the motivations, challenges, and attitudes to self-management in kidney transplant recipient (eg, graft function, graft rejection, alopecia, pain, anxiety, cardiovascular disease, diabetes, cancer, mouth ulcers, acne, cataracts, diarrhoea, physical appearance).[57]

A systematic review of patient-reported QoL outcomes in trials of immunosuppressive agents in kidney transplantations identified 2 kidney transplant-specific instruments (Kidney Transplant Questionnaire [physical symptoms, fatigue, uncertainty/fear, appearance, emotional status],[58] and a visual analogue scale to assess impact of disease) and 21 symptom-specific instruments (eg, depression, gastrointestinal problems, physical appearance, physical well-being).[21] Attributes from these will be used to inform the development of the list of outcomes for the Delphi.

All outcomes will include a plain language definition. The survey will be reviewed by the SONG Executive and SONG-Tx Steering Group, which includes kidney transplant recipients, and piloted with at least 10 kidney transplant recipients.

Survey Administration

All participants must register their name and email address via www.songinitiative.org by March 2016 to receive a standard study information sheet. Informed consent will be obtained from all participants. The surveys will be completed online via LimeSurvey. Each participant will be given a unique identifier so their responses from rounds 1 to 3 can be linked anonymously. At least 3 reminders will be sent to the participants during the Delphi Rounds in an attempt to retain at least a 70% response rate across all 3 rounds. Participants who complete all 3 rounds will receive a copy of the preliminary results to provide feedback and comment.

Delphi Survey Round 1

Participants will be asked to rate the importance of outcome domain for research in kidney transplantation (approximately 35) using the GRADE 9-point Likert scale.[59] The visual scale used for each outcome will indicate ratings 1 to 3 as “limited importance”; “4 to 6 as “important, but not critical”; and 7 to 9 as “critical importance.” An option “unsure” will also be provided. Responses to the rating questions will be mandatory. Participants can provide comments in a free-text box under each outcome. The sequence of outcomes shown will be randomized to minimize ordering bias. At the end of the round, participants can suggest new outcomes. All new outcomes that are suggested by more than 10% of participants and that do not overlap or duplicate already presented outcomes will be recoded by at least 2 investigators (N.E., B.S.) and reviewed by the SONG-Tx Steering Group, then carried through round 2.

Depending on distribution of scores across all outcomes, an outcome with a median and mean of more than 7 (with ≥ 70% of participants in both patient/family member/donor and health professionals rating the outcome 7-9; based on the OMERACT criteria for consensus in) will be retained in round 2. Any outcomes that were not retained in subsequent rounds will still be considered in the research agenda (Figure 1).

Delphi Survey Round 2:

In round 2, participants will be presented with a column graph of the distribution of scores for each outcome for the following groups: (1) patients/family members/donors, (2) health professionals, and (3) all participants (with scores weighted evenly between groups). An explanation of how to read the graph will be provided to ensure that participants can understand and interpret the graph clearly. They will also see comments from round 1 by patients/family members/donors and health professionals. In the rating scale, their own response from the previous round will be highlighted. Participants will rerate each outcome and any additional outcomes identified in round 1 using the same GRADE 9-point Likert scale. A free-text box will be provided for participants to explain reasons for their rating or to provide responses to the comments.

An outcome with a median and mean of more than 7 (with ≥ 70% of participants in both patient/family member/donor and health professionals rating the outcome 7-9) will be included in round 3.

Delphi Survey Round 3

In this final round, participants will see the distribution of scores for each outcome for all participants and by stakeholder groups and comments from round 2. Participants will see their own score from round 2 highlighted in the rating scale and rerate all outcomes. A free-text box will be provided for additional comments. In the final section of the survey, participants will be asked to complete a forced ranking exercise. They will be presented with a list of all outcomes to arrange into a list in order of importance (top being most important).

Data Analysis

For all 3 rounds, we will summarize the distribution of scores and calculate the mean, median, and proportion for ratings and rankings of each outcome. According to the OMERACT prespecified definition of “consensus” for outcomes to be included in the core set, the outcome must have at least 70% participants scoring 7 to 9 and less than 15% of participants scoring as 1 to 3.[48] Based on previous initiative, a core set includes 3 to 5 outcomes for feasibility. However, it is possible that more than 5 outcomes will meet the OMERACT threshold for inclusion. Therefore, the identification of preliminary core outcome domains will also be based on means, medians, and proportions in round 3, which will be validated against the OMERACT cutoffs based on proportions. Because it may be necessary to define consensus post hoc to some extent, the preliminary outcomes with the rationale and threshold for inclusion will be detailed in a plain language report. This will be discussed at the consensus workshop (phase 3).

Phase 3: Consensus Workshop

A consensus workshop will be convened with stakeholders to comment and critique the identification of core outcomes (including the potential set of core outcomes) and to discuss strategies for the development of outcome measures and implementation. A member of the SONG-Tx Steering Group will chair the session. We aim to have a minimum of 60 attendees, with at least 20 patients/family members. Health professionals with a range of clinical experience in kidney transplantation (nephrologists, surgeons, nursing, and allied health professionals), expertise in research (epidemiology, clinical trials in kidney transplantation, registries, quality improvement), and leadership or advisory roles in major research and policy organizations (including regulators) and industry will be invited to attend.

Before the workshop, we will send participants a copy of the results from phase 1 and phase 2, so they can reflect on the results to date and be better prepared to contribute their opinions during the workshop. The workshop will include 3 sessions:

Session 1: Introduction

We will provide a brief introduction to the SONG-Tx initiative and present the details of the SONG-Tx process and results from phase 1 and phase 2 and the preliminary core outcome set and proposed threshold for inclusion.

Session 2: Breakout groups

Participants will be allocated to 1 of 5 breakout groups with up to 12 participants in each group (including a facilitator and cofacilitator). Mixed stakeholder groups with at least 2 patients/family members will be convened to encourage a richer exchange of ideas, explanations of similar or different opinions, and breadth of discussion. A trained facilitator will ask participants to discuss the identification and implementation of core outcomes, and ensure cooperative, respectful and inclusive discussion. All facilitators will attend a briefing session and will be provided with a question guide.

Session 3: Plenary discussion

The groups will reconvene to engage in a broader discussion moderated by the workshop chair. A member from each breakout group will present a brief summary of their discussion. The wide group will be invited to give their opinions, reflections on the issues raised by other groups. The moderator will summarize key similarities and differences in the points raised across groups.

Finalization of Core Outcome Domains

The SONG-Tx process (phase 1 to phase 3) and proposed outcomes will be published in a plain language report for circulation to the participants in the Delphi (phase 2) and consensus workshop (phase 3), circulated to stakeholder groups and made available on the website for 3 weeks to obtain public comment. All feedbacks will be reviewed by the SONG-Tx Steering Group and SONG Executive Committee to finalize the SONG-Tx core outcome set.

DISCUSSION

The SONG-Tx project uses a validated and transparent process that enables broad stakeholder engagement to develop a consensus-based prioritized set of core outcome domains for trials in kidney transplantation that can also be applied to other research contexts including registry studies and quality measurement activities. For each of the core outcome domains, we will identify or develop core outcome measures using the OMERACT filter to ensure that they are valid, discriminative, and feasible.[60] We will develop and apply implementation strategies in consultation with key stakeholders so that the core outcomes will be progressively translated into all trials and research.

Consistent and complete reporting of outcomes that are highly important to all stakeholder groups is likely to improve the quality and relevance of the research evidence to better support patients and clinicians in making informed treatment decisions in kidney transplantation. The core outcome set may also serve as a catalyst in driving the research agenda to focus on outcomes that really matter to patients and health care providers.