Development and evaluation of a decision aid for family surrogate decision-makers for patients with acute kidney injury requiring renal replacement therapy (RRT) in ICUs: a study protocol.

INTRODUCTION: Shared decision making is endorsed by guidelines for both acute kidney injury and critical care medicine. However, there is still a huge need for effective interventions, especially those focusing on decisions about renal replacement therapy for intensive care unit (ICU) patients with acute kidney injury. The decision aids provide evidence-based support for shared decision making, to achieve better decisions through enhanced knowledge of treatment options and treatment aligns with patients' preferences and values. Therefore, our objectives are to develop and evaluate a decision aid systematically and rigorously for family surrogate decision makers of ICU patients with acute kidney injury who need renal replacement therapy. METHODS AND ANALYSIS: We will use a systematic development process that focuses on user-centred design to develop and evaluate the decision aid in three phases: (1) development of a draft prototype for the decision aid based on extensive literature reviews, interviews with key stakeholders and evidence synthesis; (2) alpha testing ('near live' usability) the decision aid during simulated clinical encounters to test its comprehensibility, acceptability and usability and (3) beta testing ('live' usability) to examine the aid's clinical feasibility. User testing will be conducted using mixed-methods approach to support iterative revision of the decision aid. The IPDASi (V.4.0) will be used for following qualitative assessment. All interviews will be analysed by Colaizzi's seven-step approach to qualitative analysis. The coding scheme will use to analyse user interactions. Questionnaire surveys will be analysed using paired sample t-tests when related to the before-and-after survey, otherwise using one-sample t-test. ETHICS AND DISSEMINATION: Ethical approval for this research was obtained from the Ethics Committee of the First Affiliated Hospital of Army Medical University, PLA (Ref: KY2020104). All participants will sign a formal informed consent form. The findings will be published in peer-reviewed journals and reported in appropriate meetings. TRIAL REGISTRATION NUMBER: ChiCTR2000031613. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The present study uses a systematic, rigorous protocol to develop and evaluate a decision aid for intensive care unit patients with acute kidney injury who are considering renal replacement therapy.

This study will provide evidence for integrating a user-centred design into the development of decision aids in clinical contexts.

This study will also include usability testing to provide evidence for the acceptability and feasibility of the decision aid.

This study will recruit participants from only one hospital in China. This means that the findings may not be generalised to all of China, but they will provide evidence for the design of future nationwide studies.

Introduction

Acute kidney injury (AKI) is one of the most common complications in the intensive care unit (ICU) and is associated with high morbidity and mortality.1 According to recent research in China, 51% of critically ill patients develop AKI, which increases mortality risk sevenfold as well as increasing the length of hospital stays and healthcare costs.2 Today, renal replacement therapy (RRT) is becoming a key treatment for ICU patients with AKI.3 4 RRT can be life-saving for critically ill patients with AKI.5–7 However, one study comparing patients who did and did not receive RRT found that patients with AKI who received RRT had double the risk of 28-day mortality, which accounted for 46.3%, and three times the overall ICU costs.2 Therefore, for critically ill patients with AKI, attending physicians must decide whether to initiate RRT based on the individual patient’s preferences and values. The decision to administer RRT should be based on patient preferences due to the lack of ideal clinical outcomes, regardless of treatment, for such cases.8 9 A recent study found that 60% of patients opted for RRT when clinicians recommend this treatment, while only 9% of patients received RRT when clinicians advise against this treatment.10 This is more driven by clinician and system-level factors rather than patients’ preferences and values.11 Clearly, decisions about whether to administer RRT to critically ill patient should be based on the needs and preferences of patients and their families.

Many critically ill patients are sedated or in a coma, so families must act as surrogates and make decisions for the patient. However, the suddenness and clinical uncertainty of life-sustaining treatments often make family members into decisional conflicts. These conflicts may burden family members with severe psychological syndromes such as anxiety, depression or post-traumatic stress disorder.12 13 Emotional disorders and an impaired ability to process information may lead to decision fatigue, which reduces one’s ability to make decisions.14 In addition, family members may misunderstand the prognosis of a treatment, which also affects the quality of treatment decision making.15–17 Allegretti et al18 report that over 80% of surrogates overestimate the positive prognosis of RRT, although clinicians and surrogates represent sufficiently communication and mostly understanding. Obviously, there are a range of risks and possible complications for critically ill patients with AKI,19–21 which makes it difficult for surrogates to comprehend the overall prognosis of RRT.22 23 Further intervention is needed to narrow this knowledge gap and help surrogates scientifically and efficiently understand the prognosis. More attention to surrogate engagement in decision making for ICU patients with AKI who require RRT is needed.

To help family surrogates and ICU clinicians agree on a treatment plan, verified decision aids (DAs) are needed to facilitate shared decision making (SDM) in the ICU. SDM refers to the interactions of patients or their families with healthcare professionals to weigh the pros and cons of treatment options. It highlights the importance of patients’ preferences and values for reasonably predicting patients’ best interests.24 International guidelines recommend that critical care specialists discuss treatment options with patients and/or their families as soon as possible when RRT is indicated for a patient with AKI.25 DAs are designed to inform patients and their families about treatment options, prognoses and possible decision outcomes. A DA can help patients and families evaluate different treatment options.26 A Cochrane systematic review shows that DAs can effectively improve the quality of decisions without increasing the anxiety or depression of decision makers or worsening patient or health system outcomes.27 Stacey et al26 find that DAs are associated with more knowledge of the options, decisions that better align with patients’ values, reduced conflict around decisions and higher patient participation in decisions.

Existing DAs used in the ICU context are mostly designed for surrogate decision makers who must make treatment choices for critically ill patients who are unable to participate in the decision-making process. Cox et al28 developed a web-based DA for surrogate decision-makers for patients receiving prolonged mechanical ventilation. This DA provides decision guidance but also uses an algorithm informed by surrogates’ responses to a patient values clarification exercise to indicate the preferred goal of treatment. Cox et al28 report that this DA reduced surrogate decisional conflicts during the decision-making process in the ICU. Suen et al29 also developed a family support tool to help families navigate the complexity of surrogate decision making in the ICU. This tool helps individuals understand the surrogate’s role and consider the patient’s preferences and values in treatment decisions. However, to the best of our knowledge, there is no DA designed to support decisions about the use of RRT to treat critical illness in the ICU context, although the effectiveness of other DAs for supporting families and encouraging SDM with clinicians has been verified.

In order to address this complex decisional dilemma, the present study aims to: (1) develop a DA that uses a systematic, rigorous process to support the family surrogate decision-makers for ICU patients with AKI who need RRT; (2) test the comprehensibility, acceptability and usability of the DA in a simulated decisional context (‘near-live’) and (3) verify the feasibility of the DA in ICU clinical practice (‘live’).

Methods

A systematic development process model30 and a user-centred design (UCD) will be used to develop the DA.31 This study will consist of three phases. The DA prototype will be developed in phase 1. User testing will be conducted in phases 2 and 3, and the DA will be revised based on these tests. Phase 2 will involve simulated alpha testing; beta testing in a clinical context will take place in phase 3. This study follows the IPDASi V.4.032 to ensure that the minimum quality criteria are met. The main elements of the development process are illustrated in figure 1.

Figure 1

Development process of the RRT decision aid for surrogate decision-makers in ICU. ICU, intensive care unit; RRT, renal replacement therapy.

A multidisciplinary steering committee will guide the development of the DA. A steering committee was created to ensure a systematic protocol. The steering committee consists of two experts in SDM, three professionals in critical care medicine and two patient representatives. The two SDM experts have many years of research experience in SDM and patient DAs. The critical care experts include an ICU director, a head nurse and a critical care specialist with expertise in RRT decision making and clinician and family consultation. The patient representatives are family surrogate volunteers who have experienced RRT decision making in the ICU. Steering committee meetings will be hold regularly to supervise the design, content and iterative revision of the DA.

Phase 1: development of the DA prototype

Scope: literature review

A comprehensive literature review will be conducted early to identify the purpose of DA development. First, an extensive literature review will be conducted to clarify the current state of RRT decision making and to identify the evidence for the need for a DA designed for use in the ICU. We will also conduct a competitive analysis by summarising existing relevant DA resources. This is an important step in the development of digital projects to avoid duplicating existing resources. This step will also help us determine whether the development project is compatible with our research protocol.

Design: needs assessment

Interviews with healthcare professionals

Alongside the literature review, a decisional needs assessment is critical for confirming potential DA use cases identified in the literature. Due to the complexity (as a kind of life-sustaining treatment) of RRT decision making in ICU, the decisional trigger always depends on the patients’ attending clinician. Therefore, semistructured interviews will be conducted with ICU clinicians to identify the main points that a DA might address and the opportunities to integrate the DA into a clinical workflow. We will recruit ICU clinicians who have recently (<6 months ago) taken part in clinician–family consultations addressing decisions around using RRT to treat patients with AKI. The interviews will last 30–60 min; the goal of the interviews is to elicit professional opinions on key aspects of the content, design and distribution of the DA. First, clinicians will be asked to indicate what information family surrogates need to know before taking part in these decisions. Second, they will be asked to outline the surrogate decision-making pathway to determine when and how in the process a DA would be appropriate. The individual interviews with ICU clinicians will be used to explore professional suggestions and reach consensus on healthcare professionals’ clinical needs for this DA.

All interviews will be recorded and transcribed verbatim. The interviewees’ perspectives will be written down by the interviewer; these notes will be analysed alongside the transcripts. Colaizzi’s seven-step approach to qualitative analysis will be used to code emerging categories and themes in the transcripts.33 First, all transcripts will be independently reviewed and coded by two researchers with expertise in qualitative research methods. Then, final coding will be determined via further comparison and discussion. When the two coders disagree on a code, a third qualitative research expert will arbitrate the coding decision. The computer software NVivo V.12 will be used for the analysis.

Interviews with surrogate decision-makers

UCD requires a deep understanding of the role and responsibility of the target users of the DA in development. To comply with this requirement, we will conduct semistructured interviews with family surrogates, the target users of the DA, who have taken part in RRT decision making for their loved ones. To obtain a full perspective on surrogate decision making around RRT, data from routine clinician–family encounters regarding RRT in the ICU are needed. Therefore, we will observe ICU clinician–family consultations before conducting the face-to-face interviews with surrogates to identify how a DA can meet their decisional needs. An opportunity statement exercise31 will be included in the individual semi-structured interviews to get a full understanding of surrogates’ perspectives on their RRT decisional needs. An opportunity statement exercise is a type of design thinking activity used to gather and explore users’ needs to support digital tool development. It is used to identify areas that the proposed digital tool may impact or add value to.31 Family surrogates will be asked to complete the following statement based on prepared description (summary outlines from literature review and healthcare professionals’ interviews, which describe what is a DA and what it does to help them): ‘How might we improve this DA so it can help you make the best treatment decisions based on your loved one’s preferences?’ An interview syllabus will be constructed to explore surrogates’ decisional needs around RRT and to identify factors in the current decision-making process. This will help us determine how a DA could make a measurable impact. Like the interviews with ICU clinicians, all of the audio recordings of surrogate interviews will be transcribed verbatim and qualitatively analysed along with the interviewers’ written notes.

Finally, the decisional needs of ICU clinicians and family surrogates will be translated into a general framework to be used in the next stage: detailed evidence synthesis.

Draft: working prototype

Evidence synthesis

The best evidence and other decisional information will be synthesised based on an assessment of the needs of ICU clinicians and surrogates. First, we will review current guidelines on the management of AKI and the administration of RRT to identify general treatment recommendations. Second, available evidence for treatment outcomes from the systematic literature review and meta-analysis will be used to identify information that can help users decide whether or not to choose RRT. This evidence synthesis will involve four steps: (1) identify key words; (2) systematically search for the key words; (3) screen and extract relevant evidence based on clinical needs (most included evidence will be related to treatment recommendations and prognoses) and (4) summarise the evidence into a DA script. We will search for evidence using PubMed, EMBASE, Cochrane Library, CNKI, CBM, UpToDate and the websites of clinical practice guidelines and academic associations and institutions. The precise search strategy used on PubMed is explained in detail in online supplemental file 1.

Drafting the prototype

Based on the literature review, the assessment of the needs of key stakeholders, and the best available evidence, a paper prototype of the DA will be drafted according to the criteria for developing DAs34 35 and assessed using the IPDASi V.4.0. The content of the DA will include five main topics: (1) brief instructions about the disease and treatment; (2) pros and cons of treatment options; (3) possible treatment outcomes; (4) patient stories and (5) a value clarification exercise. Once the paper prototype has been completed, the steering committee will audit it and determine which content will be included in the DA.

To ensure that the right information (regularly updated) is provided to the right person (tailored) at the right time (the appropriate point in the decision-making process),36 we will use an internet-based DA which enables content to be tailored to individual users.37 WeChat38 is the most popular social media mobile application in China. It combines instant messaging with text, image, voice and video chat options as well as payment and scan functions. More than 95% of Chinese adults have a mobile phone, and over one billion of them access WeChat at least once a day.39 WeChat is also commonly integrated with hospital services and used for making appointments with physicians and to provide health education.38 Therefore, we believe that an internet-based DA that is accessible on a mobile phone via WeChat will be a valuable tool for supporting decision making in the ICU context.

Finally, the paper prototype will be redrafted and redesigned into a working prototype, which will be an interactive web-based version that is accessible on a mobile phone via a Quick Response Code (QR code) that can be scanned using the WeChat app.

Phase 2: alpha testing

The aim of this phase is to examine the comprehensibility, acceptability and usability of the DA in a simulated decision-making scenario.30 This will improve our understanding of the interactions between DAs and target users during tasks. The participants will be family surrogates, and the test will require interaction with the DA using the think-aloud method. In this phase, the functionality and usability of the DA’s overall interface will be evaluated and problems with user interactions will be identified.

Participants

A purposive sample of surrogates not involved in the development process will be invited to participate in the alpha testing in phase 2. We will recruit family surrogate decision makers of patients with AKI from Southwest Hospital in Chongqing, China. The surrogate sample will include individuals who chose RRT for their loved one, those who considered but declined RRT, and those currently deliberating treatment options in the ICU. We expect a sample size of eight to 12 participants; this will be large enough to detect up to 80% of surface-level usability problems.40 However, the final sample size will be determined by data saturation, which will be guided by transcripts of users’ interactions with the DA. The alpha testing phase will be completed within 4 weeks.

Data collection

A researcher with expertise in health informatics and usability testing will guide each session; the sessions will be conducted in a quiet room. Data will be collected throughout participants’ use of the DA via questionnaires, recordings of the interactions and semistructured interviews. First, participants will be asked to complete a ‘before’ questionnaire, which will collect demographic information, self-perceived skill level with the internet and mobile phones, experiences with RRT decision making and knowledge41 of RRT. Second, participants will be presented with a scenario that simulates decision making about RRT for patients with AKI in the ICU. The simulations will closely match participants’ experiences. Next, participants will be given a card with a QR code that can be scanned using WeChat on a research phone (iPhone XR); this code will connect them to the DA. Participants will also receive a brief task description asking them to use the think-aloud method while engaging with the DA. In this method, participants orally describe their behaviours and thoughts while carrying out a task. Once the task is completed, participants will complete the ‘after’ questionnaire. This tool will include a knowledge survey similar to that in the ‘before’ questionnaire and an acceptability survey.42 Finally, participants will be interviewed about the experience. Interview questions will address participants’ satisfaction with the DA, pros and cons of the DA, and the quality of this decision support. The screen recording feature of the research phone will be used to record the entire user interaction (all screen activities and the user’s voice). Field notes will be taken as well to provide additional data for further analysis.

Data analysis

Due to the small sample size in this phase, data analysis will occur in conjunction with data collection. The data analysis will use a coding scheme developed by the usability engineering team from examinations of human–computer interactions and relevant literature on cognitive psychology.40 This scheme includes eleven categories of usability problems, such as navigation, graphics and layout. These categories, along with the think-aloud reports, can then be used to analyse user interactions with the DA. Additional codes that indicate other user problems generated from our testing results will be added to the scheme as needed.

The screen recordings will be watched and annotated independently by two researchers. Disagreements will be discussed to reach a consensus. The semistructured interviews will be analysed in the same way as the interviews with key stakeholders in phase 1. The knowledge surveys will be analysed using a paired t-test, and the acceptability surveys will be analysed using a one-sample t-test. SPSS Statistics V.26 will be used for the analyses.

Based on these findings, ways to improve the DA will be identified. These points will be reviewed by steering committee, which will decide which revisions to implement.

Phase 3: beta testing

Beta-testing will be conducted to test the feasibility of the DA in the ICU clinical context. The focus in this phase will be contextual factors which may influence the impact of the DA in real life. Without the developers’ control, the stage of acceptance testing will be more stable. In this phase, a mixed-methods approach will be used to conduct a before/after study in RRT decision-making procedures. The objectives of beta testing are to assess the impact of the DA on family surrogate engagement in RRT decision making and on the quality of surrogate decision making.

Family surrogate decision-makers and ICU clinicians will be recruited from the ICU clinical workflow. Participants will be recruited from the same hospital as in the alpha testing phase. We will recruit participants based on patients. If an AKI patient with indications for RRT is identified, we will first recruit clinicians to ensure that we can observe the family–clinician consultation. Then, we will invite the family surrogate to participate in the study. Surrogates will be defined as the person responsible for providing written informed consent for the patients’ medical decisions. Included clinicians will be limited to those with at least 3 years of ICU clinical experience who are responsible for guiding the decision-making process. Clinicians who are reluctant to engage in SDM with patients’ families will be excluded. Family surrogates who are not able to use WeChat on a smart phone will be excluded as well. If either the clinician or the surrogate is ineligible to participate in the study, the patient will be excluded.

A consecutive sample of 15–20 family–clinician dyads will be invited to participate. This sample size is in line with previous studies identifying the minimum sample size for calculating inferential statistics in usability testing.40 We propose recruiting as many participants as possible to generate sufficient feedback for further refinement of the DA.

Once a family–clinician dyad has been enrolled in the study, the surrogates will be introduced to the DA before the family’s decision-making consultation with the clinician. Next, the family surrogates will be invited to a formal consultation with the clinicians to discuss their loved one’s treatment. All eligible surrogates must complete the questionnaire before and after using the DA. The questionnaire will collect demographic information; it will also include the surrogate decision-making self-efficacy scale,43 the decision-making preparedness scale44 and the decisional conflict scale.45 After the consultation, family surrogates and clinicians will be interviewed independently by different interviewers. In the interviews, participants will be asked to provide verbal feedback on the decision-making process. Family surrogates will be asked whether the DA facilitated their engagement in the decision and whether it met their decisional needs. Clinicians will be asked to share their experiences and to evaluate the quality of surrogate decision making. Outlines for the interviews with surrogates and clinicians will be prepared in advance and approved by the steering committee. All interviews will be recorded and last 30–45 min. All sessions will take place in a quiet room.

Descriptive analyses will be conducted for the baseline data. A paired sample t-test will be used to assess self-efficacy, preparation and decisional conflict. CIs will be set at 95%; the significance level will be set at 5%. All interviews will be transcribed and coded using NVivo V.12.0. The interview data will then be analysed as in phase 1. Based on these findings, we will refine the DA if necessary. Finally, the steering committee will confirm the completion of the final version of the DA. The next stage will be a formal clinical trial.

Patient and public involvement

Two family surrogate decision makers were recruited as patient representatives in the steering committee. These two patient representatives will describe their decision-making stories. It is expected that these stories will help participants clarify their loved ones’ preferences. These patient representatives will engage with the entire development process and the content and design of the DA. We expect that the DA will be user-centred. The patient representatives on the steering committee will not be involved in participant recruitment.

Ethics and dissemination

Research ethics approval has been obtained from the Ethics Committee of the First Affiliated Hospital of Army Medical University, PLA (Ref: KY2020104). A statement providing information about the study will be given to each participant, and written informed consent will be obtained from all participants. Participants will have the right to withdraw from the study at any time. All significant original data collected in the study will be considered for publication in peer-reviewed journals.

Discussion

This study initiates the rigorous development and evaluation of a DA for family surrogate decision makers for critically ill patients with AKI who need RRT. We expect that the DA will improve SDM and the quality of decisions about RRT in the ICU. Over the past two decades, SDM has been introduced in China, but now public attention bounded with the paternal model in medical decision making has hardly changed. Currently, there are more theoretical studies than empirical research of clinical practice in this area. Decision making is still dominated by ICU clinicians,46 47and patients’ and family surrogates’ attitudes towards and engagement with medical decision making in China need to be explored. The State Council issued an outline for the Healthy China 2030 initiative,48 which indicates that patients’ sense of gain and security in medical care should be increased by the optimisation of treatment procedure. This initiative also proposes strengthening the human aspect of medical care by encouraging supportive doctor–patient relationships. The paternal model of medical decision making can no longer adapt to modern medical services in China today, and SDM is needed to achieve the best treatment decisions for both patients and healthcare providers.

In any case, globally, SDM is rarely implemented in RRT decision making in ICUs. A decision-making process is shared if both clinicians and patients/surrogates engage in SDM behaviour, if patients/surrogates feel that they are participants in the decision-making process, and if clinicians and patients/surrogates share their views and ideas to reach an agreement.49 However, available data show the discordance of RRT decision making between clinicians and surrogates exists.10 Furthermore, surrogates are usually unable to reasonably estimate the prognoses of their loved ones, even with adequate communication and education.18

User-centred Information Technology health tools can help users deal with complicated processes and inaccessible data.31 The goal of this DA, which will involve target users’ participation in the development stage to ensure that the final product is user centred, is to increase surrogates’ ability to engage in SDM by helping them participate more actively in clinician–family consultations. Indeed, we believe that this DA will improve surrogates’ understanding of AKI and relevant treatment options, of the risks and benefits of treatment, of possible decisional outcomes and of the importance of the patient’s preferences and values. We also expect that this DA will improve surrogate engagement in decision making by reducing decisional stress and other negative emotions.

Since most critically ill patients are incapacitated, decision making in the ICU is complex. Therefore, decision supports tailored to the unique needs of family surrogate decision makers are needed.50–53 For decisions around RRT in the ICU, key points include improving communication and support for surrogates and discussing or clarifying the preferences and values of critically ill patients.54–56 Although our plan will not develop a DA tailored to both surrogates and their loved ones, we expect that our endeavours will provide a reference for future efforts to appropriately involve incapacitated patients in treatment decision making. Of course, for this to happen, governments and policymakers must pay more attention to strategies for increasing SDM in ICU care.