Adapting a Traumatic Brain Injury Goals-of-Care Decision Aid for Critically Ill Patients to Intracerebral Hemorrhage and Hemispheric Acute Ischemic Stroke.

OBJECTIVES: Families in the neurologic ICU urgently request goals-of-care decision support and shared decision-making tools. We recently developed a goals-of-care decision aid for surrogates of critically ill traumatic brain injury patients using a systematic development process adherent to the International Patient Decision Aid Standards. To widen its applicability, we adapted this decision aid to critically ill patients with intracerebral hemorrhage and large hemispheric acute ischemic stroke.
DESIGN: Prospective observational study.
SETTING: Two academic neurologic ICUs.
SUBJECTS: Twenty family members of patients in the neurologic ICU were recruited from July 2018 to October 2018.
INTERVENTIONS: None.
MEASUREMENTS AND MAIN RESULTS: We reviewed the existing critically ill traumatic brain injury patients decision aid for content and changed: 1) the essential background information, 2) disease-specific terminology to "hemorrhagic stroke" and "ischemic stroke", and 3) disease-specific prognosis tailored to individual patients. We conducted acceptability and usability testing using validated scales. All three decision aids contain information from validated, disease-specific outcome prediction models, as recommended by international decision aid standards, including careful emphasis on their uncertainty. We replaced the individualizable icon arrays graphically depicting probabilities of a traumatic brain injury patient's prognosis with icon arrays visualizing intracerebral hemorrhage and hemispheric acute ischemic stroke prognostic probabilities using high-quality disease-specific data. We selected the Intracerebral Hemorrhage Score with validated 12-month outcomes, and for hemispheric acute ischemic stroke, the 12-month outcomes from landmark hemicraniectomy trials. Twenty family members participated in acceptability and usability testing (n = 11 for the intracerebral hemorrhage decision aid; n = 9 for the acute ischemic stroke decision aid). Median usage time was 22 minutes (interquartile range, 16-26 min). Usability was excellent (median System Usability Scale = 84/100 [interquartile range, 61-93; with > 68 indicating good usability]); 89% of participants graded the decision aid content as good or excellent, and greater than or equal to 90% rated it favorably for information amount, balance, and comprehensibility.
CONCLUSIONS: We successfully adapted goals-of-care decision aids for use in surrogates of critically ill patients with intracerebral hemorrhage and hemispheric acute ischemic stroke and found excellent usability and acceptability. A feasibility trial using these decision aids is currently ongoing to further validate their acceptability and test their feasibility for use in busy neurologic ICUs.

Shared decision-making is a collaborative approach that encourages clinicians, patients, and surrogates to work together to make treatment decisions based on the best available scientific evidence, while also considering the patient’s values, goals, and preferences (1). The Institute of Medicine and major critical care societies have strongly recommended that clinicians incorporate shared decision-making in the ICU (1, 2). Although shared decision-making has been successfully implemented for many health decisions (3), empiric research in medical-surgical ICUs has shown that only 2% of ICU clinician-family communications meet all recommended criteria for shared decision-making (4). Our previous research in surrogates and physicians in the neurologic ICU (neuroICU) has identified an urgent but unmet need for goals-of-care decision support and shared decision-making tools (decision aids [DAs]) (5). The sudden and unexpected nature of catastrophic neurologic diseases adds greatly to the surrogates’ burden of making the morally difficult life-or-death goals-of-care decision on behalf of these critically ill and incapacitated patients. This large emotional burden on families, combined with existing deficiencies in clinician-family communication in the ICU, results in poorly informed treatment decisions, high rates of psychologic distress in families, and high utilization of expensive, burdensome treatments which may be misaligned with patients’ values and preferences (4, 6, 7).

To narrow this gap, we developed a paper-based pilot DA for surrogates of critically ill traumatic brain injury (ciTBI) patients making goals-of-care decisions using a rigorous and systematic approach (8). Importantly, this DA meets International Patient DA Quality Standards (8, 9), which is an international guide on the content of high-quality DAs. During the DA’s development process, we received input from stakeholders, both surrogates and physicians, that goals-of-care DAs for other neurologic emergencies are urgently needed.

Intracerebral hemorrhage (ICH) and large hemispheric acute ischemic stroke (AIS) are the fifth leading cause of death in the United States (10, 11), and most of the deaths related to both of these diseases are due to withdrawal of life-sustaining treatments in ICUs (12). Currently, no DAs exist for surrogates of critically ill ICH and AIS patients. One other DA for strokes of all severities has recently been developed, which offers a web-based tool for both noncritically ill and critically ill patients making life-sustaining treatment decisions (13). Our DA is different from this tool in several ways: our tool purposefully includes 10 real photographs supplementing text, depicts the estimated prognosis in three levels (rather than 5) using icon arrays (rather than pie charts), all with the intention to lower the cognitive burden on stressed surrogates of critically ill patients (8, 13).

The objective of our current study was to adapt our existing goals-of-care DA for surrogates of ciTBI patients to critically ill patients with ICH and large hemispheric AIS, measure usage time, usability and acceptability, and obtain qualitative feedback. These important steps are standard and necessary in the DA development process prior to deploying the DAs in the neuroICU for feasibility.

MATERIALS AND METHODS

This study was conducted at two academic medical centers: the University of Massachusetts Medical School/UMass Memorial Medical Center and Yale New Haven Hospital. The University of Massachusetts Institutional Review Board (IRB protocol number H00015764) and the Yale University Institutional Review Board (IRB protocol number 2000023655) granted approval for this study and allowed participants to be enrolled through verbal consent due to the minimal risk posed by the study.

Goals-of-Care DA for Surrogates of ciTBI Patients

The rigorous, stepwise development process for our goals-of-care DA for surrogates of ciTBI patients, which meets International Patient DA Quality Standards criteria for high-quality DAs (9), has been previously published (8). In brief, this traumatic brain injury (TBI) goals-of-care DA is paper based, written at a sixth grade reading level, printed on 12 double-sided pages in Times New Roman font in letter size 16, and contains 10 photographs and several tables. The ciTBI DA includes the following information: TBI terminology and essential background information; the nature of the goals-of-care decision; treatment options; potential post-ICU discharge dispositions or withdrawal of life-sustaining treatments with comfort-based care; an individualized icon array displaying each patient’s unique estimated prognosis derived from the validated TBI International Mission for Prognosis and Analysis of Clinical Trials model, accompanied by text carefully explaining and emphasizing prognostic uncertainty associated with prediction models; two patient examples; and an one-page worksheet designed to prepare surrogates for the family meeting (including a values clarification exercise, decision-readiness assessment, and a tear-away page to record questions) (Supplemental Fig. 1, Supplemental Digital Content 1, http://links.lww.com/CCX/A517) (8).

Adaptation of ciTBI DA to ICH and AIS

To adapt our existing goals-of-care DA for patients with ciTBI to those with an ICH and hemispheric AIS, we first changed the essential background information and disease-specific terminology about TBI to ICH and AIS, using terminology of “hemorrhagic stroke” and “ischemic stroke.” We anticipated that additional changes to the terminology would be required based on family user feedback, which we were planning to obtain as part of the adaptation process.

Per the International Patient DA Quality Standards, all DAs should include data derived from validated outcome prediction models that carefully visualize potential outcomes and describe uncertainties (9). In accordance with these standards, and based on our prior qualitative work with neuroICU surrogates showing that estimated prognosis was a strongly desired component (14), we chose to include an individualized prognosis estimated by a validated prediction model, similar to the ciTBI DA. Of note, we also chose to provide accompanying text to clearly explain and emphasize prognostic uncertainty (Supplemental Fig. 2, Supplemental Digital Content 2, http://links.lww.com/CCX/A518), due to our awareness of the potential pitfalls of predicting an individual patient’s prognosis using prediction models derived from broad populations (8). In all of our DAs, explicit text informs surrogates that the prediction model does not take into account their loved one’s medical history, injuries to other organs, clinical course, and complications in the ICU and that they should ask the doctor how these factors may change their loved one’s prognosis.

Two board-certified neurointensivists (D.Y.H., S.M.) reviewed the current literature and carefully weighed advantages and disadvantages of existing models, while also considering patients’ and families’ desire for information on longer term outcomes. After considerable discussion, for the ICH model, we selected the previously validated 12-month functional outcomes associated with the original ICH score, given the ICH score’s widespread use and the value of its 12-month follow-up data (15). We contacted the senior author who shared the raw and deidentified 12-month functional outcome data using the modified Rankin Score for each ICH score for the purpose of our DA adaptation. With similar reasoning, after reviewing 23 different AIS prognostication models (16), we selected the most robust, high-level data, which stem from the large hemicraniectomy trials. We selected the 12-month outcome data available from the published hemicraniectomy trials Decompressive Craniectomy In Malignant Middle Cerebral Artery Infarcts, Decompressive Surgery for the Treatment of Malignant Infarction of the Middle Cerebral Artery (DESTINY), Hemicraniectomy After Middle Cerebral Artery Infarction With Life-Threatening Edema Trial (17), and DESTINY II (18).

We created icon arrays that can be individualized to the patient using the patient’s clinical variables. For ICH, we created six different icon arrays based on the ICH score ranging from 0 to 5. Of note, we did not create an icon array for the most severe ICH score of 6, since no patient received this score in the 12-month validation study (15), likely due to the fact that an ICH score of 6 is both rare and severe, such that patients did not survive to 12 months post ICH. It is also highly unlikely that a DA would be used for a patient with an ICH score of 6 due to the severity of the disease and nearly certain ultimate outcome. For hemispheric AIS, we created four different icon arrays, differentiating between patients less than or equal to 60 years or greater than 60 years, as well as those who did or did not undergo hemicraniectomy. For both ICH and hemispheric AIS, we dichotomized favorable and unfavorable modified Rankin Scale outcomes as less than or equal to 4 and greater than 4.

Acceptability and Usability Testing

Participants.

Participants were recruited from both neuroICU waiting rooms using the following inclusion criteria: age greater than or equal to 18 years, family member or friend of a patient admitted to the neuroICU with any disease, willing to read the DA with a study staff member present; English-speaking, and willing to spend approximately 45 minutes with study staff to provide verbal feedback and undergo usability and acceptability testing using validated scales (19, 20). We used a purposive sampling strategy to recruit surrogates who were either actively engaged in the process of supporting their family member in the neuroICU environment, including decision-making for their family member, but did not require surrogates to specifically be making decisions for TBI or stroke patients, given that the focus of feedback was on ease of navigation and use.

Trained members of the study team approached and recruited family members or friends of patients admitted to the neuroICU with a neurologic illness using an IRB-approved verbal consent script and printed fact sheet. After verbal consent was obtained, a member of the study team brought the participant to a private room near the neuroICU waiting room. The participant was provided with a hypothetical scenario about a patient with an ICH or hemispheric AIS awaiting a goals-of-care decision. Participants received the DA as a spiral-bound, color-printed booklet for either ICH or hemispheric AIS on an alternating schedule. The only instructions provided by the study team member were when to start reading the DA and to also complete the worksheet at the end of the DA.

Covariates Collected.

The study team member observed and timed the participant while they read through the DA, noting any signs of struggle or confusion, either nonverbal or verbal. We collected data on the age, gender, and study center of the participants.

Assessment Scales.

Following the reading of the DA and worksheet, the participant completed a short, written questionnaire assessing usability and acceptability of the DA using validated scales. We used the industry standard for assessing usability, the System Usability Scale (SUS), which is a 10-item, five-point Likert-scale with scores ranging from 0 to 100 (19). A SUS greater than or equal to 68 is an accepted cut-off for “good usability” (19). Our target SUS for “excellent usability” was greater than 80 (21). The Acceptability Scale is comprised of an nine-item, four-point Likert-scale that assesses DA content and seven additional questions assessing acceptability of length, information amount, balance of information, usefulness of the DA, understanding of prognosis, the personal worksheet, and completeness of information for decision-making. Two open-ended questions ask what participants liked about the DA and worksheet and any suggestions to improve both items (20). As in our previously published DA derivation study, we conservatively targeted greater than 70% for “good acceptability,” as our review of the literature did not reveal a validated or accepted cut-off for what constituted “good” acceptability (8). Both scales are validated and considered the industry’s standard in DA research. Finally, a short verbal feedback session was elicited by a member of the study staff.

Statistical Analysis.

Standard descriptive statistics were applied to calculate median usage times, SUS and Acceptability Scale scores, as well as proportions for response options. For the Acceptability Scale, we calculated the composite proportions for each Likert-scale point (“excellent,” “good,” “fair,” or “poor”) for the nine content questions. For each of the seven additional acceptability questions, we dichotomized the responses as “favorable” or “unfavorable.” We qualitatively analyzed the verbal feedback and grouped responses by positive comments about the DAs and suggestions for improvement. We recruited a purposive sample of 20 participants. This number was based on what was pragmatically feasible and on studies that show that 80% of usability problems can be identified with as few as 10 participants, whereas 95% of usability problems can be identified with 20 participants (22).

RESULTS

Over a 4-month period (July 2018 to October 2018), 20 participants completed acceptability and usability at both centers; 11 participants received the ICH DA, and nine received the hemispheric AIS DA. Of the participants, 15 (75%) were female, and the mean participant age was 55 (± 15) years. Median time to read the DA was 22 minutes (interquartile range [IQR], 16–26 min). Participants rated the usability of both DAs as “excellent” with a median SUS score of 84 of 100 (IQR, 61–93) with a SUS greater than 68 indicating good usability (19) (Fig. ). Most participants (89%) rated the acceptability of the DAs’ content as good (49.2%) or excellent (40.1%; multicolor bar in Fig. 1). Participants rated all other items on the Acceptability Scale favorably (length [85%], information amount [80%], balance [90%], usefulness [95%], prognosis [85%], worksheet [95%], and enough information to make a decision [85%]) (light blue bars in Fig. 1). The individual responses of the Acceptability Scale are provided in Supplementary Table 1 (Supplemental Digital Content 3, http://links.lww.com/CCX/A519).

Figure 1.

Acceptability and usability testing. Twenty participants completed usability and acceptability testing at UMass and Yale. Participants were recruited from the neuroICU waiting rooms at each center and completed testing using validated scales (19, 20). The multicolor bar represents a composite of nine questions related to the acceptability of the content of the decision aids. Eighty-nine percent of participants rated the content as good or excellent (multicolor bar). The light blue bars represent favorable responses related to the acceptability of length (85%), information amount (80%), balance (90%), usefulness (95%), prognosis (85%), worksheet (95%), and enough information to make decision (85%). The individual responses for the Acceptability Scale can be found in Supplementary Table 1 (Supplemental Digital Content 3, http://links.lww.com/CCX/A519). The dark blue bar represents the System Usability Scale (SUS), the industry standard in measuring usability of a tool (19). Participants rated usability as excellent (median SUS = 84/100), where a SUS greater than 68 is considered good (23).

Verbal feedback from participants was largely positive and acknowledged that both DAs would be very helpful for families making goals-of-care decisions on behalf of their loved ones (Table ). Several suggestions to improve the DAs were made, including using more lay terms for “intracerebral hemorrhage” and “acute ischemic stroke.” Based on this feedback, we changed the disease names in the titles of DAs to “Bleeding Stroke” and “Stroke,” respectively.

DISCUSSION

We adapted a novel goals-of-care DA for surrogates of ciTBI patients for use in surrogates of critically ill patients with ICH and large hemispheric AIS by incorporating validated, disease-specific, 12-month outcome prediction data to communicate probabilities of long-term outcome, including the uncertainties pertaining to the outcome. The DAs meet international DA standards and were rated with very high acceptability and usability by surrogates.

To our knowledge, these DAs are the first goals-of-care DAs specifically for critically ill patients with ICH and large hemispheric AIS. There is only one other DA that has been developed for patients who have developed a stroke and their surrogates (13). However, our DAs differ from this recently published DA in several ways: first, our DAs specifically target critically ill patients, rather than serving surrogates of stroke patients of all severities; second, our DAs contain 12-month validated outcomes. Surrogate decision-makers highly value long-term validated outcomes, which the authors of the other DA acknowledged as a limitation in their DA (13). The other DA uses an ordinal prognostication model predicting 90-day functional outcome, which the authors self-developed and derived from data from the Virtual International Stroke Trials Archive Plus (13). Third, our DA uses icon arrays to depict prognosis, in contrast to pie chart and horizontal stacked bars to allow portrayal of 5 ordinal levels of the modified Rankin Score, which could be overwhelming in users with lower numeracy skills; we have based our choice of icon arrays on research that has shown that icon arrays are best for risk communication (24, 25) and on direct feedback from surrogates (8). For our DA, we conducted acceptability testing in addition to the standard usability testing. Last, in contrast to the other web-based DA, our tool is paper based with many photographs, which may facilitate use among computer-illiterate surrogates (13).

One other medical/surgical ICU DA is available for patients with prolonged mechanical ventilation without a neurologic diagnosis, but a recent large clinical trial of this DA produced neutral results for the primary outcome of surrogate-physician prognostic concordance and psychologic distress in families (26). Our DA is different from this DA, because it is aimed at surrogates of neurocritically ill patients with TBI, ICH, and large hemispheric AIS, who face much more complex decision-making related to “both” the neurologic prognosis and the need for ventilation and artificial nutrition, and it includes a worksheet with values clarification, which surrogates complete. The goal is for clinicians to use this worksheet to guide the goals-of-care discussion via a “facilitated deliberation” (27–30). We based this approach on expert recommendations (31), and results from two clinical trials suggested that a knowledge approach alone will not influence emotionally laden life-support decisions, which are made intuitively using a “gut feeling” (26, 27, 32). However, our approach has not been tested in a clinical trial.

We understand that there is great risk in applying prediction models to individual patients. A gap analysis exploring prognostication in neurocritical care noted that current prognostic models are useful but limited (16). Some of the major gaps included lack of individualized patient measures, such as failure to account for medical comorbidities, and not including in-hospital events and clinical complications (16). Another study comparing formal prediction models with clinician judgment when predicting 3-month functional outcome (modified Rankin scale) in ICH patients found that clinicians judgment correlated more closely with actual patient outcome at 3 months (33). However, we included icon arrays with data derived from validated prediction models, because per the International Patient DA Quality Standards, high-quality DAs should present probabilities of potential outcomes (9) and because surrogates expressed a strong desire for it in our prior qualitative research and physicians felt it could ground other physicians given the existing variability in prognostication (14, 34–36). We mitigated the risk both by using validated models of high quality and by explicitly acknowledging the uncertainty around the projected outcome in each DA and stating that each patient has other, individual factors that influence outcome, which are not contained in the model. We also advised surrogates to consult with their doctor and ask for “best case” and “worst case” scenarios.

Our study has a number of important strengths and limitations. Strengths include the systematic development process with acceptability and usability testing and feedback from family member and surrogate stakeholders from two different medical centers. Additionally, we ensured that our DAs meet international DA standards. Of note, there is considerable variability in DA research with regards to standards of acceptability and usability testing practices (37). Similar to our published ciTBI DA development process, we recruited general neuroICU surrogate decision-makers for our acceptability and usability testing convenience sample using a hypothetical scenario and did not strictly limit the sample to only families of patients with an ICH or large hemispheric AIS in the midst of a real-time goals-of-care decision. Use of hypothetical scenarios has been a successful, practical approach in a number of prior research studies of ICU surrogates tasked with making high-acuity decisions (38–40). The experiential realism of our hypothetical scenario was enhanced by the fact that we did recruit all subjects for acceptability and usability directly from a neuroICU environment. We acknowledge that using population-based risked estimates to gauge an individual patient’s prognosis is a limitation and that we cannot completely eliminate that small risk of self-fulfilling prophecy; however, research has shown that surrogates desire outcome prognostication (14), and no DA is meant to stand alone without a clinician-family discussion. Additional potential limitations include that we did not collect race, ethnicity, or education level in our acceptability and usability testing participants. Furthermore, we excluded non-English speakers since our DA has not yet been translated and validated in other languages.

CONCLUSIONS

We adapted our goals-of-care DA for surrogates of critically ill TBI patients to surrogates of critically ill patients with ICH and large hemispheric AIS. We created two new DAs, which meet international DA standards, include disease-specific icon arrays to visualize individualized patient outcome probabilities, and have been rated highly by surrogate decision-makers for usability and acceptability. We are currently testing feasibility of DA implementation in a multicenter pilot trial.

ACKNOWLEDGMENT

We thank Dr. Claude Hemphill (University of California, San Francisco) for providing us the de-identified raw data of the 12-month validation study of the Intracerebral Hemorrhage score.

TABLE 1.

Qualitative Feedback by Acceptability and Usability Testing Participants on the Intracerebral Hemorrhage and Large Hemispheric Acute Ischemic Stroke Decision Aids

Participants Found the Decision Aid Helpful	Participants Made These Suggestions for Improvement
“I think it was very well done. I think people would find it very helpful. I think it is easy to understand. It’s heart wrenching, but it’s helpful.”	[I] “wished [there were more] footnotes [explaining] glossary terms, like what AIS means.”
“With my father, I couldn’t imagine how difficult it would be if I had to make a decision. I think this tool will make a big difference. I think it was really well put together. I think it was all well done.”	“[I] wish [there was] more information about the consequences to the family member - for example, the financial and emotional costs associated with each option.”
“I think this will be a really great thing for the families to see. It would be helpful to start thinking about what they will do before the meeting. I personally liked the icons. I see these types of images a lot in my work. I think it was clear.”	“In a digital version, there could be videos of, for example, someone on a ventilator, and that at certain points in the aid, the digital system could ask the reader ‘Did you understand ___? Yes/ No’ before allowing them to move on.”
“It’s a great survey, easy to read (especially for a layperson experiencing shock), well laid-out and explained, pretty spot-on, and easy to navigate. The aid could really help someone in that position to make a tough decision.”

We show representative quotes of the qualitative feedback by participants grouped by positive comments about the decision aids and suggestions for improvement. These were derived by either the participants providing written feedback as part of the open-ended questions of the Acceptability Scale or research staff eliciting verbal qualitative feedback and taking notes of the responses.