A patient and community-centered approach selecting endpoints for a randomized trial of a novel advance care planning tool.

BACKGROUND: Despite a movement toward patient-centered outcomes, best practices on how to gather and refine patients' perspectives on research endpoints are limited. Advanced care planning (ACP) is inherently patient centered and would benefit from patient prioritization of endpoints for ACP-related tools and studies.
OBJECTIVE: This investigation sought to prioritize patient-centered endpoints for the content and evaluation of an ACP video being developed for patients undergoing major surgery. We also sought to highlight an approach using complementary engagement and research strategies to document priorities and preferences of patients and other stakeholders.
MATERIALS AND METHODS: Endpoints identified from a previously published environmental scan were operationalized following rating by a caregiver co-investigator, refinement by a patient co-investigator, review by a stakeholder committee, and validation by patients and family members. Finalized endpoints were taken to a state fair where members of the public who indicated that they or a loved one had undergone major surgery prioritized their most relevant endpoints and provided comments.
RESULTS: Of the initial 50 ACP endpoints identified from the review, 12 endpoints were selected for public prioritization. At the state fair, 359 individuals prioritized the endpoints, the highest ranking of which was having a meaningful conversation with a physician before surgery (57%).
CONCLUSION: Using a novel combination of engagement and research techniques provided the opportunity to understand which endpoints were most important to patients and families and fostered framework development clarifying the differential contributions of engagement and research. Results from this study ultimately changed the content and evaluation of the ACP video.

Introduction

An endpoint in clinical research is defined as “an event or outcome that can be measured objectively to determine whether the intervention studied is beneficial”.1 Decision makers now recognize that patients should guide study design2,3 and be engaged in endpoint selection.4–10 This is important as patients may choose different endpoints than researchers and clinicians,11,12 given their different experiences and knowledge.13,14 The goal should be to include patients (and caregivers when appropriate) throughout all stages of inquiry to avoid token participation.10

Patient-centered evaluation requires that patients are involved in all aspects of research, including the selection of endpoints.15,16 Including patients in endpoint selection remains somewhat limited,9 and best practices are not established.17,18 Endpoint selection can occur through prioritization mechanisms such as multi-criteria decision analysis, utility eliciting, and rating/ranking methods. Ranking methods have been used to identify endpoints that patients and their family members prefer.8,12 Patients can be involved in endpoint selection through both research and engagement activities.19 Using a combined research and engagement approach satisfies a moral rationale for patients to participate in processes that inform their care.20

Our study focuses on promoting advance care planning (ACP) in high-risk surgery. ACP offers individuals the opportunity to clarify health care goals, concerns, and wishes in preparation for situations where they may be unable to make their own decisions.21 The Patient-Centered Outcomes Research Institute contracted this team to develop and evaluate a patient-centered, video-based ACP tool for patients and family members preparing for major surgery.

The aim of this research was to identify patient- and caregiver-preferred ACP endpoints for the video tool being developed. It was hypothesized that patients and their families would prefer an anxiety and depression endpoint, as previous studies highlight the presence of significant anxiety and depression in preoperative and in seriously and critically ill patients.22,23 Although anxiety and depression are often used in ACP given their responsivity,24,25 it was unknown if patients and their loved ones would want an ACP video to focus on alleviating anxiety and depression.

We also sought to highlight the difference between the objectives and outcomes of patient, caregiver, and stakeholder engagement strategies as compared with research approaches when documenting the priorities and preferences of patients and other stakeholders. More specifically, the purpose of engagement activities within this study was to identify ACP endpoints of value and to refine these endpoints so that they were appropriate for an ACP video tool in the surgical setting. The purpose of research activities was to validate the endpoints identified through engagement and to isolate which endpoint was the most meaningful to patients and their loved ones.

Materials and methods

The research team developed a framework comparing facets of engagement and research generally (Table 1) and specifically in the current study (Table 2). To develop this framework, the research team first identified five major domains (communication, ethical obligations, data collection, data analysis, and dissemination) that were integral components to both research and engagement activities. The research team then disaggregated each domain into two salient components, called “process factors”, which were common to both research and engagement. Emphasis is placed on this framework throughout the “Materials and methods” section to highlight how the shared domains and process factors are operationalized differently depending on the approach. The data collection and data analysis domains were of primary interest in the current investigation, given the overarching aim to identify an ACP endpoint based on input from diverse data sources. In the data collection domain, process factors include participant pool, ie, who takes part, and activities, ie, the structure of the participation process. In the data analysis domain, process factors include the level of inference, ie, individual vs group, as well as results interpretation, ie, the approach to “make sense” of the data.

Table 1

Conceptual comparative framework of engagement and traditional research

Domain	Process factors	Engagement	Research
Communication	Interaction	Sustained	Discrete
	Information exchange	Bidirectional, to achieve shared understanding	Unidirectional, to achieve researcher understanding
Ethical obligations	Ethical review	Non-human subject designation	Human subject designation
	Privacy	Transparent	Confidential
Data collection	Participant pool	Local and particular	Broad and diverse
	Activities	Interactive, adaptive methods	Extractive, formalized methods
Data analysis	Level of inference	Individual	Population
	Result interpretation	Stakeholder explanation, non-generalizable	Systematic analytical methods, generalizable
Dissemination	Target audience	Community members, stakeholders	Scientific community
	Time lag	None	Months to years

Table 2

Application of the comparative framework in the current study

Domain	Process factor	Engagement with stakeholders	Research with participants
Communication	Interaction	Stakeholders collaborated with investigators to conceptualize and develop the study over several years	Cognitive interview and survey participants interacted with researchers only while completing the activity
	Information exchange	Stakeholders contributed their opinions to refine endpoints	Participants’ prioritization informed final endpoint selection
Ethical obligations	Ethical review	Endpoint rating and refinement was non-human subject	Interviews and survey were human subject
	Privacy	Stakeholders provided feedback that was not anonymous	Interviews were anonymized and survey data were anonymous
Data collection	Participant pool	Purposeful partnering with stakeholders who had experience and interest	Participation by individuals with whom the research team had no prior contact
	Activities	Endpoint rating, refinement, and stakeholder meetings	Cognitive interviews and state fair surveys
Data analysis	Level of inference	Stakeholders involved in engagement; non-generalizable	Surgical patients and caregivers in Maryland; somewhat generalizable
	Result interpretation	Based on stakeholder’s experiential understanding	Based on scientific literature and statistical analysis
Dissemination	Target audience	Patients, family members, and other involved stakeholders	Scientific community through presentations and publications
	Time lag	Stakeholders informed of all findings in “real time”	Scientific community informed of results 1–3 years later

Engagement approaches

In order to identify and refine endpoints, a deliberately selected caregiver co-investigator (JM) reviewed 50 endpoints previously identified by a systematic review.24 The caregiver co-investigator had more than three decades of experience as both an informal patient advocate for her loved ones and as a formal caregiver-consultant on perioperative ACP research studies (Table 2, data collection, participant pool). The endpoints were presented in groups of four12 to the caregiver co-investigator, who rated the personal meaningfulness of the endpoints using the following criteria: one-star, unimportant; two-stars, low importance; three-stars, important; four-stars, very important (Table 2, data analysis, results interpretation). Star-rating systems are used to rate item quality in the consumer market26,27 and in health care systems.28 Given that patient participation extends beyond “having a patient sit at the table”,29 the caregiver rated endpoints prior to endpoint deliberation by a larger stakeholder team to ensure that her opinions were represented.19 Following a deliberative process, the stakeholder team (composed of palliative care clinicians, surgeons, health services researchers, and the patient and caregiver co-investigators) reflected upon its own experiences in order to contextualize each endpoint’s rating, merit, and weakness. Through group discussion (Table 2, communication, information exchange), the stakeholder team collectively agreed to omit all endpoints that were not supported by the caregiver co-investigator as a first culling (Table 2, data collection, activities).

Research team members (RAA, JFPB) grouped, labeled, and wrote descriptions for the remaining endpoints. Redundant endpoints were grouped together, and multifaceted endpoints were disaggregated into endpoints that were actionable in aim and scope. A deliberately selected patient co-investigator who had undergone major surgery (CP) reviewed the refined endpoints and suggested edits based on personal experience12 (Table 2, ethical obligations, privacy). The complete stakeholder team reviewed the updated endpoints and patient co-investigator edits. The stakeholder team participated in the same deliberative process of reflection, and ultimately culled the endpoints in a manner consistent with patient co-investigator feedback.

All engagement activities were considered non-human subjects research by the Johns Hopkins School of Medicine Institutional Review Board (Table 2, ethical obligations, ethical review).

Research approaches

Consistent with good Patient-Centered Outcomes Research (PCOR) practices,9 cognitive interviews were conducted with two sets of patient–family member dyads recruited from surgical clinics to validate engagement-identified endpoints (Table 2, data collection, activities). The dyads were informed that the research team was developing an ACP video tool and that the endpoints they would examine were possible endpoints for this video tool. The patients and their family members discussed their thoughts on the content and understandability of the endpoints presented to them. Based on this feedback, the dyads were asked to recommend improvements to the endpoints. Research team members (JFPB, ALRS, RAA) reviewed and finalized the revised endpoints and immediately presented them to the stakeholder team, completing the engagement process (Table 2, dissemination, target audience).

The project team rented a booth at the Maryland State Fair to elicit broader input on a meaningful endpoint for the preoperative ACP tool (Table 2, data collection, participant pool). State fairs, which typically have agricultural exhibitions and carnival/amusement activities, are gaining traction as research sites.30 A sign at the booth invited individuals who had undergone major surgery, or their caregivers, to participate in a survey (Table 2, communication, information exchange). Participants viewed storyboards outlining the proposed content and scope of the ACP video under development31 and were asked to anonymously (Table 2, ethical obligations, privacy) prioritize the 5 out of 12 most important endpoints for the ACP video being developed. Twelve total cards, each featuring an endpoint and its description, were presented to participants in the same order of three sets of four cards (Table 2, data collection; Table 3).12 Participants were able to seek clarification about the endpoints and the ranking activity.

Table 3

Potential endpoint labels and descriptions

No.	Label	Description
1	Anxiety and depression	Because of the video, I felt more anxious and depressed
2	Aid preferences	I like this video more than the pamphlet
3	Comfortable with aid	I was comfortable watching this video
4	Decision maker conversation	Because of this video, I have had a meaningful discussion with my surrogate about what matters most to me in life
5	Decision maker designation	Because of this video, I identified and named someone to be my surrogate decision maker
6	Decision making	Because of this video, I felt more comfortable making decisions about my health care wishes
7	Helpfulness of aid	I found this video helpful
8	Identification of values	Because of this video, I have identified what matters most to me and makes life worth living
9	Leeway discussion	Because of this video, I have talked with my surrogate about how much flexibility he/she may take in matching my wishes when making medical decisions for me
10	Leeway preference	Because of this video, I have thought about how much flexibility there is around my health care wishes
11	Legal document knowledge	Because of this video, I know more about legal medical documents such as advance directives and durable power of attorney
12	Medical form completion	Because of this video, I completed a legal medical form such as an advance directive or health care proxy
13	Medical form placement	Because of this video, I put my completed medical forms in my medical record
14	Perceived benefits	Because of this video, I have a greater sense of control over future health care decisions that may be made on my behalf if I cannot speak for myself
15	Physician discussion	Because of this video, I have had a meaningful discussion with my physician about my treatment wishes and goals
16	Recommend aid	I would recommend this video to others
17	Satisfied with aid	I was satisfied with this video
18	Substitute judgment	From watching this video, my surrogate can accurately communicate my goals and wishes
19	Successful outcome	Because of this video, I know the chances that someone who gets life-sustaining treatments (CPR and/or mechanical ventilation) would survive and get to leave the hospital
20	Telling others about decision maker	Because of this video, I have told my family who my surrogate decision maker will be
21	Treatment complications	Because of this video, I know more about the potential risks of getting life-sustaining treatments such as CPR, mechanical ventilation, and tube feeding
22	Treatment knowledge	Because of this video, I know more about life-sustaining treatments such as CPR, mechanical ventilation, and/or tube feeding
23	Treatment preference	Because of this video, I know what I would want in terms of different medical treatments
24	Well-being	Because of this video, I feel less worried (my emotional well-being was improved)

Note: Endpoints in bold were retained for prioritization at the state fair.

Abbreviation: CPR, cardiopulmonary resuscitation.

Endpoint prioritization was summarized by frequency. Two-sample t-Tests in Stata software32 assessed preference heterogeneity by sex (male vs female), age (<60 vs ≥60 years), and whether or not the surgery was for cancer (Table 2, data analysis, results interpretation). Patient and caregiver preferences were analyzed collectively as the ACP tool was intended for both audiences (Table 2, data analysis, level of inference). Participants also completed an open-ended prompt providing advice for those preparing for surgery. The 12 endpoints presented at the state fair acted as components of the analytic framework for the open-ended advice. Comments were reviewed by members of the research team with the participant’s prioritized endpoint in mind. Comments were also reviewed for any suggestions of general information to feature in the ACP video. All comments were compiled and reviewed by one research team member (ALRS), and decisions regarding incorporating comments into the video were made by two additional research team members (RAA, JFPB).

The study team received written informed consent from all participants in the cognitive interviews. Surveying at the state fair was approved as a human subject research activity exempt from full review by the Johns Hopkins School of Medicine Institutional Review Board as it was anonymous and did not contain sensitive information (Table 2, ethical obligations, ethical review). State fair participants provided implied consent to participate; the first page of the survey activity outlined that participation was voluntary and that by completing the survey, participants were providing consent.

Results

The caregiver co-investigator rated 27 of the total 50 endpoints as one star, 10 as two star, 11 as three star, and 12 as four star (Table 4). After review, the stakeholder team ultimately agreed that the 27 endpoints rated as one star by the caregiver co-investigator were not, based on their experience, meaningful endpoints to patients and families. The stakeholder team ultimately elected to discard the endpoints rated as one star (Figure 1). The remaining 33 endpoints were compacted into 24 endpoints with descriptions (Table 3). The patient co-investigator identified 12 of the 24 endpoints as being personally relevant and recommended refined labels and explanations for endpoints. The stakeholder team agreed to retain only the 12 relevant endpoints.

Table 4

Endpoint ratings for two, three, and four star endpoints as determined by caregiver co-investigator

Endpoint	Rating (out of four stars)
Accuracy of decision maker judgment	* * * *
Communication of treatment preferences	* * * *
Decided what is most important in life and health care	* * * *
Designated a decision maker	* * * *
Discussed ACP with decision maker	* * * *
Discussed ACP with family	* * * *
Identified preference for breathing tube and mechanical ventilation	* * * *
Identified preference for CPR	* * * *
Identifying what do and do not want for treatment	* * * *
Patient and provider concordance on quality of life values	* * * *
Patient and provider concordance on treatment preferences	* * * *
Placement of legal medical form in medical record	* * * *
Anxiety and depression associated with using aid	* * *
Completion of a living will	* * *
Discussed ACP with physician	* * *
Health status (mobility and physical and social activity) as result of using aid	* * *
Knowledge of different levels of care (life-prolonging, limited, or comfort care)	* * *
Knowledge of life-sustaining treatment options	* * *
Knowledge of potential complications from life-sustaining treatments	* * *
Knowledge of the chances that someone survives life-sustaining treatments	* * *
Mental health as a result of using aid	* * *
Satisfaction with the decision aid	* * *
Uncertainty in decision making	* * *
Comfort with decision aid	* *
Completion of a health care power of attorney form	* *
Decided how much flexibility to give decision maker	* *
Discussed decision maker’s flexibility with the decision maker	* *
Helpfulness of decision aid	* *
Knowledge of legal medical forms	* *
Recommendation of decision aid	* *
Sense of control over future health care decisions	* *
Sharing information from legal medical documents with family	* *
Told others about decision maker	* *

Abbreviations: ACP, advance care planning; CPR, cardiopulmonary resuscitation.

Figure 1

Endpoint identification and selection results.

The patient–family member dyads agreed that the 12 endpoints were personally meaningful to them, and the stakeholder team subsequently finalized the following endpoints: anxiety and depression, comfort watching the video, decision making, decision maker conversations, decision maker designation, helpfulness of the video, leeway preference, perceived benefits, physician discussion, identification of values, telling others about decision makers, and well-being. Endpoint descriptions were intended to serve as the basis for how each endpoint might be operationalized in a video evaluation study and are provided in Table 3.

Over 11 days at the state fair, 359 people completed the survey (Table 5). Most participants were female (63%), aged >50 years (77%), had at least some college education (79%), were Caucasians (81%), and not reporting on experiences from cancer surgery (74%). Older participants more highly prioritized naming an alternate decision maker (p=0.01; Table 6) and identifying what makes life worth living (p=0.01; Table 6). No differences were observed by sex or by cancer surgery.

Table 5

Demographic characteristics of participants at the Maryland State Fair, N=359

Demographics	% (n)
Sex
Female	63.2 (227)
Male	36.8 (132)
Age, years
18–49	24.5 (88)
50–89	75.5 (271)
Race
White/Caucasian	81.3 (292)
Black/African American	11.4 (41)
Hispanic American	1.7 (6)
Asian/Pacific Islander	3.3 (12)
American Indian/Alaskan Native	2.2 (8)
Education
Some high school or high school graduate	21.2 (76)
Some college or college graduate	53.5 (192)
Some graduate school or completed graduate school	25.3 (91)
Who had surgery (all that apply)
Me	74.9 (269)
My spouse or partner	23.4 (84)
My parent	29.8 (107)
My sibling	10.9 (39)
Other	20.6 (74)
Surgery for cancer	25.6 (92)

Table 6

Frequency of endpoint selection by participants at the Maryland State Fair in 2014

Endpoint	% All(N=359)	% Age <60(n=186)	% Age ≥60(n=173)	p-value
Physician discussion	57	59	55	0.44
Well-being	51	53	49	0.45
Decision maker designation	48	42	55	0.01
Decision making	47	50	44	0.26
Perceived benefits	45	47	42	0.34
Decision maker conversation	43	42	44	0.70
Identification of values	42	36	49	0.01
Telling others about decision maker	41	41	41	1.0
Comfortable with video	40	39	41	0.70
Helpfulness of video	35	35	34	0.84
Leeway preference	21	25	17	0.06
No endpoint chosen	17	17	16	0.80
Anxiety and depression	13	15	12	0.41

The most valued endpoint was having a discussion with a physician about treatment wishes and goals prior to surgery (57%; 95% CI: 52–62; Table 6). In participants’ advice to others preparing for surgery, they described the importance of this discussion. One participant urged people preparing for surgery to “have open conversations with people included in your care (especially surgery)”. Another participant advised to “never be afraid to ask (the) doctor questions”. Only 13% of participants prioritized the anxiety and depression endpoint within their top five (95% CI: 10–17; Table 6). Although some advice reflected value in the anxiety and depression endpoint, such as to “have a positive attitude”, another participant explicitly recommended removing this endpoint.

Discussion

Patients’ and caregivers’ preferences for endpoints can be elicited through engagement and research approaches. The most highly prioritized endpoint for the ACP video was having a meaningful discussion with a physician rather than the hypothesized anxiety and depression endpoint. Differences between older and younger participants’ preferences for ACP have been previously observed,33,34 but it is notable that preference differences by age were not observed for the prioritized endpoint of physician discussion. That the oft-used anxiety and depression endpoint was not the most valued endpoint contributes to a growing body of literature demonstrating that professionals and patients may have different endpoint preferences,12,13 further validating the need to include patient perspectives early in the research process.

A meaningful conversation between a patient and a surgeon prior to surgery was not initially hypothesized as a meaningful endpoint by the research team, given the highly biomedical nature of surgical care. Physician discussion is a relevant endpoint as communication skills with patients (inherent for good ACP conversations) are positively associated with patient satisfaction,35 and patients often prefer doctors who participate in communication-based shared decision making.36 Studies specifically highlight potential challenges for ACP communication in the preoperative setting as surgeons and patients may enact an implicit “covenant of care” during the consent process in which patients transfer decision making about life-sustaining perioperative treatments to the surgeon.37,38 Patient–surgeon communication is of particular importance, given the complexity of conversations that must occur in advance of surgery.39 Communication is a well-documented facet of patient-centered care;40,41 however, surgeons often do not explore emotional concerns of patients.42 Given this evidence, it is significant that results support that patients and family members place a high value on a preoperative ACP video tool that fosters meaningful patient–physician discussions about treatment wishes and goals, suggesting that many surgery patients and family members may be ready to talk about ACP.

Patient-centered research does not stop at the identification of endpoints; rather, it actively incorporates the perspectives of patients and family members into the research process. Patient prioritization of meaningful physician discussion over reducing anxiety and depression, therefore, resulted in a substantial redesign of both the ACP video and its evaluation trial. The team negotiated with its funder to change the primary outcome of the video’s evaluation from anxiety and depression to having a meaningful discussion. Meaningfulness of the discussion was operationalized by the measure “patient-centered nature of the patient–surgeon conversation” in the resulting video evaluation trial, procedures for which have been previously described.43 This change in outcome also required that language be added to the video to encourage open communication between the patient and surgeon.43

In the practice-oriented context of the current study, using both engagement and research approaches in endpoint selection provided an innovative means of identifying and prioritizing endpoints. The conceptual framework presented in this paper adds knowledge to the current literature by depicting how engagement and research compare across various domains. This conceptual model may serve as guidance for future studies, considering the value and purpose of engagement and/or research to identify the endpoints for PCOR.

Interpretation of the study must consider its limitations. First, although the endpoints were initially identified through an environmental scan, all relevant endpoints may not have been included for consideration. Second, although the patient-centered nature of the intervention is the strength of the study, the patient and caregiver-driven selection of endpoints may have looked different had different patients and caregivers been included in the selection process. Although reproducibility and replication are highly important in research, it is important to highlight that engagement purposively involves “insiders” because of their unique experiences. Engagement is a pragmatic approach for eliciting patient and caregiver input into formative stages of larger research studies. The largest limitation of the study pertains to the generalizability of the research findings at the state fair. Even though a large number of people attending completed the survey, this self-selected convenience sample may not be representative of all patients preparing for major surgery, and future studies using a dual engagement–research approach should strive to maximize generalizability during endpoint prioritization. It is noteworthy that the results from the state fair are certainly more generalizable than those that would have emerged from engagement activities alone and that the full generalizability of these findings will be further evaluated during the trial of the developed ACP intervention.43

From its inception, researchers on this project planned to use a novel patient-centered method to identify, refine, and select endpoints in the absence of best practices on how to do so. Using a combined research and engagement approach represents a commitment to the hypothesis that gathering patient perspectives makes research more relevant to patient and family member decisions and is more likely to lead to the outcomes that matter most to them.7