Engaging Stakeholders to Develop a Patient-centered Research Agenda: Lessons Learned From the Research Action for Health Network (REACHnet).

BACKGROUND: Patient-centered research requires a focus on the needs and priorities of patients. Because patient engagement can result in the discovery of important topics not currently prioritized by research programs, topic generation, and prioritization activities conducted with patients, caregivers, and other stakeholders are essential. To develop patient-centered research agendas for obesity and diabetes, the Research Action for Health Network conducted topic generation and prioritization activities with multistakeholder research advisory groups.
OBJECTIVES: The purpose of this case study was to demonstrate how methods for engaging patients in topic generation and prioritization can be implemented in practice for the development of a patient-centered research agenda. RESEARCH
DESIGN: Four multistakeholder groups comprising patients, clinicians, and researchers met 4-5 times between November 2014 and July 2015 to generate and prioritize topics for obesity and diabetes research. Topics were prioritized using an iterative engagement process, in which themes were identified and resulting topics were refined and ranked over multiple meetings. PARTICIPANTS: Sixty-four patients, clinicians, and researchers participated in 2 obesity and 2 diabetes advisory groups. The majority of participants (64.0%) were patients, followed by clinicians (23.4%), researchers (9.4%), and parents of children with diabetes (3.1%).
RESULTS: Ten and 12 priority topics were identified for obesity and diabetes, respectively. The resulting research agendas were disseminated to patients, researchers, and clinicians.
CONCLUSIONS: Patient engagement has the potential to enrich our understanding of patient priorities for research. The results from this process suggest that convening in-person multistakeholder groups can be an effective way to generate research topics that reflect patients' priorities. Engagement strategies should be focused not only on the development of patient-centered research topics but also on the implementation of these topics into research studies.

The Affordable Care Act brought new attention to patient-centeredness in research through the creation of the Patient-Centered Outcomes Research Institute (PCORI) in 2010.1,2 PCORI promotes the active engagement of patients, clinicians, and other stakeholders for conducting patient-centered outcomes research (PCOR).2 Patient-centered research focuses on topics that reflect the needs and priorities of patients; this approach ensures that research is relevant to patients and increases the likelihood of implementing research findings to health care.1,3–8 Because patient engagement can result in the discovery of important topics not currently prioritized by research programs,9,10 topic generation and prioritization activities conducted with patients and other stakeholders are essential for developing a patient-centered research agenda.

Previous studies have used a variety of patient engagement methods for topic generation and prioritization. Stewart et al4 performed a systematic review of 148 studies that identified patient and clinician priorities for research. The majority of these studies involved patients and clinicians working separately to generate topics; only 9 studies engaged both patients and clinicians to generate research questions collaboratively.4 In these studies, face-to-face meetings were the most common method of engagement.4 Methods for patient prioritization of research topics have included individual rating of topics, voting, scoring, and consensus processes.4,11–17 PCORI recommends a topic generation process that includes collaboration between patients and clinicians, peer consultations, data analysis, and consensus-building approaches.18 Previous research has also described the use of iterative processes for effective stakeholder engagement, in which the same participants are engaged over time to generate and prioritize research topics.19–21 Such a process can ensure that resulting priorities have been created and refined by stakeholders themselves.

The Research Action for Health Network (REACHnet), formerly the Louisiana Clinical Data Research Network (LaCDRN), is one of 13 CDRNs that comprise PCORnet, created in 2014 with the purpose of conducting large-scale, patient-centered comparative effectiveness research. During the first 18 months of the initiative, each CDRN was required to develop a patient-centered research agenda through robust engagement with patients and other stakeholders. To fulfill this aim, REACHnet used an iterative patient engagement process involving multistakeholder groups. This paper describes the implementation of these methods through the engagement process, from the formation of advisory groups through dissemination of the final patient-centered research agenda.

METHODS

Recruitment of Advisory Groups

Between July and October of 2014, REACHnet formed 4 multistakeholder research advisory groups in New Orleans and Baton Rouge: 2 focused on diabetes and 2 focused on obesity. At the time of this initiative, REACHnet included only health systems in Southeast Louisiana; the network has since expanded to include systems around Louisiana and Texas. Diabetes and obesity are conditions with an enormous health burden in Louisiana.22,23 REACHnet investigators and health systems leaders nominated clinicians and researchers for participation on the basis of their commitment to the engagement process, expertise in the condition of interest, and willingness to participate in regular meetings. Clinicians were physicians or nurses actively involved in care for patients with diabetes or obesity. Researchers were PhD level research scientists currently studying topics related to obesity or diabetes. Patients were eligible if they were diagnosed with obesity, diabetes, or prediabetes; parents of children diagnosed with diabetes were also eligible to participate. Two different approaches were used to recruit patients. In the first approach, physicians and nurses recruited patients during clinic visits, explaining the purpose of the group and the benefits of participating. In the second approach, patients were recruited from active support groups for managing the condition of interest.

Meeting Format

Between November 2014 and August 2015, REACHnet convened advisory groups regularly to generate, refine, and prioritize research topics. The diabetes advisory groups, held in New Orleans and Baton Rouge, met 4 times each (8 total meetings), and the obesity advisory groups, held in New Orleans and Baton Rouge, met 5 times each (10 total meetings). These locations were selected to maximize representation of participants across REACHnet health systems. Two REACHnet personnel attended each meeting to act as a facilitator and a note-taker. Each meeting agenda included activities pertaining to the research topic generation and prioritization process, which was conducted over the series of meetings. Meetings lasted 1.5 hours and took place at a public library, community center, or other public space. Topic generation activities followed the format of a focus group. Stakeholders were encouraged to speak openly, ask questions, and contribute ideas at any time. During the last 5–10 minutes of each meeting, participants completed a brief anonymous survey to provide feedback on the structure and content of the meeting. Participants were given a $50 gift card for each meeting and were provided with a healthy dinner.

Because participating stakeholders possessed varying levels of knowledge and experience with the research process, the first meeting included a research training session. The session, facilitated by a REACHnet investigator, was based on the “Research 101” training module developed at Vanderbilt Institute for Clinical and Translational Research (used with permission).24 Advisory group members received an overview of the research process and common research terminology. The training session was modified to include principles of PCOR and an overview of REACHnet.

Composition of Groups

Table 1 shows the composition of the research advisory groups categorized by condition, location, and type of stakeholder. In New Orleans and Baton Rouge combined, a total of 36 stakeholders and 28 stakeholders participated in the obesity and diabetes advisory groups, respectively. Of the 64 participants, 41 (64.0%) identified as a patient with diabetes or obesity, 2 (3.1%) identified as a parent of a pediatric patient with diabetes, 15 (23.4%) identified as a clinician, and 6 (9.4%) identified as a researcher. Meeting attendance confirmed that patients comprised more than half of participants during each meeting. Because participants were members of an advisory board, and were not considered research subjects, additional demographic information was not collected.

TABLE 1

Composition of Research Action for Health Network Research Advisory Groups for Obesity and Diabetes

Topic Generation and Prioritization

Figure 1 shows an overview of the topic generation and prioritization processes. The topic generation process began with an unstructured brainstorming session involving active dialog among all group members. The meeting facilitator asked patients and parents to share concerns related to the condition of interest that are most important for themselves and their family members when managing their condition. The meeting facilitator assured participants that there were no right or wrong answers and encouraged them to share their ideas freely. Discussions were lively and there was a high level of rapport among group members.

FIGURE 1

Flow diagram showing topic generation, prioritization, and alignment processes. REACHnet indicates Research Action for Health Network.

During the initial meetings, 33 themes were identified for obesity and 24 were identified for diabetes. REACHnet personnel then grouped the originally proposed themes into topics (19 for obesity, 17 for diabetes) and presented the list back to each group at the subsequent meetings to collect feedback. During the next 2 meetings in each group, topics were added, eliminated, and refined by participants. This process culminated in 2 final lists of 10 topics for obesity and 12 topics for diabetes. The resulting lists were used for the topic prioritization exercise.

To assess which of the topics were most important to patient stakeholders, REACHnet staff developed a simple worksheet that listed the final, refined topics. Different versions of the worksheet (ie, varying order of the topics) were presented to the New Orleans and Baton Rouge groups. This prioritization activity occurred during each group’s third and fourth meetings. Participants were asked to indicate the 3–5 topics that were most important to them, and they were also invited to add any additional topics that had not yet been discussed. The framing of this worksheet aligns with PCORI’s standard process for prioritizing patient-centered research questions, which asks stakeholders to judge topics based in part on the topic’s importance to the individual themselves.25

Each topic was scored to identify the final research topics of highest priority to patient stakeholders. Topics that were selected as “important” by more than half of the respondents were included in the final, prioritized lists—a total of 5 topics for obesity and 3 topics for diabetes. The final lists were shared with participants at the fifth meeting of each group to confirm that the results accurately represented the groups’ collective opinion. Members were invited to provide qualitative feedback on the accuracy of the results based on their participation in the topic identification and ranking exercises. Table 2 shows the outputs of the iterative engagement process at each stage, from topic generation to final priority lists.

TABLE 2

Outputs of Engagement Process During Each Iterative Stage

RESULTS

Prioritized Research Agenda

Table 3 shows the final list of prioritized research topics. For obesity, 2 topics were ranked as important for 74% of participating patients were. These topics were: (1) the effectiveness of weight loss programs for different types of people (personality, body type, preferences, age, etc.); and (2) barriers to physical activity and ways to overcome these barriers. Other topics selected as the highest priorities included: (3) the relationship between mental health and weight outcomes; (4) primary obesity prevention through teaching children healthy habits; and (5) effectiveness of technology for weight loss.

TABLE 3

Final List of Prioritized Research Topics

For diabetes, the 2 topics ranked as important by 69% of participants included (1) methods for improving patients’ understanding of diabetes tests and vocabulary and (2) preventing the progression of diabetes complications. Using educational tools and technology for improving diabetes management was selected as the third highest priority topic.

Implementation of Findings

To disseminate the results of this process, the final research agendas were made publicly available on the REACHnet website as a formal report as well as a patient-friendly infographic, the format of which was also informed by the advisory groups. The finalized research agendas were shared with investigators, health system leaders, and patient stakeholders through a monthly newsletter and in REACHnet meetings with clinicians, researchers, and health system leaders. In addition, REACHnet created policies for research participation that promote the prioritized research agendas for obesity and diabetes. Investigators wishing to use REACHnet to conduct research must explain how the proposed study aligns with the patient-centered research agenda. Using an adapted version of PCORI’s standardized rubric, this alignment is evaluated by REACHnet leadership and community stakeholders (patients, clinic staff, or community educators) to inform decisions about network participation. As REACHnet’s patient-centered research agendas were finalized and disseminated, local researchers have proposed a total of 5 studies that align with research priorities for either weight or diabetes. As of May 2016, all 5 studies have been funded.

DISCUSSION

There has been considerable discussion about “tokenism” in patient engagement, characterized by low levels of engagement and lack of meaningful participation by patient stakeholders.1,18,26,27 We took several measures to ensure genuine and meaningful engagement of patients. First, there was continued involvement of the same participants throughout the topic generation and prioritization process. Participants were therefore able to build trust and rapport over time. REACHnet staff also communicated regularly with advisory group participants to enhance commitment to the process. This sustained engagement over time was essential for the successful completion of this project. Second, a multiple iteration approach was used throughout the entire process, which positively influenced stakeholder commitment, trust in the process, and the quality of the final product. During each meeting, participants reviewed the progress made during the previous meeting and worked toward completing next steps. Third, the ratio of patients to clinicians and researchers was high. This, coupled with a patient-centered approach by facilitators, ensured that the topic generation process remained focused on patient priorities rather than on existing research priorities of clinician or researcher participants. Staff involvement was limited to organization of topics by theme and summarizing and disseminating meeting results.

Creation of a patient-centered research agenda has little impact without implementation. Abma and Broerse28 suggest that matching research topics with investigators is one way to achieve successful implementation of a patient-centered research agenda. In our approach, we searched for alignment of the research agenda with national research priorities, with the goal of identifying potential funding sources. For diabetes, each of the final research topics corresponded to at least 2 national research priorities identified by PCORI, Institute of Medicine (IOM), and the American Diabetes Association.29–33 For obesity, each final topic corresponded with at least 4 research priorities identified by PCORI, IOM, the American Heart Association (AHA), the National Eating Disorders Association (NEDA), the National Collaboration on Childhood Obesity Research (NCCOR), and the National Institute of Health National Institute of Diabetes and Digestive and Kidney Diseases (NIH/NIDDK).29,30,34–37 In total, we found 23 national priorities that aligned with our finalized topics for diabetes and 34 for obesity. These findings indicate that priorities for patients in Southeast Louisiana are similar to patient priorities nationally for these conditions.

Although many studies have engaged patient and clinician stakeholders for topic generation and prioritization, a relatively small number have used groups of combined stakeholders groups to achieve this.4,17 More commonly, studies have created separate groups of clinicians and patients for identifying research priorities, which may then be integrated in a later stage.4,10,28,38 Because our process only included multistakeholder groups, we cannot compare how priorities differed among stakeholder groups.28 Nevertheless, a multistage process with varying stakeholder types such as ours has several advantages. First, it allows for participants to take into account the opinions of other stakeholders, which can improve understanding of the condition as well as increase the feasibility of the resulting research topic. Second, involving researchers and clinicians in stakeholder advisory groups may allow for an improved chance of implementation of the resulting research agenda.24 Finally, interaction between researchers, clinicians, and patients is essential for increasing trust and improving collaboration.

Several important limitations of these methods warrant discussion. First, because all advisory groups included a mix of patients, researchers, and clinicians, it was not possible to assess the extent to which clinicians and researchers may have influenced the final selection of topics. However, because group discussion was patient-led, the majority of each group were patients, and only patients completed the final ranking activity, we believe that the final topics accurately reflect patient priorities. Second, the groups included highly motivated patients, many of whom had previously participated in research. As a result, the list of priority research topics may not be generalizable to all patients with the conditions of interest. Third, REACHnet personnel facilitated meetings in addition to reviewing meeting minutes and worksheets, which may have introduced bias. This potential bias could be eliminated during a replication of this process by having staff that did not attend organize and refine research topics. Finally, while this process benefited from regular high attendance from participants, it is unclear whether this level of participation and engagement could be achieved without offering patient incentives. Previous studies have found that monetary incentives are effective for increasing participation, particularly among those who are less likely to participate in research.39–41 Altruism, perceived health benefits or education, recruitment by a doctor or nurse, and therapeutic aspects of the research process have also been cited as motivating factors for participation in qualitative research.42–44

Patient engagement has the potential to enrich our understanding of patient priorities for research. Research institutions, networks, and investigators can learn a great deal from previous approaches to stakeholder engagement. The results from this process suggest that convening in-person multistakeholder groups is an effective way to generate research topics that reflect patients’ priorities. Given the current focus on developing patient-centered research questions, standardized approaches should be defined to meaningfully involve the patient community in this process. Successful approaches will build trust in the patient-research partnership, ensure that patients are meaningfully engaged throughout the process, and capture the diversity of patient experiences and perspectives. The described framework achieves this end and provides a blueprint for groups interested in pursuing a collaborative approach in which multistakeholder groups work together to identify research priorities.