A Community Engagement Advisory Board as a strategy to improve research engagement and build institutional capacity for community-engaged research.

PURPOSE: Community engagement is deemed as critical to the success of the CTSA program. In 2009, to improve research engagement and build capacity for community-engaged research across the translational spectrum, the Center for Clinical and Translational Science at the University of Illinois at Chicago created a Community Engagement Advisory Board (CEAB). Here, we report results of our ongoing evaluation efforts.
METHODS: CEAB activities are evaluated using mixed methods. Annual CEAB evaluation surveys were completed from 2010 to 2016 (n=106 respondents). In 2014, two 90-minute focus groups were conducted with a subset of recent CEAB members (n=19).
RESULTS: Survey data suggest respondents perceive their consultations to be helpful in improving the capacity of researchers (90%) and the quality of research projects (80%). Further, CEAB members perceive themselves to have personally benefitted from their involvement including obtaining new knowledge (84%), expansion of their networks (76%), and forming new community linkages (51%). Results of the qualitative data were consistent with survey data.
CONCLUSIONS: Our CEAB has improved research engagement and developed institutional capacity to conduct community-engaged research in several ways. Our findings can inform the establishment or enhancement of community engagement services for CTSA-affiliated researchers and community partners. © The Association for Clinical and Translational Science 2018.

Introduction

The Institute of Medicine (IOM) report, The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research, offers several key recommendations and opportunities for action [1]. Community engagement has been highlighted as critical to the continued success of the CTSA program [2] and is increasingly recognized as central to the continuous feedback loop necessary for translating knowledge gained from laboratory science into clinical practice to improve health [3, 4]. For example, in T1 research (Translation to humans), community engagement is essential to inform research needs and priorities and identify ethical concerns associated with a research project [5]. Major community engagement goals of T2–T3 research (translation to patients and practice) are to partner with patient stakeholder/advocacy groups and community-based organizations to increase the cultural appropriateness of research objectives and to engage and recruit research participants [6]. Community engagement goals of T4 research (translation to populations) include determining the best methods for dissemination and adoption of new treatments, establishing community monitoring groups to ensure equitable access, and identifying unintended consequences for vulnerable patient groups [7, 8].

In recognition of the broad benefits of community involvement in research, the IOM report stressed the need for innovation in community engagement approaches [1]. In 2014, Holzer and Kass [4] conducted a study to evaluate changes in the community engagement strategies of NIH-funded CTSA programs following the increased emphasis on community engagement. Researchers identified 2 distinct types of engagement strategies within the CTSA grantee institutions: (1) capacity-building strategies and (2) research engagement strategies. As described by the authors, capacity-building strategies were efforts to prepare faculty and the staff of the institution for community engagement and prepare community members for such partnering around research. Research engagement strategies included efforts to develop and implement engagement strategies for specific research projects. Each of these is important, and institutions could benefit from innovations that simultaneously seek to build capacity among researchers and community stakeholders while also offering expert input on community engagement strategies for a wide range of research projects [9].

In 2009, the Center for Clinical and Translational Science (CCTS) at the University of Illinois Chicago (UIC) established a Community Engagement Advisory Board (CEAB) as a working group within the CCTS Recruitment, Retention, and Community Engagement Program (RRCEP). The CEAB is a free consultative service provided to university faculty, postdoctoral fellows and doctoral students from UIC, university faculty from other area academic institutions, and researchers from community agencies [10]. CEAB members are invited community and patient advocates, members of key community-based organizations, and research faculty and staff members from UIC and other academic institutions with significant experience and expertise in community-engaged research. The role of the CEAB is to provide constructive input on specific issues as defined by the researcher requesting the consultation. CEAB members (n=31) are distributed across 2 separate boards that meet on alternating months. CEAB members serve 3 year terms and participate in an average of 3.5 consultations annually (consultations are held in 9 out of 12 calendar months). CEAB consultation meetings last about 2 hours and typically include 2–3 consultations of 30–45 minutes each. Projects can be at any stage of development. Since 2009, the CEAB has provided 123 consultations to investigators seeking input on a variety of issues including identifying community partners, refining research methods, developing recruitment and retention plans, identifying culturally appropriate engagement strategies, forming community advisory boards and overcoming barriers to participant engagement (for a full description of the CEAB development and implementation see Matthews et al [10]).

A CEAB differs from a traditional community advisory board in that it advises on a variety of research projects and consists of diverse membership in expertise, demographics, and roles. Further, unlike a traditional community advisory board, a CEAB can support capacity building for community-engaged research among academic and community partners, for example, by enhancing bi-directional relationships between community members and academic researchers as mandated as part of the National Institutes of Health Clinical and Translational Science Awards (CTSA) program [11]. Despite the acknowledged importance of community engagement, to our knowledge, no studies have examined the potential of a CTSA-affiliated CEAB to both develop institutional capacity for community-engaged research and provide input regarding engagement strategies for specific research studies.

To address this gap in the literature, the purpose of this study was to assess how the CEAB has contributed to both research engagement and institutional capacity building for community-engaged research. CEAB members also provided suggestions for ways to increase capacity building. The findings from this study have implications for the further development and refinement of this innovative approach to community engagement in the context of CTSA-affiliated services for researchers.

Methods

Overview

Data for the evaluation of CEAB research engagement and capacity building were derived from 2 sources: an annual evaluation survey of CEAB members (years 2010, 2012–2016) and 2 focus groups conducted with a subgroup of CEAB members in 2014. Methods and procedures for the survey and focus groups are described below.

Annual Evaluation Survey

In order to obtain feedback from CEAB members on their experiences, the CCTS RRCEP developed a survey in collaboration with the CCTS Evaluation and Tracking (E&T) Team. The Evaluation and Tracking program develops and maintains data collection systems, process evaluation data, and report on activities, outputs, and outcomes of each of the core CCTS programs. Questions focus on meeting attendance and preparation, perceptions of the quality and benefits of the consultations provided, perceived benefits of participation, and overall level of satisfaction with their involvement. Members are also asked annually whether they plan to continue their membership. The survey is administered online using Qualtrics Research Suite (Qualtrics, Provo, UT), an online survey tool that allows researchers to build, distribute, and analyze online surveys in real time [12].

The evaluation survey has been continuously implemented from 2010 to 2016 with the exception of 2011 (evaluations were halted in 2011 for survey and system updates). Each year, CEAB members are sent an email with a link to the survey. A second reminder email is sent 2 weeks later. There are a total of 106 responses across 6 years. The average annual survey response rate is 61% (range 43–75%). Some individuals responded in more than 1 year (the term for CEAB membership is 3 years and some members have served multiple terms). Deidentified data derived from online surveys were initially collected for quality assurance purposes and so this activity was determined to be exempt by the UIC institutional review board (IRB).

Focus Group Procedures

We conducted focus groups in 2014 to obtain feedback from CEAB members regarding their experiences and to identify topics for future education and training [10]. RRCEP leadership and staff developed the focus group protocol and discussion guide collaboratively with a CEAB member who also serves on the administrative team. In June 2014, all current CEAB members (n=32) were invited via email to participate in a focus group. Interested participants contacted staff by telephone. Two 90-minute focus group sessions (n=19) were held at a university location, with 9–10 participants each. Before the discussion, written informed consent was obtained by a research staff member, and participants completed a brief (5–10 min) self-administered paper-and-pencil survey that asked about the communities in which they work, the health issues addressed by their organization, engagement in CEAB activities, and other opportunities that had resulted from CEAB membership.

The focus groups were conducted according to established methodology [13]. Questions aimed to elicit participants’ thoughts regarding their experiences as CEAB members, observations about the research projects for which they have provided consultations, and recommendations for how to improve the quality of consultations and training of new and current CEAB members as well as researchers, particularly those who are new to community engagement. A trained moderator (A.K.M.) guided the structured discussion. Focus groups were audio-recorded and professionally transcribed; the facilitator and note-taker immediately debriefed after each session to highlight important findings, and focus group transcriptions were carefully reviewed. All focus group participants received a $50 gift card. The study was approved by the UIC IRB.

Data Analysis

Standardized descriptive statistics (frequencies, means, percentages) were used to analyze the survey data. Two authors reviewed the focus group transcripts for key themes across groups. While keeping the original evaluation questions in mind, statements were categorized into broad themes and subthemes. Coding categories were then used to summarize key ideas in the combined focus groups as described by Stewart et al. [14]. To protect confidentiality, all tapes, documents, and transcripts were logged using participant numbers, and electronic data were password protected. Further, the list of codes identifying individual participants was stored separately in a locked filing cabinet.

Results

Results of Annual CEAB Member Surveys

A total of 106 respondents across all 6 years completed a survey of their experiences providing CEAB consultations. Table 1 displays the summary of CEAB survey results from 2009–2010 and 2012–2016. The majority of CEAB members believe that the consultations they provide to researchers are very or extremely helpful (90%) and that the consultation results in an improvement in the researcher’s project always or most of the time/always (90%). In total, 98% of CEAB members reported that they would recommend a research consultation with the CEAB. Respondents reported being very or extremely satisfied (98%) as CEAB members. In total, 90% of all CEAB members reported that they plan to continue as members the following year.

Table 1

Community engagement advisory board member survey (2010–2016, n=106)

Survey question	Yes [% (n)]	No [% (n)]	Not at all [% (n)]	Slightly [% (n)]	Moderately [% (n)]	Very [% (n)]	Extremely [% (n)]	Not sure [% (n)]
Benefit of consultations offered
How helpful are consultations to researchers	–	–	0% (0)	0% (0)	6% (6)	36% (38)	54% (57)	4% (5)
Do consultations improve a researcher’s project	–	–	0% (0)	3% (3)	6% (6)	46% (49)	34% (36)	11% (12)
Recommend a consultation to a researcher	98% (104)	2% (2)
Overall satisfaction with CEAB membership
Feel welcome as a member	–	–	0% (0)	0% (0)	3% (3)	34% (36)	63% (67)
Feel comments are valued	–	–	0% (0)	2% (2)	10% (11)	64% (67)	24% (25)
Level of satisfaction with membership	–	–	0% (0)	0% (0)	2% (2)	41% (44)	57% (62)
Plan to continue as a member	91% (98)	2% (2)						7% (8)
Increased capacity as a result of membership
How much have you learned	–	–	0% (0)	1% (1)	15% (16)	40% (42)	44% (47)
Have you been able to apply new information to your setting	79% (82)	13% (14)						8% (8)
Have you been able to make new community linkages*	51% (49)	29% (28)						20% (20)
Have you been able to expand your networks*	76% (72)	11% (10)						13% (13)

CEAB, Community Engagement Advisory Board.

This question was not asked in 2010; Responses options for each question were measured on a 5-point Likert scale with 1=more negative perception and 5=more positive perceptions. The qualitative wording of each scale varied somewhat based on the question asked.

In terms of their own personal and professional capacity building, over 80% of members said that they had learned “a lot” or “a great deal” from their experience on the CEAB. In total, 76% of members said they were able to expand their networks due to membership. Additionally, 79% of the members stated that they have been able to apply what they learned as a CEAB member to their own setting or organization. Slightly more than half (51%) of all members reported establishing new community linkages as a result of their involvement with the CEAB.

Focus Group Findings

Key qualitative findings are described in the next section and organized based on broad themes and subthemes. Quotes illustrate main points (Table 2).

Table 2

Summary of key qualitative findings

Theme	Subthemes	Illustrative quotes
Contributions of the CEAB Consultation Service	1. Role of CEAB members	“It’s really important, I believe, to be the voice of those that we don’t usually hear. That’s the residents. I’m here, speaking not only for the university, as a researcher, but I’m really an advocate for community residents.”
	2. Value of CEABconsultations	“What the committee has shared with these researchers is really from the perspective of the community. Understanding the culture, understanding the language, understanding what will work, what won’t work, and voicing that, so that the investigators can listen to what we are saying.”
CEAB Membership and Capacity Building	1. Increased personal knowledge	“Individually and, I think for the community, the research projects have often time informed me of the latest scientific theories about treatment and so on, about things that are associated with positive outcomes.”
	2. Increased organizational capacity	“For the years I’ve been here, its allowed me to be able to take back a lot of what is going on in the UIC community that’s not always taken back to the people who need it the most …not just to patients and consumers but even to my peers and people who don’t get to participate.”
Improving the Quality of CEAB Consultations	1. Include more lay members	“people who are in communities that are commonly targeted. It would be great to have those residents here, to hear how their communities are going to be asked to do something. They can give firsthand – and like you say, we all represent some of those communities, but in a different way.”
	2. Improve training of new members	“Coming in cold, and not really having-because even though I was here [have served on CABS], I never really participated like this. I kinda had a sense, but I didn’t really know what the expectation would be of me personally.”
	3. Preparation for consultations	“I definitely appreciate the handouts and receiving them beforehand…”
	4. Feedback and evaluation	“It would be nice to have some type of evaluation of our service, whether it’s useful or whether it’s not. If there is some other way that we can improve it would be nice to know that. Otherwise, we just hitting, taking aim in the dark.”

CABS, Community Advisory Boards; CEAB, Community Engagement Advisory Board.

Perceived Contributions of the CEAB Consultation Service

In framing their contributions, CEAB members were first asked to discuss how they define their roles. CEAB members emphasized that they are community representatives, community gatekeepers and protectors, and liaisons between researchers and the larger community. As community representatives, members emphasized that they have knowledge of the lived experiences of many different community groups. Members felt that their knowledge of the culture and norms of the community was an important role of the CEAB in improving the conduct of research by providing an “on the ground observation of what’s the reality.” However, many CEAB members felt that in reality, they were serving as proxies for “the real community,” that is, those members of under-resourced and underserved communities that experience the highest levels of health disparities. As one CEAB member said:

“It’s really important, I believe, to be the voice of those that we don’t usually hear. That’s the residents. That’s the people in the community who we are trying to impact their lives. I’m here, speaking not only for the university, as a researcher, but I’m really an advocate for community residents.”

CEAB members also perceived themselves to be community gatekeepers, which they described as someone who provides researchers with access to a community via the specificity of language, location, and cultural information. Other members described the role of “gatekeeper” as someone who excludes or keeps out researchers that do not seem prepared to work with community or whose research does not offer obvious benefits to community members. One CEAB member said:

“They [researchers] wanna know how we feel and how do they access your community. Be that gatekeeper. Tell ‘em what language to use. Who should come to the alley [potential locations for recruiting participants]. How you should present it [the objectives of the research study], you know? I hope we get it across here [to the researchers].”

Another stated:

“There are people (researchers) who are very technical, and there’s no relationship conversation. There’s no discussion of relating to the group. There’s no engagement aspect to it. There’s no population—none of the population has been invited to the table, even in the formative stages. There’s been no discussion with anybody from that particular population. They don’t have a sense of what area to target, or what community-based organizations, or who the stakeholders are. Those are, to me, signs that you don’t really understand who it is you’re trying to engage with.”

Members also expressed a perceived responsibility to protect against potential harms associated with research, especially if they were instrumental in connecting community members to researchers. Specific examples of perceived harms included over researching members of vulnerable communities or failure to follow-up on promises (e.g., services that would be offered as part of the funded project). A CEAB member said: “I guess I see it as my responsibility, … If I’m inviting somebody in, then how do I make sure that they’re following up? It’s on my responsibility to do that.”

Finally, CEAB members described themselves as liaisons between the researchers and community members but emphasized the importance of community advocacy in the research process.

Value of CEAB Consultations

After defining their roles, CEAB members discussed their contributions to community-engaged research, and the effectiveness and usefulness of the consultations that they provide both individually and as a group. Each of these subthemes are discussed below.

Despite some concerns about being the “authentic” voice of the community, when asked about the specific contributions of the CEAB in improving research, the majority of members readily pointed to the CEAB’s ability to provide valuable input and guidance on strategies for working with and engaging community members. For example, CEAB members mentioned that they could explain cultural norms, anticipate what would be feasible and acceptable to community members, and discuss the history of abuses associated with research and resultant community mistrust. Members also point to their ability to observe and understand what researchers from various institutions have done that has resulted in a failed or successful engagement with community—and pass on this wisdom in their consultations. Once CEAB member described this wisdom:

“What the committee has shared with these researchers is really from the perspective of the community. Understanding the culture, understanding the language, understanding what will work, what won’t work, and voicing that, so that the investigators can listen to what we’re saying.”

While the majority of members felt that the quality of consultations was high and the feedback and recommendations provided to investigators were valuable, some expressed concern about the ability of their contributions to improve the quality of research because many on the CEAB were new to research and do not hold advanced degrees. That being said, members saw themselves and other members of the advisory board as providing meaningful contributions despite this:

“We’re still learning, but we’re at the level where we need to be, so we can intelligently work together, and also get out of it what’s needed to really advance our quality of life in our community.”

“I don’t have to have a PhD, but I can still engage in these things in a meaningful way.”

CEAB Membership and Capacity Building

Finally, CEAB members reflected on their participation on the advisory board and the ways in which involvement has increased their own personal capacity and their ability to increase the capacity of the communities or organizational groups that they represent. Each of these subthemes is discussed below.

Participants uniformly reported that they had personally benefited through their involvement in the CEAB. Some indicated that they were learning more basic knowledge about research and new scientific advancements. For example:

“Individually and, I think, for the community, the research projects have often time informed me of the latest scientific theories about treatment and so on, about things that are associated with positive outcomes”.

CEAB members report sharing information with members of their communities, organizations and family members. CEAB members also noted that they had learned more about the different kinds of research taking place in various Chicago communities, information that is not always easy to find. They reported that the other members of the CEAB represent a network for the exchange of resources and ideas.

In addition to increasing personal knowledge, CEAB members felt that their involvement increased their capacity to bring resources to the communities and organizations that they represent. For example, some members have applied the knowledge that they have obtained to mentor other organizations to obtain grant funding. Others reported learning specific technical skills that they have used to evaluate and enhance the programming offered at their own organizations:

“For the years I’ve been here, its allowed me to be able to take back of lot of what is going on in the UIC community that’s not always taken back to the people who need it the most… not just to patients and consumers but even to my peers and people who don’t get to participate.”

One of the lay CEAB members secured her own grant for community engagement and on multiple occasions received input for her proposed research from the other CEAB members. This was a source of empowerment for the group. As she said:

“Well, for me, I got my own study. Had it not been a member of this board, I would not have had access to the type of consultations that I received. It made a real difference in what I’ve been able to do”.

Strategies for Improving the Consultations Provided by CEAB Members

Focus group participants described their initial experiences joining the CEAB and potential approaches for including, training and onboarding new members, strategies for preparing for consultation meetings, and the desire for ongoing evaluation of their services. Each of these subthemes are discussed below.

In discussing their role in providing consultation, CEAB members discussed that it would be good to include more

“people who are in communities that are commonly targeted. It would be great to have those residents here, to hear how their communities are going to be asked to do something. They can give firsthand—and like you say, we all represent some of those communities, but in a different way.”

Onboarding and training of new members was also seen as critically important to the functioning of the CEAB. Many members reflected on their early experiences joining the board and the uncertainty they felt about their new roles and responsibilities. Although orientation sessions were offered, participants identified the specific types of information that could be included in future orientation sessions for new members to help them fulfill their expected roles. Many participants described that they would have benefitted from exposure to some of the most frequently encountered concepts or terms that are used by researchers. Some felt this would be best achieved by offering a brief “Research 101” handout with terminology defined and providing a mentor within the group to help orient new members. Other members were most interested in knowing about the structure and mechanics of an actual consultation and would have benefitted from either participating in a mock review or observing a few consultations before they were expected to contribute. As one CEAB member said:

“Coming in cold, and not really having—because even though I was here, I never really participated like this. I kinda had a sense, but I didn’t really know what the expectation would be of me, personally. What can I bring? What can I share? What could I possibly add to the conversation?”

These suggestions have been incorporated into a new member orientation session as well as continuing education for members as new concepts and issues arise in consultations (i.e., precision medicine, tissue banking).

CEAB members noted that given the relatively short time allotted for each consultation (45 min) researchers need to be concise in providing information to the board in order to maximize the relevance of consultation input. Specifically, members suggested a standardized template for summarizing the goals and objectives of the study and specific questions for the board to be sent to CEAB members to review before the consultation meeting. In addition, they recommended a similar standardized format for researchers’ PowerPoint presentations, highlighting the problem, key research questions, study methods, target population, and specific consultation questions.

Every investigator that attends a CEAB consultation completes an evaluation, and overall evaluation of the services provided has been very high [10]. However, leadership had not been sharing this information with CEAB members but rather using it for their own evaluation purposes and reporting to NIH. During the focus group, CEAB members discussed that they would like to hear how investigators evaluated the consultations in order to refine the feedback they are providing and to determine whether additional types of expertise should be sought for inclusion on the advisory board. As they stated:

“It would be nice to have some type of evaluation of our service, whether it’s useful or whether it’s not. If there some other way that we can improve it would be nice to know that. Otherwise, we’re just hitting, taking aim in the dark.”

“Well, sometime I feel I talk a lot here. I’m not sure how often if its effective”.

Finally, a concern was raised and echoed by several members about the extent to which the input of the CEAB was being incorporated into the consulting investigator’s research planning and execution:

“That’s just a concern that I’ve had. This wealth of information and contributions that’s made by this body, is it actually being incorporated into the work that the investigators have asked for?”

Discussion

Patient and community engagement are now understood as cornerstones of translational research and should be integrated into all phases of the translational spectrum [1]. The convening of community advisory boards to provide input on community-engaged research has been shown to improve the success and quality of a variety of research protocols [15-17]. Here, we demonstrate that the formation of a CEAB not only provides opportunities to build community engagement capacity among CTSA researchers and but also among participating community members. In addition to the service that the CEAB provides to UIC clinical and translational investigators, CEAB participation also builds capacity among members and their communities. For example, members indicated enhancement in personal knowledge, information relevant to their community or organizational settings, and the creation of new or expanded networks. The linkages that formed among CEAB board members illustrate the ripple effect of the CEAB program to capacity building for both UIC investigators and board members.

The findings of this mixed-methods study of a long-standing CEAB serving the needs of CTSA-affiliated researchers have implications for CTSAs that want to establish or enhance community engagement services for researchers and community partners. Diversity of perspectives is a strength of the UIC CCTS CEAB. From its inception, the CEAB has included key stakeholders from religious groups, community-based organizations, research staff with significant community engagement expertise, members of specific health-focused organizations, and community advocates. This range of expertise was seen as central to the tasks being asked of the CEAB members. However, some members felt that while they were able to advise researchers on issues related to the community, the “conversation would be richer” if more lay community members were involved, especially in terms of educating researchers about issues facing underserved communities. Finding and retaining such individuals can be difficult, and more information is needed regarding best practices [18-20].

Ongoing evaluation of the needs and experiences of CEAB members is also recommended. Annual satisfaction surveys of board members and focus group sessions allow CEAB members to provide feedback on their experiences and to ensure that potential improvement to board communication processes and education are identified and implemented. Several suggestions for improving the consultation preparation and experiences of CEAB members were identified as part of our focus group interviews. Key among those was the establishment of systematic procedures for onboarding and training of new members. Although one of the strengths of community advisory boards is that they understand the perspectives and needs of community members [21], the nonacademic members of our CEAB were interested in obtaining more education and training in the key principles associated with the conduct of health-related research. Specifically, many participants described that they would have benefitted from earlier exposure to some of the most frequently encountered concepts or terms that are used by researchers. It is important to note that the CEAB boards differ in their goals and composition compared with traditional community advisory boards. Our CEAB members are stakeholders with varying levels of research experience and serve as part of a formal CCTS consultation service. Many of our lay community members were specifically recruited because of their prior experiences with community advisory boards. Despite this prior experience, several CEAB members reflected on their early experiences joining the board and the uncertainty they felt about their new roles and responsibilities. Nonacademic or lay CEAB members voiced a desire to have orientation sessions and continuing education that would allow them to have the ability to be “peer members” of the CEAB as opposed to “community members.” Suggestions from the CEAB membership that have been implemented, and which are all designed to assist the membership in performing their roles and responsibilities, include new member orientation sessions, a member mentoring program, plans for future educational programming, and new organizational items such as notebooks, action pages, and consultation abstracts. These new board development ideas and the involvement from the membership will serve to strengthen future board consultation services and should be implemented as part of newly developed CEAB board trainings.

Limitations

The research study included a small sample of the target population from a single geographical location, thus additional studies are required. Although generalizability is not a goal of qualitative research, nonparticipants may have other opinions or experiences germane to understanding the experience of CEAB members.

Conclusions

Community engagement is essential to the successful translation of interventions and other healthcare advances into community settings. More research should be conducted to evaluate and improve the experiences of community advisory board members as they contribute to engaging communities and advising researchers at all (and especially the early) stages of the translational continuum [17].