Making the Random the Usual: Appreciative Inquiry/Boot Camp Translation-Developing Community-Oriented Evidence That Matters.

Background: The evidence underlying clinical guidelines arising from typical scientific inquiry may not always match the needs and concerns of local communities. Our High Plains Research Network Community Advisory Council (HPRN CAC) identified a need for evidence regarding how to assist members of their community suffering from mental health issues to recognize their need for help and then obtain access to mental health care. The lack of evidence led our academic team to pursue linking Appreciative Inquiry with Boot Camp Translation (AI/BCT). This article describes the development and testing of this linked method. Method: We worked with the HPRN CAC and other communities affiliated with the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP) practice-based research networks to identify 5 topics for testing of AI/BCT. For each topic, we developed AI interview recruitment strategies and guides with our community partners, conducted interviews, and analyzed the interview data. Resulting themes for each topic were then utilized by 5 groups with the BCT method to develop community relevant messages and materials to communicate the evidence generated in each AI set of interviews. At each stage for each topic, notes on adaptations, barriers, and successes were recorded by the project team.
Results: Each topic successfully led to generation of community specific evidence, messages, and materials for dissemination using the AI/BCT method. Beyond this, 5 important lessons emerged regarding the AI/BCT method: Researchers must (1) first ensure whether the topic is a good fit for AI, (2) maintain a focus on "what works" throughout all stages, (3) recruit one or more experienced qualitative analysts, (4) ensure adequate time and resources for the extensive AI/BCT process, and (5) present AI findings to BCT participants in the context of existing evidence and the local community and allow time for community partners to ask questions and request additional data analyses to be done. Conclusions: AI/BCT represents an effective way of responding to a community's need for evidence around a specific topic where standard evidence and/or guidelines do not exist. AI/BCT is a method for turning the "random" successes of individuals into "usual" practice at a community level.

Background

The “translation” of health-related scientific discoveries into clinical guidelines and everyday clinical practice can take years or decades, and many discoveries may never make it into daily practice.[1,2] The advent of evidence-based guidelines in contrast to consensus-based guidelines has been a major advance.[3] Many of these discoveries and guidelines pertain to the wide spectrum of health conditions and ages encountered in primary care. However, discoveries and guidelines do not exist for all health conditions that are of priority to communities, and in some cases the evidence and guidelines might apply only to a limited group (such as certain ages, cultures, or ethnicities). Indeed, the original articles describing strategies for classifying evidence acknowledge the existence and need for observational evidence.[45] In these instances, a method is needed to develop community relevant evidence from patients and community members who have found their own pathways to success through perseverance or trial and error, instead of using an academic, top down, barrier-focused approach that ignores the resourcefulness, skills, and knowledge of patients and community assets.

We first became aware of these potential evidence limitations through the work of our long-standing partnership between the High Plains Research Network (HPRN) Community Advisory Council (CAC) and the University of Colorado Denver’s Department of Family Medicine (CU-DFM). The HPRN CAC had previously identified mental health as a priority issue in their region of eastern Colorado. To help focus their goals for patient-centered research and action, the group undertook a year-long process of learning about mental health from various experts who covered definitions and epidemiology of major conditions, integrated behavioral health, suicide prevention, and policy.

As explained by HPRN CAC member, Chris Bennett,

We began studying mental health in eastern Colorado by looking at lots of research to improve mental health outcomes. Most of it revolved around implementing evidence-based guidelines for specific treatments for specific illnesses or system-level changes. As each expert talked, we routinely kept coming back to, “That’s awesome work, but not quite what we think our community needs.” We realized that we wanted to tackle the problem of “How do we help people overcome the barriers to access the mental healthcare system and support before they have a mental health crisis?” This was the question that wasn’t sufficiently answered in the research or evidence-based guidelines.

Maret Felzien, another member of the HPRN CAC, added,

The existing guidelines and information didn’t introduce a way forward for us. We needed a way for us to work on improving and preventing mental health outcomes from a locally informed base. We needed a round peg for a round hole versus a square peg in a round hole.

Inspired by the experience of the HPRN C.A.C., the CU-DFM’s SNOCAP (State Networks Of Colorado Ambulatory Practices & Partners) practice-based research networks’ Community and Patient Advisory Councils and 2040 Partners for Health (2040) worked to fill this methods gap. We intended to address the methods gap for sourcing and translating community and practice-based evidence by identifying and testing the use of the new method described in this report. We call this method Appreciative Inquiry/Boot Camp Translation (AI/BCT). AI is a method that uncovers and learns from successful occurrences and conditions that foster success.[6-8] In AI, participants are interviewed about their successes, and the qualitative analysis of these successes seeks to identify common themes that play an important role in the participants’ successes. BCT is a participatory method developed by the partnership between the HPRN CAC and CU-DFM to translate medical jargon into messages and materials that are locally relevant, actionable, and attainable. BCT has proven itself with multiple sets of guidelines and evidence as an effective, evidence-based method for translation of these into locally relevant messages and materials.[9-13] The resulting materials and their dissemination have impacted colorectal cancer screening rates, asthma exacerbations, and delivery of self-management support among others.[12,14,15] The joining of these methods into a new combined method called AI/BCT is designed to access local patient and community members’ successes and translate those random successes into evidence and interventions that will relevant and become more usual within a community. We obtained Methods Award funding from the Patient-Centered Research Outcomes Institute (ME-1303-5843) to pilot the AI/BCT method and identify aspects and lessons learned that would facilitate use of the method by other academic/patient partnerships.

The purpose of this article is to primarily describe the rationale for AI/BCT, while briefly describing the projects we used to pilot and evaluate the method with a discussion of important lessons learned from this initial formal evaluation of the AI/BCT method. We will describe the 5 topic areas for which “on the shelf” evidence was lacking, our AI/BCT approach to address the gap in evidence for each, an overview of the products that emerged from each, and the key learnings from our study. For those interested in a full description of our methods and their formative evaluation, we will present those in a subsequent paper.

Methods

We began our investigation of AI/BCT by working with our community partners to identify topic areas with which we could implement and evaluate AI/BCT. To identify the topics for AI/BCT projects, we used a Delphi method approach to reach consensus on the important health issues of priority within each community.[16] We supplemented this discussion with a review of the academic literature on the community-identified topics to determine need, feasibility, and current evidence base. This yielded the 5 topic areas in Table 1.

Table 1.

Health Topics Identified by Community Partners for Appreciative Inquiry/Boot Camp Translation (AI/BCT).

Topic	Community	Partner Organization	Evidence Gaps
Accessing mental health care and support	Denver metropolitan area	2040 Partners for Health	How to shorten the time to sustained, successful access to mental health care
Accessing mental health care and support	Rural eastern Colorado	High Plains Research Network Community Advisory Council	How to facilitate access to and the entry process into mental health treatment and support
Chronic pain management	Denver metropolitan area	Colorado Research Network Patient Advisory Council	How to successfully manage chronic pain and maintain function
Primary care practice transformation	Denver metropolitan area	University of Colorado practice transformation team	How to successfully implement the myriad of resources surrounding practice transformation
Diagnosis and treatment of sleep apnea	Denver metropolitan area	2040 Partners for Health	How to shorten the time to diagnosis and treatment of sleep apnea

For each individual topic we worked with the associated community partners in Table 1 to identify the specific evidence gaps, design recruitment materials, design an AI interview guide, and recruit eligible participants. The study protocol was approved by the Colorado Multiple Institutional Review Board (COMIRB). As approved by COMIRB, verbal informed consent was obtained from each AI interview participant.

Successful access to mental health care provides an illustrative example of a topic, identified by our community partners, for which we found an evidence gap. Mental health care access is frequently conceptualized as being related to an individual’s ability to pay for services or having direct access to a mental health care provider. However, our communities were instead concerned with aspects of access related an individual’s recognition of the need for help and their subsequent and sustained access to help in the form of professional care. Think of this as crossing the boundary between individual recognition of symptoms and seeking and receipt of care. Indeed, the challenges associated with crossing this boundary were amply illustrated by the stories told by our interviewees. Successful access, diagnosis, and treatment also underlay the topic of sleep apnea. For the topic of chronic pain management, our community partners identified that evidence was lacking as to how individuals successfully gain access to, and navigate through, the multitude of potential treatment modalities. Finally, there was a desire to uncover the elements of successful primary care practice transformation to patient-centered care for practices that have experience in support programs but may still exhibit resistance to transformation.

The AI interviews were conducted by our study team staff, proceeding from the key question for each topic as shown in Table 2. The majority of the interviews were with one participant, and all interviews were recorded and transcribed for qualitative analysis to identify key themes. The qualitative analysis for each topic was led by 2 of our team members (JSH and TLH) in a grounded theory approach, using a combination of editing and immersion-crystallization styles.[17,18] This consisted of iterative review of codes and themes by the entire study team, including members of our community partnerships.

Table 2.

Appreciative Inquiry/Boot Camp Translation (AI/BCT) Topics, AI Key Questions, Resulting Themes and BCT Products.

Topic	AI Key Question	AI Interview Themes	BCT Products
Accessing Mental Health Care (Denver)	What happened that helped you or someone you know get a service or some type of care for mental health?	Importance of an advocateAcceptanceInsurance implications	Posters and flyers carrying the message “Somebody Cares . . . Talk.”
Accessing Mental Health Care (Rural)	Think about when you successfully accessed mental health care. Tell us that story. What worked? Who helped? What resource? What happened next?	AdvocatesImportance of the Other Person (initiates help, uses informal approach, found in safe venues, is a connector)“Handle it on your own” approach to mental health	● Messages around community responsibility/permission to ask; “Be the Other Person” theme● Changing Our Mental and Emotional Trajectory (COMET) community-based intervention that uses a new 5-point conversational version of the Patient Health Questionnaire (PHQ) to address mental and emotional health needs before crisis; aims to foster social connectedness and conversations to support mental health
Chronic Pain Management	What helped you live with chronic pain so that the pain—and the management of it—does not dominate your everyday life?	ContextDiagnosisTreatmentOutcome	● Messages: (1) Create goals that are more important to you than your pain experiences, (2) You can’t always go it alone; find people who will help you be who you were before you pain, and (3) Try, try, try; you have permission to try more than one strategy.● Materials: Chronic Pain Management Roadmap to Success and cover letter paper, both to be distributed to primary care practices and community settings
Primary Care Practice Transformation	What led up to the practice starting patient-centeres medical home (PCMH)? What facilitated your PCMH work? What was it about the practice or situation that helped?	TrustAttitudesKey role (usually not someone in a “traditional” leadership position)Workflow changes/protocolsPractice and individual qualities	● Main messages: “Change isn’t coming. Change is here;” “That was then. This is now;” “Engage all team members;” and “Thinking of retiring? What kind of practice will you leave?”● Materials: brochure, PowerPoint presentation, short storyboard video to reach near-retirement providers and precontemplative practices; expand messages on sense of duty to patients, trust, key roles, openness to change, and timeliness.
Diagnosis and Treatment of Sleep Apnea	Tell us the story about when you were properly diagnosed with sleep apnea and successfully treated. (Reframe if needed.) What is the story about how you were diagnosed and your successful treatment?	SymptomsMultiple health care providersTreatmentRealizing seriousnessPositive outcomes	● Key messages: Sleep is an important part of health and well-being. Poor sleep can lead to numerous health issues and poor quality of life. If you are not sleeping well, then you ought to talk to your doctor.● Materials: Brochure and poster to help guide conversations with care provider or as stand-alone products.

Following the AI analysis, the BCT phase began for each topic. Using a typical BCT process, we recruited a set of 11 to 14 BCT participants for each topic from the corresponding partner community. Two study team members facilitated each BCT with additional participation by one of the qualitative leads, who served in the role of an “interview data content expert,” much as a topic content expert participates in the usual BCT process. Beyond this variation for each topic, a typical BCT process was followed, and each topic had results of community relevant messages and materials.

Figure 1 summarizes the AI/BCT process. A detailed description of our methods will be presented in a subsequent paper.

Figure 1.

Overview of the Appreciative Inquiry/Boot Camp Translation (AI/BCT) process.

Results

We found AI/BCT is a useful method to create locally relevant evidence for addressing important health issues where little, if any, evidence currently exists. We successfully conducted AI/BCT processes around our five community identified topics of interest. Table 2 summarizes for each topic the key AI interview question, the main themes that emerged from the AI interviews, and the final BCT messages and sample products. Each iteration of our AI/BCT process yielded AI findings and themes that were then translated by a BCT process into messages and materials that can be used by the community to address the health topic or concern.

Beyond the results of the messages and materials produced by each AI/BCT, we developed several important lessons about the application of the AI/BCT method: Researchers must (1) first ensure whether the topic is a good fit for AI, (2) maintain a focus on “what works” throughout all stages, (3) recruit one or more experienced qualitative analysts, (4) ensure adequate time and resources for the extensive AI/BCT process, and (5) present AI findings to BCT participants in the context of existing evidence and the local community and allow time for community partners to ask questions and request additional data analyses to be done (Figure 2) .

Figure 2.

Key, practical learnings for Appreciative Inquiry/Boot Camp Translation (AI/BCT) implementation.

First, selection of topics suitable for AI is important; AI requires stories of success. Identifying priority health topics in communities should begin with an open-ended inquiry, which is then focused with the help of academic partners. As lists are refined with the Delphi method,[16] feasibility of the topics becomes an important consideration. Does the community believe finding stories of success will be difficult? If so, the topic may not work. Does the topic have an existing significant evidence base that is relevant to the community partners? If so, community-academic partnerships should opt to use the traditional BCT process alone. If neither of these apply, AI/BCT could be a good fit for the topic chosen. Scope is important to consider as well. Examples of priority topics that are beyond the scope of AI/BCT include large community issues such as making health insurance accessible and addressing pharmaceutical costs. Topics that are typically a good fit for AI are locally relevant; in other words, they are common issues in the community and a priority need in the minds of community members; and feasible for a community-level intervention.

Another important step to determine if a topic is a good fit for AI is whether success can be clearly defined. For example, the AI/BCT on chronic pain management began with the broad topic of chronic pain. After discussion, the team specified a focus on the management of pain. With help from community partners, we chose to define success as managing chronic pain in a way that asks, “Are you able to do most of what you want to do on most days of the week?”

Second, a focus on “what worked” for AI interviews is important. Interviewers should be trained in the AI approach. Focusing on “what worked” during interviews can be challenging, especially to researchers trained in problem-focused approaches. Utilizing 2 co-interviewers for data collection efforts can help ensure that interviews are complete and capture elements of success. Within the AI data collection component, we also found that individual interviews delivered more in-depth and robust stories of success than small groups. Group interviews are not recommended as obtaining individual stories of success was more challenging in a group format than individual interviews.

Third, selecting one or more experienced qualitative analysts for the AI data is crucial. The entire AI/BCT research team does not need to be skilled in AI data analysis. However, the researcher leading the analysis of the AI data should have advanced qualitative analysis skills. Coding, identifying common themes, and organizing results require an analyst that remains in the “what worked” frame of mind. Further, identifying themes and organizing data for BCT require an analytic lens that comes from a solid understanding of both AI and BCT.

Fourth, ensure adequate time and team resources for the process. Researchers using AI/BCT should allow a minimum of four months to collect, code, analyze, and organize the AI interview data. The time requirements should be factored into budgeting on the academic side and made transparent for community partners as well to avoid expectations of more rapid results. The analyst works with a team, particularly in the early stages of reviewing transcripts and creating a coding system. This coding team should include at least one interviewer, if the analyst did not participate in interviews. Coding analyst teams should incorporate members with specific subject matter expertise to shed light on terms or procedures, if applicable. For example, the Practice Transformation AI/BCT coding team included a physician to provide insights and clarifications that only a practicing physician could offer.

Finally, take care in how the AI data is presented during the BCT stage. AI findings are incorporated into the BCT process as the new, locally generated evidence base for the selected topic. We tested and confirmed the value of a 2-part educational presentation at the BCT kick-off meeting, which is slightly altered from traditional BCT. First, a topic expert provides a presentation on what is known about the topic. This is followed by a presentation of the AI process and results, including themes of how successes in that particular community have been achieved. The BCT facilitation team and AI data analyst need to be able to articulate both the local relevance of the AI data and the scientific rigor used to collect and analyze the data. It is also important at this stage to allow the community partners to request additional data be analyzed for review and presentation at a later meeting. This came into play during the chronic pain management AI/BCT when BCT participants wanted to know more about why more women were interviewed than men—was this due to convenience sampling or that women were more often likely to tell their story?

Discussion

We conceived AI/BCT as a method to address health concerns that are of importance to communities, but for which evidence of how to address the topic is either lacking or a poor fit due to the unique circumstances of a particular community. The contextual relevance of evidence and solutions to community health concerns is critical. As discussed in the introduction, our experience with one of our communities’ concerns regarding access to mental health care led us to devise a method that could leverage isolated successes of community members to create approaches that could be applied across a community.

AI is just beginning to be applied to the health arena. However, much of the focus of this work is on the organization of health care.[7,8,19-24] We believed that AI had potential as a method to uncover local community members’ and patients’ solutions to their health concerns, in effect generating patient-centered evidence for what works. In our study we successfully used AI to generate stories of success from community members struggling with health concerns. These formed the basis of themes that were successfully translated by BCT participants into interventions that can be broadly used in their communities to address the health topics and concerns.

Key understandings that emerged from our study were the following: AI/BCT is not a method for any health topic. Where evidence exists for a particular health topic and simply needs translation into locally relevant messages and materials, the standard BCT process is very appropriate. For broader topics that may require policy solutions, AI/BCT is not a good fit. AI requires a focus on what works during interviews. Qualitative analysts must understand the AI approach and focus on successes. The AI part of the process adds significantly to the time and resources required. Finally, AI findings must be presented to BCT participants along with additional context of what is already known about the health topic. This helps weave together current knowledge and locally relevant evidence to address each topic. If a trained BCT facilitator is not available, we recommend interested academic/community partnerships obtain training. We hold trainings regularly for that purpose.

AI/BCT addresses a methods gap, providing a method for generating and translating locally relevant evidence of success related to a particular health issue. AI/BCT is a method for turning these “random” successes into “usual” practice at a community level. AI/BCT identifies and speeds up the translation of successful local discoveries into daily practice, reduce suffering, and improve the health of local communities. While our study was limited to implementation of the AI/BCT method across five topics in a single state, we believe our work will help to identify and accelerate the translation of local discovery into regular practice, further reduce unnecessary suffering, and make an overall positive impact on the improved health of local communities touched by this work. To benefit other communities wanting to embark on this work, we have conducted a training in the method and plan to further apply the method with our own community partners.