The Delivery Science Rapid Analysis Program: A Research and Operational Partnership at Kaiser Permanente Northern California.

INTRODUCTION: Health care researchers and delivery system leaders share a common mission to improve health care quality and outcomes. However, differing timelines, incentives, and priorities are often a barrier to research and operational partnerships. In addition, few funding mechanisms exist to generate and solicit analytic questions that are of interest to both research and to operations within health care settings, and provide rapid results that can be used to improve practice and outcomes.
METHODS: The Delivery Science Rapid Analysis Program (RAP) was formed in 2013 within the Kaiser Permanente Northern California Division of Research, sponsored by The Permanente Medical Group. A Steering Committee consisting of both researchers and clinical leaders solicits and reviews proposals for rapid analytic projects that will use existing data and are feasible within 6 months and with up to $30,000 (approximately 25-50% full-time equivalent) of programmer/analyst effort. Review criteria include the importance of the analytic question for both research and operations, and the potential for the project to have a significant impact on care delivery within 12 months of completion.
RESULTS: The RAP funded 5 research and operational analytic projects between 2013 and 2017. These projects spanned a wide range of clinical areas, including lupus, pediatric obesity, diabetes, e-cigarette use, and hypertension. The hypertension RAP project, which focused on optimizing thiazide prescribing in Black/African-American patients with hypertension, led to new insights that inform an equitable care quality metric designed to reduce blood pressure control disparities throughout the KPNC region.
CONCLUSIONS: Programs that actively encourage research and operational analytic partnerships have significant potential to improve care, enhance research collaborations, and contribute to the building and sustaining of learning health care systems.

INTRODUCTION

Health care researchers and delivery system leaders share a common mission to improve health care quality and outcomes. A key goal for health services researchers, particularly for those embedded within learning health systems, is to improve health care by addressing delivery science research questions that are strategically important to health care organizations. Clinical and operational leaders also routinely work to improve care delivery through ongoing organizational quality improvement initiatives. However, despite these common interests, efforts to form research and operational partnerships may be hampered by a number of factors, including disparate priorities, timelines, and funding models.1 Research funding models that are specifically designed to foster these partnerships and facilitate the translation of care innovations and knowledge into practice are still developing.2 Traditional investigator‐driven approaches to research may fail to adequately engage clinical and operational stakeholders, or create interventions that are not sustainable in real‐world environments.1 Research that is focused on understanding and addressing priorities within health care delivery systems may be more likely to lead to evidence‐based innovations in processes and outcomes of care that can be diffused on a large scale.3 Participatory research models that actively engage clinical and operational stakeholders in designing and conducting research have significant potential to increase the likelihood that research questions will be relevant to health care systems, and that research findings are translated quickly into systematic action.4 Despite this need, there are no published accounts of research funding mechanisms specifically designed to encourage research and operational partnerships and put the findings of stakeholder‐engaged research into action.

The Delivery Science Rapid Analysis Program (RAP), created in 2013, is an innovative funding mechanism designed to provide analytic and scientific resources to address rapidly researchable analytic questions with high value to both quality improvement leaders and researchers within Kaiser Permanente Northern California (KPNC). The KPNC Division of Research (DOR), which is like an academic department embedded within the health care delivery organization, creates generalizable knowledge to improve health and health care nationally. The DOR is primarily externally funded, with almost 80% of its 2016 annual budget of $87 million coming from the NIH and other external funders, mainly via competitive grants such as R01s. The Permanente Medical Group (TPMG) contracts with Kaiser Foundation Health Plan, the insurer organization within Kaiser Permanente, and exclusively provides physician services to KPNC's members. Through its Quality and Operations Support (QOS) Department, TPMG routinely works to address questions on how to improve care delivery in its mission to support quality improvement initiatives throughout the KPNC region. Many of these questions are of mutual interest to DOR, TPMG, and QOS and often can be answered quickly through targeted analysis of EPIC integrated electronic health record data. Although DOR and other TPMG groups have a long history of collaboration, efforts to work together have often been hampered by challenges common to research and operations partnerships. In addition, a lack of designated programming resources to conduct strategic analytics is a significant barrier to these partnerships. In response to these challenges, DOR investigators and TMPG clinical leaders created a new funding mechanism, the RAP, specifically designed to address these issues. Our hypotheses were that a program to proactively identify and rapidly address important analytic questions would advance the work of both DOR and TPMG/QOS, deepen the partnership between research and operations, and improve care quality. Furthermore, we believed the processes developed and lessons learned in implementing such a program had the potential to expand the evidence base on how to improve research and operational partnerships in general.

The purpose of this paper is to describe the Delivery Science Rapid Analysis Program as a potential framework for stimulating evidence generation that directly informs clinical decision making and practice. We discuss the types of proposals the program seeks to support and describe the mechanisms of how RAP proposals are solicited and funded. In addition, this paper provides a vignette on one of the first RAP funded projects to address health disparities in hypertension care and outcomes, and the project's region‐wide impact on quality improvement and quality measurement. Finally, we discuss our challenges, limitations, and lessons learned and the key elements of the program we believe are necessary to translate this type of funding and partnership mechanism to other health care delivery systems.

METHODS

Conceptual framework

The Delivery Science Rapid Analysis Program (RAP) supports the research mission of DOR and the quality improvement goals of TPMG and QOS by soliciting and funding projects that address “rapidly researchable” questions of strategic interest to both research and operations. In addition to funding research analysis, the RAP works to disseminate individual project findings to stakeholders throughout KPNC and to use these findings to further the research and quality improvement objectives of the health care system.

The underlying premise for the Delivery Science Rapid Analysis Program (RAP) is that there is a critical intersection in 3 areas of health care quality improvement: strategic questions that the health care delivery system clinical and operational leaders are interested in answering, scientific questions that investigators are interested in pursuing, and questions that are both researchable and feasible to address in a relatively short amount of time (see Figure 1). The RAP specifically targets the intersection of these 3 domains and focuses on identifying questions that meet the following criteria:

Figure 1

Rapid Analysis Program target research questions

Strategically relevant to TPMG (eg, provides information that helps meet quality goals)

Covers a topic area and/or analytic method where a DOR scientist has relevant expertise

Can be expressed as 1 to 2 research aims with testable hypotheses

Can be primarily addressed using existing data sources (eg, electronic health record data)

Addressable with analyses requiring ~6 months to complete

Will lead to a research product (eg, publication, abstract, pilot data for grant proposal)

Projects are specifically excluded if they require:

Significant data outside of what is available through existing data sources

Analysis that cannot be completed in ~6 months

Topic area or analytic expertise that is not available at DOR or QOS

RAP structure

The goal of the RAP is to identify and complete 2 to 4 projects annually that are of high strategic value to both DOR and QOS. The RAP is led by a steering committee with representatives from DOR, TPMG, and QOS and seeks to solicit research questions from a broad constituency across these groups. Chosen projects have both DOR and operational leads that work together to define the research questions and create the analysis plan. Once projects are identified, the designated programmer/analyst at the DOR works with the lead DOR investigator on the project and the TPMG/QOS project leads to generate results and disseminate findings to project stakeholders. The RAP project manager works to ensure that each project receives timely institutional review board (IRB) approval as necessary, and that all deadlines are met. Each proposed project can submit a budget of up to $30,000 in total costs (without indirects). This amount is intended to primarily cover the costs of a programmer/analyst part time, or 25% to 50% full‐time equivalent depending on salary. A small portion (5% full‐time equivalent maximum) for the DOR scientist participating in the project can also be requested in the RAP budget.

Deliverables for each individual project include a final report for stakeholders and at least 1 specific research product: examples include a peer‐reviewed publication, a conference abstract, or pilot data for a future collaborative grant‐funding opportunity. Metrics of success for the RAP as a whole include whether results inform quality improvement efforts, number of research products generated, increased sense of partnership between DOR and QOS, and leadership satisfaction with the program.

Process for receiving and selecting RAP project concepts

The RAP Steering Committee with membership from both DOR and TPMG/QOS issues calls for proposals to assess project ideas for consideration. All proposals are submitted to the program's project manager, who then works with the RAP lead (Dr. Schmittdiel) to distribute the ideas to the Committee for discussion and review. Once a concept is reviewed by the committee, a vote decides whether it should go forward as an approved RAP research project based on the selection criteria outlined in Section 2.1. The committee also has the opportunity to suggest clarifications or modest revisions to the proposals to help them adhere as closely as possible to the program's objectives.

Once a project concept is approved, the DOR investigator serving as the individual project lead works with the RAP manager to obtain IRB approval. If the project is determined to be a quality improvement initiative not requiring IRB approval, this is noted.

Generating RAP project ideas

DOR investigators and clinical and operational leaders are invited to submit proposals for consideration by the RAP. To ensure that all research concepts are of interest to both DOR and QOS, a submission has to have both a designated DOR investigator who serves as the individual project lead, and a TPMG/QOS clinical or operational lead. A brief (1‐ to 2‐page) concept piece describing the research questions, methods, and timeline, as well as the research product resulting from the collaboration, is prepared by the DOR individual project lead in partnership with their TPMG/QOS collaborators.

The RAP aims to encourage broad participation in generating ideas from a cross section of topic areas across the departments. Scientists at DOR are invited to generate ideas through their department‐level meetings and through DOR‐wide investigator/biostatistician meetings, which also provide a forum for discussing the RAP's goals and processes. The associate executive directors at TPMG and the RAP Steering Committee members solicit input from operational leaders and stakeholders regarding which questions should be considered for the program.

At the end of the RAP's first year, the Steering Committee saw a need to do more to publicize the program and encourage applications that addressed a wide range of clinical and operational topics. During the second year of the initial RAP funding period, we worked closely with the TPMG and QOS representatives of the steering committee to generate a list of sample priority topics to be distributed along with the funding cycle announcements, and efforts were made to distribute both the “request for proposals” (RFP) and the sample list more broadly.

RESULTS

Impact of the RAP

The RAP was initially funded for a 2‐year period from 10/1/2013 to 9/30/2015. During that time, 4 RFP cycles were administered by the RAP Steering Committee, resulting in the following 4 awards:

Finding Opportunities to Reduce Disparities in Blood Pressure Control: A Focus on Adequate Dosing of Thiazide Diuretics

Pediatric Obesity and Weight Tracking in the Get Healthy Action Plan (GHAP) Population

Comparative Effectiveness of Therapies to Prevent Nephritis in Systemic Lupus Erythematosus (SLE)

Identifying Electronic Nicotine Delivery Systems (ENDS) Use References Within Free‐Text Data Fields in the Electronic Health Record.

The pediatric obesity project provided critical information to the pediatric leadership team on redesigning their current strategies for obese pediatric patients and resulted in a manuscript.5 The lupus project's results are being used to inform strategies to address care gaps in SLE and have prompted new strategies to increase telephone contact between patients and their rheumatologists. The RAP‐sponsored work on identifying ENDS use is being used to identify and target interventions toward populations vulnerable to this form of nicotine use.

The following case study provides a description of our first RAP funded project and outlines the impact the project has had on regional quality improvement and measurement.

“Finding Opportunities to Reduce Disparities in Blood Pressure Control: A Focus on Adequate Dosing of Thiazide Diuretics”

Despite notable improvements in cardiovascular disease management and outcomes in recent years, disparities in morbidity and mortality persist in the United States.6, 7, 8, 9 An important challenge for health care systems is the identification of modifiable determinants of suboptimal blood pressure management among Blacks/African Americans.10, 11

Thiazide diuretics may be especially effective in the control of hypertension and in the prevention of strokes among Black/African American patients.12, 13, 14 However, strategies for promoting the use of these therapies at clinically effective doses in clinical practice are needed.

Using Richmond medical center as a test case, this 6‐month collaborative project between DOR investigators, QOS leaders, and TPMG clinicians assessed the feasibility of a quality improvement project aimed at reducing disparities in blood pressure control through targeted clinical intensification with thiazide diuretics. The specific aims of the project were (1) to examine thiazide diuretic dosing in Richmond by race and ethnicity using univariate and bivariate analysis and (2) to evaluate the feasibility of integrating data on thiazide use and dosing among black patients into routine team quality reports to facilitate clinically appropriate treatment intensification.

We created an algorithm to identify patients eligible for clinical intensification of thiazide diuretics. Baseline data revealed that 90% of the subset of Black/African American patients with uncontrolled hypertension with no documented allergy to thiazide therapy were not being prescribed a thiazide or were being prescribed a submaximal dose of a thiazide. Over the course of 3 months, Black/African American patients in poor control at Richmond were targeted for clinical intensification using thiazide diuretics. We compared rates of thiazide diuretic treatment, defined as any use (non–dose specific) and whether patients already prescribed a thiazide diuretic were at the maximal dose, at the end of this 3‐month pilot intervention to rates at baseline.

The intervention was conducted in stages. First, the RAP programmer on the project (Ms. Dyer) used the algorithm developed by the research team to identify eligible patients. These patients were then called by the medical assistant at Richmond to encourage a repeat blood pressure test. Patients who retested and whose updated blood pressure remained high were then counseled by the pharmacist about starting thiazides or dose escalation.

The Richmond team was successful in obtaining new updated blood pressure measurements for two thirds (n = 332) of the 536 patients targeted for the pilot intervention. Notably, 63% (n = 210 out of 332) pilot participants with a new blood pressure were found to be “in control.” However, 78% (n = 420 out of 536) of pilot participants had no change in thiazide diuretic dosing during the follow‐up period. Of those not using thiazide diuretics at baseline (n = 316), 11% started the medication by the end of follow‐up. Thiazide diuretic dosing was increased for 7% of the 220 patients who were at a submaximal dose at baseline.

To evaluate the feasibility of integrating thiazide use and dose data into routine reports and care plans, the RAP team conducted a debriefing meeting in January 2015 with the Richmond implementation team to discuss the program. Most notably, the care team reported that the introduction and use of the thiazide data provided by the RAP project did not require a lot of additional work and integrated well into the usual work flow.

The implementation team identified several factors that may impede clinical intensification using thiazide diuretics, including provider barriers to clinical intensification, patient nonadherence and failure to follow up, lack of in‐person contact between pharmacists and patients, and challenges in reconciling information about medication regimens from different sources in real time. Opportunities for improving the potential impact of this quality improvement intervention may include changes to existing work flow as it relates to the blood pressure clinic, the addition of medication adherence measures to the population care tools, and additional provider education regarding options for diuretic prescribing.

This first project from the RAP was so successful in proving both the existence of opportunities to increase thiazide dosing, and the feasibility of providing information on inadequate thiazide dosing for Black/African American patients to outreach teams, that it informed strategies pertaining to a region‐wide equitable care quality metric for reducing blood pressure disparities. This metric assesses whether members eligible for thiazide treatment are receiving thiazide treatment and, if thiazide is received, whether it is prescribed at a clinically effective dose. To facilitate this measurement, region‐wide enhancements to the KPNC population care data systems were introduced for use at each facility. Dr. Adams' TPMG partner in the RAP project, Hypertension Clinical Lead (Dr. Young), was instrumental in this adoption process. Drs. Adams and Young are currently conducting an evaluation of the impact of this new quality metric on disparities and blood pressure outcomes in KPNC.

DISCUSSION

The RAP is a new, specific mechanism to actively identify rapid‐cycle analytic questions of strategic interests to both research and operations groups within KPNC, stimulate collaborations between researchers and clinical leaders, and provide direct funding to researchers and programmers to conduct this work. In addition, as discussed in our case study above, the RAP program has facilitated region‐wide quality improvement strategies that have had an organization‐wide impact on efforts to improve hypertension care for Black/African Americans patients.

Challenges, limitations, and lessons learned

Our main challenge in implementing the RAP program was letting potential applicants know about the new program and encouraging a range of applications across clinical topics and organizational priorities. While informing DOR investigators about the program was relatively straightforward, we realized that knowing who to reach out to on the operations side, and how to facilitate new partnerships between researchers and clinical leaders in areas where there had been less collaboration in the past, required additional effort. As noted above, creating a list of potential priority topics and expanding the distribution of the RFP helped to address this issue. We also found that when Steering Committee members began personally reaching out to their own networks of leaders and researchers across KPNC, it was very helpful for increasing the number and variety of applications received from across the organization.

Although we believe that the RAP has been successful at engaging researchers and clinical leaders and addressing patient‐level processes and outcomes of care, it is important to point out that the RAP was not designed to specifically incorporate patient perspectives in prioritizing and selecting RAP projects. The need to further involve patients in all aspects of research,15 including potentially these types of rapid analytic projects designed to inform quality improvement, is something that could be considered in future iterations of the RAP program.

The RAP is an innovative, flexible strategy that has been successful in achieving its goals furthering research and operational partnerships and increasing the speed of research and quality improvement data analysis within KPNC. A specific goal in creating the RAP was not only to increase the level of collaboration between DOR with QOS and other clinical quality leaders within TPMG, but also to develop generalizable knowledge on improving health care research and operational partnerships nationally. Based on our experience in designing and implementing the program, we believe there are 3 key elements that are required to spread models like the RAP to other systems:

Dedicated programming resources and expertise for rapid cycle projects

Although many potential quality improvement collaborations between researchers and operational leaders may not require extensive funding or staff, a huge barrier to conducting these projects can be the lack of analytic time or “bandwidth” to conduct the work. By setting aside modest amounts of resources and personnel time, the RAP has been able to address a wide range of questions of importance to many stakeholders throughout the organization. Other organizations seeking to adopt a similar model should be encouraged by the fact that the targeted investment in the RAP program cost a fraction of many quality improvement initiatives.

Program leadership from operations and research

The RAP Steering Committee that reviewed and selected projects for funding consisted of researchers, clinical leadership, and operational leads; DOR scientists make up a minority of the committee's members. By ensuring that organizational leaders who understood the KPNC health care objectives and strategies were a key part of the decision‐making process, the RAP has been able to successfully solicit, select, and fund high priority rapid analysis projects. In addition, this partnership at the steering committee level served to model the research and operational collaborations that the RAP was designed to encourage, and helped to build trust and reputation across a number of clinical and scientific areas that were of strategic priority to the organization. We recommend that health care delivery systems interested in developing a RAP‐type program begin with this high level of collaboration in designing and implementing the program itself.

Adhere to the principles of learning health system

Learning health systems seek to leverage their data systems, clinical knowledge, and experience toward improving health care for all of their patients.16 A critical component of the success of the RAP is that it is embedded within a care delivery system that is committed to these principles, and eager to translate findings into action. In order for a program like RAP to succeed, an organization has to be invested in learning from project findings and in following through on both implementing and de‐implementing strategies based on the results. We believe rapid analytic partnerships can not only thrive within learning health care systems but also increase the capacity of the system to learn and grow.

CONCLUSION

Programs that actively encourage research and operational analytic partnerships have significant potential to improve care, to further research collaborations, and to contribute to the building and sustaining of learning health care systems. Health care delivery systems should consider adopting such programs as a way to improve health care nationally.

NOTE ADDED IN PROOF

We would also like to gratefully acknowledge the contributions of Dr. Philip Madvig (Associate Executive Director, The Permanente Medical Group) for his guidance and support for the program.