Evaluation of a learning collaborative to advance team-based care in Federally Qualified Health Centers.

Practising team-based primary care allows Federally Qualified Health Centers (FQHC) in the USA to be accredited as patient-centred medical homes, positioning them for value-based models of shared savings in healthcare costs. Team-based care (TBC) involves redesign of staff roles and care delivery processes to improve efficiency and effectiveness, which requires a systematic and supportive approach to practice change over time. Thirteen FQHC primary care teams participated in an 8-month learning collaborative with a goal of providing teams with the knowledge, skills and coaching support needed to advance TBC in their organisations. The primary aim was to evaluate self-reported changes in FQHC teams' assessment of their practice relative to key concepts of TBC. The secondary aim was to evaluate how teams used the collaborative to develop new skills to advance TBC, and the implementation, service and patient outcomes they achieved. Site visits were conducted with three teams 6 months postcollaborative.
Results: Two teams withdrew. The remaining teams embarked on 15 TBC improvement initiatives. Nine teams submitted a total of 11 playbooks to guide other staff in changes to their practice. Three teams reported improved efficiencies at the service level (screening and scheduling), and one improved outcomes in patients with diabetes. The nine teams that completed precollaborative and postcollaborative self-assessments reported improvements in their practice and in coach and team skills. Site visits revealed that actionable data were a barrier to improvement, coaching support from the collaborative was highly valued and FQHC leadership support was critical to improvement. Leadership investment in developing their primary care teams' quality improvement, coaching and data analytical skills can advance TBC in their organisations. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Team-based care (TBC) is one of the 10 building blocks of high-performing primary care, and has been shown to result in better health outcomes, higher patient satisfaction, decreased provider burnout and improved patient access.1–5 It clarifies roles and responsibilities of clinicians and staff, reduces duplication of tasks and poor communication, and standardises day-to-day processes such as scheduling and screening. Care teams provide patients with greater continuity of care and management of chronic conditions, such as diabetes.1–5 TBC is a component of the patient-centred medical homes (PCMH) programme in the USA.1 2 4 For Federally Qualified Health Centers (FQHC), which care for 29 million poor and underserved patients in the USA, accreditation as a PCMH positions them for the shift from fee for service to value-based ‘mixed models’ of shared savings, and case-based care.6–9

Problem

For FQHCs, advancing TBC requires restructuring and redesign of staff roles and responsibilities, as well as workflows, in order to improve efficiency and clinical effectiveness.10–13 Such a transition requires a systematic and supportive approach over time, as it is often difficult to change long-established patterns of practice. The approach described in this paper was a learning collaborative for TBC in FQHCs.

Available knowledge and rationale

The learning collaborative was adapted from the Institute for Healthcare Improvement’s Breakthrough Series, which are designed to kick-start organisations’ improvement efforts.14 15 Consistent with that model, our collaborative included knowledge about the central concepts of TBC, practice self-assessment tools relative to TBC for participants to identify areas for improvement, and development of quality improvement (QI) and data analytical skills to make those improvements.16–20 We also provided a structured coaching programme21 to guide the primary care teams in their redesign work. This type of coaching, also called facilitation, has been demonstrated to be key to the success of organisations’ efforts to improve practice.22 23

Aim

Our goal was to provide primary care teams with the knowledge, skills and support needed to advance TBC in their organisations, a process that would continue beyond the collaborative. The primary aim was to evaluate self-reported changes in FQHC teams’ assessment of their practice as a result of learning about and applying the central concepts of TBC. The secondary aim was to evaluate how teams used the collaborative to develop new skills to advance TBC, and the implementation, service and patient outcomes they achieved by the end of the collaborative.

Methods

Context: setting and participants

The TBC learning collaborative was supported by a National Cooperative Agreement (NCA) grant from the US Department of Health and Human Services Health Resources and Services Administration (HRSA) to Community Health Center, Inc (CHCI) and its Weitzman Institute (WI), the research and education division of CHCI. CHCI is a multisite FQHC and level III PCMH in Connecticut providing comprehensive care to over 100 000 medically underserved patients annually.

The opportunity to apply for learning collaborative participation was publicised in the Primary Care Digest, a weekly HRSA publication. Structured interviews were held with 20 applicant FQHCs. Table 1 describes the 13 FQHCs selected based on alignment of TBC with organisational goals, commitment that their teams would have the time and data to participate in the collaborative, and confirmation that teams could make decisions about practice change at the team level. Previous experience with QI skills was preferred but not required. Leadership in selected organisations acknowledged these expectations in writing.

Table 1

The 13 FQHCs that participated in the learning collaborative

Participating organisation	State	Practice sites (n)	Number of patients(2018 UDS data)	Role of coach in the organisation
Site 1	VA	19	43 303	Registered nurse risk management coordinator
Site 2	TX	4	15 445	Family nurse practitioner
Site 3	MA	8	26 635	Registered nurse quality improvement coordinator
Site 4	CA	15	2300	Clinic operations manager
Site 5	PA	5	21 833	Patient services centre manager
Site 6	MD	2	6582	Registered nurse
Site 7	VA	3	9360	Paediatric nurse practitioner and medical director
Site 8	MI	5	6519	Director of programmes
Site 9	NJ	9	28 735	Regional practice director
Site 10	CT	23	49 527	Practice coach
Site 11	MI	22	15 017	Operations director
Site 12	WA	3	29 568	Quality and accreditation coordinator
Site 13	CA	20	57 930	Physician assistant

FQHC, Federally Qualified Health Center; UDS, Uniform Data Set.

The 13 FQHCs committed a core and extended primary care team from a practice site within their health centre to participate in the collaborative. The core team consisted of the three to seven individuals who work together every day to care for a panel of patients, such as primary care providers, behavioural health specialist, nurses, medical assistants and non-clinical staff. The extended team consisted of clinicians and staff who work with multiple teams, such as a pharmacist or nutritionist. Each FQHC also designated a team member as the improvement coach to guide the team through the work of the collaborative, and to participate in the structured coaching programme. Coaches were not required to have previous experience.

Patient and public involvement

Patients were not involved in the development of the learning collaborative or its evaluation.

Description of the intervention

Table 2 describes the structure and content of the learning collaborative, which was implemented by the NCA team at CHCI/WI. We began with a 2-day in-person training for team coaches, followed by an 8-month programme of seven videoconference interactive learning sessions, weekly meetings of the FQHC teams at their sites and weekly sessions between FQHC improvement coaches and their NCA mentor-coaches (coach-mentor calls). Self-assessment tools allowed teams to measure their practice against best practices in TBC.

Table 2

Structure and content of learning collaborative on team-based care

Intervention	Description
Two-day in-person coaching and quality improvement (QI) boot camp	FQHC coach and at least one other team member attended the boot camp, which focused on the art of coaching, and using QI and data analytical skills to systematically undertake improvement initiatives towards the goal of advancing team-based care.
Seven live interactive videoconference learning sessions, 90 min each	Learning sessions included: didactics on the central concepts and practices of TBC; FQHC progress reports; refreshers in QI methods; and discussion. Learning sessions were video recorded and posted to an online learning community.
Self-assessment tools	FQHC teams are assigned the following tools to complete, and to discuss the results with their teams during initial meetings: Primary care team guide assessment29 Coach skills self-assessment30 Team skills self-assessment31
Team assignments	Teams were expected to meet weekly for an hour working their way up the improvement ramp as a guide to changing their practice, and to upload seven completed QI tools onto an online learning community by the end of the collaborative.
Online learning community and curricular syllabus	Content posted included: a syllabus with links to self-assessment tools; QI materials and resources on team-based care; a discussion board; and folders for each team to upload assignments accessible by all teams.
Weekly live videoconference coach-mentor calls, 60 min each	Two experienced CHCI QI coaches (NCA coach-mentors) held weekly videoconference calls with a group of FQHC team coaches to assess team progress, provide feedback and support regarding assignments, QI tools and managing team dynamics. Extensive notes were taken during these weekly calls.

CHCI, Community Health Center, Inc; FQHC, Federally Qualified Health Center; NCA, National Cooperative Agreement; TBC, team-based care.

The central concepts of TBC included the building blocks of primary care4 11: clarifying roles and responsibilities of core and extended team members to capitalise on their training; team huddles to improve communication; tracking cycle time for efficient office visits; previsit planning to improve compliance with scheduled tests and screenings; and population health.

The QI skills focused on using the improvement ramp, a data-based systematic approach to changing practice.24 25 First, teams assessed their practice, and then used fishbones and process maps to focus on a specific improvement initiative, such as improving the efficiency of routine screening. They developed global and measurable specific aim statements for the initiative, followed by conducting Plan-Do-Study-Act cycles to trial new workflows and responsibilities. At the top of the ramp, teams developed a step-by-step guide, called a playbook, for staff to spread the improvement across their health centre.

Description of the intervention: execution of the learning collaborative by the NCA team

The NCA team met weekly to focus on lead measures related to team progress26–28: team participation in learning sessions and weekly coach-mentor calls; issues discussed during those calls; number and quality of completed assignments; and FQHC coach contact with the NCA team for further support. All NCA team members listened to the weekly coach-mentor calls.

Study of the intervention: measures and analysis

The primary aim to evaluate self-reported changes in FQHC teams’ assessment of their practice used the measures listed below before and after participation.

Primary care team guide assessment29 has 24 items representing TBC activities in 15 domains. Items are rated 1–12 across four levels of performance, with higher scores indicating higher performance: A (rating of 10–12); B (rating of 7–9); C (rating of 4–6); and D (rating of 1–3). Total scores could range from 24 to 288.

Coach skills self-assessment30 has 38 items in five skill domains. Each item is rated 0–10 across four levels of performance, with higher scores indicating higher performance: (0) want to learn; (1–2) novice; (3–4) advanced beginner; (5–6) competent; (7–8) proficient; (9–10) expert. Total scores could range from 0 to 380.

The team skills self-assessment31 has 21 items in five skill domains. Each item is rated as follows: (1) strongly disagree; (2) disagree; (3) agree; (4) strongly agree. Total scores could range from 21 to 84, with higher scores indicating higher level of performance.

The secondary aim to evaluate how teams used the collaborative to develop new skills and advance TBC was measured using Proctor’s32 implementation, practice and patient outcomes. First, we measured the implementation outcome acceptability, defined as participant satisfaction with the collaborative, using surveys after the 2-day coaching and boot camp and each learning session.

Second, we measured the implementation outcome adoption/uptake, defined as actions by the individual teams to use the activities and resources of the collaborative: per cent attendance at learning sessions and at coach-mentor calls; consistency of hosting team weekly meetings; and number of QI assignments completed for teams’ improvement initiatives. Third, service outcomes and patient outcomes were self-reports by teams related to these initiatives, such as improved efficiency in screening and improvement in measures of patient health status respectively.

Site visits were conducted with three teams 6 months after the collaborative. Structured interviews addressed facilitators and barriers related to advancing TBC: coach and team functioning; use of QI and data skills; spread and sustainability of TBC model; and organisational leadership and support.

Results

Primary aim

The primary care team guide assessment29 was completed by all 13 teams precollaborative and nine teams postcollaborative. For these nine teams, a one-tailed t-test assuming unequal variance did not support significant improvements in total scores (p=0.057) overall, despite an average increase of 24 points across the teams. Figure 1 shows precomparison and postcomparison of the teams’ scores. Table 3 lists the precollaborative and postcollaborative levels of performance in 15 domains for the nine teams. There were shifts in five domains to higher levels of performance: practice team overall, registered nurse, laypersons, population management and medication management. While no teams reported a level A (highest level) on the practice team domain precollaborative, four did so postcollaborative.

Figure 1

Prescore and postscore on the primary care team assessment. TBC, team-based care.

Table 3

Prechange and postchange in levels on the primary care team guide assessment

Primary care team guide assessment: n= 9 sitesChanges in level before/after in 15 domains by site and by domain
Domain	Sites
	Site 2	Site 3	Site 4	Site 5	Site 6	Site 9	Site 10	Site 12	Site 13
The practice team	B	B	B	↑ B/A	↑ B/A	↑ B/A	↑ B/A	C	B
Medical assistant (MA)	B	↓ A/B	↑ B/A	B	B	↑ B/A	B	↓ B/C	B
Registered nurse (RN)	↑ D/C	↓ A/B	↑ B/A	D	↑ B/A	↑ D/C	↑ C/B	C	↑ D/C
Laypersons	↑ C/B	↓ A/B	↑ D/A	B	B	↑ C/B	A	↑ C/B	↑ D/B
Pharmacist	↑ B/A	B	↓ C/D	↓ B/D	↓ A/B	B	B	A	B
Enhancing access	↓ A/B	B	A	↑ B/A	B	↑ B/A	B	↓ B/C	A
Self-management support	↓ B/C	B	D	↑ C/B	B	B	A	D	B
Population management	↓ B/C	A	↑ B/A	C	↑ B/A	↓ A/B	↑ C/B	↓ C/D	↑ D/C
Planned care	↓ B/C	A	↓ B/C	↑ B/A	↑ B/A	↓ A/B	B	↓ C/D	↑ C/B
Care management	↓ B/C	B	↑ C/B	↑ B/A	A	A	↑ B/A	C	↑ C/A
Medication management	↑ C/A	B	↑ C/B	↑ C/B	↑ B/A	↑ C/B	↓ A/B	↓ A/C	C
Referral management	B	B	↑ B/A	↑ B/A	A	B	B	D	B
Behavioural health integration	↓ A/C	↑ C/A	↓ A/B	A	A	↓ A/B	↑ C/A	D	A
Communication management	B	B	B	↑ B/A	B	↓ A/B	A	↑ C/B	↑ C/A
Clinic-community connections	C	↓ A/B	↑ C/B	B	A	B	A	↑ C/B	A

↑ Indicates increase in self-reported performance Pre/Post. ↓ Indicates decrease self-reported performance Pre/Post. Single letter indicates no change Pre/Post.

Most teams also reported decreased levels of performance postcollaborative in some domains, which might be attributed to social desirability bias.33 That is, teams’ self-assessment of their practice prior to the learning collaborative may have been overly favourable, and that as they learnt more about TBC, they applied a more rigorous interpretation of TBC to their practice.

The coach skills self-assessment30 tool was completed precollaborative and postcollaborative by eight respondents from seven FQHC sites; site 5 had two coaches. A two-tailed paired t-test34 by respondent supported a significant increase in total scores precollaborative to postcollaborative (p=0.0032, 95% CI −157.04 to −46.95) (figure 2). After collaborative, the eight respondents rated themselves higher in all five domains increasing their total scores.

Figure 2

Prescore and postscore on the coaching skills self-assessment.

The team skills self-assessment31 was completed by 25 respondents both precollaborative and postcollaborative, representing eight FQHC sites. A paired t-test34 by respondent supported a significant increase in total scores precollaborative to postcollaborative (p<0.000, 95% CI −12.4 to −5.9). When the data were examined by domain rather than respondent, significant increases were found in all five domains of team skills: data skills; effective meetings; implementing daily huddles; improvement skills; and teamwork.

Secondary aim

Satisfaction with the 2-day in-person coaching and QI boot camp was high, with 91%–100% agreeing/strongly agreeing on all 11 items that it was helpful. Satisfaction with the seven learning sessions was also high, ranging from 92% to 100% for agree/strongly agree on 11 of the 12 items. The 12th question was worded in a way that made responses difficult to interpret.

The secondary aim was to evaluate how teams used the collaborative to develop new skills, and the implementation, service and patient outcomes they achieved by the end of the collaborative. Table 4 summarises implementation, service and patient outcomes at the team level for the 13 FQHC teams selected for the collaborative. Teams met with varying consistency and two teams ultimately withdrew due to competing priorities. The teams embarked on 15 improvement initiatives, 11 of which resulted in playbooks. Eight teams submitted all seven QI assignments for their initiatives. Three teams reported data supporting improved efficiency in service outcomes: routine screening (2), and rescheduling patients who did not show up for appointments (1). One team reported data supporting improved outcomes among patients with uncontrolled diabetes.

Table 4

Summary of outcomes for practice teams

Site	Implementation outcomes: adoption/uptakeActions to engage in the work of the collaborative				Improvement initiatives	Service outcomes or patient outcomes
	% attendance Learning sessions (n7) (median71%)	% attendance Mentor-coach calls* (median 71.5%)	Number of assignments submitted (of 7) (median 7)	Regularity of weekly team meetings	Types of initiatives†	Reported changes in service efficiency or in patient health
1	71% (5/7)	93% (25/27)	7	Most of the time	Cervical cancer screening†
2	86% (6/7)	83% (24/29)	7	Most of the time	Diabetes care (DidNotShow letters)†
3	100% (7/7)	76% (22/29)	6	Irregular meetings	Efficient office visit workflow (cycle time)
4	71% (5/7)	83% (24/29)	7	Most of the time	Cervical cancer screening†	Increased cervical cancer screening rates from 32% to 40%
5	86% (6/7)	85% (23/27)	7	Almost all of the time	Efficient office visit workflow†Early arrival time communication†
6	86% (6/7)	72% (21/29)	7	Almost all of the time	Mammography screening†	Increased mammography screening rates from 67% to 71%
7	71% (5/7)	31% (9/29)	7	Most of the time	Reducing no shows†	Improved rate of rescheduling patients who had not shown up for an appointment, from 37% to 57%
8	57% (4/7)	21% (6/29)	7	Stopped meeting	Access to appointment process
9	71% (5/7)	44% (12/27)	6	Most of the time	Patient experience and clinical workflows redesign (chart prep for post emergency room visits)†
10	71% (5/7)	63% (17/27)	5	Irregular meetings	Improving control of hypertension and low-density lipoprotein (LDL)
11	29% (2/7)	14% (4/29)	2	Stopped meeting
12	86% (6/7)	37% (10/27)	6	Most of the time	Pre visit planning†Referral process†
13	100% (7/7)	78% (21/27)	7	Most of the time	Uncontrolled diabetes intervention†	Decrease in HbA1c in population of patients with diabetes, from 10.44 to 8.6, in 6 months

*One National Cooperative Agreement (NCA) coach held 29 calls, the other NCA coach held 27 calls.

†Indicates a playbook was submitted for the improvement initiative.

Site visits

Structured interviews during visits to sites 2, 3 and 5 identified facilitators and barriers to advancing TBC, echoed in the recorded notes from the weekly coach-mentor calls.

Actionable data were a major barrier to pursuing improvement initiatives. Data reports generated by the electronic health record were either not readily available, not reliable or difficult to interpret. Many team members lacked data analytical skills prior to joining the collaborative.

Leadership support for participation in the collaborative and for teams’ ability to make decisions was cited as critical. The higher performing teams (sites 2 and 5) were given time to meet regularly and had access to data. Site 5 was ‘given the autonomy to drive the change and choose their intervention [initiative]… ideas came from the team and staff and were not top-down…ownership by the team of changes also made it more successful’. In contrast, the low-performing team at site 3 struggled to find time to meet throughout the collaborative and had little data. They noted ‘if teams want to propose changes to workflow, they submit them to the Leadership and the Board for approval’ which caused delays and dampened enthusiasm.

Finally, coaches at all three sites endorsed the importance of the weekly coach-mentor calls: ‘hearing what other participants in the collaborative were working on and learning from their best practices was such a valuable component of the Collaborative’.

Discussion

The results suggest that primary care teams that committed to the work of the learning collaborative on TBC gained knowledge to assess and improve an area of practice and developed coaching and QI skills to advance TBC. All of the teams and coaches that completed before and after self-assessments—primary care team guide assessment and coach and team skills self-assessments—reported improvements. While some were statistically significant, the small numbers make generalisability a challenge, a common problem for short-term learning collaboratives which typically consist of a dozen teams.14 Nine teams submitted a total of 11 playbooks, three reported improved efficiencies at the service level and one improved patient-level outcomes, thus attaining high levels of achievement on the improvement ramp. The data submitted by these four teams were not expected to be statistically significant; rather, the data demonstrated that teams learnt how to measure changes.

Eight months is insufficient time to achieve durable statistically significant patient outcomes when a learning collaborative is designed to help teams develop the skills to achieve those outcomes and access to reliable data is a barrier. It is enough time to demonstrate uptake of those skills needed to advance TBC. Even the decrease in some domains of the primary care team guide assessment suggests that as participating teams became better informed about TBC, their self-assessments became more valid and reflective of their values.33 It is also important to note that each team in our collaborative embarked on a different improvement initiative. By contrast, other learning collaboratives spend years with a focus on a single area of practice to achieve sustainable results, such as neonatal intensive care35 or cardiovascular disease.36

As with any learning collaborative, there were high and low-performing teams. Nembhard37 has identified determinants of teams’ success in collaboratives that support our own findings: a combination of interorganisation learning activities (activities in which organisations have the opportunity to learn from each other), intraorganisation learning activities (when teams work with colleagues in their organisation) and organisational support. For our learning collaborative on TBC, the interorganisation learning activities included the 2-day in-person boot camp, the seven interactive learning sessions and the weekly coach-mentor calls. For intraorganisation learning activities, we can cite consistency of weekly team meetings, use of the practice self-assessment tools and submission of team QI assignments related to improvement initiatives.

Teams needed organisational support to engage in these activities, and despite their leaderships’ written commitment, some teams did not get it. Teams at sites 8 and 11 stopped participating due to competing organisational priorities. While site 3 struggled to engage in the intraorganisational activities, their coach remained active in the interorganisational activities, and submitted most of the QI assignments on her own. By contrast, sites 5 and 6 were actively engaged in all activities throughout the collaborative; they both produced playbooks and site 6 reported improved screening rates. Leadership from site 5 was so pleased with their teams’ work in the TBC Collaborative that they created a new role for one of their two coaches to spread QI and TBC throughout the organisation. By comparison, the coach from site 3, discouraged, left her job.

An important strength of the learning collaborative on TBC was using both external (NCA) and internal (FQHC) coaches.25 The coaching by the NCA coach-mentors and the proactive execution of the collaborative by the NCA team provided highly valued support outside of the teams’ own organisations. We did not provide technical assistance by request; we were proactive partners in the teams’ efforts.

Future evaluation of learning collaboratives would benefit from more attention to coaching/facilitation and to execution, a factor in implementation science38 that has not been adequately studied in healthcare. The structure and content of collaboratives vary widely39 as does the complexity of the organisations that participate in them. Published protocols often focus on clinical outcomes, but not how the collaborative was executed, or how coaching was done, if at all. It is also important to understand how the intended change in practice was implemented by the participants in the unique context of their organisations’ structure, resources, leadership engagement and skill set for implementing change.17 19 20 38 40–45

Conclusion

In this paper, we report on the extent to which 13 primary care teams from FQHCs in the USA availed themselves of the activities and resources offered by an 8-month learning collaborative for TBC. We examined self-reported changes in practice, and outcomes at the implementation, service and patient levels. The collaborative advanced the ability of engaged teams to assess their practice against central concepts of TBC, and to use QI skills to change that practice. A key characteristic of the 8-month collaborative was the role of the NCA team, and especially the mentor-coaches, whose weekly support for the FQHC improvement coaches was proactive.

Developing team capacity and capabilities for successful and sustainable TBC requires organisational investment in not just the short-term collaborative learning activities. It also requires long-term work, including protected time for teams to meet regularly, access to reliable data and coaching. Innovations in practice, such as TBC, require context conducive to change and attention to details in making the change,46 such as reliability of data, steps in critical processes and clarification of roles. There is no substitute for time on task and the skills, teamwork and support to make good use of that time.