Outcome mapping for health system integration.

Health systems around the world are implementing integrated care strategies to improve quality, reduce or maintain costs, and improve the patient experience. Yet few practical tools exist to aid leaders and managers in building the prerequisites to integrated care, namely a shared vision, clear roles and responsibilities, and a common understanding of how the vision will be realized. Outcome mapping may facilitate stakeholder alignment on the vision, roles, and processes of integrated care delivery via participative and focused dialogue among diverse stakeholders on desired outcomes and enabling actions. In this paper, we describe an outcome-mapping exercise we conducted at a Local Health Integration Network in Ontario, Canada, using consensus development conferences. Our preliminary findings suggest that outcome mapping may help stakeholders make sense of a complex system and foster collaborative capital, a resource that can support information sharing, trust, and coordinated change toward integration across organizational and professional boundaries. Drawing from the theoretical perspectives of complex adaptive systems and collaborative capital, we also outline recommendations for future outcome-mapping exercises. In particular, we emphasize the potential for outcome mapping to be used as a tool not only for identifying and linking strategic outcomes and actions, but also for studying the boundaries, gaps, and ties that characterize social networks across the continuum of care.

Introduction

Health systems aiming to deliver integrated patient care face myriad multilevel and context-specific challenges.1–3 Integrated care can be defined broadly as care that is coordinated across multiple health care professionals, organizations, and sectors and that is attuned to patient needs and preferences.4 Political, financial, geographical, technological, interorganizational, and interprofessional factors influence the capacity and motivation of different components of the system to work together. The literature suggests that prerequisites for integrated care include a shared vision, clear roles and responsibilities, and a common understanding of how the vision will be realized.3,5–8 In the context of multiple meanings and approaches to integration, stakeholder agreement within system boundaries has been identified as a necessary antecedent to developing effective and sustainable integrated systems.5,9,10

One approach for building these conceptual prerequisites to integrated care is to use outcome-based thinking. Outcome-based thinking involves identifying desired outcomes and the capabilities needed to achieve them. Outcome-based thinking has been used extensively by the international development sector,11,12 as well as by both business and health care managers, as a basis for performance measurement, management, and change.13–18 A range of different terms has been used to describe the product of outcome-based thinking, including outcome maps, logic models, value maps, means-ends diagrams, and strategy maps. As the differences in terminology suggest, a number of distinct techniques exist for developing and applying outcome-based thinking. We use the term outcome map.

An outcome map visually depicts how particular actions or processes (shown as rectangles in the map) contribute to improvements in higher-level outcomes (shown as circles) (see Figure 1). Each activity creates an immediate outcome that is directly linked to it, and this relationship is illustrated using an arrow. Immediate outcomes contribute to achieving higher-level outcomes – relationships that are also shown using arrows. The result is a schematic that leads from actions on the left-hand-side of the page to the goals on the right. Since all actions and outcomes are linked toward achieving the goals, the outcome map ensures that goals and actions are aligned.

Figure 1

System integration outcome map for the Central Local Health Integration Network (LHIN).

Note: The boxes represent actions or processes, and the circles represent outcomes. The arrows depict the pathway from actions to immediate and higher-level outcomes.

The outcome-mapping process described in this paper is premised on the idea that behavioral change is fundamental to improvement and is facilitated by a learning-based and stakeholder-driven approach.19–20 Thus, dialogue is emphasized. While facilitators guide the process, the map is created by stakeholders who have a shared interest in addressing a challenge, and who use the map as a vehicle to talk about their shared goals and what needs to be done to achieve them. The map summarizes this conversation. The map is visual, rich in detail, and uses a simple graphic language to represent a complex strategy on a single page. The map also links dialogue to action. Action without dialogue fails to take advantage of the knowledge that resides in a system, and action is often misplaced. Dialogue without action fails to follow through on good intentions. Through outcome mapping, talk leads to action. Finally, the process facilitates relationship-building as participants build trust and discover ways to collaborate in creating a desired future, seeing explicitly how their individual actions support and reinforce the actions of others. Outcome mapping is thus particularly suited to multistakeholder initiatives – such as partnerships and interorganizational networks – where alignment, collaborative commitment, and shared purpose are urgently needed.21

Despite the potential benefits of outcome mapping to strategy implementation, change management, and performance improvement, few examples exist of its use as a tool to facilitate health system integration. In this paper, we describe an outcome-mapping exercise we conducted at a Local Health Integration Network (LHIN) in Ontario, Canada, using consensus development conferences. We draw from two theoretical perspectives – complex adaptive systems and collaborative capital – in explaining the value of outcome mapping and in interpreting and critiquing the outcome map. We argue that outcome mapping may be used as a tool not only for identifying and linking outcomes with actions, but also for making sense of a complex system, building collaborative capital, and studying the boundaries, gaps, and ties that characterize social networks across the continuum of care. Health systems aiming to transform fragmented services into integrated services may benefit from this process of mapping pathways to desired outcomes while identifying specific professional and organizational roles and social ties that influence outcome achievement.

Theoretical perspectives on integrated care

Researchers have evoked a range of theories and concepts in their studies of integrated care, including organizational design, strategy, leadership, culture, learning, change and innovation, decision making, and team, organizational, and network performance, as well as environmental contingency, transaction cost, resource dependence, structure and agency, and institutionalism.6,22–28 More recently, the concepts of complex adaptive systems and social capital have been applied to integrated care, though empirical research remains limited.29–33 Due to their growing significance in the field and their relevance to outcome mapping, we briefly describe these two perspectives below in order to frame our study and to highlight their value to outcome mapping for health system integration.

Complex adaptive systems

Complex adaptive systems (CASs) are open systems with fuzzy boundaries comprised of numerous, diverse, highly interactive agents.34 In CASs, patterns of interaction and continuous adaptation contribute to innovative and unpredictable behaviors and events; these systems are thus characterized as emergent and self-organizing.34 Several scholars argue that health systems are complex and adaptive.35–41 Like CASs, health care delivery involves diverse professional groups and organizations, evolving patient needs, conflicting interests, and changing technologies and treatments – all embedded within a particular social, political, and economic context. This complexity simultaneously demands and defies integration.42,43

A CAS perspective is increasingly being applied to the conceptualization and practice of integrated care, to help explain and reframe the barriers to integration and to promote facilitative, indirect leadership approaches.29,30,33,44 A CAS lens suggests that the health system is not constituted merely of the sum of its parts, but also of the interrelationships among its parts. Dividing the system into manageable units to be analyzed separately, with the hope of understanding the system’s complexity, destroys what one is seeking to understand, since complexity results from interactions and is manifested at the level of the system itself.39 Similarly, dividing the system into manageable units to be controlled, with the hope of improving system performance, fails to consider how strategies and activities in different parts of the system may interact to produce unexpected or counterproductive system outcomes. Based on these insights, scholars recommend that leaders and managers understand the culture(s) of the environments in which they work, provide a context that supports group-based and cross-boundary experimentation and learning, share control, and refrain from top-down mechanisms of forced change.30 While these suggestions align with current knowledge of the enablers and barriers to integrated care, there are few, if any, concrete tools available to facilitate the application of a CAS perspective and management approach to integration efforts. We propose that outcome mapping may fill this gap.

Collaborative capital

Social capital is defined as the goodwill available to individuals, groups and/or organizations as a function of their social relations.45 Social capital can facilitate access to information, enhance power and influence, and contribute to solidarity, thereby reducing the need for formal controls.45 Collaborative capital is a form of social capital that supports collective action and change through “dense networks of information sharing, trust and reciprocation norms.”46 The concept of social capital emphasizes individual benefits, whereas collaborative capital focuses exclusively on collective actions and outcomes,46 though the two levels are interrelated. The dimensions of social and collaborative capital include the structural (network ties, network configuration, and network stability), the cognitive (shared goals and shared culture), and the relational (trust).47

Although much of the literature on integrated care highlights the roles of interorganizational and interprofessional relationships, and discusses cultural conflicts,48 the concepts and associated literature of social and collaborative capital are rarely applied. One exception is a conceptual paper outlining indicators that may be used to measure and monitor social capital in integrated care networks.32 Similarly, Scott and Hofmeyer synthesize network theory and social capital to argue that efforts to transform systems and support collaboration must be based on an understanding of how “structures influence, and are influenced by, the way that people relate to one another.”49 A recent editorial also calls for the increased use of social network analysis in the design and assessment of integrated care.31 Social network analysis measures the relationships and ties between individuals, groups, and organizations within a network. The study of “gaps” and “disconnections” between formal and informal clusters of people may also offer insights into the social structures that characterize interactivity, or the lack thereof.50 While the influence and value of social capital to integrated care initiatives is increasingly recognized, specific mechanisms or tools for fostering social capital in this context have yet to be explicitly outlined and examined. We propose that outcome mapping may help fill this gap.

Methods

In 2006, 14 geographically defined Local Health Integration Networks (LHINs) were mandated into existence by Ontario’s Ministry of Health and Long-Term Care. The LHINs plan, fund, and manage services within their regions using integrated health service plans developed in collaboration with local health care providers and community members. In 2008, we began the outcome-mapping project in collaboration with the Central LHIN. The Central LHIN is the most populous of the 14 regions and among the fastest growing, with the highest proportion of immigrants in the province.51 The outcome-mapping exercise was initiated for the purposes of creating stakeholder alignment around a shared vision and producing a comprehensive, clear, and actionable roadmap to guide decisions and actions.

A steering committee was created consisting of Central LHIN representatives, including a senior director, an epidemiologist, a board member, three researchers, and senior directors from a local hospital and a community agency. A consulting team with extensive experience in the participative approach to outcome mapping was recruited to facilitate dialogue and the application of outcome-mapping techniques. A consensus development conference (CDC) method was used to elicit the experiences and perspectives of 45 health care professionals from across the health care sector. A CDC involves face-to-face discussion and debate between stakeholders, and is a research tool used to enhance decision making, identify strategic directions, and advance stakeholder ownership and engagement.52 Participants were selected from organizations within the Central LHIN using purposive sampling, and consisted of professionals with multiple years of experience in integrated care settings and/or knowledge based on research and implementation of integrated care initiatives. These professionals spanned managerial (n = 18), clinical (n = 15), administrative (n = 9), and research (n = 3) roles, and represented the full continuum of care, including the LHIN, acute care, rehabilitation and complex continuing care, primary care, home care, and community support agencies.

A semistructured interview guide was used to facilitate dialogue among the participants at the first CDC, using questions focused on health system goals, indicators of success, and barriers to and enablers of goal attainment. The consultants acted as neutral receivers of participant responses, encouraged storytelling, and ensured that the discussions were not dominated by any small number of vocal individuals. Responses were recorded on a flip chart, were summarized in brief phrases or a few words, and were visible to the entire group. After the list was completed, participants were given the opportunity to clarify, dispute, and discuss any points on the list. Areas of agreement and disagreement were noted. The consultants’ approach to conflict management involved asking clarifying questions, not making declarative statements, and focusing the conversation on outcomes rather than on actions; at the end of the process, there were no major disagreements. To ensure that responses were accurately captured within the flow of the group discussion, the conversations were tape-recorded and transcribed verbatim. Through content analysis of the transcripts, the themes generated at the CDC were further refined and categorized into outcomes and activities for inclusion in the map.53 This data was used in the development of the first draft of the outcome map for review by the steering committee and for respondent validation in our next CDC.54 In addition, LHIN documents were used to cross-validate some of the information.

The first draft of the map was then presented to the participating stakeholder group at the second CDC, where they were asked to comment on whether the map reflected reality. They were further asked to comment on anything wrong or missing and on what should be modified or added, and why. The outcome map was displayed on a large poster board and markers were available to participants to enable them to graphically display their comments, concerns, or suggestions, for the entire group to see. Participants were asked to verbalize what they were doing and why they were doing it. It was felt this was important, in order that group participants could hear each other’s understanding of the actions, connections, and relationships as these were displayed on the map.54 Dialogue was focused on clarifying high-level goals and objectives, adding missing outcomes, perfecting the language used to describe outcomes, and correcting the logic of the causal links. All comments, explanations, and discussions were tape-recorded and transcribed verbatim to inform the next iteration of the outcome map.

Following this CDC, all participants, including those who could not attend, were sent a copy of the map to allow for further reflection and feedback, as a form of member checking.53 It took an additional iteration to finalize the map, in dialogue with participating stakeholders and the steering committee; thus, the outcome-mapping process used was interactive and emergent in identifying the existing consensus. At each stage, the map became richer and more robust. All 45 participants attended at least one of the three half-day CDCs, held at York University and the Central LHIN; this ensured continuity of understanding and intention as the outcome map developed. The consensus among the stakeholders that there was “little left to talk about” signified the end of the process and the final stage of development. The outcome map represented a graphic illustration of strategic outcomes, capabilities, enabling outcomes, and supporting actions that contributed to the functioning of the health care system. The map reflected the participating group’s agreement on the outcomes to be achieved and the actions that needed to be taken, to realize the desired goals.

Results

Participants identified the end-goal of health care as “caring communities, healthier people, and health system sustainability.” They identified the key strategic outcomes as (1) clients have access to an integrated health care system, (2) service is delivered by the appropriate provider in the appropriate location, and (3) information, support, advocacy, and other help are provided as people move through the continuum of care. The capabilities required to achieve these outcomes were categorized into four broad areas: (1) integration, coordination, and information access, (2) service delivery, which encompasses capacity, breadth, efficiency, access, and effectiveness, (3) adaptation, which refers to continuous performance improvement and capacity for change, and (4) sustainability. Due to the size of the outcome map and the focus of this paper, only the portion specific to system integration is provided in Figure 1, which will be summarized below.

According to participants, and as depicted in the outcome map, in order to achieve a “health system that functions as an integrated system,” 30 enabling outcomes must be attained, supported by 21 action areas that span governance structures, policy and legislation, collaborative culture, partnerships, and role clarity and conflict. Arrows on the map suggest linkages between actions, precursor outcomes, and the end goal of an integrated health system, rather than cause-and-effect relationships. The “system integration” portion of the outcome map was linked closely with the related areas of “information access” (focused on eHealth and information technology) and “system coordination” (focused on patient referrals and transfers), which collectively support the common objective of “clients have access to an integrated health system.” The two enabling outcomes that generated the most discussion and interest were “role confusion and conflict are eliminated” and “a culture of collaboration rather than competition is created,” suggesting that these outcomes may be more challenging and complex to achieve.

In the closing discussion of the final CDC, participants said that the outcome map highlighted the complex interlinkages present in the system, while the consultation process revealed pools of knowledge – groups and organizations – they did not know existed, thereby enabling them to develop new relationships and commitments to collaborative work. These shared sentiments suggest that outcome mapping helped strengthen stakeholder awareness and alignment within a complex system and build collaborative capital to support the delivery of integrated care.

Discussion

The outcome map provides insight into the interrelated factors that facilitate integration from the perspectives of local stakeholders. The growing literature on integration supports the roles of governance, policy, collaboration, partnerships, culture, and role clarity in achieving integrated care. For example, a systematic review of health system integration identified ten key principles for integrated care, including aligned governance, accountability and policy structures, interprofessional teamwork, cross-sectoral partnerships, and cultural and leadership commitment to integration, among other enablers.3 Similarly, a quality management model for integrated care developed through a Delphi study highlights the importance of delivery system structures, role and task clarity for collaboration, commitment and transparency, and shared learning, among other elements.55 While the themes are comparable, the outcome map is unique in that it provides a visual diagram of action-to-outcome pathways that help stakeholders create a shared vision for the health system and enable them to see how their work contributes to system goals.

Although the literature on integration emphasizes the importance of promoting a collective understanding of the vision and developing trust between professional groups and organizations, no specific methods or tools are proposed for how to achieve these aims, other than generic recommendations for cultural change, open communication, and distributed leadership.56–58 This paper thus addresses a gap in the literature on integrated care and may foster discussion and knowledge exchange regarding other tools and methods that are being used or should be applied in the future.

The outcome map can be used by leaders and managers as the basis for planning, communication, and performance measurement, by plotting projects, initiatives, and relevant indicators on the map. For example, indicators that capture the structural, cognitive, and relational aspects of social ties may be used in combination with other structural, process, and outcome measures to provide a more complete view of system functioning and performance.

The theoretical perspectives of CAS and collaborative capital, which were underlying themes in many participant comments, are also helpful in interpreting and critiquing the outcome map. The outcome map incorporates activities of relevance to the diverse actors in the health system, from policy makers and regional leaders from the Ministry of Health and Long-Term Care and the LHIN respectively, to the clinicians, board members, and managers who work in a variety of organizations across the continuum of care. The outcome map emphasizes the interdependence of varied processes in achieving the desired outcomes in a way that simplifies the complexity of the health system while maintaining a holistic, rather than a narrow and divided, view of health system functioning. A CAS perspective implies that leaders should be mindful of how interventions in one area of the map may result in positive or negative effects elsewhere. This suggests that incremental change be pursued and that actors within the system (both individuals and organizations) be afforded the flexibility to experiment.

In addition to aligning, in many ways, with a CAS perspective, outcome mapping may also serve as a practical tool for building collaborative capital. As our participants noted, the relationships forged as part of the outcome-mapping process may be as important a product for integrated care efforts as the outcome map itself. New social ties (interpersonal and/or interorganizational) reflect the structural dimension of social and collaborative capital while new understandings of and attitudes toward other professional groups, organizations, ideas, goals and ways of working reflect the cognitive and relational dimensions of social and collaborative capital. The outcome-mapping process may enable individuals who identify strongly with their profession to expand their “in-group” to include diverse professionals working within the same multidisciplinary team or interorganizational partnership.59 However, although the outcome-mapping process appears to help foster collaborative capital, the outcome map itself falls short of explicitly incorporating and clarifying the roles of interorganizational and interprofessional relationships in outcome achievement.

In Table 1, we outline the key principles of CASs and collaborative capital, and suggest the implications of each assumption for improving the outcome-mapping process and product. Both perspectives emphasize the role of relationships and social networks in system performance and change, while the CAS lens also highlights the emergent and unpredictable nature of the system.18 Although consideration for social relationships is often not integrated into the outcome-mapping process in business applications, it is a fundamental part of how the international development sector uses outcome maps.11 Development programs view all participants as potential agents of change and examine each actor’s sphere of influence as part of the outcome-mapping process. In future outcome-mapping exercises, we recommend plotting stakeholder groups and organizations, and their interrelationships, on the outcome map to clarify who, not just what, contributes to outcome achievement. Doing so will also highlight flows of communication, segmented groups or communities, and the links between different capability areas on the outcome map. This information can be used to identify existing alliances that may be leveraged to mobilize or support action.

Table 1

Complex adaptive systems and collaborative capital: implications for outcome mapping

Theoretical principle	Implication for outcome mapping
Complex adaptive systems
Change is non-linear	Used judiciously, different line weights can help to emphasize the strength of a relationship, and feedback loops can be used to depict increasing returns in a reinforcing relationship among key outcomes
Systems self-organize and adapt spontaneously; the future is uncertain	Consider the outcome map a “living document” that requires regular checks and modifications through stakeholder consultation
Interactions among agents (individuals and organizations) shape the emergent properties of the system	Map the social ties, formal and informal, that exist among individuals, groups and organizations within the system
Agents are opportunistic, spontaneous and capable of learning	Keep action items in the outcome map broad to allow agents to find their own ways of contributing to desired outcomes
Collaborative capital
Change occurs through a sequence of actors connected by their relationships	Include relevant individuals, groups and organizations on the outcomes map to clarify who, not just what, contributes to outcome achievement
Social ties influence behaviours and the transmission of information	Map the social ties, formal and informal, that exist among individuals, groups and organizations within the system

Based on this critique, we promote a two-step approach to outcome-based thinking for health system integration, based on the work of Shaxson and Clench.60 First, stakeholders in dialogue must clarify the intended final goal of integrated care, and work backwards to identify the enabling outcomes and actions that contribute to the desired result. Once an outcome map has been produced, a social network framework can be developed that aligns with the content and layout of the outcome map. Social network frameworks may also be developed for a specific policy issue or for engaging a formal network of organizations at a more strategic level.60 The principles of participation and iterative learning that characterize the outcome-mapping process must be maintained throughout the process, as it is through discussion and engagement that stakeholders may achieve a shared vision and strategy, and an enhanced understanding of the health system and their related roles in delivering integrated care.

This study has limitations, some of which suggest opportunities for future research. First, the validity and reliability of CDC findings are partially dependent on the composition of the group and the facilitators’ approach to structuring the interaction.52 Although participants reflected a diverse range of professional roles and organizations, it is possible that some stakeholder groups were missed. In particular, considering the most recent definition of integrated care, which incorporates elements of both coordination and patient centeredness,4 the perspectives of patients and their informal caregivers should be incorporated into the outcome-mapping process. Second, the external validity of our findings is limited by the focus on the Central LHIN; the value of CDCs and outcome maps to integrated care planning and management could be further tested in other local, national, and international contexts. It is also unclear what the long-term effects (or the lack thereof) of the outcome-mapping project were in the Central LHIN, or how outcome mapping compares with other tools or approaches for stakeholder engagement and consensus-building for integrated care. Future empirical work may conduct follow-up surveys or interviews to gather stakeholder reactions to the outcome-mapping process, and to ascertain how and to what extent an outcome map has been used (or not) at system and organizational levels. It would also be helpful for researchers to engage with practitioners to identify the tools and methods they use (if any) to support interorganizational and interprofessional communication, and to compare these techniques and management experiences with outcome mapping. Finally, our assumptions regarding the value and relevance of CASs and collaborative capital to outcome mapping for health system integration require further development and validation through practical applications and empirical research.

Conclusion

Outcome mapping may be used by leaders and managers as a tool for building the prerequisites to integrated care, defined in the literature as a shared vision and a common understanding of how the vision will be realized.3,5–8 Outcome mapping via CDCs may also help stakeholders make sense of a complex system and foster collaborative capital, a resource that can support information sharing, trust, and coordinated change toward integration across organizational and professional boundaries. The concepts of CASs theory and collaborative capital also suggest ways in which current outcome-mapping methods may be enhanced, particularly in the context of efforts to integrate care. The participative approach to outcome mapping described in this paper may be used to identify and link outcomes with actions and to study the boundaries, gaps, and ties of social networks across the continuum of care as we move toward an integrated health care system.