Partners for the optimal organisation of the healthcare continuum for high users of health and social services: protocol of a developmental evaluation case study design.

INTRODUCTION: Case management allows us to respond to the complex needs of a vulnerable clientele through a structured approach that promotes enhanced interaction between partners. Syntheses on the subject converge towards a need for a better description of the relationships between programmes and their local context, as well as the characteristics of the clienteles and programmes that contribute to positive impacts. The purpose of this project is thus to describe and evaluate the case management programmes of four health and social services centres in the Saguenay-Lac- Saint-Jean region of Québec, Canada, in order to inform their improvement while creating knowledge on case management that can be useful in other contexts. METHODS AND ANALYSIS: This research relies on a multiple embedded case study design based on a developmental evaluation approach. We will work with the case management programme for high users of hospital services of each centre. Three different units of analysis will be interwoven to obtain an in-depth understanding of each case, that is: (1) health and social services centre and local services network, (2) case management programme and (3) patients who are high users of services. Two strategies for programme evaluation (logic models and implementation analysis) will guide the mixed data collection based on qualitative and quantitative methods. This data collection will rely on: (1) individual interviews and focus groups; (2) participant observation; (3) document analysis; (4) clinical and administrative data and (5) questionnaires. Description and comparison of cases, and integration of qualitative and quantitative data will be used to guide the data analysis. ETHICS AND DISSEMINATION: The study protocol was approved by the Ethics Research Boards of the four health and social services centres (HSSCs) involved. Findings will be disseminated by publications in peer-reviewed journals, conferences, and policy and practice partners in local and national government. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

The case study and the developmental evaluation approach are adaptable to the complexity and evolution of the context.

Various strategies will be used to facilitate good validity and to help us determine qualitative impacts attributable to the programme:24

Triangulation of data sources and evaluators to deepen our understanding of the question;

Reflexive process and evaluation minded all along the data interpretation phase;

Integration of qualitative and quantitative data allowing us to directly question people concerned on other aspects potentially influencing results.

The study will be carried out in only one region of Québec. However, multiple case studies will allow us to ensure transferability to other contexts, through the theoretical enlightenment provided and the reproducibility of observations in many cases.27

Introduction

Much like other industrialised countries,1 2 the Health Services Research Institute observed that close to 80% of the costs of healthcare at the Canadian level is attributable to 10% of the population.3 Many of these people frequently use hospital services for increasingly complex health needs4–6 arising from such factors as multimorbidity, psychiatric comorbidities and psychosocial issues, or a combination of these factors.5 7 8 Requiring care and services from many partners in the health and social services care system as well as the community network, these high users are more at risk of encountering difficulties in the integration of care,9 and more at risk for incapacity and mortality.10 Faced with this issue, case management (CM) is increasingly being recognised internationally as an appropriate intervention to improve satisfaction and quality of life,11 and to reduce costs associated with high users of services.1 4 11–16 CM allows better response to the complex needs of a very vulnerable clientele through a structured approach that promotes enhanced interaction between partners of the health and social services system as well as with the community network.

CM is defined as a dynamic and systematic collaborative approach to ensure, coordinate, and integrate care and services for a clientele. An intervention geared towards interdisciplinarity in which a key practitioner or navigator (nurse, social worker or others) evaluates, plans, implements, coordinates and prioritises services based on people’s need in close collaboration with concerned partners.17 CM emphasises four main components: (1) in-depth evaluation of the person's needs and resources; (2) establishment and follow-up of an individualised services plan that is person-centred; (3) coordination of services between partners to improve services integration; and (4) self-management support of the person and his or her family.18 19

Although results appear correlated with programme intensity,12 characteristics of CM programmes present much variability in regard to, for example, their implementation contexts, targeted clienteles and duration of follow-up.20 21 Syntheses on the subject converge towards a need for a better description of the CM programmes and the relationships between programmes and their local context, as well as the characteristics of the clienteles and programmes that contribute to positive impacts.20–22

In 2004, the creation of local services networks (LSN) in Québec aimed to bring services closer to the population and to make them more accessible and better integrated. At the heart of each LSN, an establishment called a health and social services centre (HSSC), including hospital, community and long-term services, acts as the basis or foundation for the LSN ensuring access, continuity, coordination and quality of the services intended for the population of its local territory.23 In 2008, the Saguenay-Lac-Saint-Jean health and social services agency appointed the six HSSCs of its territory to deploy CM programmes for high users of hospital services.

The aim of this project is thus to describe and evaluate the CM programmes of four HSSCs in the region in order to inform their improvement while creating knowledge on CM that can be useful in other contexts.24 Specifically, this study, funded by the Canadian Institutes of Health Research (CIHR) within its Partnerships for Health System Improvement programme, aims to answer the following questions over the course of three evaluation cycles while providing feedback to key decision-makers over the 3 years of the project: (1) What are the different components of the CM programme of each HSSC: their structure, their actors (targeted clientele and practitioners), their operating process and their predictable effects? (2) What are the strengths and aspects to improve of each programme from the perspective of the concerned actors in view of a better services integration? (3) What characteristics of the clientele and the CM programmes contribute to positive impacts on use of services, quality of life, patient activation and patient experience with care?

Methods and analysis

Conceptual framework

The research question as well as the data collection (interview and discussion guides) and analysis will rely on the conceptual framework suggested by Chaudoir et al25 to guide research on the implementation of innovations. This framework proposes five broad categories of factors to consider in the evaluation of the implementation of an innovation (programme), that is: (1) environmental factors; (2) organisational factors; (3) factors related to the practitioners; (4) factors related to the patients and (5) programme-related factors. Environmental factors refer to the larger context in which the organisation evolves, such as, for example, their mandates and allocated funds. Organisational factors include different aspects associated with the organisation in which the programme is implemented, such as organisational culture, type of leadership and climate. Factors associated with the practitioners represent the characteristics of these individuals who interact with patients within this programme, for example, attitudes towards the innovations or capacity in adapting to change. The factors related to the patients correspond to the characteristics of these patients that can influence the results of the programme, such as motivation to get involved in the programme. Finally, factors linked to the programme represent the aspects of the implemented programme. These five broad categories of factors will be used to guide us in the identification of characteristics that can potentially contribute to the impact.

The integration of CM services into each LSN will be examined according to the integrated care model recently suggested in England.26 This model proposes six essential dimensions of services integration based on patient experience: (1) consideration of patient and family needs; (2) communication with the patient and between practitioners; (3) access to information; (4) involvement in decision-making; (5) care planning; and (6) transitions between various health professionals and practitioners.

Research design

This longitudinal research relies on a multiple embedded case study27 design based on a developmental evaluation approach (figure 1).28 29 Multiple case study is preferred as this design is well adapted to respond to a research question focused on the ‘how’ in a complex system (LSN), and in dynamic and varied contexts at the time of the study.27 30 We will work with four cases, the ‘case’ being the CM programme for high users of hospital services of each HSSC. The number of cases, fixed at four, appears optimal to obtain good diversity of contexts while ensuring the feasibility of the proposed approach. The four HSSCs selected are the first four in the region to have implemented CM in their organisation. Three different units of analysis will be interwoven to obtain an in depth understanding of each case, that is: (1) HSSC and LSN (‘macro’ level); (2) CM programme for high users of services (‘meso’ level) and (3) patients who are high users of services (‘micro’ level). In addition to allowing for an in depth analysis of each case, the multiple case study design will offer analysis strategies to systematically compare trends observed between cases.

Figure 1

Research design and project outline. CM, case management; HSSC, health and social services centre; QUAL, qualitative data; QUAN, quantitative data.

The team proposes to use a developmental evaluation approach in response to decision-makers’ needs for ongoing access to information required to inform and orient their decisions. Developmental evaluation that builds on an efficient partnership between researchers and decision-makers helps support adaptive learning in emerging and complex initiatives.31 It consists of collaboratively asking evaluative questions and collecting data allowing for feedback, and to support decision-making and modifications to be made to improve the programme.32 Considered as a rigorous evaluative approach, it allows for the required flexibility in a context of evolving programmes in real clinical settings.32 33 According to this approach, the three main roles of the researchers in the team are to: (1) accompany the team to facilitate the project's evolution; (2) collect qualitative and quantitative data; and (3) ensure continuous feedback of results to the Director-generals and programme directors and to the committees that are designated to organise services for high users, committees that will be identified with the Director-general of each HSSC.33

Data collection strategies associated with each research question (primary and secondary outcomes)

Two strategies for programme evaluation (logic models and implementation analysis)34 will guide the mixed data collection. This data collection will rely on five methods (three qualitative and two quantitative) explained further in the text. A database will be created in order to organise the data collected during the case study. It will contain the raw data to be used to write the case history, but will remain distinct and be used by an independent reviewer if need be, thus improving the reliability of the study.27 The database will include: field notes, collected documents and other material (verbatim, observation notes, quantitative data). Reliability will also be ensured by different strategies27 to maintain an explicit chain of evidence: (1) the case history will refer to the pertinent citations in the database; (2) the database will contain sufficient information on the data collection; (3) the data collection will follow the procedures announced.

Question 1 (secondary outcome)

What are the components of the CM programme of each HSSC: structure, actors, operating process and predictable outcomes?

To answer the first question, the logic model for the CM programme of high users of services of each HSSC will be described35 to present its structure, its actors (targeted clientele and professionals/practitioners) and its processes, and to illustrate what it aims to accomplish (its effects/outcomes).36 To be coherent with developmental evaluation, these models will be updated in years 2 and 3.33 The data collection methods will involve interviews and focus groups with the various stakeholders (table 1) and analysis of the documents related to the implementation of each programme.

Table 1

Type of interviews planned according to stakeholder category

Stakeholder category	Individual interviews (total number of interviews in each HSSC)	Group discussions (number of groups of 6–10 participants in each HSSC)
Decision-makers and managers	▸ Director-general Agency (1)▸ Director-general HSSC (4)▸ Programme directors CM (4)▸ Programme managers CM (4)	▸ Programme coordinators with links to CM programmes (4)
Case managers	▸ Practitioners playing an important role in CM (12)
Medical teams	▸ Directors of professional services (4)▸ Presidents of the physicians, dentists and pharmacists council (4)	▸ Executive members of the physicians, dentists and pharmacists council (4)▸ Territorial tables on primary care medicine (4)▸ Territorial tables on specialised medicine (4)
Professional teams (nurses, social workers, others)		▸ In close contact with frequent users in HSSC (4)▸ In close contact with frequent users in the community (primary care) (4)
Pharmacists and community organisations		▸ Community pharmacists (4)▸ Community organisations (4)
Patients and family caregivers	▸ In years 2 and 3: frequent users having experienced the most and the least improvement in quality of life (32)	▸ Frequent users of services (4)▸ Family caregivers (4)

CM, case management; HSSC, health and social services centre.

Question 2 (secondary outcome)

What are the strengths and areas for improvement of each programme from the concerned actors’ point of view in the perspective of a better integration of services?

Question 3 (primary outcome)

What characteristics of the clientele and CM programmes contribute to a positive impact on use of services, quality of life, patient activation and patient experience of care?

To answer questions 2 and 3, an implementation analysis will focus on the internal dynamics of the programmes by examining the influence of the interaction between each programme and its implementation context in an attempt to explain the variations observed in its effects.34 Implementation analysis is “particularly important when the intervention analysis is complex and composed of sequential elements with which the context may interact in different ways.”34 The case study design is totally appropriate for the analysis of complex intervention implementations.34 37 The logic models formulated in question 1 will be compared to identify the common and distinct aspects between HSSCs, allowing us to hypothesise on the characteristics potentially having an impact on use of services, quality of life and care experience, hypotheses that will be explored in the implementation analysis. The conceptual framework presented previously will also be used to identify significant characteristics. In addition, implementation analysis will address conditions for transferability of programmes to other contexts while providing information on the characteristics of these contexts more likely to generate positive impacts.34

Data collection methods

Answers to questions 2 and 3 (implementation analysis) will be obtained through a mixed data collection based on the five following methods:

Individual interviews and focus groups (qualitative data)

The main actors involved in CM and the care continuum of high users of services will be engaged through purposive sampling38 in each HSSC, at the beginning of years 1, 2 and 3. Various strategies were suggested by the HSSC partners to promote participation and facilitate exchanges: integration of discussions into existing meetings; planning discussions over a meal if and when appropriate; sending personalised invitations through leaders in the areas of interest. All individual and group interviews (table 1), conducted using interview guides composed of open questions adapted to the group of interest, will be audio recorded and transcribed verbatim. The interview guides will address the five main categories of factors to consider in the implementation of a programme (described in the conceptual framework). Patient experience with care will be operationalised according to the six dimensions presented in the model of services integration. Data saturation is not the goal for each group, but the diversity of actors engaged will provide a complete representation of each case.39 In addition to the group discussion planned with the high users of services in each HSSC, additional samples will be recruited in years 2 and 3 for individual interviews among people who have had the most and least improvement in quality of life over a 1-year period (total n=8 in each HSSC). These interviews will allow us to examine the factors that contributed to or hindered an impact on this variable.

2. Participant observation (qualitative data)

The developmental evaluation approach28 proposes the active participation of the research team (research assistants and principal investigators) within the partner HSSC. This participant observation will contribute to keeping the HSSC informed periodically of the results of the evaluative process and to observe changes occurring in the field40 to better assess transformation processes.33 Participant observation will take place during quarterly meetings (principal investigators) with the Director-generals, CM programme directors and two clientele representatives in the HSSC and during meetings of the committees designated to organise care for high users (research assistants). Data will be collected using a logbook and field notes.38

3. Document analysis (qualitative data)

The analysis of documents on the subject of programme implementation will serve to corroborate and complete the information obtained through the other data collection methods.40 Two main sources of documents will be used: documents on the CM programme of high users of services and meetings records of the committees designated with the organisation of care for high users.

4. Clinical and administrative data (quantitative)

Utilisation of services will be considered from an organisation perspective. This way, the number of high users of emergency and hospitalisation services and the ratio of high user visits/total visits will be determined monthly for each HSSC using the already operational Magic Chronique computer application, which uses interfaces that are similar in all four HSSC. In the absence of a consensus definition of frequent users in the literature, the one retained at a regional scale will be used (six visits or more to the emergency room in the past year or 3 hospitalisations or more). The ratio compiling all high user visits/total visits will also be determined for each HSSC. Quality of data will be controlled using an integrated model of information quality and using a series of algorithms for the validation of data. These data will also be collected retrospectively for all HSSCs (on a monthly basis in the year preceding the start-up of the project).

5. Questionnaires (quantitative data)

Each project year will coincide with the recruitment of the new cohort of high users in the CM programme of each HSSC for a period of 1 year. The French-language questionnaires, in which metrological qualities are well documented and adequate, will be administered, following informed consent, to all persons (100 patients from the HSSC of Chicoutimi and HSCC of Jonquière, and 75 patients from the HSSC of Alma and HSSC of La Baie) at entry into the CM programme (sociodemographic questionnaire, health literacy, patient activation, multimorbidity and quality of life) and at 6 months and 1 year (quality of life). The sociodemographic questionnaire will assess age, sex, income and education of the participants. Literacy will be measured using the Newest Vital Sign41 and patient activation with the Patient Activation Measure.42 43 Multimorbidity will be evaluated with the Disease Burden Morbidity Assessment by self-report44 45 and quality of life, using the SF-12v2.46 47

Data and statistical analyses plan

In the first step, the qualitative data will be analysed by the research assistants and principal investigators according to a qualitative content analysis process in three iterative steps, to identify emerging themes and trends: data reduction, data organisation (matrix-schemes) and their interpretation.48 The evolving context and inability to control the environment in which the programmes will be evaluated render the use of an experimental design inappropriate to evaluate quantitative effects (use of services and quality of life).22 49 Rather than aim to perform a non-biased estimation of the extent of the effects of CM programmes, the quantitative data will first be analysed, then interpreted in integration with the qualitative data. For use of services, we will use an interrupted time series evaluation approach,50 where monthly measures (12 measures each year) over the year preceding the start-up and during the carrying out of the study will first allow us to perceive trends and their stability over time.51 Regression analysis by segment will then allow us to explore a change in trend or level between each study cycle (each year).51 For quality of life, we will perform multiple regression analysis for each HSSC linking change (SF12v2at entry—SF12v2one year later) in quality of life (dependant variable) to participant characteristics while introducing the ‘cohort’ variable (1, 2 or 3) as an independent variable to explore if year of participation in the programme seems to have an impact on change in quality of life. The quantitative analyses will be performed using the SAS V.9.2 software.

Two strategies will be used to guide the second stage of the data analysis: description and comparison of cases, and integration of qualitative and quantitative data.52 We will first proceed with the isolated analyses of each of the four cases using all the qualitative and quantitative data. One case history grouping all the relevant qualitative and quantitative data will be drafted throughout the process for each HSSC, thus allowing us to manage the large amount of qualitative data collected.27 Triangulation of data, at the data source level and at the level of the different evaluators, will ensure validity of the case histories and allow us to integrate the two types of data for a better understanding of the results. This triangulation will also ensure a certain coherence with the search for significance of the developmental evaluation approach.33 The four case histories will then be used as a basis for the comparison between cases at the end of the study to answer the third research question with the help of descriptive and interpretative multiple level matrixes allowing for systematic comparisons between cases and between the three units of analysis (macro, meso and micro).48 Different analytical techniques for the multiple case studies will be used, such as comparison of patterns, search for rival explanations and the construction of explications.27 Data management and reduction will be realised with QSR*NVIVO 10 software. All team members will participate in key stages of the analysis to ensure a good interpretation.

Potential biases

Given the changing environment and the multitude of variables that can influence the measured quantitative variables (use of services, activation, quality of life), it will be difficult to measure the direct impact of the programme using these variables. It is for this reason that the quantitative data will first be analysed, then interpreted in integration with the qualitative data.

A second important concern is external validity. It represents a potential bias if we try to transfer our results to different contexts. However, multiple case studies will allow us to ensure transferability to other contexts, through the theoretical enlightenment provided and the reproducibility of observations in many cases.

Ethics and dissemination

Informed consent will be obtained from each person recruited for the interviews and group discussions as well as from the patients who complete the questionnaire. Confidentiality will be respected and data will be stored following the rules currently applied with respect to duration and security. All publications will respect confidentiality.

Findings will be disseminated by publications in peer-reviewed journals, international, national and regional conferences, and policy and practice partners in local and national government.

Status of the study

The full study is expected to last 3 years, from September 2014 to August 2017.

Discussion

The project will have a lasting impact on CM programmes of the partner HSSCs. First, because of the developmental evaluation approach, decision-makers were engaged significantly, at an early stage, facilitating knowledge translation.24 28 Then, the early and constant feedback to stakeholders will allow us to provide evidence that may positively influence decisions to improve programmes, while at the same time maximising their chances for sustainability. The researchers’ role will play out well beyond the data collection and analysis; they will be able to actively intervene to influence the course of the process by informing decision-making and by facilitating learning.33 Finally, decision-makers will be able to put forward the characteristics identified in the clienteles and CM programmes to contribute to a better impact on use of services, quality of life and patient experience.

Considering the organisational and major financial impact of high users of hospital services and considering that CM is now proposed by many bodies2 53 54 to better respond to the complex needs of this clientele, the answer to the research question will be of interest to many decision-makers in the healthcare system. This project will provide relevant results, more specifically in regard to characteristics of the clientele and of the programmes contributing to positive impacts on organisations and patients, as this topic remains unanswered in the literature. Many strategies will thus be undertaken during the case study, so that project conclusions can be transferable to other contexts: theoretical enlightenment; reproduction of observations in many cases;27 and in depth description of programmes, actors and context.48