Emergent programme theories of a national quality register - a longitudinal study in Swedish elderly care.

RATIONALE, AIM, AND
OBJECTIVE: This study aimed to explore programme theories of a national quality register. A programme theory is a bundle of assumptions underpinning how and why an improvement initiative functions. The purpose was to examine and establish programme theories of a national quality register widely used in Sweden: Senior alert. The paper reports on how programme theories among change recipients emerge in relation to the established programme theory of the initiator.
METHODS: A qualitative approach and a longitudinal research design were used. To develop programme theories among change recipients, individual semistructured interviews were conducted. Three sets of interviews were conducted in the period of 2011 to 2013, totalling 22 interviews. In addition, 4 participant observations were made. To develop the initiator's programme theory, an iterative multistage collaboration process between the researchers and the initiator was used. A directed content analysis was used to analyse data.
FINDINGS: The initiator and change recipients described similar programme logics, but differing programme theories. With time, change recipients' programme theories emerged. Their programme theories converged and became more like the programme theory of the initiator.
CONCLUSIONS: This study has demonstrated the importance of making both the initiator's and change recipients' programme theories explicit. To learn about conditions for improvement initiatives, comparisons between their programme theories are valuable. Differences in programme theories provide information on how initiators can customize support for their improvement initiatives. Similar programme logics can be underpinned by different programme theories, which can be deceptive. Programme theories emerge over time and need to be understood as dynamic phenomena.

INTRODUCTION

With its approximately 45 000 approved users, Senior alert (SA) is a national quality register (NQR) with wide use in Sweden.1 Senior alert is a process‐oriented NQR supporting healthcare staff in their use of evidence‐based risk assessment scales to systematically prevent pressure ulcers, malnutrition, bad oral health, bladder dysfunction/incontinence, and the tendency to fall among the elderly.2 Senior alert and 3 other NQRs were included in a national programme aiming to improve the quality of care for frail elderly with complex health issues. The programme lasted from 2010 until 2014 and comprised multiple subprojects based on a holistic view of healthcare. Based on the governmental decision to introduce performance‐based incentives, remunerations were used to stimulate healthcare organizations' affiliation to the NQRs.3 The unique civic registration numbers, all Swedish residents have facilitated registry work. The objective of NQRs is to support evidence‐based work and quality improvement and to enable comparisons and research.4 The number of NQRs has increase significantly in the last decade.5

In 2016, 270/290 municipalities and 16/20 county councils/regions were affiliated to SA. Despite the wide dissemination on a national level, the use of SA varies markedly within, and between, affiliated organizations.1 The variation of use highlights that the elderly are not offered risk‐prevention activities in similar ways.

Varied and high failure rates reported in studies of improvement initiatives highlight the need to study in more detail the conditions favourable for change.6 In line with this, some healthcare researchers have actualized the necessity of studying theories underpinning improvement initiatives. Knowledge of these theories is argued to lead to deeper understanding of the behaviours of change recipients, the functionality of the improvement initiatives, and the results they deliver.7, 8 The proposal to study theories behind improvement initiatives is supported in this paper, and the concept of programme theory (PT) is used in this effort. In this paper, “change recipients” refers to clinical microsystems (CMSs) at work units taking on SA. Clinical microsystems are groups of patients, families, and care teams working regularly together to improve care for specific patient groups.9 “Initiator” refers to individuals with extensive knowledge of SA. These SA experts are employed by the NQR to represent, develop, and initiate dissemination nationally.

The purpose of this study is to examine and establish the PTs of SA in CMSs at work units in elderly care. By comparing their PTs with that of the initiator, the paper reports on how PTs in CMSs emerge in relation to the established PT.

THEORETICAL FRAMEWORK

Scholars have used various concepts for theories underpinning improvement initiatives, and there is a certain conceptual ambiguity. In particular, the concepts of programme logic and PT have been mixed together. Astbury and Leeuw10 have offered definitions of the adjacent concepts. They have argued that programme logic is the description of how an improvement initiative is outlined, often as a causal process map visualizing the components of the improvement initiative.

A PT is deeper and concerns the underlying assumptions of how and why the components of the programme logic function and interrelate. The notion of the PT was introduced by evaluation researchers and was described as “the construction of a plausible and sensible model of how a program is supposed to work.”11 All improvement initiatives are based on PTs, but they are seldom explicit.7, 12, 13 Without understanding of the PT, change recipients risk copying routines and behaviours from the initiator, without knowledge of how to contextualize the change. This reduces change recipients' possibilities to succeed with the improvement initiative.14 Understanding of PTs also supports the generalizability of improvement initiatives.13 Thus, knowledge of PTs is an essential condition for improvement initiatives.7, 15

Programme theories have mainly been studied as static phenomena, as if they stay unaffected over time. This ontology is in opposition to sense‐making theory, which claims that interpretations are prone to change. According to sense‐making theory, interpretations develop retrospectively, based on shared experiences.16 Thus, taking a sense‐making perspective, the importance of studying timely changes of PTs to gain deeper understanding of the emergent conditions for improvement initiatives is emphasized in this paper. The importance of including the perspectives of both the initiator and change recipients is also underlined. Scholars have made transparent that a discrepancy between how initiators and change recipients interpret a change hampers the possibilities for success.17 Hence, to gain a detailed understanding of the conditions for SA, both the initiator's and CMSs' PTs are studied in this paper.

METHODS

Research design and included work units

In this study, a qualitative approach was used, and the research design included 2 perspectives of PT. One perspective covered the PT of the initiator of SA, and the other the PTs of CMSs. The research was longitudinal on a microsystem level and extended over a period of 3 years (2011‐2013). It was a collective case study, and 3 cases (CMSs at work units) were included. It was an instrumental collective case study, meaning that the cases were used to study a specific phenomenon within the cases, rather than the cases as such.18 In this research, the cases were used to study PTs of SA.

The choice of work units was based on a nonprobability and purposive choice strategy.18 External change agents with knowledge of the maturity of SA work suggested municipal nursing homes that were in the initial stages of SA work.19 Three work units agreed to participate and to protect their confidentiality, their names were substituted with letters in this paper. “A” was a municipal nursing home for 50 residents in a rural area in mid Sweden. “B” was a municipal nursing home for 38 residents in a rural area in southern Sweden, and “C” was a large municipal nursing home for 100 residents situated in a larger city southwest of Sweden. The managers of the included work units suggested CMSs for the study. The included participants in the CMSs were individuals who had volunteered to participate.

Data collection

To develop the initiator's PT, an interactive approach was used.20 The data collection method was an iterative multistage collaboration process between the researchers and the 2 SA experts. On the basis of information on SAs webpage,1 the researchers made the programme logic of SA explicit. From the programme logic, 4 generic concepts were extracted: change recipients, material and artefacts, actions, and outcomes. The generic concepts, together with the programme logic, guided the researchers as they collaboratively developed a draft PT of SA. The draft was sent to one SA expert for revision. With proposed changes implemented, the PT was sent to the second SA expert. Once again, the PT was revised, and the researchers collaborated to develop a PT they could agree upon. This version was sent to the second SA expert, and after minor changes highlighted in a telephone meeting with the first author, the PT was approved.

The primary data collection method to develop CMSs' PTs was individual semistructured interviews. The interview guide was developed in collaboration between the researchers and included questions enabling participants to talk about and reflect on their perceptions and routines for prevention, experience with SA, and outcomes. The interviews lasted approximately 40 minutes and were tape‐recorded and transcribed verbatim. Three sets of interviews approximately 1 year apart were conducted (2011‐2013), totalling 22 interviews. Fifteen individuals were interviewed: 3 managers, 2 nurses, and 10 assistant nurses. Because of flexible staff schedules and round the clock staffing, it can be difficult to interview the same individuals each year in a longitudinal study. Thus, the longitudinal aspect of the study is not on the individual level but on the CMS level. However, 5 individuals participated in 2 or 3 interviews. Four participant observations at CMS meetings were also conducted, and the dialogues in these CMS meetings were transcribed verbatim.

Coding and analysis

The elements of the initiator's PT were transformed into a coding scheme by the first author. In collaboration, the authors refined the coding scheme, and the final version was entered into a software supporting qualitative research (NVIVO). The first author made a deductive coding of the transcribed material (interviews and CMS meetings) and registered the coding systematically in the software. The software summarized the coding into a report for each CMS, with meaning units sorted after codes and years. The first and third authors read the reports and discussed differing opinions regarding coding until consensus was reached. The first and third authors collaborated in a directed content analysis, including both latent and manifest content.21 For all CMS, the researchers made a close reading of meaning units under each code. Coherent content of meaning units was discussed and compared with the corresponding element in the initiator's PT. The lack of meaning units under codes was also interpreted, and the comparisons were written up as key sentences. To detect timely changes, differences in key sentences between years were analysed. These timely changes were also articulated in key sentences. Finally, the key sentences were used to write up each CMS' PT, including how the PT developed over time (see SUPPORTING INFORMATION). In the last step of the analysis, the CMS' PTs were compared with the initiator's. The findings are illustrated by citations from participants, translated by the authors.

Ethical considerations

The study was based on several research ethical considerations.22 Researchers' objectivity, the participants' experience of the voluntary nature of the study, and their benefits from participating in it were subjects for close examination. The considerations did not reveal any ethical obstacles to the study. The participants were given oral information about the purpose and procedures of the study, and the principles of confidentiality and voluntariness were emphasized. The participants gave verbal consent to participate.

FINDINGS

The initiator's PT

In Figure 1 (below), the generic concepts and the initiators' programme logic and PT are depicted.

Figure 1

Senior alert (SA), the initiator's perspective. CMS, clinical microsystem

At the first level, generic concepts highlight 4 concepts constituting the general structure of the model. Reading from the left to the right, the concepts highlight that “change recipients” together with some kind of “material and artefacts” perform “actions,” which in turn lead to “outcomes.” At the second level of the model, the generic concepts are used as a structure to outline the programme logic of SA. Reading horizontally from the left to the right, change recipients are CMSs, healthcare staff, leaders, and elderly/patients. The material and artefacts needed for the preventive work are computers, proposals for measures, risk assessment scales, and standardized outcome reports. The actions of prevention are risk assessing, registering in SA, taking preventive actions, following up outcomes, and improving. These actions are expected to lead to improved outcomes. The programme logic is underpinned by several assumptions, constituting the PT at the third level of the model.

Reading from the left to the right, the PT highlights the assumptions that change recipients are motivated to work with SA. It is also assumed that change recipients are knowledgeable (informed about responsibilities), supportive (to others), and driven by determination (not giving in to resistance). Change recipients are believed to be cooperative and courageous (taking independent responsibility for prevention). Under the concepts material and artefact, the PT highlights the assumptions that computers are available and that processes and routines are clear and customized. Furthermore, the PT comprises assumptions regarding SA: that the NQR is integrated into the preventive work, that it provides transparency, and supports collaboration and improvement. Under the action concept, the PT highlights several assumptions regarding what change recipients do. Finally, under the generic concept of outcomes, the PT highlights the assumption that prevention activities need to be improved.

The CMSs' PTs

Change recipients

The initiator's PT included assumptions of change recipients having the motivation, determination, and knowledge to support SA work. This clearly diverged from the CMSs' own PTs. Clinical microsystems assumed change recipients, with the exception of a few key persons, avoided rather than supported SA.

But the rehab staff are the ones I can see reacted the strongest; they see that this is extra work, or we do already do this; so why should we? Participant 1, CMS in C, 2011.

However, CMSs' assumptions concerning change recipients developed over time and increasingly resembled that of the initiator. This was especially true for levels of knowledge.

I think that all my work really is about being a little bit ahead and seeing. Actually, SA is great as it gives me control. Participant 15, CMS in A, 2013.

Material and artefacts

Assumptions about materials and artefacts initially differed between the initiator and CMSs, but the differences were smaller and declined with time. During the study period, the processes and routines of SA were customized and incorporated, and SA provided enhanced transparency.

In 2013, SA was integrated in daily work and supported improvement activities. However, CMSs hardly saw how SA supported collaboration.

No, we have not talked about Senior alert, actually.

Participant 4 in C, 2012.

Actions

Clinical microsystems' assumptions concerning actions developed and came to correspond more with the initiator's assumptions over time. Clinical microsystems customized and incorporated processes and routines. To run improvement tests, 2 CMSs (A and B) discussed and reflected on processes, routines, and outcomes.

Otherwise, we have constant communication, verbal communication about changes and what we should do about it.

Participant 9, CMS in B, 2013.

Now it's been said that we should recall data ourselves and show our staff.

I should show this at meetings, so we need to learn how to create these graphs. Participant 12, CMS in A, 2013.

In 2011, CMSs could not recognize that SA supported risk prevention for the elderly. Furthermore, the leadership support was weak, and the staff hardly ever overviewed the preventive work on the process/microsystem level. However, during the study period, the learning developed and included new areas and larger cohorts of healthcare staff. The CMS in unit B gained improvement knowledge that could be beneficial in other improvement efforts.

Outcomes

There were no major differences between the initiator's and CMSs' assumptions concerning outcomes. Both parties assumed that outcomes had or needed to improve. Clinical microsystems' assumptions regarding what to expect from the operations became broader, including both staff prerequisites and patients. At first (2011‐2012), CMSs' assumptions mainly concerned staff prerequisites for improved outcomes: that knowledge, information, statistics, communications, and a structured work process were important. Later (2013), the assumptions included patients, and it was assumed important to include oral health, medication, pain reduction, wellbeing, and patient safety in the preventive work. Clinical microsystems' assumptions also became more demanding during the study period. They increasingly assumed that there was an urgent need to improve and that it was crucial to shift from a reactive to a proactive work approach.

That the routines become good and efficient, try to work with continuous improvements and to identify the … work processes.

We'll try to create more quality for the care recipients.

Participant 9, CMS in B, 2013.

All CMSs improved their outcomes. Levels of SA registration improved and included the majority of patients. Clinical microsystems believed that they had fewer fractures, better weight control, better nutrition, fewer pressure ulcers, fewer falls of the elderly, better oral health, and a better patient focus.

Detection is now much earlier; if someone is just a little red, we immediately work at full throttle to check up.

Participant 10, CMS in B, 2013.

ANALYSIS

In the analysis, (dis)similarities, confusion, and (dis)connections between the initiator's and CMSs' PTs appeared.

Similar programme logics but dissimilar programme theories

The initiator and CMSs described similar programme logics; however, the analysis revealed several differences concerning underpinning assumptions, which was why the surface level similarity could be misleading.

Confusion between motivation and discomfort

Clinical microsystems reported lack of motivation. Despite this, they improved their SA work and increased their expectations regarding outcomes, which indicated that they were motivated. Clinical microsystems assumed that they would not be supported in their work, leading to feelings of discomfort.

Confusion between teamwork and solitary registration

Clinical microsystems assumed SA did not support collaboration. They assumed computer work to be solitary work, instead of seeing SA work as a team‐based process that was summarized and reported into the system by a single healthcare worker.

Connection between improved outcomes and patient‐oriented expectations

Clinical microsystems assumed administration, processes, and routines needed to improve and did not mention the need for better patient‐oriented outcomes. These assumptions developed, which implies that internal work on processes and routines led to higher expectations regarding patient‐oriented outcomes. The structured SA work thus seems to have made transparent that patient‐oriented outcomes could improve.

Emergent and converging programme theories

Clinical microsystems' PTs initially differed from the initiator's PT in central aspects. However, CMSs' PTs emerged over time. This was evident for all CMSs, even if their PTs took somewhat different trajectories. With time, CMSs' PTs converged and became more like the initiator's.

Disconnection between learning, improved work, and outcomes

Clinical microsystems' learning deepened and came to include both organization focused‐ and patient‐oriented aspects of prevention.23 Clinical microsystems' outcomes improved. The CMSs also recognized that SA provided transparency, and CMSs had greater expectations of what results they could achieve. However, CMSs had difficulties in connecting these improvements with their SA work. During the gradual development, CMSs continuously internalized the improvement activities in their regular work and thus had difficulties seeing SA as an external change agent.

DISCUSSION

Senior alert is a process‐oriented NQR helping CMSs to work systematically with evidence‐based prevention. The design of the register is in line with the so‐called safety 2 perspective.24 With this perspective, the focus is not only restricted to measuring and reducing defaults in care but also strengthens practices going well. In SA, the safety 2 perspective is manifested by the close interest in a systematic work approach and the performance of complete risk prevention work processes. This is an example of how the design of an artefact's interface can pave the way for a specific action.25 However, the results of this study also indicate that SA influenced how CMSs conceived their work, in terms of their emergent PTs. Not only did CMSs' behaviours change but also their assumptions. To learn what improvement initiatives lead to, it is important to study effects on both manifest and latent levels, and from different perspectives. Otherwise, improvement initiatives can lead to wrong decisions, undesirable effects, or a lack of knowledge about how the initiatives actually work. When changes occur on latent levels, they are likely to be unrecognized by change recipients.26 This could explain why the CMSs in this study had difficulties connecting learning, improved work, and outcomes with SA.

Complex interventions are demanding to describe and evaluate, and one complicating factor is emergence. In complex interventions, the outcomes and the means to achieve them emerge during the change process.27 To capture the developed understanding that a complex intervention process leads to, its PT may need to develop during the process. However, to our knowledge, previous research has primarily addressed PTs as static phenomena.

To capture complexity of interventions, it is also common to develop a PT that is so broad that it captures various stakeholders' assumptions.27, 28 One complication of this approach is that the PT risks reflecting some kind of “average assumptions” instead of pointing out how stakeholders' assumptions differ.

The design of this study paid attention to these complications. Instead of incorporating different assumptions into one general PT of SA, parallel PTs were developed. The initiator's PT served as the basis for the examination of CMSs' PTs, revealing that CMSs had several diverging assumptions regarding the improvement initiative. There are no claims that the initiator's PT was right and that the CMSs were wrong. On the contrary, to succeed with the improvement initiative, the initiator needed to customize the support according to how CMSs understood the initiative and not according to the initiator's own understanding.

One notable finding is that CMSs' PTs converged and with time, became more like the initiator's PT. Clinical microsystems' initial PTs were based on their previous experiences of prevention and IT tools, and as the prolonged engagement with SA provided them with new experiences, their PTs developed. Clinical microsystems developed, eg, a sense of urgency regarding further improvements, and in this sense, the improvement initiative “worked.” However, CMSs' PTs did not turn into replicas of the initiator's PT; several differences remained. Understanding of these differences could have influenced how the initiator introduced and supported the improvement initiative.

This study was preceded by methodological considerations. A directed content analysis starts in a pre‐existing theory and key concepts from the theory can be used as coding categories. When data are collected by interviews, open‐ended questions can be complemented by targeted questions regarding the predefined categories.21 In this study, the initiator's PT underpinned the analysis of CMSs. Programme theories are based on assumptions, and individuals can partially be unaware of their assumptions. Thus, it was presumed doubtful for participants to reply on straightforward questions regarding assumptions. To avoid influencing the participants with questions explicitly revealing the initiator's PT, the interview guide included semistructured questions helping participants to talk about and reflect on their work.

To examine the emergence of PTs, a longitudinal research design was chosen. This research design was considered to support the study of how CMSs continuously developed their assumptions, leading to emergent PTs.

The convergence of PTs calls for closer scientific examination in future. By studying the emergent nature of change recipients' PTs, scholars can examine latent effects of improvement initiatives.

This study has practical implications. It has revealed that it is not enough that initiators and change recipients share programme logic, because it may be underpinned by diverging PTs. Thus, initiators need to seek understanding of how change recipients perceive the improvement initiative. However, one challenge with assumptions is that people are not totally aware of them. When asked, change recipients risk responding on the basis of prevailing norms and not on the basis of their own assumptions. Consequently, to get a comprehensive picture of the conditions and results of improvement initiatives, initiators need to complement explicit metrics and discussions with analyses on interpretive and latent levels, eg, in forms of PTs.

National quality registers are designed for long‐term continuous improvement, and the quality of data is dependent on nationwide dissemination.29 With knowledge of discrepancies between initiators' and change recipients' PTs, initiators can customize dissemination support more efficiently. In the effort to identify discrepancies, the model in this study can function as a template. In the technical development of NQRs, knowledge of PTs can provide a basis for how the interface should be designed to support the desired development at both manifest and latent levels.

Leaders have been acknowledged as key to supporting quality improvement with regard to the use of a well‐established NQR.30 However, in a recent study, colleagues asking for local results were identified as the most influential factor regarding the use of NQR‐data in improvement work.31 In this study, the CMSs assumed that they would not get sufficient leadership support and they also had difficulties seeing how SA supported teamwork. This indicates that individuals in the CMSs felt abandoned, and consequently, it is critical for leaders to improve their support for SA work and teamwork.

CONCLUSIONS

Scholars have emphasized the need to make PTs underpinning improvement initiatives explicit. This study has demonstrated the importance of making both initiators' and change recipients' PTs explicit, because they can differ. To learn about conditions for improvement initiatives, comparisons between initiators' and change recipients' PTs provide important knowledge for both parties. Programme theories emerge over time and need to be understood as dynamic phenomena and latent effects of change.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

Clinical microsystems' program theories of Senior alert

Click here for additional data file.