The contribution of case study research to knowledge of how to improve quality of care.

BACKGROUND: Efforts to improve the implementation of effective practice and to speed up improvements in quality and patient safety continue to pose challenges for researchers and policy makers. Organisational research, and, in particular, case studies of quality improvement, offer methods to improve understanding of the role of organisational and microsystem contexts for improving care and the development of theories which might guide improvement strategies.
METHODS: This paper reviews examples of such research and details the methodological issues in constructing and analysing case studies. Case study research typically collects a wide array of data from interviews, documents and other sources.
CONCLUSION: Advances in methods for coding and analysing these data are improving the quality of reports from these studies.

The gap between the knowledge of what works and the widespread adoption of those practices has become a major preoccupation of researchers and a challenge for funders and policy makers.1–3 Recognition of this ‘quality chasm’ (the term that the US Institute of Medicine used to describe the distance ‘between the healthcare we have and the care we could have’4) has led to an increased focus on quality improvement and implementation science to advance understanding of how to promote evidence-based practice. In turn, the focus on implementation has led to the development of multiple theories and frameworks to guide implementation,5–7 but no framework has demonstrated widespread results in practice.

There seems to be no immutable formula for successful implementation of innovations. While rational decision-makers would like the effectiveness of new technologies (including new work routines, devices and medications) to be the primary determinant of their adoption, research suggests otherwise. Healthcare systems are complex and variable. While some teams or organisations provide a ‘receptive context’ for innovation,8 others resist, having limited interest or abilities to implement new ideas. Decades of research in organisational and social sciences suggest that the nature of the innovation and the organisational, professional and health system contexts into which they are introduced influence their adoption.7 9–11 Thus, creating more effective, evidence-based care relies not just on developing and disseminating the evidence, but also on building knowledge of the ways in which innovations can be embedded into ongoing practice. Understanding the structures and processes of change is as critical as the knowledge of what works. In this paper, we outline how case study research can contribute a more detailed understanding of how to improve care. Case study methods are underutilised in quality improvement research, and given the growing calls to understand how innovation works in different contexts12–14 these methods could be a valuable addition to current approaches. We begin by illustrating the insights from case study research, and then examine the contribution of case study research to theory. Next we discuss strategies for analysing case study data and the scientific soundness of such information, ending with a discussion of the need for case studies to enhance the scientific understanding of quality improvement.

Insights from case study research

Three examples of how qualitative organisational research informs our understanding of the adoption of healthcare innovations illustrate the value of this research. Denis and colleagues15 studied the adoption of four innovations in several Quebec hospitals. They found that the strength of evidence of the innovation was not the only factor influencing adoption. Organisational arrangements, clinical skills and other more ambiguous elements that were open to interpretation and negotiation were also critical. In another study examining innovations in acute care and primary care settings in the UK, Ferlie16 identified the critical role of boundaries between professional groups. Unlike some prior studies where high levels of professionalisation facilitated adoption of innovations, Ferlie's research found that the varying roles, social boundaries and distinctive cognitive styles of different professional groups can limit the adoption of new technologies. For example, the introduction of an anticoagulation service was slowed by disagreements between cardiologists, primary care physicians, nurses and IT system designers about the appropriate indications for treatment.

The adoption of minimally invasive cardiac surgery for coronary artery bypass graft or valve replacement surgery in 16 US hospitals provides a third example. Edmondson and colleagues17 found that successful implementation depended on team learning processes rather than resources, academic status or innovation history. Innovative procedures like minimally invasive cardiac surgery disrupt established work routines. Establishing the necessary new routines for minimally invasive cardiac surgery depended on staff perceptions of psychological safety (the sense that ‘well-intentioned interpersonal risks will not be punished’), team stability and a collective learning process supported by leaders.

Each of these research projects used case study methods to identify the novel aspects of the process of implementing innovation. The research teams collected and analysed data from interviews, clinical data and documents. These research projects examined individuals or teams in context; they were embedded multiple case designs.18 Although the researchers had detailed knowledge of potentially relevant factors, these were primarily exploratory studies, examining which aspects of the innovation, the individuals and teams and the larger organisations influenced the adoption of the innovation.

The case study methods used in these three studies offer valuable tools in exploring the effectiveness of quality improvement more broadly. While case study research is a well-established method in organisational research, it appears to be less common in organisational health services research. Case study research designs involve the collection of qualitative (and often quantitative) data from various sources to explore one or more organisations or parts of organisations and the characteristics of these contexts.19 Some criticise case study research because they believe that the small sample size and lack of controls undermine the ability to generalise,20 while others worry that the analysis of case study data is often unsystematic.21 Yet case studies, because they detail specific experiences in particular contexts, offer the opportunity to learn more about the relationship of organisational processes and context to the success or failure of quality improvement efforts.

Contributions of case studies to theory

Case studies can inform the development of more robust theory that identifies the links between problem, intervention and outcome. Robert Yin, in his classic book,22 notes that case study research is particularly helpful when researchers want to answer questions of how or why things work in real life contexts. Theory generated from cases may help to make sense of the complex relationships that underline healthcare practice and elucidate why efforts to improve care succeed in some circumstances, but not in others.

Christensen and Carlile23 note that theory building (the creation of a ‘body of knowledge’ or understanding) occurs in two ways or stages; first there is a descriptive or inductive stage where researchers observe phenomena and describe and measure what they see (see figure 1). Based on these observations, researchers develop constructs that abstract the essence of what has been observed, classify or categorise these observations, and identify relationships between them. Through these activities, researchers develop theories or models which organise the aspects of the world they study. Second, in a deductive process, researchers test and improve these theories by exploring whether the same correlations exist in different data sets. This hypothesis testing allows the theory to be confirmed or rejected, and it also permits further specification of the theory to define the phenomena more precisely or specify the circumstances under which correlations hold. Where the goal of research is discovery or new explanations, case studies may offer a more powerful research design than experimental methods.24 25

Figure 1

Process of building theory.

Edmondson and McManus26 add to Christensen and Carlile's outline of the process of theory building and testing by identifying the importance of ‘methodological fit’ between theory building and different research methods. They suggest the appropriateness of different types of data varies depending on the research questions posed, the current state of the literature and the contribution envisaged from the research. Qualitative data, including interviews, observation and document analysis, are most appropriate for research where theory is nascent, and the research questions are exploratory. On the other hand, where theory is mature, survey methods and statistical testing focused on confirmation of hypotheses are more appropriate.

Organisational case studies have been an effective way to build theory in organisational research.18 Eisenhardt and Graebner27 note that ‘[a] major reason for the popularity and relevance of theory building from case studies is that it is one of the best (if not the best) of the bridges from rich qualitative evidence to mainstream deductive research. Its emphasis on developing constructs, measures and testable theoretical propositions makes inductive case research consistent with the emphasis on testable theory within mainstream deductive research.’ Some authors28 argue that single case studies provide more detail and offer ‘better stories’ which are helpful in describing phenomena. But others assert that multiple case studies provide a stronger base for theory building.22 27 Multiple case studies are powerful, since they permit replication and extension among individual cases. Replication enables a researcher to perceive the patterns in the cases more easily and to separate out patterns from change occurrences. Different cases can emphasise varying aspects of a phenomenon and enable researchers to develop a fuller theory. Fitzgerald and Dopson19 identify four common types of multiple case study designs, each based on a different logic. These include (1) matching or replication designs intended to explore or verify ideas; (2) comparison of differences, including cases selected for their different characteristics; (3) outliers, comparison of extremes to delineate key factors and the shape of a field; and (4) embedded case study designs where multiple units are examined to identify similarities and differences.

Despite growing numbers of studies on quality improvement in healthcare, there is limited growth in a more general theory about improvement. For example, there is a growing view that improvement interventions should be tailored to potential barriers. Yet, as Bosch notes,29 in many cases it is difficult to assess whether such tailoring was done based on a priori barrier identification, and explicit use of theory to match the intervention to the identified barriers. Bosch adds that ‘the translation of identified barriers into tailor-made [quality improvement] interventions [and their] implementation is still a black box for both educational and organisational interventions’ (p. 161). Case studies might contribute useful information to develop relevant theory. More broadly, case study research provides methods to examine organisational processes over time, examining the interplay of interventions with team dynamics or leadership strategy. For example, studies by Baker30 and Bate31 of high-performing healthcare organisations illustrate the challenges of creating, spreading and sustaining effective practice in organisations. Some case study research has followed organisations over extended time periods repeating interviews with key informants (eg, Denis' work on strategic change40 41). Unlike survey research and RCTs, case study research can analyse the process of implementation and unpack the dynamics of change.

Data collection and analysis

Organisational case studies can include a wide array of data, including interviews, documents, ethnography, survey data and observations. Although the case study is generally viewed as a qualitative method, it may include quantitative data. For example, Greenhalgh's study of the impact of ‘modernisation initiatives’ of the delivery of care in London42 used a wide range of methods and data, including interviews, document analysis and ethnography. Other organisational case study research17 32 40 43 has adopted a similar mix of data sources.

Case study research typically generates large quantities of data, which makes analysis critical, but complex. Moreover, the methods for aggregating data across projects are not well developed. Coffey and Atkinson note that the use of coding and sorting, and the identification of themes are ‘an important, even an indispensable, part of the qualitative research process.’44 Yet, there are challenges to such methods, since coding individual experiences can lead to ‘decontextualisation,’ fragmenting such meanings and making them difficult to identify.45 These problems are accentuated in multiple cases where results may reflect differences between the methods used, or the interests and orientation of various researchers. Even within the same research project, different investigators may take the lead in different cases. Dopson adds several other considerations about chronology: ‘Were the studies synchronous? Were they prospective or retrospective? Were they longitudinal or cross-sectional? How variable were the political and organisational contexts?’ (p. 6).32 Multiple case studies are difficult to report, given the space constraints for journal publication,27 and the use of extensive tables risks mimicking the presentation of quantitative data, stripping the illustrative detail from the case presentations.19

Synthesis across studies can help to build a more generalisable understanding of organisational strategies to support improvement. Yet views vary on whether we can synthesise research from multiple case studies undertaken independently. In their review of studies examining efforts to integrate evidence into clinical decision-making in UK healthcare, Dopson and colleagues32 compared and synthesised their findings reanalysing the original studies to identify themes, recoding their reports and then assessing the outputs generated by the five researchers involved (see table 1). Such tables offer a bird's-eye view of the extent to which common themes inform different case studies, but such summaries are divorced from understanding how these issues are inter-related within each case.

Table 1

Identifying research themes across studies of innovation diffusion32

Theme	Dopson and Gabbay33 (1995)	Wood et al34 (1998)	Dawson et al35 (1998)	Clinical Standards Advisory Group36 (1998) (Gabbay et al)	Fitzgerald et al37 (1999)	Dopson et al38 (1999)	Locock et al39 (1999)
1. Evidence is not sufficient	3	2	3	3	3	3	3
2. Evidence is socially constructed	2	3	3	2	3	3	3
3. Evidence is differentially available	2	3	2	3	3	1	2
4. Hierarchies of evidence exist	3	3	3	3	2	3	3
5. Other sources of evidence	2	2	3	3	3	2	3
6. The importance of professional networks	2	3	3	3	3	3	3
7. The role of professional boundaries	2	3	3	3	3	2	2
8. Context as an influence	3	2	3	3	3	3	3
9. The role of opinion leaders	2	2	3	3	3	3	3
10. The enactment of evidence	3	3	3	3	3	3	3

1, Theme is present; 2, strong evidence of theme; 3, very strong evidence of presence.

Methodological rigour

Efforts to create such syntheses raise issues about methodological rigour. For those researchers who adopt a positivist framework, the test of good case studies builds on four criteria used to assess the rigour of field research: internal validity, construct validity, external validity and reliability.22 These criteria might be applied to case studies in the following ways (see table 2).

Table 2

Framework for an investigation of the methodological rigour of case studies40

Internal validity	Construct validity	External validity	Reliability
Research framework explicitly derived from literature (diagram or explicit description of causal relationships)	Data triangulation (archival data; interview data; participatory observation; direct observation)	Cross-case analysis (either multiple case studies or nested cases within one organisation)	Case study protocol (how was case study conducted)
Pattern matching	Review of transcripts and drafts by peers	Rationale for case study selection (and why appropriate in view of research questions)	Case study database (with all available documents, interviews transcripts, etc)
Theory triangulation (different theoretical lenses or bodies of research used as research framework or for interpretation)	Review of transcripts and drafts by key informants	Details on case study contexts	Organisation's actual name provided explicitly in the report
	Clear chain of evidence
	Clear explanation of data-collection circumstances
	Clear explanation of data analysis procedures

Gibbert and colleagues46 reviewed case studies published in the organisation/management literature between 1995 and 2000. They found research procedures enhancing external validity in 82 of 159 papers, and procedures supporting reliability in 27 of these papers. Few papers provided evidence of internal or construct validity. Yin proposes pattern matching; explanation building; addressing rival explanations and using logic models as strategies to address internal validity.22 Eisenhardt offers a series of questions that reflect on the match between method and results: ‘Have the investigators followed a careful analytical procedure? Does the evidence support the theory? Have the investigators ruled out rival explanations?’18 (p. 548). Non-positivist researchers employ other methods to ensure the soundness of their findings; for example, see Lincoln and Guba.47

An alternative measure of the rigour of case study research focuses on how good the theory is that emerges from this research. Pfeffer48 suggests that good theory is parsimonious, testable and logically coherent. Good theory should also address critical issues of interest to organisations and interested parties. Insights from other disciplines and attempts to seek out anomalies in other authors' work that might inform research in different areas are other strategies that may enrich the quality of case study research, improving the theory that results.48

Despite the need for more robust theory, why are there so few organisational case studies of quality improvement? Some candidate explanations might include: (1) the limited number of organisational scholars working in this area; (2) the dominance of alternative research paradigms that dismiss case study research; (3) difficulties in securing funding; (4) the lack of publication outlets; and (5) the absence of a clear understanding of the relationship of case study research to the development of theory, and the testing of theory using randomised control trials and other methods. Still, the emergence of several strong research groups in the UK, Canada and the USA, and growing numbers of high-quality publications offer hope. What is missing in quality improvement research is a clear understanding of how case study research could contribute to the broader research enterprise, enriching the qualitative understanding of the complex processes of improving healthcare delivery.

Conclusions

Comparative case study research provides useful methods for identifying the factors facilitating and impeding improvement. Although valuable in their own right, such methods also offer the opportunity to enrich more traditional approaches to assessing interventions, helping to explain why some interventions are unsuccessful, or why they seem to work effectively in some contexts but not in others. Efforts to improve patient safety and quality of care need to take into account the complexities of the systems in which these improvements are being introduced. Case study methods provide a robust means to guide implementation of effective practices.