Taking account of context in systematic reviews and guidelines considering a complexity perspective.

Systematic review teams and guideline development groups face considerable challenges when considering context within the evidence production process. Many complex interventions are context-dependent and are frequently evaluated within considerable contextual variation and change. This paper considers the extent to which current tools used within systematic reviews and guideline development are suitable in meeting these challenges. The paper briefly reviews strengths and weaknesses of existing approaches to specifying context. Illustrative tools are mapped to corresponding stages of the systematic review process. Collectively, systematic review and guideline production reveals a rich diversity of frameworks and tools for handling context. However, current approaches address only specific elements of context, are derived from primary studies which lack information or have not been tested within systematic reviews. A hypothetical example is used to illustrate how context could be integrated throughout the guideline development process. Guideline developers and evidence synthesis organisations should select an appropriate level of contextual detail for their specific guideline that is parsimonious and yet sensitive to health systems contexts and the values, preferences and needs of their target populations.

In systematic reviews and guidelines about complex interventions, considering context is challenging because it is diversely defined and dynamic.

Context is typically caricatured as a location in time, in an organisational level or within a geographical space. Reviews and guidelines rarely provide sufficient detail of the context of an intervention in terms of its location in social, cultural, political or economic space.

The extent to which reviews and guidelines of complex interventions consider context is limited by inadequate information on context in included primary studies.

Collectively, existing tools help systematic reviewers to explore or report on context holistically. However, individually the tools differ in use of terms such as context, setting, environment and location and in what they consider as relevant.

Reporting standards and templates, as well as improved reporting of context in primary studies, are needed to better account for context in systematic reviews and guidelines taking a complexity perspective.

Introduction

Complex interventions, whether clinical, health service, public health or from beyond the health sector, include multiple components that interact along non-linear causal pathways within a complex adaptive system.1 2 Interventions may share a common label and yet contain diverse content and variable components and use diverse modes of delivery, making effects unpredictable. By intervention, we include diverse procedures, practices, programmes and policies operating at different levels of system change. Yet the context or setting within which the intervention is introduced and interacts, ‘a dynamic multidimensional environment’3 constitutes the most significant aspect of this complexity.4 Decision makers need to know when, why, how and in what circumstances complex interventions work well.5 6 If a decision-maker does not understand the interplay between context and an intervention or policy (or more accurately the mechanisms by which the intervention or policy achieves its effect), then they can have only limited confidence in whether they would see the same effect in their own context.7 Such a complexity perspective poses particular challenges in a global context when producing and implementing statements that include ‘recommendations intended to optimise patient care or population health, informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’.8 Local decision makers need to decide whether a particular guideline is relevant at all, whether it is likely to achieve a better or suboptimal effect and whether the intervention needs to be modified for contextual sensitivity.

Complex interventions interact with and modify the context within which they are introduced. Sometimes complex interventions adapt to the context in which they are implemented while some interventions explicitly aim to change the contexts within which they are implemented.2 Complex interventions may alter a discrete aspect of practice (eg, how doctors interact with patients) or something more radical (eg, smoke-free legislation).8 Key decisions when producing a particular guideline are whether it is helpful and informative to view an intervention as complex.2 9 10 Not every topic justifies the added time and resource use required to explore complexity;2 11 guideline developers may conclude that contextual variation is less critical to intervention effects than intervention complexity. Where complexity is to be considered, guideline developers must decide which characteristics or sources of complexity are to be prioritised. Evidence to Decision (EtD) frameworks offer a potential menu but not all aspects need to be addressed in every review or guideline.10 Having decided whether a review will gravitate towards a complex intervention perspective or towards a complex system perspective, a review team can develop specific review questions to consider complexity and select evidence, synthesis methods and review tools accordingly.12 Qualitative13 and other forms of evidence14 may help in understanding how an intervention interacts with, reshapes and is reshaped by its target context.

This paper reviews a selection of tools for considering context and evaluates their suitability for use in reviews and guidelines that take a complexity perspective. The paper offers guidance on good practice when considering context in systematic reviews and guidelines. The paper addresses the following questions: How is context defined and understood?; Why is context important in reviews and guidelines of complex interventions?; What tools and methods are available to handle context in systematic reviews? and What might represent good practice for considering context in reviews and guidelines?

How is context defined and understood?

A plethora of definitions exists for ‘context’, each varying in the extent to which they adopt an ‘intervention perspective’ or a ‘system perspective’;2 making it problematic to privilege a single frame of reference.15 UK Medical Research Council guidance defines context as ‘factors external to the intervention which may influence its implementation, or whether its mechanisms of impact act as intended’.16 Further guidance defines context as ‘any feature of the circumstances in which an intervention is conceived, developed, implemented and evaluated’.17 Methodologically, a tension exists between the technical advantages of deciding on a single standardised definition of context and the conceptual richness of diverse and competing definitions. A systematic review should clearly articulate its internal definition of context, justify its selection and apply this definition consistently through the review process.

Context and setting, although used interchangeably, are not synonymous. The former typically carries an overarching sense while the latter is often locally bounded (eg, an acute hospital setting in a low-income or middle-income development context). Research and evaluation, broadly speaking, seek either to ignore, control, document or explore contextual variation. Many intervention and evaluation designs seek to ensure that the context for the intervention group is the same as that in the control group, so that the intervention is the only meaningful difference between groups. Certain contexts are more easily explored by randomised controlled trials (eg, secondary or tertiary care)18 but become less amenable to controlled studies on implementation. Current design and statistical approaches do not adequately deal with contextual variations within source studies and imperfectly represent ‘real world’ conditions within the target context.19 Demarcation between the outer setting (the economic, political and social context within which an organisation resides) and the inner setting (features of structural, political and cultural contexts through which an implementation process will proceed) are ‘not always clear’, given that the interface is ‘dynamic and sometimes precarious’.20 One concept analysis states: ‘Context reflects a set of characteristics and circumstances that consist of active and unique factors, within which the implementation is embedded’.21 Subsequently, the concept analysis unpacks context into eight domains (ie, locational, geographical, epidemiological, sociocultural, socioeconomic, ethical, legal and political). Context typically incorporates the institutional context (setting), the wider environment (eg, epidemiological, genetic, geographical, sociocultural, health system specific, political and others22) or a composite of both.

Authors of primary and secondary research find context challenging to operationalise, research and evaluate.23 Primary studies are criticised for failing to characterise context,17 yet many guidance documents recommend that contextual factors are examined within systematic reviews and syntheses.24 Review teams face particular challenges resulting from (1) the lack of contextual detail in included studies and (2) unsophisticated approaches to handling context within existing review tools.

The Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) reporting standards,25 an evidence-based minimum set of items to help authors improve reporting of systematic reviews and meta-analyses, refer to ‘context’ as the circumstances that explain why a review is currently needed (ie, the need for decision support and the prevalent state of knowledge). PRISMA does not even reference ‘context’ in its common usage as ‘the population targeted by implementation’. The PRISMA-CI extension for Complex Interventions, a supplemental set of items for reporting systematic reviews of complex interventions, partially addresses such limitations,26 27 extending to include the elements of Time and Setting. Nevertheless, this ‘intervention perspective’ proves unable to accommodate a complex systems perspective.2

Adding an additional component to a complex intervention (eg, a practice, behaviour or a way of organising care) almost invariably displaces something else, whether planned or not, thereby challenging conventional evaluation.4 While the context of an intervention is the focus for change, it is important to acknowledge that ‘usual care’ equally operates within a complex adaptive system. Interventions may not transfer to other contexts because usual care in those contexts is already more effective than the intervention or at least more responsive to local needs. Furthermore, studies frequently offer only a sketchy picture of what constitutes ‘usual care’.

Why is context important in reviews and guidelines of complex interventions?

Key tools and methods used in the systematic review process similarly lack clarity and consensus surrounding the term ‘context’. Context is typically either reported in insufficient detail or omitted altogether from many systematic reviews. Systematic reviews are frequently characterised as seeking to ‘strip away’ context.28 Transferability of review findings, particularly to Low and Middle Income Countries (LMICs), depends on resources, health systems, cultural norms and the physical environment.29 Wider considerations include geographic, seasonal, environmental, cultural, economic, sociopolitical, regulatory and organisational inner (eg, payment policies) and outer (eg, organisational cultures) settings.30 Differences between national health services and insurance-based health systems are similarly important within high income countries.30 Above all, those examining context should seek to extend analysis beyond the caricaturisation of time and place to explore other, less salient, features of contextual variation.

Table 1 characterises three different evaluation perspectives ranging from an intervention perspective, through a more hybrid ‘partial system’ approach, to the system perspective. The table reveals increasing complexity with subsequent rows increasingly being likely to require, and to reward, examination of contextual variation.

Table 1

Perspectives mapped to role of context and evaluation activities

Perspective	Role of context	Characteristic evaluation activity	Important elements of context to consider
Intervention perspective	Context tends to have a ‘disruptive’ role in enhancing or impairing intervention effects	Health Technology Assessment*	Characteristics of usual careExisting skills, training and expertiseResources acceptability and feasibility
‘Partial’ system perspective	Contextual variation potentially explains differential effects across organisations	Health Services Delivery Research	Organisational context, culture and valuesAcceptability and feasibility
System perspective	Contextual variation represents a form of ‘chaos’ against which evaluators identify common mechanisms	Health Systems and Public Health Research	As above, but extended to include economic, cultural, social, gender and political factors

The above perspectives lie on a continuum; a ‘partial system perspective’ characterises a scenario that acknowledges selective aspects of a wider system but does not engage with the full range of system factors.

*Increasingly initiatives such as EUnetHTA or INTEGRATE-HTA endorse a societal perspective of health technology assessment (HTA). Organizations such as the WHO are increasingly moving from clinical recommendations to clinical/health system (hybrid) guidelines, further blurring such distinctions.

Systematic review and guideline production seeks to ‘globalise the evidence, localise the decisions’.31 This globalise-localise strategy distinguishes three ‘contexts’ of evidence production: (1) the context of the review question (the review context); (2) the contexts of included studies (the source contexts) and (3) the implementation context into which recommendations are introduced (the target context). So, a review focused on low-income and middle-income countries (the review context) may access and include studies conducted in Africa and South America (source contexts) to inform practice in South East Asia (the target context). The context within which a study is conducted may differ considerably from the target context within which study results are later used.32

What tools and methods are available to handle context in systematic reviews?

Context is currently factored into systematic review processes in two main ways:

Contextual items included within existing review tools (eg, specific tools for describing interventions and for assessing complexity).33 Many review tools already include one or more items for context.

Use of a supplementary framework to explore contextual issues.30 A guideline group can use a context framework when planning reviews (eg, when formulating questions within a review) or while conducting reviews (eg, when representing subgroups within the synthesis).

Table 2 provides examples of known tools and frameworks for taking account of context. Full text searching of Google Scholar for references to ‘context’ within articles reporting systematic review methodology was used to identify candidate tools. This paper seeks to illustrate well-known or recently developed approaches to be used at different stages of the systematic review process and to summarise their potential strengths and weaknesses. While not seeking to be exhaustive, the table can be used to identify composite approaches that explore complementary perspectives of context.

Table 2

Tools and frameworks for handling context, mapped to stages of a systamatic review process

Stage of the review	Tool or framework	Strengths	Limitations
Throughout	Realist evaluation7/Realist synthesis34	Offers a method for integrating quantitative and qualitative data. Focuses on context as one line of inquiry, specifically addressed when assessing programme outcomes.15	Resource intensive; requires reviewers to prioritise which explanations would benefit from further exploration.
	Systems thinking4	Offers holistic picture of complex phenomena and interactive elements including context.	No specific review or synthesis tools or methodologies have been designed for this perspective.
	Evidence to Decision frameworks (eg, WHO-INTEGRATE10 or GRADE EtD38 framework)	Seek to identify elements of context throughout entire evidence production and guideline development process.	Comprehensive coverage makes it difficult to use entire tool within a tight systematic review time window. Reviewer must prioritise items relevant to specific review.
	Logic models2	Provide a graphical representation of context surrounding an intervention.	Offer simplistic perspective with a focus on inputs, processes, outputs and outcomes. Context is only identified as a backdrop to programme activities.
Planning the review
Question formulation and defining eligibility criteria	Question structures:11 PICOC,41 CIMO,42 SPICE,43 PerSPE©TIF11	Include important elements of context/setting when compared with intervention-based PICO formulation.	Adopt a simple linear framework for representing review question.
Protocol development	PROSPERO (International prospective register of systematic reviews) template45	Item #23 (context) requires summary details of the setting.	Focuses on context as it relates to study eligibility.
	Use of frameworks (see itemised list below)	Offer prompts to ensure appropriate issues are considered when conducting the review.	Few available frameworks for systematic consideration of context.
	PRISMA-P44	Reporting standard for all elements of a protocol.	Superficial approach to context (setting and time frame).
Conducting the review
Data extraction and/or analysis	Framework for the design and reporting of cluster RCTs.46	Requires careful description of context to help interpret findings.	Designed as a framework for primary research. Requires evaluation within a synthesis context.
	CFIR20	Itemises 39 factors influencing implementation of interventions offering taxonomy for classifying factors.20 Terminology reported as clear and coherent.91	Complex framework that may include too many factors to be practical for data extraction reflecting that it has been designed for primary studies.91
	CICI framework22	Based on concept analysis therefore offers broad coverage of context issues.	Limited conceptual thinking on the inter-relationships of framework elements of context and implementation.
	Diffusion of innovations40	Considers temporal issues of context and mechanisms of sustainability and spread.	Does not offer practical framework for most aspects of context.
	iCAT-SR33	Requires review team to make judgements on the extent to which the effects of the intervention depend on the context or setting within which it is implemented.	Limited specific detail on context although supplemented by embedded items with contextual implications, for example, organisational level.
	Normalisation process theory49	Acknowledges wider societal and contextual barriers and organisational and social factors. Defines context at early stages of intervention; implementation and evaluation.	While recognising context as a dynamic environment, and as a process, not a place; challenging to operationalise within a synthesis context.
	PARIHS47	Context is core construct and interacts with other constructs.	Focuses on implementation setting, not wider social, political or economic environment. Originally designed for primary studies.47
	PROGRESS-Plus53	Includes population-based, social and cultural factors to explore/explain variation in intervention feasibility, acceptability, meaningfulness and effectiveness.	Broad population focus at expense of less common but equally important factors resulting in individual variation.
	RE-AIM framework48	Uses stakeholders to prioritise aspects for particular focus.	Primary study focus. Extensive resource use when exploring all aspects within a synthesis. Focuses mainly on number individuals adopting evidence as main contextual factor.16
	Theoretical Domains Framework50	Specifically designed for implementation contexts. Includes Environmental context and resources and Social influences. Extensive use in systematic reviews.	Focuses on organisational level context with passing acknowledgement of external factors in the environment that cause stress.
	TIDieR55	Includes a WHERE question to elicit different levels of contextual detail.	Focuses on intervention description; offers exemplar for how context might be developed. Focuses on spatial aspects of context.
Literature searching	Citations, Lead authors, Unpublished materials, (Google) Scholar, Theories, Early examples, Related projects (CLUSTER) technique58	Identifies additional ‘sibling studies’ to expand detail of context.	Potentially time-consuming and dependent on linked publications.
	Web searches for supplementary data on context not present in published reports	Offers accompanying detail to complement published studies.	Requires opportunistic searching; challenging to document.
Data extraction	Context templates	Prompts review team to compile proforma assessments to standardise reporting of context.	Missing data leads to inconsistent reporting and impaired comparison.
	iCAT_SR tool33	Includes context among 10 dimensions that examine interplay of factors contributing to complexity.2	Context dimension is optional, not required.92 Other elements implicitly relate to context.
	TIDieR55	As discussed above.	As discussed above.
Making use of review findings
Producing the report	Reporting Standards (eg, CONSORT;62 PRISMA;25 PRISMA-CI;26 ENTREQ63)	Prompts to identify and elicit missing data on context.	Context or setting typically constitutes a single (or limited) number of items.
Grading of findings	GRADE65	Global influence supported by established methods group. Recently addressed contextualisation for (1) guidelines and (2) HTAs/systematic reviews.93	Lacks structured approach for incorporating non-epidemiological evidence on context and implementation.39
	GRADE-CERQual66	Includes relevance element and broadens beyond time and place. Work in progress.	Requires further development to extend beyond less salient aspects of context.
Research recommendations	EPICOT-dts84	Optional elements include Disease burden or relevance and Time aspect of core EPICOT elements.	Little evidence of uptake and utilisation (excepting NICE and isolated WHO guidelines).
Implementation	Oxford Implementation Index77	Contextual factors is largest of four domains: intervention design, actual delivery by trial practitioners, uptake of intervention by participants.77	Sheer number of contextual elements requires prioritisation, particularly within context of evidence synthesis.
	Case studies, profiles, vignettes	Designed for use within local implementation context and to permit local tailoring.	Only focus on isolated simplistic aspects of context.
	Spreadsheets, calculators (eg, NICE Physical Activity Return on Investment Tool)75	Enables user to evaluate a portfolio of interventions in their geographical area (eg, region, county or local authority) and model different payback returns. Interventions are mixed and matched to identify best 'value for money'.	Underpinning assumptions behind model may change. Requires periodic updating.
Adaptation	SUPPORT tools59	Offer a comprehensive and systematic guide for considering local context throughout the decision-making process.	Limited number of examples. May oversimplify decision making process—lack a complex system perspective.
	ADAPTE and other adaptation guidelines94	Offer alternatives to developing de novo guidelines (eg, ADAPTE covers organisational and cultural considerations).	Frameworks need to be evaluated for rigour, efficiency and transparency of process.

CFIR, Consolidated Framework for Implementation Research; CICI, Context and Implementation of Complex Interventions; CIMO, Context-Intervention-Mechanisms-Outcome;CONSORT, Consolidated Standards of Reporting Trials; ENTREQ, Enhancingtransparency in reporting the synthesis of qualitative research; EPICOT, Evidence, Population, Intervention, Comparison, Outcomes, and Time Stamp; EtD, Evidenceto Decision; GRADE, Grading of Recommendations, Assessment, Development, and Evaluation; GRADE-CERQual, Grading of Recommendations, Assessment, Development, and Evaluation-Confidence in Evidence from Reviews of Qualitative research; HTA, Health Technology Assessment; iCAT-SR, Intervention Complexity Assessment Tool for Systematic Reviews; LMIC, NICE, National Institute for Health and Clinical Excellence; PARIHS, PromotingAction on Research Implementation in Health Services; PerSPE©TIF, Perspective-Setting-Phenomenon-Environment-Comparison-Timing-Findings; PICOC, Population-Intervention-Comparison-Outcome-Context; PRISMA-P, PreferredReporting Items for Systematic Reviews and Meta-Analyses for Protocols; PRISMA-CI, Preferred Reporting Items for Systematic Reviews and Meta-Analyses for ComplexIntervention; PROGRESS-Plus, Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital;RCT, randomised controlledtrial; RE-AIM, Reach, Effectiveness, Adoption, Implementation, and Maintenance; SPICE, Setting-Perspective-Interest,-Phenomenon-of-Comparison-Evaluation; TIDieR, Template for Intervention Descriptionand Replication.

How to consider context when planning, conducting and implementing reviews

Review authors have numerous opportunities to consider contextual details when planning and conducting the review and, subsequently when making recommendations (box 1).

0. Use a review method (eg, realist synthesis or meta-ethnography) that allows exploration of context and/or

1. Consider contextual details during question formulation and when defining eligibility criteria.

2. Describe contextual considerations in the review protocol.

3. Extract contextual details as part of data extraction.

4. Seek missing contextual details.

5. Consider contextual characteristics during statistical and qualitative analyses and explore contextual heterogeneity when appropriate.

6. Report contextual details in supplementary materials.

7. Develop support tools to facilitate implementation and adaptation.

8. Describe implications for future research.

Throughout the review

One approach to handling context within systematic reviews and guidelines is to use a review method that explicitly articulates context within its programme logic. Realist synthesis34 may identify contextual variation to be explored through complementary use of quantitative and qualitative data sources.16 Realist synthesis seeks to explain why interventions may or may not work, within which contexts, how and in what circumstances’.35 Specific techniques for juxtaposing quantitative and qualitative evidence are elaborated elsewhere in this series.13 14 Synthesis methods that specifically examine context, as with realist review, or that seek to preserve context when ‘translating’ across studies (as with a meta-ethnography alongside an effects review) offer a holistic approach. However, they struggle to deliver within the constrained time and resource timetable36 that characterises guideline production.

The value of systems thinking4 is explored elsewhere. However, although applications of system thinking in public health are emerging,37 few systematic reviews have encapsulated systems thinking within their review methods. EtD frameworks, such as the GRADE EtD38 framework and the WHO-INTEGRATE framework,10 itemise many contextual factors impacting on decision-making. Nevertheless, exhaustive frameworks may prove resource-intensive when harnessed to a systematic review. Guideline developers and the review team should consider which, if any, elements from this expansive menu will prove informative for their particular guideline/review.

Logic models are graphic descriptions of a system and are designed to identify important elements and relationships within that system.2 In seeking to portray multiple interacting elements, they represent a light touch response to context; the focus is on the intervention or programme itself, together with associated inputs and activities, and context offers a frame for intervention delivery. So a logic model may be useful for initiating discussion of context but will probably require further, more granular, specification.

Planning the review

Consider contextual details during question formulation and when defining eligibility criteria

Methodologists emphasise that a review team should consider context from initiation of the review.11 39 Assumptions about context first surface within the Population, Intervention and Comparison elements of a review question; a review may define its context as ‘low and middle-income settings’, as ‘Africa’ or as a named country. However, a review question may favour geopolitical aspects of context over equally important considerations such as rural-urban differences, ethnicity, infrastructure and policy. Health systems frequently characterise context as primary, secondary, tertiary and community but this demarcation may run counter to a systems perspective.

Question formulation frameworks represent flat, linear representations of context11 that do not accommodate a systems perspective. In contrast, a systems perspective, where macrolevel environmental factors and microlevel factors of setting dynamically interplay, may require a comprehensive question framework.40 Few question formulation frameworks specify context (or associated subconcepts, eg, setting or environment). Nevertheless, question structures such as Population-Intervention-Comparison-Outcome-Context (PICOC),41 Context-Intervention-Mechanisms-Outcomes (CIMO)42 and Setting-Perspective-Interest, phenomenon of-Comparison-Evaluation (SPICE)43 offer alternatives to the PICO structure, which omits context (environment or setting) altogether and may prompt guideline developers or review authors to articulate context concerns. A review team may incorporate question structures that include environment/setting within their inclusion and exclusion criteria and the data extraction form. An innovative, yet untried, question formulation, PerSPE©TiF11 may prompt a review team to identify elements of context not typically articulated within an intervention perspective.

Describe how context is to be defined within the review protocol

Other aspects of the review protocol have an implicit role in defining context, for example, inclusion and exclusion criteria, the choice of data sources and search terms as well as language and date restrictions. However, two principal structures for codifying the review protocol, PRISMA-P44 and PROSPERO45 noticeably lack detail of context. PRISMA-P, a reporting standard for review protocols, simply lists setting and time frame alongside the PICO elements. No detail is given on specific details to be included. The template used to register reviews in PROSPERO, an international registry of systematic reviews, requires contextual details ‘if these form part of the review’s eligibility criteria but are not reported elsewhere in the PROSPERO record’. Examples include: ‘studies in hospital accident and emergency departments’ or ‘research in low-income and middle-income countries.’ Detailed reporting in a protocol template can help in agreeing the scope of the review and to surface assumptions of similarity and difference across contexts (eg, combining studies with broadly comparable generalisable study features, eg, low-income and middle-income countries) and splitting (only combining studies with highly comparable populations for example, East Africa).45 However, such templates require the review team to preidentify contextual differences before starting to undertake the review.

Conducting the review

Extract contextual details as part of data extraction

Numerous purpose-specific frameworks exist that offer a potential structure for data extraction forms. Generic frameworks, designed for primary research, are suggested in the Medical Research Council Process Evaluation Guidance.16 Examples include the Cluster randomised controlled trials46 framework, PARIHS47 and RE-AIM.48 However, frameworks designed for use within primary studies may include too much detail to sustain data extraction for a review. Other frameworks remain experimental such as the Context and Implementation of Complex Interventions (CICI) framework, produced for a European Union-funded project as a generic framework for exploring for the context of any health technology.22 Similarly, the WHO-INTEGRATE framework enumerates the ‘Health system and broader context’.10 When conducting a systematic review of a complex intervention, a team must prioritise important contextual elements considered important for data extraction rather than seeking to use frameworks in their entirety.

Implementation frameworks (such as Normalisation Process Theory49 and the Theoretical Domains Framework50) offer an alternative lens on context. A systematic review of facilitators and barriers to team working in primary care used Normalisation Process Theory to identify the importance of contextual variation in funding models and colocation.51 The Theoretical Domains Framework was used in a systematic review on barriers and enablers to acute stroke management in the emergency department revealing important contextual differences in social influences.52 Frameworks must optimise usability and detail; the Diffusion of Innovations framework40 does not appear sufficiently nuanced for data extraction whereas the meta-framework, Consolidated Framework for Implementation Research,20 appears overly detailed. Craig and colleagues have combined and consolidated a comprehensive list of features of context.17

Potential is offered by PROGRESS-Plus, devised by the Cochrane Equity Methods Group to examine any intervention or programme, where a review team identifies concerns of equity.53 The PROGRESS-Plus framework includes contextually sensitive details such as race, religion and ethnicity and may therefore help in isolating important variations that impact on the effects of a complex intervention.54

The Intervention Complexity Assessment Tool for Systematic Reviews (iCAT-SR)33 and the Template for Intervention Description and Replication (TIDieR)55 specify elements of context alongside salient details of the target intervention. The iCAT_SR tool,33 which seeks to locate an intervention on a simple-complex continuum, references context as how ‘the effects of an intervention may be dependent on the societal, political, economic, health system or environmental context in which the intervention is delivered’. Such an approach helps to ‘embed’ context alongside intervention complexity.

The TIDieR55 checklist is used for describing interventions in sufficient detail for replication and can help to extract information on WHERE an intervention was delivered, which is important in considering context. The TIDieR framework encourages a detailed description of location including any necessary infrastructure or relevant features.55 The importance of specific factors depends on the type of health decision and the decision-making context, but the TIDieR framework may be helpful when extracting detailed information about spatial aspects of context.55 A subsequent paper evaluating TIDieR56 contrasts full reporting of interventions (to enhance replicability) with detailed reporting of context (to surface implementation issues). TIDieR, as the first major reporting tool specifically to examine implications when reporting context, signals an opportunity to recalibrate away from seeking to replicate context to the more realistic aspiration of seeking to understand and interpret its effects.

While these tools and frameworks present possible structures for extracting data on context, they tend to artificially separate intervention characteristics and contextual considerations, which is unhelpful when reviews and guidelines specifically need to identify and take account of sources of complexity.

Seek missing contextual details

Where the review team considers that important details about context are missing from the included studies, there are three principal options for locating this information.57 The CLUSTER technique58 has been devised for searching systematically for associated (‘sibling’) reports, such as pilot, feasibility studies, doctoral theses, book chapters, process evaluations and qualitative research, that offer supplementary detail beyond that retrieved from a systematic subject search.

Alternatively, a team can conduct supplementary web searches to build up context templates from routine government data and from policy sources for the study context of each ‘index study’. Policy and sociodemographic characteristics appear as national profiles on websites of government agencies or of international organisations such as the World Bank or WHO. Institutional websites also host organisational details, for example, the number of hospital beds or the catchment area for a health service. Finally, the team can harness the collective knowledge and expertise from an expert advisory group, but such approaches pose challenges to the need to be both systematic and explicit.

Consider contextual characteristics during statistical and qualitative analyses and explore contextual heterogeneity when appropriate

Where appropriate and feasible, a review team should consider context, as identified a priori in the protocol, when grouping studies, conducting analyses and exploring heterogeneity.59 Higgins and colleagues describe statistical and non-statistical methods available for synthesising quantitative data,19 including subgroup analyses and meta-regression for examining sources of heterogeneity including context. Flemming and colleagues describe approaches for qualitative evidence synthesis, particularly helpful for exploring variation in context and explaining why certain interventions work in some contexts and not others.13 Noyes and colleagues address methods for mixed method synthesis and how such evidence can help a decision-maker to understand how context may affect feasibility and acceptability.14 All these methods possess advantages and disadvantages; essentially, though, the choice of methods centres on the extent to which a review team aims to gain a greater understanding of the study context (by examining potential explanatory factors within the study data) or the extent to which the team seeks to focus on transferability from the study context to the target context (by exploring shared contextual characteristics).

Making use of the review findings

Report contextual details in supplementary materials

Reporting standards for primary studies seek to encourage more complete reporting of context; however, they prescribe diverse levels of suggested reporting detail.60 Craig and colleagues affirm that ‘indicating and describing clearly the contexts in which the research was carried out, how they affect implementation and variation in impacts and the limits to the applicability of the results should make reports of primary studies and evidence syntheses more useful’17 to researchers and decision-makers alike. Shoveller and colleagues report from a snapshot of current journals that context was typically treated as a ‘black box’ or problematised as something to be ‘controlled for’.61 A systematic review of a complex intervention can only report contextual details if they are available in the index primary trial reports or in supplementary publications. CONSORT (2010) suggests reporting ‘other information about setting or location that could have influenced the observed results’62 yet offers limited guidance on how context should be reported.

Currently, PRISMA25 demonstrates superficial engagement with the need to report details of context. The specific PRISMA-CI extension for complex interventions does, however, extend coverage of contextual detail and hint at potential future directions.26 27 Thus, the PICO structure becomes a six-element PICOTS mnemonic to include Time and Setting.27 Furthermore, PRISMA-CI recognises organisational features, geographic location; financial setting and clinical setting and, optionally, ‘rival activities’27. In contrast, the Enhancing Transparency in Reporting the synthesis of Qualitative research (ENTREQ) reporting standard63 does not include any item specifically targeted at context and, indeed only refers to population and country. ENTREQ does, however, advocate the need to specify context when formulating the review question and also alludes to concerns of transferability.

The WHO has recently sought to enhance reporting of programme level detail, recognising that the impact of many social and behavioural programmes is ‘tied to the local context (eg, sociocultural, socioeconomic, geographical, legal, political, health system) and to the processes of implementation, which may not be easily described’.64

Tools for assessing the quality (or certainty) of evidence contained in systematic reviews, that is the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for quantitative outcomes65 and the Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) approach for synthesised qualitative findings66 facilitate post hoc consideration of context when moving from evidence to recommendations. GRADE assessment criteria recognise contextual variation (eg, geographic setting, culture, subcultures and resources) and also differences in perspective (patients, providers, public health and society and of individual vs population based evaluation frames).67 Both the GRADE domain of indirectness68 and the GRADE-CERQual component of ‘relevance’ require a reviewer to assess the extent to which study contexts match the target contexts.69

Grading systems are constrained by their reliance on the level of detail in the source primary studies; detail that is often absent. Furthermore, they depend on the ability of the reviewer to identify salient contextual detail a priori and to make this detail the focus for analysis. Finally, summary of findings65 66 are limited in the contextual detail they can accommodate, necessitating difficult choices.

Develop support tools to facilitate implementation and adaptation of systematic review findings

Applicability or generalisability checklists, of which more than 25 frameworks exist,70 help when translating systematic review findings into specific contexts. The SUPPORT tools (https://health-policy-systems.biomedcentral.com/articles/supplements/volume-7-supplement-1), written for decision makers and those who support these decision makers, include tools on assessing the applicability of review findings71 and using local contextual evidence.72

Tools to support implementation include case studies, country profiles and vignettes which prompt the user to consider potential variation in context. Individual country case studies can help decision-makers to visualise differences between a source context and a target context or to benchmark across countries.73 However, numbers of case studies are necessarily limited and they may lack the depth required to unearth critical differences. Country profiles require a study author to make an a priori ‘editorial’ judgement on what is ‘relevant’. Vignettes use narrative techniques to characterise complex circumstances and to trigger critical thinking about similarities and differences with local practice. Typically vignettes simplify population characteristics (eg, age or severity of a condition) but they could equally capture contextual characteristics. For example, Gauld et al, (2012) use vignettes to characterise qualitatively the primary care systems of 12 different countries, supported by routine quantitative data on key indicators.74

Spreadsheets and calculators require users to add local contextual data to be factored within resultant calculations and projections. So, for example, the National Institute for Health and Clinical Excellence in the UK offers return on investment tools that allow local health providers to configure optimal packages of interventions for physical activity and alcohol or tobacco control for their populations.75 The NICE Guidelines programme routinely produces three types of implementation support tool to accompany each guideline:76

a baseline assessment tool,

clinical audit tools,

a costing report and costing template, or a costing statement.

Such tools use common software, such as Microsoft Excel, to improve uptake.

The Oxford Implementation Index offers a rare example of a tool to help reviewers identify and extract contextual information associated with implementation from studies included in a systematic review.77 Other tools include implementation workbooks and worksheets.78 79 The ADAPTE toolkit80 highlights criteria for assessing whether a recommendation is applicable to a local context.

Considering contextual factors on completion of a review, using applicability checklists and decision-making frameworks, helps to make review findings more useful to guideline developers. Alternatively, strengthening the conduct of the systematic review19 29 can lead a review team to consider a complexity perspective a priori using an EtD-making framework.10

Context is pivotal when localising guideline recommendations. Guidance from the GRADE Working Group directs guideline developers in deciding when to adopt (use a guideline ‘as is’), adapt (introduce local evidence and expertise to rework guidelines) or contextualise (reframe recommendations in light of local resources and constraints) and when to initiate guideline development de novo. 81 82

Describe implications for future research

An important outcome of any systematic review, or guideline, is articulating gaps in knowledge or evidence on the topic. However, often reports of reviews and guidelines provide only general observations and miss the opportunity to identify important gaps relating to different contexts. Where important contexts are missing, resulting in ‘absolute gaps’ or ‘synthesis gaps’83 the review team can identify implications for future research. The Evidence Population Intervention Comparison Outcome Timestamp (EPICOT) framework, developed by organisations involved in commissioning or summarising research, is recommended for formulating research recommendations.84 Optional elements in the framework (current state of evidence (E), appropriate study type (s), disease burden and relevance (d) and timeliness (t)) help to ensure more detailed definition of context.84 The WHO Guidelines on Indoor Air Quality85 used this EPICOT reporting structure. However, the EPICOT framework shares limitations of the PICO question framework in providing a static linear framework and it does not take into account other important characteristics of the immediate setting or the wider environment.

What might represent good practice for considering context in reviews and guidelines?

This paper reaffirms that, where possible, systematic review authors and guideline developers should identify relevant aspects of context from the beginning of the process, when the scope of the review or guideline is determined and the questions identified.86 Sandelowski offers some useful ‘rules of thumb’ on how to anchor text to context in restatements of findings.87 Little consensus exists on how best to identify and select relevant aspects of context that may be important in reviews and guidelines considering a complexity perspective. Other papers2 10 in this series offer practical suggestions on how to achieve this.

A review team should recognise that different types of evidence can contribute to an improved understanding of context. For example, Craig and colleagues illustrate how inclusion of qualitative evidence, in the form of a meta-ethnography can help in exploring context.17 Natural experiments and uncontrolled studies can present opportunities for understanding context and yet are frequently excluded from systematic reviews.17 Systematic reviews can further be designed to incorporate implementation and outcome evidence, rarely reported within a single primary study but usefully juxtaposed within implementation reviews. Ideally, a review team will plan a priori analyses based on anticipated differences in context.88 Nevertheless, if they have not preplanned such analyses, exploratory post hoc analyses, clearly identified as such, may be appropriate in generating further hypotheses.88

From initiation of a review, the review team should look out for studies reporting interventions from highly disparate contexts, whether differences are openly apparent or masked by incomplete reporting.77 Variations in context impact on the review question, the search strategy, the inclusion and exclusion criteria, the data extraction forms and any subgroup analyses and the resultant recommendations. Inevitably, further contextual features emerge throughout the review; a review team should document these, together with implications for the final report. Ongoing interaction with stakeholders is one way to ensure that contextual factors do not surface unexpectedly during the course of a review.

Box 2 illustrates how a joined-up approach to handling context might be built within an existing guideline process. The chosen example of a complex intervention, antenatal care, is delivered in diverse ways across different countries and settings. Therefore, results from individual trials may not generalise to other types of population and setting.89 Box 2 illustrates multiple points at which context might be introduced within a systematic review while illustrating the diversity of approaches to context employed by each tool or framework.

Planning the review

A guideline development group on antenatal care selects the WHO-INTEGRATE framework10 to identify and discuss the importance of context throughout the development process. The group uses the PerSPE©TiF11 question formulation (From the Perspective of a Pregnant Woman in the Setting of Rural Communities how does the Phenomenon of Facility Based Care within an Environment of Poor Transport Infrastructure and Distantly-Located Facilities (Compare with Traditional Birth Attendants at Home) up to and including the Time of Delivery in terms of Findings related to the Woman's Perceptions and Experiences?) to define the scope of the review. Using the PerSPE©TiF structure, they craft a review protocol together with inclusion and exclusion criteria to define and anticipate salient aspects of context.

Conducting the review

The review team takes details from the protocol and selects an appropriate data extraction framework (the Context and Implementation of Complex Interventions (CICI) framework,22 deciding that broader environmental concerns are more critical to this population-based guideline than the specific setting. They access websites to elicit extra contextual data and identify ‘clusters’ of associated studies using the CLUSTER technique.58 Where critical details are missing, the review team contacts authors of included studies requesting additional information, setting a deadline for response. The review team juxtaposes results from a quantitative meta-analysis and a qualitative synthesis using a logic model.95 Important details of context inform plans for statistical and qualitative analyses and the statistical team explore contextual heterogeneity across predefined subgroups.19

Making recommendations

The review team provides appendices detailing the context described in each included study. The guideline development group pays particular attention to factors relating to Indirectness and Relevance when formulating their GRADE65 and GRADE-CERQual66 recommendations. The group uses an adaptation framework (ADAPTE94) to provide contextually-sensitive recommendations for major regions of the world, allowing each country to tailor country-specific recommendations, where feasible. The guideline developers provide accompanying spreadsheets and vignettes to facilitate roll-out and the group frames recommendations or using the EPICOT-dts framework.84

Avoiding interpretation errors in connection with context

Based on the above scenario (box 2), we have identified and labelled three ways in which readers may misinterpret context within systematic review findings; variously termed the ‘universal fallacy’, the ‘focal fallacy’ and the ‘evidence availability fallacy’ (also see online supplementary appendix):

A reader may think that a review finding that is not qualified by contextual descriptors (eg, time, place, socioeconomic factors, religious beliefs and so on), is a universal finding (ie, applying to all contexts) (the universal fallacy).

A reader may think that a finding only applies to the context that is the focus of the review question (when it is actually more generalisable) (the focal fallacy).

A reader may think that a finding only applies to contexts studied within included studies (when this simply characterises where research has been undertaken) (the evidence availability fallacy).

To avoid these interpretative errors, review authors and guideline developers can better articulate context-sensitivity when framing review findings and subsequent recommendations:

By Exception—A review team could highlight that exceptions (divergent study contexts) are different from all the other study contexts or from all other contexts encompassed by a review question, for example, ‘This finding seems to apply irrespective of context, except in X, Y, Z where primary care systems continue to be disrupted by conflict and migration’.

By Inclusion—A review team could explicitly state contexts to which findings apply. This flags the distinction between ‘We have evidence that this only applies to these contexts’ and ‘We only have evidence that this applies to these contexts’.

By Exclusion—A review team could explicitly state contexts to which findings do not apply.

By Assumption—A review team could define relevance according to the presence or absence of common mechanisms or contextual factors, for example, ‘This finding is likely to apply in those contexts with a developed method for priority setting for nutrition’.

Supporting recommendations by assumption should be treated with caution, comparable to circumspect use of indirect comparison in quantitative research. Nevertheless, the value of explanatory theory is increasingly recognised and its contribution undoubtedly extends beyond the theories of change or behavioural and social theories that are the focus of this paper.90

Limitations of this analysis

Our brief selection of high-profile or recently developed frameworks and tools for taking account of context in systematic reviews and guidelines reveals that such tools remain limited in their facility to accommodate a complex systems perspective.2 This selection is necessarily arbitrary given that information retrieval strategies to identify review tools that target context systematically do not currently exist. Highlighted tools and frameworks are illustrative requiring that the example given in box 2 is hypothetical and, indeed aspirational.

Conclusion

Approaches to context within systematic reviews typically share the following limitations:

Tools typically capture context at a single timepoint and are barely able to capture prevailing trends.

Many tools derive from use in primary research and either have limited utility within synthesis or have not been fully evaluated.

Particular tools focus on a single specific level of context, particularly true of implementation tools that target the direct setting within which the intervention is delivered.

We have also identified a potential tension given that tools that are best equipped for considering context may, by their very nature, not prove practical as a structure or framework for data extraction or analysis. In the absence of empirical evidence on features of context that impact most on intervention success, we observe that tools tend to become increasingly expansive. This, in turn, makes data extraction and analysis more time consuming and resource intensive.

For reviewers and guideline developers, this paper emphasises the practical need to select from this toolkit judiciously. Context should be addressed early in guideline development and then revisited throughout the review process. Frameworks offer a useful structure for ensuring that data on context are extracted for the review. Checklists prompt decision-makers to consider important factors and reporting standards ensure that, where reported, requisite details are present in the report or in supplementary material accompanying each report. Knowledge translation tools such as vignettes, country profiles, calculators and spreadsheets can help in translating between the study contexts and the target context. With such diverse approaches, the current way forward requires composite use of different tools at appropriate stages of the process. Our brief scan of illustrative tools reveals a need for a coherent approach that runs throughout the review process and that underpins the individual tools used at each stage. Approaches to context to date suggests that the components already exist; the outstanding methodological challenge is to agree a coherent approach within which existing tools might be integrated.

Recent years have seen collective recognition of the importance of context when selecting, evaluating and implementing complex interventions. This paper consolidates collective learning in identifying and using tools for handling context and meeting the ever more complex demands of guideline developers and other decision-makers. This overview signals how context might possibly be addressed within systematic reviews and guidelines. For the present, review teams are encouraged to extract contextual details when available, elicit them when feasible and reconstruct them when available data allows. Review teams and guideline producers should seek to navigate a path that is parsimonious in terms of resource and data extraction and yet sensitive to their specific contexts and the values, preferences and needs of their target populations.