Mapping the evidence about what works to safely reduce the entry of children and young people into statutory care: a systematic scoping review protocol.

INTRODUCTION: The increasing number of children and young people entering statutory care in the UK is a significant social, health and educational priority. Development of effective approaches to safely reduce this number remains a complex but critical issue. Despite a proliferation in interventions, evidence summaries are limited. The present protocol outlines a scoping review of research evidence to identify what works in safely reducing the number of children and young people (aged ≤18 years) entering statutory social care. The mapping of evidence gaps, clusters and uncertainties will inform the research programme of the newly funded Department for Education's What Works Centre for Children's Social Care. METHODS AND ANALYSIS: The review uses Arksey and O'Malley's scoping review methodology. Electronic database and website searches will identify studies targeting reduction of care entry, reduction of care re-entry and increase in post-care reunification. Supplementary searching techniques will include international expert consultation. Abstracts and full-text studies will be independently screened by two reviewers. Ten per cent of data abstraction will be independently conducted by two reviewers, with the remainder being extracted and then verified by a second reviewer. Descriptive numerical summaries and a thematic qualitative synthesis will be generated. Evidence will be synthesised according to primary outcome, intervention point (mapped across socioecological domains) and the realist EMMIE categorisation of evidence type (Effectiveness; Mechanisms of change; Moderators; Implementation; Economic evaluation). ETHICS AND DISSEMINATION: Outputs will be a conceptual evidence map, a descriptive table quantitatively summarising evidence and a qualitative narrative summary. Results will be disseminated through a peer-reviewed publication, conference presentations, the What Works Centre website, and knowledge translation events with policy-makers and practitioners. Findings will inform the primary research programme of the What Works Centre for Children's Social Care and the subsequent suite of systematic reviews to be conducted by the Centre in this substantive area. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first scoping review to map the gaps, clusters and uncertainties in the international evidence base addressing what works to safely reduce care entry for children and young people.

Published and unpublished studies will be identified via a robust and sensitive search strategy. Searching techniques will include searching of electronic bibliographic databases and international expert consultation.

The realist EMMIE framework will guide the evidence synthesis, providing a contextually relevant summary that will support policy-makers and practitioners in understanding the appropriateness of the evidence base.

No formal assessment of study quality will be conducted.

Stakeholders have been consulted on the review and will be involved in dissemination.

Introduction

Preventing the need for children and young people to enter statutory social care is a significant social, health and educational priority. In the past 20 years, there has been a consistent increase in the number of children residing in the English care system, rising from 50 900 in 1997 to 72 670 in 2018.1 2 Care-experienced individuals experience a range of adverse outcomes across the life-course compared with the general population, including higher rates of psychological disorders, poorer educational attainment and lower rates of employment.3–5 Some consensus has emerged around the need to do more in preventing the problems leading to care entry.6 This will ensure compliance with the principles of the United Nations Convention on the Rights of the Child and the Children Act 1989,7 both of which mandate a child’s rights to be cared for by their parents. It will further address important economic considerations, as out-of-home placements incur significant costs, with an average annual spend per head of £29 000–£33 000 for foster care and £131 000–£135 000 for residential care in England.8

Efforts to reduce care entry may be contested, however, due to evidence that pre-care experiences may be the primary contributory factor to adverse life-course outcomes.9 10 Indeed, for many individuals, statutory care may be protective. Thus, there remains a complex challenge to statutory care systems in safely reducing care entry, while ensuring the appropriate identification and referral of those who necessitate intervention. A number of approaches have proliferated internationally, spanning the full range of socioecological intervention points.11 These include but are not limited to interpersonal interventions that focus on the communications within the family (eg, Intensive Family Preservation Services), organisational interventions that modify social work practice and ethos (eg, Reclaiming Social Work), and national policy strategies (eg, Department for Education’s (2016) Putting Children First). Despite such developments, however, there are limitations with the extant evidence base, which has prevented a comprehensive understanding of how best to prevent care entry. This understanding is imperative given the current public spending context, where there has been a substantial decrease in spending on preventative services, amidst increases in the proportion of children services’ budget being spent on statutory care and child protection.12

A key issue is the extensive variation in the methodological conduct and quality of evaluations, which makes it difficult to clearly ascertain the evidence base for any particular intervention. Information on much current social care practice is often limited to local descriptions of innovation.13 Where efforts have been made to conduct robust evaluations, such as the widely implemented Family Group Conferencing,14 the use of randomised controlled trials has been contested.15 This may be partially linked to debates about whether social care should be an evidence-based or experientially based discipline16 17 Furthermore, the heterogeneity in social care systems and resource structures makes it difficult to understand the international relevance and transferability of evidence-based approaches, and the likelihood of replicating effectiveness in new contexts. In alignment with realist and complex systems perspectives,18–24 interventions may be understood as disruptions to the system into which they are introduced and are thus inseparable from the context in which they operate. As such, intervention implementation practices will likely vary across systems, leading to the differential activation of causal mechanisms, and hence the potential realisation of different outcomes. To redress these limitations, there is a need to review the international evidence on what works in safely reducing the number of children and young people in care.

In 2017, the What Works Centre for Children’s Social Care was established by the Department for Education (DfE) to develop evidence-based approaches to improve outcomes for children at risk of entering statutory care, and to ensure the translation of evidence into routine practice through meaningful engagement with policy and practice stakeholders. Reducing the need for children and young people to enter care is the first priority area of the Centre. The Centre’s programme of work includes the conduct of scoping and systematic reviews to identify, appraise and synthesise the extant evidence base. The present scoping review will inform the programme of research to be undertaken by the Centre. Specifically, the review findings will support the formulation of specific and discrete research questions to be addressed within a subsequent suite of systematic reviews focused on the safe reduction of children and young people in statutory care.

Study objectives

We will conduct a scoping review to map the evidence base on what works in safely reducing the need for children and young people to enter into statutory care, in order to identify key evidence clusters, gaps and uncertainties.25 The review will scope the evidence across three areas: the safe reduction of the need for children and young people to enter statutory care; the safe reduction of the need for children and young people to re-enter care; the safe increase in children and young people’s re-unification with their family following a period in out-of-home care. We note that the review is not concerned with the absolute reduction of the number of individuals in care, but rather the ‘safe’ reduction, while ensuring the correct identification and support of those necessitating statutory intervention. Preliminary searches have been undertaken and no existing scoping reviews have been identified.

Methods and analyses

Conceptual model

Given the complexity and contextual contingency of the effectiveness of social care interventions, the scoping review will adopt a realist approach to evidence mapping.18 20 26 Rather than a focus on absolute measures of intervention effects, realist approaches consider the question of what works, for whom, in which circumstances. Evidence reviewed is not then singly appraised or synthesised according to aggregate intervention effect sizes but can be considered in relation to the composite assessment proscribed by the EMMIE framework,26 which supports the interrogation of a heterogeneous and complex evidence base.

The EMMIE framework comprises five dimensions that will be used for evidence mapping according to the review questions: Effect (E) of an intervention; Mechanisms (M) through which an intervention is expected to have an effect; the contexts that Moderate (M) if these mechanisms will be activated to generate the intended effect; system level Implementation (I) barriers and facilitators; Economic (E) cost-effectiveness.26 These dimensions have been identified as pragmatic and meaningful in presenting evidence for policy-makers and commissioners. To date, the framework has primarily been employed with review of reviews or systematic reviews of primary evidence,27 28 and we understand this to be the first example of its use with a scoping review.

Patient and public involvement

There was no patient or public involvement in the scoping review. The scoping review was designed to find secondary evidence. The findings inform further primary studies exploring the system-level mechanisms identified in the scoping review. These studies have a significant focus on involvement of and consultation with care experienced young people, social workers and social care managers.

Protocol design

The scoping review methodology is structured and reported in accordance with Arksey and O’Malley’s methodological guidance29 and Levac et al’s30 methodological enhancement. There are six composite stages: (1) identification of the research question; (2) identification of relevant studies; (3) study selection; (4) charting of the data; (5) collation, summary and reporting of results; (6) consultation with relevant stakeholders. Protocol components have been cross-referenced with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) Checklist to ensure completeness.31 32

Stage 1: identification of the research questions

The broad remit of the scoping review was set by the funder of the What Works Centre (WWC) for Children’s Social Care, the DfE, which was developed following a period of consultation with the sector on the aims and foci of the Centre. The multidisciplinary systematic review research team at Cardiff University, the research partner of the WWC, distilled this policy interest into a set of operational research questions. Questions were refined through sector engagement by the WWC, consultation with the expert panel of the WWC, consideration of key publications and academic journals, and preliminary searching of relevant databases.

The aim of the scoping review is to map the evidence base in regard to the following three questions:

What is the nature and quantity of evidence for interventions that aim to safely reduce the number of children and young people entering statutory care?

What it the nature and quantity of evidence for interventions that aim to safely reduce the number of children and young people re-entering statutory care?

What it the nature and quantity of evidence for interventions that aim to safely increase the reunification of children and young people with their families following a period in out-of-home statutory care?

As part of a scoping review comprises the elicitation and clarification of key variations in nomenclature, concepts and outcome measurements, we are also interested in mapping corollary and proximal outcomes that may support the reduction of children and young people in care. We understand that some studies may assess more specific aspects of the care process (eg, reduction in number of care and supervision orders or care plans). With extensive international variation in these, it is important to develop a map of intermediary measurements and how they might relate to the primary outcomes.

Stage 2: identifying relevant studies: eligibility criteria, information sources and search strategy

Eligibility criteria

The eligibility criteria were developed in accordance with the PICO (Population, Intervention, Comparator and Outcome) format.31 To incorporate the EMMIE framework, an additional Evaluation (E) criteria was included, with studies being eligible if they reported evidence mapping onto one or more of the EMMIE dimensions (table 1).

Table 1

PICO (E) scoping review eligibility

PICO (E)	Inclusion criteria
Population	Children and young people who are in need of care or have been in care when ≤18 years old
Intervention	Interventions are defined as a disruption to the system.11 23 They can operate across a single or multiple socioecological domain(s): intrapersonal; interpersonal; organisational; community; policy
Comparator	Usual care; alternative intervention; no comparator
Outcome	No of children and young people entering care No of children and young people (re-)entering care No of children and young people re-unified with their families following a period in statutory care  Corollary or proximal outcomes that support three outcome measures
Evaluation	Included if Effectiveness of the intervention is evaluated

The expanded inclusion criteria that will guide study selection is presented in the appendices (online supplementary appendix 1).

To ensure review results are relevant to the UK setting, inclusion is limited to research conducted in the following countries: England, Wales, Scotland, Northern Ireland, USA, Canada, Australia, New Zealand, France, Germany, Sweden, Finland, Norway, Denmark, Netherlands and Ireland. While there are differences in the legal and social frameworks, research from these countries was deemed more likely to be applicable.

Information sources

The following 18 databases will be searched: ASSIA, British Education Index, Child Development & Adolescent Studies, CINAHL, Embase, ERIC, HMIC, IBSS, Medline (including Medline in Process and Medline ePub), PsycINFO, Scopus, Social Policy & Practice, Social Services Abstracts, Sociological Abstracts and Web of Science (Social Sciences Citation Index, Conference Proceedings Citation Index—Social Science & Humanities, Emerging Sources Citation Index). Grey literature will be identified through the following online resources: Action for Children, Barnardo’s, Care Leavers’ Association, Children’s Commissioners’ offices for four UK nations, Children’s Society, Child Welfare Information Gateway, Department for Education, Early Intervention Foundation, Joseph Rowntree Foundation, National Institute for Health and Care Excellence, OpenGrey, REES Centre, Samaritans and Thomas Coram Foundation. Experts will be contacted to identify relevant published and unpublished studies.

Search strategy

A search strategy designed to balance sensitivity and specificity will be developed to identify research evidence in any language from 1991 (in accordance with the enactment and introduction of the UK Government’s Children Act in 19897) to 2018. The strategy will be developed in Ovid Medline and adapted to the functionality of other databases (see online supplementary appendix 2).

Stage 3: selecting studies

Selection processes

A subset of studies will be independently screened by all members of the review team to calibrate the inclusion criteria and ensure consistency in approach. Following this, study title and abstracts will be independently appraised against the inclusion criteria by two reviewers. A safety first approach will be adopted whereby if one reviewer included at title/abstract, then the full text will be examined.33 Reasons for exclusion will be recorded at full text. Discrepancies will be resolved by consensus and, where this is not possible, a third reviewer will arbitrate. Where it is unclear from the full text if a study meets the inclusion criteria, the authors will be contacted for additional information.

Data management

Retrieved studies will be imported into EndNote and de-duplicated, before being uploaded to Rayyan QCRI online application for study selection.34 Completed screening records will be converted to RIS files and transferred to EndNote for storage. Full text of included studies will be exported to NVivo for charting.

Stage 4: charting the data

Data extraction

Data across the following domains will be extracted: outcome (care entry, care re-entry, reunification); intervention type (intervention activities and resources; socioecological domain of intervention (community, policy, organisational, family or interpersonal level)); EMMIE dimensions addressed (table 2); study characteristics (authors; year of publication; country; study design; target population, eg, social workers, family, children, young people; analysis approach).

Table 2

Operationalisation of EMMIE for identifying whether a study has the type of evidence for each dimension of EMMIE (adapted from Johnson et al 26)

EMMIE dimension	Is at least one of the following reported in source
Effect	Effect size
	Measurement/consideration of unanticipated effects
Mechanism/mediator	Map of possible mechanisms/logic maps
	Mediator or mechanism-based moderator analysis
	Assessment/statements of most likely mechanisms and any contextual conditions (these can be narratives)
Moderator/context	Context-based moderator analysis/subgroup analysis (analysis testing the differences that context makes to outcome; theoretically driven/conducted due to data availability/not theoretically driven/not mentioned prior to analysis)
	Statements qualifying contextual variations (these can be narratives)
Implementation	A list/statement/description of key components effecting implementation success (including fidelity issues, barriers and facilitators to implementation, acceptability, feasibility etc)
	A list/statement/description of key components deemed necessary for replication elsewhere
Economic	Quantification of inputs to the intervention/intervention outputs
	Quantification of intensity (eg, spend per head)
	Estimate of cost of implementation (including by subgroup)
	Estimate of cost-effectiveness by unit output or by subgroup
	Estimate of cost–benefit (including by subgroup)

Data extraction will be piloted and calibrated with a subset of included studies. Due to the complexity of the data extraction, four reviewers will independently extract outcome, EMMIE, intervention and socioeconomic data and then discuss decisions in a group from 10% of studies to ensure consistency. Data will be extracted from the remaining studies independently by three reviewers, with a fourth reviewer to resolve issues. Regular meetings to discuss emerging issues will ensure ongoing consistency. Where additional data about the intervention are required, the authors will be contacted. If it is not possible to obtain this data, the citation will be added to an appendix of ‘potentially relevant’ research. Study characteristics will be extracted by additional research and administration staff as available. Data on overarching study characteristics will be extracted to an Excel file to support the numerical summary of findings.

Data within each paper will be coded with the support of NVivo V.12.35 A hierarchical coding tree will be indexed according to these domains with a subset of studies and will be refined and confirmed with the review team. Memos will be generated to ensure reviewer reflexivity.

Risk of bias

In line with prescribed scoping review methodology, study quality will not be appraised. Scoping reviews intend to map the concepts underpinning a research area and the main sources and types of evidence available,36 rather than an assessment of the quality of individual studies.

Stage 5: collating, summarising and reporting the results

Data analysis

A descriptive numerical summary and a qualitative thematic analysis will be undertaken.30

Descriptive numerical summary analysis: Using information from the Excel data extraction form, studies will be grouped numerically and hierarchically according to outcome, intervention type, socioecological domain, study type and evidence type (Effect, Mechanisms, Moderators, Implementation and Economic). This mapping will quantify the spread of the extant research evidence and identify key evidence gaps.

Qualitative analysis: Coded data will be grouped into outcome, intervention type, socioecological domain, study type and evidence type. Data will be grouped according to the system-level mechanism (the first M of the EMMIE dimensions) through which the intervention aims to change care outcomes.

Reporting the results

Results will be reported in three ways: a summary table mapping the quantity and nature of the data; a narrative summary of key evidence clusters around system-level mechanisms, gaps and uncertainties; a conceptual map to visually depict the spread of the identified evidence (figure 1).

Figure 1

Example conceptual map of type and quantity of evidence across child’s pathway in/out of care (background squares), socioecological domains (nested circles), intervention type (blue writing) and the spread of evidence across the five EMMIE dimensions (pie charts).

Stage 6: consulting with relevant stakeholders

Patient and public engagement

The WWC will consult on the review findings with the English social care sector via policy and practitioner panels and continued knowledge translation events. From the scoping review, the review team will identify a range of candidate research questions and systematic reviews, based on there being sufficient quantity of research evidence to warrant synthesis and the absence of similar or recent systematic reviews. These will be the subject of prioritisation by care-involved families and frontline social care practitioners according to their importance and relevance to the English social care sector. Focus groups will be recruited through the WWC’s partner Local Authorities (LAs). Stakeholders will be invited to support the appropriate interpretation, presentation and dissemination of review findings to ensure relevance to the diverse range of policy and practitioner audiences.

The scoping review findings and subsequent systematic reviews and primary studies will also inform the research agenda for the wider Centre. They will support the identification of de novo interventions that warrant robust scientific evaluation by the Centre or evidence-based interventions that have demonstrated effectiveness elsewhere and require adaptation and/or evaluation replication within the UK context. Through the application of the EMMIE framework, the review will also progress understanding of the contextual contingencies of intervention effects and how the social care system can best support the implementation of approaches aimed at safely reducing the number of children and young people in statutory care.

Ethics and dissemination

The scoping review methodology reviews data from publicly available materials and so does not require ethical approval.

The present scoping review constitutes the first phase of research as part of the What Works Centre for Children’s Social Care’s aim to safely reduce the number of children and young people residing in care in England. The results of the scoping review will inform the conduct of a suite of systematic reviews and primary studies with a more focused consideration of the evidence base for specific interventions, populations or outcomes. These reviews will also include a robust appraisal of evidence quality.