Interventions to reduce suicidal thoughts and behaviours among people in contact with the criminal justice system: A global systematic review.

BACKGROUND: People who experience incarceration die by suicide at a higher rate than those who have no prior criminal justice system contact, but little is known about the effectiveness of interventions in other criminal justice settings. We aimed to synthesise evidence regarding the effectiveness of interventions to reduce suicide and suicide-related behaviours among people in contact with the criminal justice system.
METHODS: We searched Embase, PsycINFO, MEDLINE, and grey literature databases for articles published between 1 January 2000 and 1 June 2021. The protocol was registered with PROSPERO (CRD42020185989).
FINDINGS: Thirty-eight studies (36 primary research articles, two grey literature reports) met our inclusion criteria, 23 of which were conducted in adult custodial settings in high-income, Western countries. Four studies were randomised controlled trials. Two-thirds of studies (n=26, 68%) were assessed as medium quality, 11 (29%) were assessed as high quality, and one (3%) was assessed as low quality. Most had considerable methodological limitations and very few interventions had been rigorously evaluated; as such, drawing robust conclusions about the efficacy of interventions was difficult.
INTERPRETATION: More high-quality evidence from criminal justice settings other than adult prisons, particularly from low- and middle-income countries, should be considered a priority for future research. FUNDING: This work was funded by the Australian government's National Suicide Prevention Taskforce. RB is supported by a National Health and Medical Research Council (NHMRC) Emerging Leader Investigator Grant (EL2; GNT2008073). MW is supported by a NHMRC Postgraduate Scholarship (GNT1151103). SF was funded by the NIHR HTA Programme (HTA Project:16/159/09).

Evidence before this study

One previous review had synthesised the literature regarding the effectiveness of interventions during incarceration, but no studies had investigated the effectiveness of interventions to prevent suicidal thoughts and/or behaviours among people in contact with the multiple other settings in the criminal justice system. We searched Embase, PsycINFO, and MEDLINE on 1 June 2021 using variants and combinations of search terms relating to suicide, self-harm, prevention, and criminal justice system involvement (suicide, self-injury, ideation, intervention, trial, prison, probation, criminal justice).

Added value of this study

Our review identified gaps in the evidence base, including a dearth of robust evidence regarding the effectiveness of interventions across non-custodial criminal justice settings and from low- and middle-income countries. We identified the need for studies examining suicide prevention initiatives for people who were detained in police custody, on bail, or on parole/license, those serving non-custodial sentences, and those after release from incarceration. Furthermore, our findings suggested an absence of interventions which considered specific population groups with diverse needs, such as women, First Nations people, and young people.

Implications of all the available evidence

Significant gaps in the evidence base exist. To reduce the high rates of suicide currently observed among people in contact with the criminal justice system, high-quality future research across criminal justice settings beyond prisons – particularly from low- and middle-income countries – should be considered a priority.

Alt-text: Unlabelled box

Introduction

People who come into contact with the criminal justice system (referred to as justice-involved people) die by suicide at a higher rate than those who have no criminal justice system exposure.,2, 3, 4, 5,6, 7, 8, 9 This is, in part, because the life trajectories of many people in contact with the criminal justice system are characterised by chronic instability, abuse, neglect, and intergenerational disadvantage,10, 11, 12 all of which increase the risk of suicidal thoughts and behaviours.,, This risk is further compounded by an increased prevalence of complex, co-occurring health conditions and risk-taking behaviours in this population, such as elevated rates of mental disorder, (including major depression and personality disorders)18and substance use disorders. The increased suicide rate is even more pronounced among young people (i.e., those aged <25 years), exposed to the criminal justice system, with one Australian study finding that one third of all deaths in young people released from adult prisons were due to suicide. Additionally, in many colonised societies, such as Australia, New Zealand and North America, Indigenous people have higher rates of suicide, and are disproportionately affected by the criminal justice system compared to non-Indigenous people., Given the markedly increased risk of self-harm and suicidal behaviour among justice-involved people, particularly among young and Indigenous people, measures to reduce this risk are important to public health and to addressing health inequalities.

Preventing suicide during incarceration has been identified as a priority by the World Health Organization (WHO) and several countries have published national guidelines for suicide prevention in custodial settings. While previous reviews have examined interventions to prevent suicides in prison,, little is known about the effectiveness of interventions to reduce suicide (and suicidal thoughts and behaviours) in other criminal justice settings, such as in youth detention, courts, police custody, or for people serving non-custodial sentences (e.g., under community-based supervision) or on parole/licence. Further, the risk of dying by suicide is markedly increased after release from incarceration, when many people are exposed to numerous stressors and often have limited support or access to services.,, The risk of self-harm is also elevated after release from prison,31, 32, 33 further increasing the (already elevated) risk of dying by suicide., Despite this, in many settings there are no formalised policies regarding suicide prevention after release from prison, or in non-custodial criminal justice settings. Synthesising the literature across these settings is necessary to inform development of such guidelines. In this review, we aimed to identify and synthesise evidence regarding the effectiveness of interventions to reduce suicide and suicide-related behaviours among people in contact with all settings of the criminal justice system.

Methods

Overview

Our review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42020185989).

Search Strategy

We searched Embase, PsycINFO, and MEDLINE using variants and combinations of search terms relating to suicide, self-harm, prevention, and criminal justice system involvement (appendix pp. 1-3). We reviewed the reference lists of included studies and used the authors’ professional networks to identify additional eligible studies. Searches were limited to studies published between 1 January 2000 and 1 June 2021 to ensure that the review reflected contemporary evidence. Conference abstracts and reviews were excluded. To identify relevant grey literature, we used keyword searches in Google, the Aboriginal and Torres Strait Islander Health Bibliography, the Indigenous Justice Clearinghouse, and the Health Issues in Criminal Justice database (appendix p. 4). We also used keyword searches on the websites of relevant international organisations, including the American Foundation for Suicide Prevention, the International Society for the Study of Self-Injury, beyondblue, and the Black Dog Institute in Australia.

Study selection criteria

Studies were eligible for inclusion if they presented original data regarding the effectiveness of interventions to reduce suicide and/or related outcomes (including self-harm, suicidal ideation, and suicide attempts) among people who had come into contact with any component of the criminal justice system. We included studies of people who had contact with one or more of the following criminal justice settings: in a custodial setting (e.g., prisons, jails, youth detention facilities), in police custody, in the community whilst awaiting trial, in court facing charges, participating in diversion and/or appearing before special courts (e.g., problem-solving court or a court exercising therapeutic jurisprudence), remanded in custody (pre-trial, awaiting sentencing, or held in custody through inability to meet bail conditions), on parole/licence, serving community-based supervision orders/sentences (e.g., probation), on community forensic orders, or in a secure forensic hospital. Studies were excluded if participants had not come into contact with the criminal justice system, participants were (or had been) detained for reasons not related to the criminal justice system (e.g., immigration detention), or if suicide and/or related outcomes were not reported.

Study selection

Identified citations were imported into EndNote reference management software and duplicates were deleted, before being uploaded into the citation management software Covidence for screening. Titles and abstracts of potentially eligible studies were reviewed by two of four trained researchers (AC, EJ, AB, MW). After title and abstract screening was completed, the full text of each remaining article was screened by two of the same four researchers. Non-English language papers were translated for full text screening using Google Translate, which has been previously demonstrated to be a viable and accurate tool for translating articles published in other languages into English and abstracting data for systematic reviews. Uncertainty regarding eligibility was resolved through discussion with the senior author.

Data extraction

Data were extracted by four researchers (AC, EJ, AB, MW). The following information was extracted: year of publication; country; intervention type; duration of the intervention; outcome(s) and method(s) of measurement; participant demographics; number of participants; criminal justice system setting; evaluation design; duration of follow-up; key findings; and study limitations (as determined by both the study authors and by our research team).

Quality assessment

The Joanna Briggs Institute (JBI) Critical Appraisal Tools were used to assess the methodological quality of all primary research publications by evaluating the extent to which they addressed the possibility of bias in a number of areas of study design, conduct, and analysis. Five different versions of the JBI tool were used depending on the study design (e.g., quasi-experimental studies, analytical cross-sectional, cohort, randomised controlled trial (RCT), and qualitative studies). The number of assessment domains varied among the different versions of the tool and ranged from 8-13. Each of the domains received a quality score of 0 (indicator not present or met), 1 (indicator unclear or not applicable), or 2 (indicator applicable and met). A total quality score was calculated by summing the individual domain scores. Each study received a low, medium, or high score for quality based on an established cut-off score for each study design (appendix p. 5). The same four researchers independently assessed the quality of included publications, with one researcher assessing each publication and any uncertainty in their assessment resolved through further discussion. We did not exclude studies based on quality; considering the limited extant evidence base, we believed that it was important to critically review all available literature.

Data synthesis

The Sequential Intercept Model (SIM) is a framework that considers the interface between the criminal justice and mental health systems. Central to the SIM is the idea that each intercept point within the criminal justice system represents an opportunity to prevent suicide and divert people with mental disorders away from the criminal justice system. We had planned a priori to synthesise the literature according to criminal justice system setting; however, due to the small number of studies conducted in some settings, and the diversity of interventions, this was not possible. Instead, we undertook a narrative synthesis of included studies, grouped according to intervention components using a typology that emerged from the data (Table s1; appendix p. 6]), and prioritised the narrative synthesis of studies that were of higher quality and stronger study design. Given the diversity of study designs and outcomes measured, we were unable to meta-analyse the findings.

Role of the funding source

This work was funded by the Australian government's National Suicide Prevention Taskforce. The funder approved the study design, data collection, and analysis methods, but had no access to the dataset and no role in the interpretation of the findings, decision to publish, or preparation of the manuscript. AC, AB, MS, EJ, and RB had access to the dataset and all authors were involved in the decision to publish the manuscript.

Results

The electronic search yielded 2014 articles, with an additional 7 identified through other sources. Of these, 1518 articles remained after duplicates were removed, and the full text of 167 articles was examined for eligibility. The final review comprised 38 studies: 36 primary research articles and two grey literature reports (Figure 1). An overview of included studies is provided in Table 1.

Figure 1

Flow diagram of study selection.

Table 1

Initiatives to prevent suicide and suicidal behaviours in criminal justice system settings.

Author(s), year, citation	Country (region)	Study Design	Sample	Setting	Intervention description	Measures	JBI score grouping	Key findings and limitations
Model of care
Bistodeau & Daigle (2000)1	Canada (Quebec)	Mixed methods, quasi-experimental pre-test post-test, qualitative component	24 adult males	Prison	Service with two main components: promotion of mental health and suicide prevention. Service activities include group therapy revolved around problem solving therapy and/or peer helper training. Referral to the service takes an average of 6 months and individual needs of the person determines where they are referred.	Suicidal ideation measured by the Suicide Probability Scale and qualitative data from interviews.	Medium	Quantitative results were limited due to the small sample size. Qualitative results revealed that the service attracted people experiencing suicidal ideation who did not currently participate in other activities. Qualitative interviews indicated that peer helpers were well received by staff and their positive impact in the prison community. Authors commented that it was unclear as to whether the observed reduction in the prison suicide rate was related to the service.
Fortune et al., (2010)2	UK	Qualitative design.	30 adult men and women; 26 of 30, aged 22–56, 87% white	Community Forensic Order	Three new medium-secure forensic services to provide treatments to address mental health needs for people with borderline personality disorder. Service 1 medium-secure unit and a residential service. Service 2 inpatient medium-secure unit and a community team. Service 3 inpatient medium-secure unit, a community team and a residential service, consisting of two hostels.	Client perspectives on changes in self-harm behaviour as a result of intervention	Medium	Limited qualitative evidence that all three services helped service users to reduce the frequency of self-harm.
Rivlin (2010)3	UK	Mixed methods: prospective cohort with a qualitative component.	Qualitative component sample: 24 incarcerated males, 4 incarcerated male listeners, 13 members of staff. 83% white = 83%	Prison (HMP Grendon)	Model of care: Democratic Therapeutic Community (DTC).	Self-harm	Medium	DTC had rates of self-harm (29 incidents per 1,000 incarcerated people per year) less than a quarter of the rate at non-TC prisons in England and Wales (137 and 130 incidents per 1,000 incarcerated people in 2004 and 2005, respectively). This finding could not be explained.
Glowa-Kollisch et al., (2016)4	USA (New York)	Retrospective single-group cohort, n = 90.	1718 adults, males = 59; female = 31; mean age = 30; 97.8% aged >19.	Prison	New treatment units for people with serious mental illness; Clinical Alternative to Punitive Segregation (CAPS) unit; designed to offer a full range of therapeutic activities and interventions for these patients, including individual and group therapy, art therapy, medication counselling and community meetings.	Rates of self-harm	High	Individuals who spent time in both CAPS and Restrictive Housing Unit (RHU) had self-harm rates five times higher when in RHU (0.98) than when in CAPS (0.19). Small sample size limits ability to detect meaningful differences, no control of confounders, did not stratify outcome data by youth/adult.
Ford et al., (2020)5	USA (New York)	Retrospective cohort, non-equivalent control group design (propensity score matching used, pairs = 302).	302 adult males, median age = 36 years; Hispanic = 27%; white = 11%;Non-Hispanic Black = 55%;Non-Hispanic Asian = 4%Other/ missing = 4%	Prison	PACE (Program for Accelerating Clinical Effectiveness) units – designed to have large, open spaces, confidential interview rooms, adequate space for protected group activities, staff offices, and as much natural light as possible. Duration of program ≥ 14 days, overall median length of stay in PACE was 58.5 days.	Self-injury	High	PACE participants had non-statistically lower rates of self-injury, compared with control participants, at both 30 and 60 days.
Glowa-Kollisch et al., (2014)6	USA (New York)	Quasi-experimental non-equivalent control group pre-test post-test design.	898 adult males, treatment group (TG) = 218; control group (2011) = 267; control group 2010 = 413. Non-Hispanic white = 18%;Non-Hispanic Black = 54%;Hispanic = 3%;Asian/Pacific Islander = 1%;Other/unknown = 25	Prison	Beyond the Bridge aimed to improve mental health services for adults in the facility's dedicated mental health units. Group therapy, individual encounters with social workers, psychologists, psychiatrists, and discharge planners. Duration: 6 weeks.	Acts of self-harm, placement on suicide watch	Medium	Inconclusive – TG had significantly reduced self-injurious behaviour when compared to 2011 controls. However, no difference between TG and 2010 controls.
Maguire et al., (2018)7	Australia (Victoria)	Quasi-experimental single-group pre-test post-test design.	28 adult males, ages 23-70	Forensic Hospital	The Safewards model consists of six domains: staff team, physical environment, outside hospital, patient community, patient characteristics, and the regulatory framework. Duration: 12 months.	Incident reports for self-harm	High	Very low number of self-harm incident reports pre and post. A decrease was observed, though this could be a chance finding.
Group programs
Johnson et al., (2019)8	USA	Experimental design, RCT.	181 adults, 117 males, 64 females; ages 18-65; IPT + TAU group = 91; TAU = 90. African American/Black = 20%; Asian = 1%; Native American/ Alaskan Native = 4%; 62% white, 13% other = 13%	Prison	Interpersonal Psychotherapy (IPT) delivered using 20 group therapy sessions of 90 min each over 10 weeks, plus four individual sessions.	Suicidal ideation9	High	No differences between IPT and Treatment As Usual (TAU) groups. Limitations: too few participants reported suicidal ideation for meaningful comparisons.
Black et al., (2013)10	USA (Iowa)	Single-group pre-test post-test design.	77 adults, 18% male, 82% female, ages 19-50, 89% white; 10% African American; 1% Native American	85% mediumsecurityprison, 15% community corrections	STEPPS program for persons with borderline personality disorder combines cognitive behavioural elements with skills training. 20 × 2-hour weekly sessions.	Suicidal and self-harm behaviour	Medium	Program significantly reduced suicidal behaviours. Limitations: one group, follow-up was at week 20 only, only for people with BPD.
Baybutt et al., (2019)11	UK	Qualitative, focus groups with participants and staff.	16 adult males, aged over 21 years	Prison	Greener on the Outside for Prisons (GOOP) is a social and therapeutic horticulture and environmental programme.	Self-harm	Medium	Weak, anecdotal evidence suggesting GOOP reduced the frequency of self-harm for some participants.
Eccleston & Sorbello (2002)12	Australia (Victoria)	Quasi-experimental single-group pre-test post-test design (though only qualitative anecdotal evidence relating to self-harm available).	Adults, males (age and number of participants not reported)	Remand centre	The Real Understanding of Self-Help (RUSH) program uses cognitive behaviour therapy (CBT) to validate participants’ past and current emotional, cognitive and behavioural responses to stressful situations and life experiences. 20 × 2-hour sessions, delivered twice per week for 10 weeks.	Self-harm	Low	Weak, anecdotal evidence from correctional officers that support effectiveness of RUSH in reducing self-harm.
Gee and Reed (2013)13	UK	Quasi-experimental non-equivalent control group pre-test post-test design.	62 adult females (of whom 29 completed the program)	Prison	Modified dialectical behaviour therapy [DBT]: group skills training, individual therapy, and team consultation meeting, delivered over 8 weeks.	Suicidal ideation and self-harm risk	Medium	DBT reduced suicidal ideation and self-harm risk among program completers (n = 29) with comparison to non-completers (n = 33). Not statistically significant and did not provide explanation for low completion rate.
Jackson (2003)14	USA (Georgia)	Quasi-experimental single-group pre-test post-test design.	61 adults, aged over 19, sex unclear	Prison	Psychoeducational program which aims to provide coping skills training.	Fear of suicide (no further explanation provided)	Medium	No significant change in the fear of suicide following program completion. Limitations: brief report - lacking in discussion about all of the measures.
Long et al., (2011)15	UK	Quasi-experimental non-equivalent control group pre-test post-test design.	44 adult females	Medium secure Forensic hospital	Dealing with Feelings Skills Group Training; CBT group treatment adapted from DBT for women with either a primary or secondary diagnosis of personality disorder.	Suicidality	High	Following treatment, program completers had lower suicidality scores. Limitations: The results compared program completers with non-completers.
Pratt et al., (2015)16	UK (Northwest England)	RCT. 31 in CBSP and 31 in treatment as usual (TAU) group.	62 adult males, aged 21-60; 85% white, 6% Black	Prison	Cognitive Behavioural Suicide Prevention (CBSP) – structured, time-limited psychosocial intervention developed to treat individuals experiencing suicidal ideation and/or behaviour. 20 sessions, delivered twice weekly initially, then once weekly.	Self-injurious behaviours (SIBs)	High	At 6 months the CBSP group achieved a significantly greater reduction in self-injurious behaviours compared to TAU group (6-month mean [SD], TAU: 1.48[3.23] vs CBSP: 0.58[1.52], p = 0.003), with a moderate treatment effect (Cohen's d = -0.72, 95%CI: -1.71 to 0.09; baseline mean [SD], TAU: 1.39[3.28] vs CBSP: 1.06[2.10]). At the end of treatment, over half (10/18, 56%) of participants in the CBSP group achieved a clinically significant recovery, compared with a quarter (5/22, 23%) of the TAU group (χ2 = 4.55, p = 0.03). A clinically significant recovery for participants was indicated for total scores of <67 on the suicide probability scale.
Rasmussen et al., (2018)17	Australia (Queensland)	Retrospective cohort study design.	335 Indigenous adult men	Prison	Aboriginal art program for incarcerated people located in a separate cellblock and facilitated by a cultural liaison officer. The cultural space provided incarcerated Aboriginal people with a social environment to practice Aboriginal art, socialise and make contact with visiting elders from the local community.	Suicide/self-harm risk factors drawn from Initial Risk Needs Assessments recorded within Integrated Offender Management System	High	The findings of this study are limited due to the exclusion of participants currently at risk of suicide/self-harm from participation in the program. The authors found, after adjusting for suicide/self-harm history, there was strong evidence that an increase in attendance the Aboriginal art program was associated with reduced incidence of suicide/self-harm. Each day of attendance to the Aboriginal art program contributed to an average 19% (CI 95%: 12–25%) reduction in the rate of suicide/self-harm risk assessment.
Riaz & Agha (2012)18	Pakistan (Karachi)	Quasi-experimental single-group pre-test post-test design.	9 adult females, ages 21-50	Prison	CBT administered over 12 sessions; one session per week for four months; each session lasted between 45 to 60 minutes.	Self-harm	Medium	Intervention was not successful in reducing self-harm.
Walker et al., (2017)19	UK	Qualitative design, participant interviews.	13 adult females, ages ≥18; white = 12; mixed/minority = 1.	Prison	Group letter program: receipt of a good-bye letter at the end of brief psychodynamic interpersonal therapy (PIT).	Participant perceptions of impact on self-harm	High	Although some qualitative evidence suggesting the good-bye letter prevented self-harm in participants, impact of program unclear.
Webb (2020)20	USA (Tennessee)	One-group, pre-test post-test	64 adult males	Prison	2-week mindfulness program where participants received between 1 to 6, 90-minute sessions with between 2 to 14 participants in each session.	Suicidal ideation using the Patient Health Questionnaire (PHQ-9)	Medium	The mean suicidal ideation scores post-test were significantly reduced when compared with pre-test. Authors also reported moderately strong, negative, partial correlation between mindfulness and depression and suicidal ideation scores after controlling for pretest mindfulness, whereby participants with the most change in mindfulness experienced significant change in depression and suicidal ideation.
Peer support programs
Hall and Gabor (2004)21	Canada (Alberta)	Retrospective cohort design.	322 adults, sex unclear	Prison	SAMS in PEN peer suicide prevention. Inmate volunteers can apply for training. A request to meet a SAM is made directly between potential client and peer volunteer.	Suicide, attempted suicide and suicide risk	Medium	Inconclusive - low base frequencies.
Perry et al., (2021)22	England	Prospective cohort design.	828 male adults	Prison	An existing problem-solving therapy (PST) skills intervention was adapted for a prison cohort. 36 incarcerated adults were trained to become peer-support mentors (PSM) and deliver the PST to other incarcerated adults over 4 months.	Self-harm	Medium	For the 130 (16%) of peers who received the full delivery of the PST skills from PSMs (because 698, 84% received promotion only), a significant reduction in the incidence of self-harm was observed (rate of – 0.25 per month 95% CI -0.37 to -0.13, p < 0.001). However, there was no evidence of prison-wide reductions in self-harm (p = 0.253) or for those that received promotion only (p = 0.883). Findings limited by lack of a control group.
Individual/tailored treatment programs
Nee & Farman (2005)23	UK (Portsmouth)	Quasi-experimental single-group pre-test post-test design.	Adult females, sample size unclear somewhere between 19-30, ages 19-49; All but three participants were white	Prison	DBT for women diagnosed with BPD. 12-month programs in two prisons, and a shortened program format that ran in one more prison (one 16 and two 12 weeks during a 20-month pilot period).	Self-harm incidents, measured by a 'hand trawl of prison self-harm records	Medium	Reduction in self-harm from pre-DBT to during DBT (number of incidents recorded). Small sample size so conclusions limited. Follow-up data for 14 DBT participants and 5 control participants.
Perry et al., (2019)24	UK (Yorkshire and Humber)	Qualitative.	Adult males & females, 48 received intervention	Prison, sentenced and awaiting sentence	Adaptation of an existing community-based problem-solving skills intervention delivered by prison staff in four UK prisons.	Participant and staff perceptions (self-report on self-harm)	Medium	Some evidence to suggest the intervention reduced self-harm. Unclear how many recipients of the intervention were interviewed.
Walker et al., (2017)25	UK, England	RCT	113 women, aged 18 – 52 (mean 29.92), 56 received intervention, 57 received Active Control (AC)	Prison	A six-week Psychodynamic Interpersonal Therapy (PIT) adaption delivered over 4-6 × 50 min sessions which uses a client-therapist relationship as a tool for resolving interpersonal issues. Participants were randomised to the PIT or AC trial arm using a stratified block method to achieve balanced characteristics in each arm. AC trial arm received a session that consisted of being taken out of their cells and having non-prison staff company. They had four sessions over 4 weeks that lasted 50 minutes.	Suicidal ideation (Beck's Scale for Suicidal Ideation)	Medium	Both intervention groups had improved suicidal ideation at post-intervention. No difference in suicidal ideation between PIT and AC. Limitations: Use of select samples – included individuals at risk of self-harm or suicide. Attrition issues limited power of the study.
Western Australian Mental Health Commission (2015)26	Australia (Perth)	Quasi-experimental single-group pre-test post-test design.	No information provided.	Mental Health Diversion Court	The Start Court Program delivered by multidisciplinary team: magistrate, police prosecutor, duty lawyer service, court coordinator, psychiatrist, psychologist, clinical nurse specialists, and senior social worker. Six month individually-tailored program reflecting the needs of the individual.	Risk of suicide	Medium	67% of participants were assessed as being at lower risk of suicide following the program. Not tested for significance.
Low et al., (2001)27	UK (Woodbeck)	Quasi-experimental single-group repeated measures design.	10 adult females	Forensic hospital	DBT – weekly, individual psychotherapy combined with group behavioural skills training over the course of 1 year.	Self-harm and suicide ideation	Medium	Significant reduction in self-harm during therapy, which was maintained at 6-month follow-up.
Camp et al., (2018)28	UK (South East England)	Retrospective single-group pre-test cohort study	35 adult males, mean age 33; 71% white = 71%; 29% Black, Asian, Minority Ethnicities	Prison, sentenced and remand	Flexible, individualised psychosocial program which aimed to reduce custodial violence and disruption. Delivered by a small multidisciplinary team of clinical staff and a prison officer for 8 to 10 weeks.	Self-harm	Medium	Frequency of self-harm decreased following intervention but this decrease was not significant. Study limited by high attrition.
Multi-component programs
Freeman & Alaimo (2001)29	USA (Illinois)	Descriptive	Adult, male	Prison	Cermak Mental Health Services - Cook County Department of Corrections Suicide Prevention Program. Multifaceted throughcare suicide prevention program with different components from reception stage through to release.	Suicide rates	Medium	Impact of multifaceted throughcare suicide prevention program on suicide rates unclear. Limited by lack of a control group.
Sarotar et al. (2018)30	Slovenia	Descriptive	Adult, male (n = 520 at the time of the study but varies across years); age range 21-74	Prison	Prison wide anti-suicide Strategic Plan in 2004 and the implementation of screening for suicidal behaviour at the entrance point for every incarcerated person in 2012	Suicide rates	Medium	Limited by low base level of suicide before and after the strategic plan implementation. There were 13 suicides from 1995 to 2005, 7 suicides from 2005 to 2012 and no suicide from 2012 to 2015.
Legislation/policy changes
Kovasznay et al. (2004)31	USA (New York)	Retrospective, cross-sectional (repeated) relating to outcome of interest	Adult males; 15.6% white; 50% African American	Prison	Policy: environmental modifications, changes to clinical and administrative policies and procedures, and enhanced staff training	Suicide rates	Medium	Impact on suicide rates inconclusive. Suicide rates gradually decreased since measures were implemented. Rates reached low of 10.2 per 100,000 in 2001.
Shaw & Humber (2007)32	UK	Retrospective cohort study design.	Adult males and females	Prison	Policy: NHS took over prison health care in the UK in April 2006. Tailored assessment & care plan Program: An Assessment, Care in Custody and Teamwork (ACCT) approach.	Suicides	High	Impact on suicides inconclusive. Policy and tailored assessment & care plan decreased suicide rates by 14% (127 per 100,000 in 2004 to 90 per 100,000 in 2006), though authors note that it could be a chance finding.
Fruehwald et al., (2000)33	Austria	Retrospective cohort design.	Adult	Prison and remand	Various legislation changes in Austria including criminal law reform 1975, criminal law amendment, efforts to offer better therapeutic facilities - criminal law amendment, Increased employment of psychologists and social workers.	Prison suicide rates over time	Medium	Impact on suicides inconclusive. Suicide rates increased in prisons following changes (though likely due to confounding factors). A parallel decrease of the average daily prison population and incarceration of only people who had been convicted of highly violent offences.
Slade and Forrester (2015)34	UK (London)	Mixed methods: Retrospective case study design, factor identification, qualitative interviews.	Adult, male	Prison	National Suicide Prevention Strategy (1991–2008) and Local Suicide Prevention Strategy (Multiagency and cultural change) (2009-2011).	Suicide	Medium	Impact on suicide unclear. The prison's suicide decreased following implementation of the strategy. The study provides support for two pivotal factors: senior management supported cultural change and cross-professional collaborative working that led to this reduction.
Programs in youth detention settings
Department of Justice (2009)35	USA	Retrospective survey design	79 youths; mean age = 15.7	Youth detention	Surveys of juvenile facilities that sustained suicides in the study period (1995-1999) to determine the facility characteristics including suicide prevention components in use (staff suicide prevention training; written suicide prevention policy, training, intake screening, CPR certification, observation, safe housing, and mortality review). Juvenile facilities included juvenile detention centers, reception centers, training schools, ranches, camps, and farms—operated by state and local governments and private organizations.	Suicide rate	Medium	36.7% of the suicides occurred in juvenile detention centres. Suicide prevention training and all seven critical components of suicide prevention policy decreased numbers of suicide. Although 78.5 percent of victims died in facilities that maintained a written suicide prevention policy at time of suicide, only 20.3 percent of victims were in facilities that had all seven suicide prevention components. Of all the suicides that took place in youth detention centres, 10.3 percent occurred in facilities that had all seven components.
Gallagher & Dobrin (2005)36	USA	Retrospective cohort study	Juvenile Residential Facility Census (n = 3690 facilities).	Youth detention	Intake screening components including the timeliness of screening after reception and percentage of people screened at reception. Facilities housing youth who are: awaiting adjudication, on probation, in youth detention, or youth in the adult criminal court.	Suicide attempts	High	Models indicated a significantly lower odds of suicide attempts in facilities that screen the entire population and screen within the first 24 hours.
Wakeman (2011)37 Study #1 and Study #2	USA	Quasi-experimental single-group pre-test post-test design.	Study 1:8 youth, all female; ages 14-18. 88% African American; 12% Caucasian Study 2: 38 youths, all female, ages 13-18; 58% African American; 16% Caucasian; 3% Biracial	Youth detention	Dialectical Behaviour Therapy's Core Mindfulness skills. Four principle modalities: group skills training, individual psychotherapy, telephone consultation, and a therapist consultation team. Program delivery is 4 weeks.	Suicidal ideation	Low	Study 1: No significant differences in suicide ideation following program completion. Very small sample with high attrition. Study 2: Suicide risk scores decreased over the course of the study.
Welfare & Mitchell (2005)38	UK (Suffolk)	Quasi-experimental single-group pre-test post-test design.	16 youths, male, age ranges 15-18	Youth detention	Access program was delivered over the course of 12 sessions (over 4 weeks), each session consisting of an hour of classroom-based group work, a short break and an hour in the gym. Each session built on what had taken place previously.	Suicidal ideation	Low	Decrease in mean Beck Hopelessness Scale by 3 points. Small sample. Suicide ideation not directly measured, but rather measured through Beck Hopelessness Scale.
Rohde et al., (2004)39	USA	Experimental design, RCT.	Male, aged 12-22; 64% white non-Latino; 6.7% African American; 2% Asian; 14% Latino; 10% Native American, 1.5% other	Youth detention	The coping course: 16 treatment sessions conducted over 8-week period in class-like fashion, with group leaders teaching adolescents a variety of skills. Intervention group = 46; Control group 1 = 30; Control group 2 = 62.	Current suicidal behaviour	Medium	Intervention had no effect on suicidal behaviour.

Characteristics of included studies

Of the 38 included studies, 16 were conducted in the United Kingdom (UK),,43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57 13 in the United States (US),58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70 four in Australia,71, 72, 73, 74 two in Canada,, and one study in each of Austria, Pakistan, and Slovenia. Thirty-three studies (87%) focused on adults and five (13%) focused on youth. The majority of interventions (n=27; 71%) were set in adult prisons, five (13%) were set in youth detention settings, three (8%) were set in forensic hospitals, one (3%) involved people serving community corrections orders, one (3%) involved both people serving a community forensic order and those serving a prison sentence, and one (3%) evaluated a suicide prevention intervention in a court setting (Table 2). No studies were identified which examined suicide prevention interventions for people detained in police custody, people on bail or parole, or people in the community who had been released from prison, youth detention, or a forensic hospital. Most studies (n=31, 82%) did not have a control or comparison group,,43, 44, 45,,,51, 52, 53, 54, 55,57, 58, 59, 60, 61, 62,,,,,70, 71, 72, 73, 74, 75, 76, 77, 78, 79 and only four studies were RCTs.,,,

Table 2

Overview of the effectiveness of interventions in reducing suicidal thoughts and behaviours for populations in different criminal justice system settings.

Setting	# of studies	Studies with some evidence of beneficial intervention effect	# of studies	Studies with no observed effect	# of studies	Studies with inconclusive findings	# of studies
# of studies		9		4		27
Police custody (current)	0		0		0		0
Awaiting trial in community (bail) (current)	0		0		0		0
Courts (current contact) – including specialist courts.	1		0		0	START Mental Health Court Diversion and Support Program**26	1
Parole (current)	0		0		0		0
Remanded in custody (current)	2		0	Tailored program delivered by multidisciplinary team**28	1	Cognitive Behaviour Therapy (CBT) & Dialectical Behaviour Therapy (DBT) RUSH program*12	1
Serving non-custodial sentence or community-based supervision order (current)	1		0		0	STEPPS Group Program**10	1
Community forensic order (current)	1		0		0	Three new medium secure forensic services**2	1
Prison/jail (current)	27	Cognitive behavioural suicide prevention program***16Problem-solving intervention**24Treatment units for severe mental illness***4Peer-support program**22Group mindfulness program**20	5	Group CBT program**18Model of Care for Accelerating Clinical Effectiveness (PACE)***5	2	STEPPS group program for borderline personality disorder**10Tailored program - delivered by multidisciplinary team**28Democratic Therapeutic Community (DTC) impact on self-harm compared with non-TC prisons**3The impact of legislative changes**33National Suicide Prevention Policy**34Multifaceted throughcare program**29Group Interpersonal Psychotherapy (IPT)***8Multifaceted suicide prevention strategy**30Model of care – promotion of mental health and suicide prevention**1Policy – environmental modifications, changes to clinical and administrative policies and procedures, and enhanced staff training**31Policy and tailored assessment & care plan on suicide rates***32Beyond the Bridge mental health unit's impact on self-injurious behaviour**6Therapeutic horticulture and environmental program**11DBT tailored program**23DBT group program**13Psychoeducational and CBT group program**14Group Aboriginal art program***17Peer suicide prevention program**21Goodbye letter group program***19Group Psychodynamic Interpersonal Therapy**25	20
After prison/jail (ever, lifetime contact)	0		0		0		0
Youth detention (current)	5	Facilities with all critical components of suicide prevention programs**35Facilities with appropriate intake screening*36	2	The coping course group program*39	1	Group and individual DBT**37Multifaceted classroom based and physical activity program**38	2
After youth detention (ever, lifetime contact)	0		0		0		0
Forensic hospital (current)	3	Tailored DBT program**27	1		0	The Safewards model with 6 domains (staff team, physical environment, outside hospital, patient community, patient characteristics, and the regulatory framework)***7Group CBT & DBT program***15	2
After forensic hospitalisation (ever, lifetime contact)	0		0		0		0

***= high quality study design; **= medium quality study design; and *= low quality study design.

Characteristics of participants

Of the 38 included studies, 33 (87%) sampled adults,43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,55, 56, 57, 58, 59, 60,62, 63, 64, 65,69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79 and five (13%) sampled youth.,,66, 67, 68 Among the 87% of studies that sampled adults, ages ranged from 18 to 74 and 17 included only males,,,,,,,,,,,,70, 71, 72, 73,, seven only females,46, 47, 48,,,, seven both males and females,,,,,, and two where the sampled sex was unclear., Four studies sampled adult participants from people detained in forensic hospitals,, or people on community forensic orders. Most studies did not report ethnicity, however for the eleven that did, eight reported a majority of participants (62 – 92%) that self-reported as white,,,,,,,, with a minority who self-reported as African American (10-20%), Native American (1-4%), Asian (1-4%), Black (5%) and other or missing (4%).,, Two studies reported that a majority of participants (54-55%) were Hispanic black,, and one study reported the majority to be African American (50%). Among the 13% of studies that sample youth aged ranged from 14 to 21 years, one of which had a majority identifying as African American (57.9-87.5%) and one a majority of white youth (64.2%). Two studies sampled male youths only,, two included all sexes,, and one sampled female youths only.

Quality of the evidence

Most studies (n=26, 68%) were assessed as medium quality according to our categorisation of the JBI, ,43, 44, 45, 46,,,,54, 55, 56, 57, 58,,63, 64, 65,,,,74, 75, 76, 77, 78, 79, 11 (29%) were assessed as high quality,,,,,,,,,,, and one was assessed as low quality. Although not formally tested, we found that there was a protective effect of interventions reported by authors in 29 (76%) of the included studies, suggesting that there might be a publication bias in the available literature.

Models of care

Seven studies investigating the impact of different models of care in custodial settings, forensic hospital settings, and community-based forensic settings were included.,,,,,, Two of these studies concluded that their model of care reduced self-harm,, whilst the remaining studies reported conflicting or non-significant,, findings. One observational study reported lower rates of self-harm in a therapeutic community prison in England than in conventional UK prisons, although the factors contributing to this finding were not identified.

Group-based treatment programs

Twelve studies investigated group-based treatment programs in adult correctional settings.,,,,,,,69, 70, 71,, Most of these studies had methodological limitations, including the use of a single-group, pre-test post-test design,,,,,,, excluding participants currently at risk of self-harm, and reporting the impact on self-harm using weak anecdotal evidence from correctional officers. One high-quality RCT reported a reduction in self-injurious behaviours and another found no difference in suicidal ideation. Pratt et al.’s trial of incarcerated adult males reported that, relative to the treatment as usual group, at 6-months follow-up the 31 participants who completed a 20-session cognitive behavioural suicide prevention (CBSP) course achieved a significantly greater reduction in suicidal behaviours (6-month mean [SD] TAU: 1.48[3.23] vs CBSP: 0.58[1.52], p = 0.003) with a moderate treatment effect (Cohen's d= -0.72, 95%CI: -1.71 to 0.09; baseline mean [SD] TAU: 1.39[3.28] vs CBSP: 1.06[2.10]). Further, at the end of treatment, more than half (10/18, 56%) of participants in the CBSP group achieved a clinically significant recovery, compared with just under a quarter (5/22, 23%) of the TAU group (χ2 = 4.55, p = 0.03). A clinically significant recovery for participants was indicated for total scores of <67 on the Suicide Probability Scale. Johnson et al.’s trial, investigating the impact of a 10-week interpersonal psychotherapy (IPT) group intervention for 117 incarcerated men (65%) and 64 women (35%) with major depressive disorder, found no difference in suicidal ideation between the 87 participants in the IPT group and the 86 control participants at three-month follow-up.

Peer support programs

One prospective quasi-experimental study and one retrospective cohort study investigated the impact of peer support programs on suicidal thoughts and behaviours in prison settings. One study investigated the impact of a “prisoner listener” scheme, whereby incarcerated adults were trained to provide confidential suicide prevention support to their peers. The impact of the program was unclear due to low base frequencies of the main outcomes. Perry et al.’s quasi-experimental study investigated the impact of a peer-led problem-solving therapy (PST) skills intervention on self-harm in prison, whereby incarcerated men were trained in PST and became peer-support mentors. Participants who received PST skills training from the mentors at least once reported fewer self-harm episodes per month following the intervention (-0.25; 95%CI: -0.37 to -0.13, p<0.001), whilst participants who did not receive the PST intervention reported no reduction. The lack of control group was a limitation of the study.

Individual treatment/tailored programs

Six studies investigated the impact of individual and/or tailored treatment programs (e.g., dialectical behaviour therapy) among adults in contact with the criminal justice system.,,,,, Walker et al. conducted an RCT investigating the impact of a six-week Psychodynamic Interpersonal Therapy (PIT) delivered to 56 women in prison over 4-6 sessions. Fifty-seven participants were randomised to the control arm using a stratified block method to achieve balanced characteristics. Both the PIT and control arms had reduced rates of suicidal ideation post-intervention, but there was no difference between the two groups.

The remaining five individual treatment interventions had methodological limitations and reported inconclusive findings. Four were conducted in custodial settings either for sentenced adults,,, and/or for those remanded in custody, one was conducted in a high-security forensic hospital, and one was delivered to participants in a mental health court. Four of the studies were single-group pre-test post-test design without a control group.,,, Findings were limited by the absence of testing for statistical significance, descriptive reporting, small sample sizes (<20 participants), and/or the use of qualitative evidence with no control group to support claims of a reduction in self-harm.

Multi-component programs

Two multi-component suicide prevention programs (i.e., those integrating multiple suicide prevention components as part of a broader systems approach) were included., In their retrospective cohort study, Freeman and Alaimo's program in a US men's prison included screening, crisis intervention, and – when deemed in the best interests of the individual – detention in a state hospital. In their descriptive study of 520 incarcerated males in Slovenia, Sarotar et al. reported a reduction in the prison's suicide numbers between 2005 and 2015 following the implementation of a prison-wide suicide prevention plan in 2004. Both studies were assessed as medium quality, and constrained by the lack of control groups,, observational design, and low base frequencies of suicide both before and after implementation of the strategic plan.

Changes in legislation or policy

Four studies investigated the impact of a policy or legislation change on suicidal thoughts and behaviours in adult custodial settings.,,, Changes included criminal law reform, environmental modifications, and changes to clinical and administrative policies and procedures, and the UK's National Health Service assuming responsibility for the prison health service. Due to the uncontrolled, retrospective nature of three studies,,, it was not possible to reliably determine the impact of the changes on suicidal thoughts and/or behaviours. In their retrospective mixed-methods case study, Slade and Forrester aimed to identify factors associated with a sustained reduction in suicide rate in a London, UK prison from 2008-2011 following implementation of the National Suicide Prevention Strategy (1991–2008) in male prisons, and a Local Suicide Prevention Strategy (multi-agency and cultural change) in 2009. Neither the national strategy nor the local strategy was described in detail and, as such, the contribution of the individual components to suicide reductions remains unclear.

Programs delivered in youth detention

Five studies,,66, 67, 68 evaluated interventions for adolescents in youth detention settings; one RCT, two single-group pre-test post-test studies,, and two retrospective cohort studies., Rohde et al. conducted an RCT investigating the impact of participation in ‘the coping course’, an 8-week, 16-session course for 46 adolescent males who learned a variety of skills (e.g., social skills, cognitive restructuring, problem-solving). Course completion had no impact on suicidal behaviours when compared to the two control groups comprising 30 and 62 adolescent males. Gallagher and Dobrin's retrospective cohort study investigated the association between suicide screening practices and suicide attempts in juvenile justice facilities. Findings suggested that in the 3% of studies that conducted universal screening (i.e., for the entire facility population), suicide screening within the first 24 hours after arrival was linked to lower odds of suicide attempts (OR: 0·39 [95%CI: 0·23-0·65]), irrespective of facility size or referral pathway. Findings were limited by the use of selected samples, whereby most facilities (97%) that reported screening indicated that only young people assessed as being at risk of suicide were screened.

Discussion

This review examined the effectiveness of interventions to reduce suicidal thoughts and behaviours in people who had contact with the criminal justice system. Due to methodological limitations of most primary studies (e.g., the absence of control groups, small sample sizes, and short follow-up periods), coupled with the absence of rigorous evaluations, drawing definitive conclusions about the efficacy of interventions was difficult. Despite these limitations, our findings provide evidence for the feasibility of some approaches in preventing suicidal thoughts and behaviours among incarcerated people. Evaluating programs in criminal justice settings can be complicated by political and structural considerations, resource limitations, duty of care, and ethical considerations. Additionally, given the complexities of most criminal justice settings and uncertainty regarding clinical equipoise, randomising participants to the intervention or control arm may not be feasible or ethical. However, such challenges are not insurmountable and independent, published evaluations of interventions in this context should be considered a priority.

Although our review included all criminal justice system settings, most of the literature focused on adult custodial settings. Despite evidence that the vast majority of suicides among justice-involved people occur in the community,,, most studies evaluated interventions to prevent suicide in custodial settings, where the State has a duty of care. We identified no studies examining suicide prevention initiatives for people who were detained in police custody, on bail, or on parole/licence. Additionally, despite previous research indicating that rates of self-harm and suicide are considerably higher after incarceration than in either youth detention or adult prison,,, we identified no eligible studies that followed participants from custody into the community. As such, evidence on effective interventions to reduce the rate of self-harm or suicidal behaviour in people released from prison remains a critical gap in the literature. Future evaluations should incorporate longer data collection periods that include follow-up after release from custody. We did identify one ongoing RCT in the US that aims to address the lack of evidence on the post-release period by investigating the impact of a suicide risk reduction intervention on suicide at multiple timepoints in the first year after release from prison.

RCTs investigating the impact of interventions on suicide face considerable challenges, given that large sample sizes are required to detect meaningful differences. In order to prevent suicidal thoughts and behaviours in people who move through the wider criminal justice system, rigorous evaluations of interventions are required in all criminal justice settings, including the community into which people are released following contact with the justice system. Longitudinal studies of this population are both challenging and resource intensive; however, multi-sectoral data linkage (e.g., linking criminal justice records with other databases such as ambulance, emergency department, hospital, or death records) is a robust and validated method for efficiently following large samples of vulnerable individuals over time, including within randomised and quasi-experimental designs., Such linked data can also reliably be used to ascertain health service contacts resulting from self-harm and/or suicidal ideation,, and identify people at increased risk of future self-harm following release from incarceration.

There is a pressing need for improved routine investigation and data collection on deaths after release from incarceration which, in turn, may inform preventive efforts. Opportunities exist to learn more from past events through systematic analysis by clinical academics; for example, all deaths occurring during and after release from custody could be investigated and documented by an independent body. In settings such as Australia and the United Kingdom, where some deaths in custody are investigated by a coroner, deaths occurring after release from prison could be investigated to identify possible opportunities for prevention. Particular resources could be directed to investigating deaths occurring within the first 12 months, a period which has been identified as an increased risk for suicide., Routine recording and investigation of suicides following release from custody would provide a database which could be used for research purposes, such as the stratification of suicide risk after contact with the criminal justice system as has been implemented in Australia. This method has since been validated and identified as an approach that could be adopted by other countries.

Research focusing on interventions for specific populations of people with diverse needs was absent from the literature. For example, in response to the growing number of women incarcerated globally and the documented trauma experienced by the majority of these women, targeted research into patterns of suicide and self-harm among women is essential to facilitate the design of programs that are evidence-based, trauma-informed, and responsive to their unique needs. Similarly, research should consider cultural-specific needs of ethnic minorities and First Nations people in colonised countries, to ensure that interventions accommodate cultural diversity. Young people also represent a priority group who span both youth detention and adult custodial settings, with those aged 18-24 years often treated as adults and subsequently excluded from evaluations of youth (despite evidence suggesting that this age group should be considered ‘young’ based on their developmental stage). Evaluations of interventions for justice-involved young people are therefore essential due to their increased risk of self-harm and suicide94, 95, 96, 97 that may require different responses to adults due to differences in their cognitive and developmental stage and social context. Finally, and despite the inherent challenges in implementation, research should incorporate to the greatest extent possible the expertise of people with lived experience of criminal justice systems and processes to improve the design, operation, and evaluation of all interventions.

Our review is the first to systematically examine the effectiveness of suicide prevention initiatives at all points of the criminal justice system, for people of any age and in any country globally. However, it also has limitations. First, we focused on interventions which explicitly aimed to reduce suicide and/or related outcomes. Programs that do not directly measure the impact on suicidal behaviour may also have the potential to reduce suicide by reducing risk factors (e.g., homelessness, or food insecurity), and improving protective factors (e.g., prosocial relationships, opportunities for education, employment, improved mental health). Second, most included studies were conducted in high-income, Western countries, which may limit the generalisability of our findings in other settings. Third, our review was limited to criminal justice settings. Although it is possible that interventions in the general population may be effective in these settings, both the study population and settings are markedly different from the general community, given the controlled environment, increased vulnerability of the population, and elevated rates of self-harm and suicide. We identified very limited evidence of effective interventions, suggesting that merely applying existing and evaluated community interventions to these settings may be inadequate, and is an area in urgent need of further research. Finally, given that studies reporting the impact on both suicide and self-harm (suicidal behaviours) were included, there is heterogeneity in the study outcomes. However, given the strong correlation of the two, and in light of the relative dearth of literature in this space, we believed it was important that both outcomes were included.

Whilst we identified many interventions designed to prevent suicidal thoughts and behaviours among people in prison, most contained significant methodological limitations, and few had been rigorously evaluated. More high-quality evidence from other criminal justice settings, particularly from low- and middle-income countries, should be considered a priority for future research. In the absence of this evidence, high rates of suicide among marginalised people in contact with the criminal justice system will persist.

Declaration of interests

SF is an expert member of the UK's Independent Advisory Panel on Deaths in Custody, outside the submitted work. All the other authors report no conflicts.