Literature DB >> 34377359

A meta-analysis of dropout from evidence-based psychological treatment for post-traumatic stress disorder (PTSD) in children and young people.

Caroline Simmons1,2, Richard Meiser-Stedman1, Hannah Baily2, Peter Beazley1.   

Abstract

Background: Despite the established evidence base of psychological interventions in treating PTSD in children and young people, concern that these trauma-focused treatments may 'retraumatise' patients or exacerbate symptoms and cause dropout has been identified as a barrier to their implementation. Dropout from treatment is indicative of its relative acceptability in this population. Objective: Estimate the prevalence of dropout in children and young people receiving a psychological therapy for PTSD as part of a randomized controlled trial (RCT).
Methods: A systematic search of the literature was conducted to identify RCTs of evidence-based treatment of PTSD in children and young people. Proportion meta-analyses estimated the prevalence of dropout. Odds ratios compared the relative likelihood of dropout between different treatments and controls. Subgroup analysis assessed the impact of potential moderating variables.
Results: Forty RCTs were identified. Dropout from all treatment or active control arms was estimated to be 11.7%, 95% CI [9.0, 14.6]. Dropout from evidence-based treatment (TFCBTs and EMDR) was 11.2%, 95% CI [8.2, 14.6]. Dropout from non-trauma focused treatments or controls was 12.8%, 95% CI [7.6, 19.1]. There was no significant difference in the odds of dropout when comparing different modalities. Group rather than individual delivery, and lay versus professional delivery, were associated with less dropout. Conclusions: Evidence-based treatments for children and young people with PTSD do not result in higher prevalence of dropout than non-trauma focused treatment or waiting list conditions. Trauma-focused therapies appear to be well tolerated in children and young people.
© 2021 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

Entities:  

Keywords:  CBT; EMDR; PTSD; TEPT; TF-CBT; abandono; acceptability; aceptabilidad; adolescentes; adolescents; children; dropout; niños; psicoterapia; psychotherapy; 可接受性; 孩子; 年轻人; 心理治疗; 退出

Mesh:

Year:  2021        PMID: 34377359      PMCID: PMC8344790          DOI: 10.1080/20008198.2021.1947570

Source DB:  PubMed          Journal:  Eur J Psychotraumatol        ISSN: 2000-8066


Introduction

Many children and adolescents are exposed to traumatic events throughout the world, with around 15.9% of those exposed going on to develop Post Traumatic Stress Disorder (PTSD) (Alisic et al., 2014). PTSD is characterized by the re-experiencing of traumatic events, avoidance of reminders of the trauma, hypervigilance to threat and increased physiological arousal (International Classification of diseases for mortality and morbidity statistics (11th revision) (ICD-11) World Health Organization, 2019)). Untreated, PTSD can result in severely impaired social, academic and occupational functioning, which can persist into adulthood (Yule & Bolton, 2000). It is fortunate, therefore, that a number of psychological treatments have demonstrated efficacy in this area. In particular, a range of trauma-focused cognitive behavioural interventions, and to a slightly lesser extent, Eye Movement Desensitization and Reprocessing (EMDR) therapy have well-established empirical support confirmed by numerous meta-analyses (e.g. Gutermann et al., 2016; Mavranezouli et al., 2020; Morina, Koerssen, & Pollet, 2016). As such, they are the recommended treatment in a number of national treatment guidelines, e.g. the UK National Institute for Health and Care Excellence (NICE) which recommends trauma-focused cognitive behaviour therapies as the first-line intervention, with EMDR to be considered for those who do not respond (NICE, 2018); and the International Society for Traumatic Stress Studies (ISTSS) who recommend both trauma focused cognitive behaviour therapy and EMDR as first-line interventions. (Bisson et al., 2019) It has been widely noted, however, that despite this strong evidence base, there continues to be an under-utilization of these approaches in clinical settings (Borntrager, Chorpita, Higa-mcmillan, Daleiden, & Starace, 2013; Clark, Sprang, Freer, & Whitt-Woosley, 2010; Eslinger, Sprang, Ascienzo, & Silman, 2020; Finch, Ford, Grainger, & Meiser-Stedman, 2020a; Finch, Ford, Lombardo, & Meiser-Stedman, 2020b). Rates of young people dropping out from treatment for PTSD are significant (Dorsey et al., 2017). A number of authors have linked these two phenomena to suggest that concerns that some treatments may precipitate dropout may lead clinicians to avoid trauma-focused interventions (Borntrager et al., 2013; Feeny et al., 2003; Foa, Zoellner, Feeny, Hembree, & Alvarez-Conrad, 2002; Ruzek et al., 2014; Ruzek, Eftekhari, Crowley, Kuhn, & Karlin, 2017; van Minnen et al., 2010). A definition of trauma-focused cognitive behavioural interventions can be found within the UK’s NICE guidance, which considers elaboration and processing of trauma-related memories and emotions, restructuring of trauma-related meanings for the child or young person, and help to overcome avoidance as key features (NICE Guideline NG116; 2018). This definition encompasses a range of treatments including Trauma-Focused Cognitive Behaviour Therapy (TFCBT), Cognitive Processing Therapy (CPT), Narrative Exposure Therapy (NET) and Prolonged Exposure Therapy (PE). The same guidelines recommend that clinicians consider EMDR for children and young people, if they do not respond to, or engage with, TFCBT (NICE Guideline NG116; 2018). Both approaches involve explicit exposure to the trauma memory, be it through ‘trauma narration’ (a detailed re-telling of event and accompanying thoughts and feelings), in vivo exposure to trauma-relevant objects or places, or imaginal exposure (bringing to mind and focusing on the details of the event). It is exposure techniques in particular, that have been most frequently implicated in the suggestion that some treatments can exacerbate symptoms and are particularly poorly tolerated in people with PTSD (Feeny et al., 2003; Foa et al., 2002; Lancaster et al., 2020; Larsen, Wiltsey Stirman, Smith, & Resick, 2016; Olatunji, 2009; Ruzek et al., 2014).). To date, six meta-analyses have considered dropout from PTSD treatments in adults, with mixed results. Bradley, Greene, Russ, Dutra, and Weston (2005) reported some data that implied there was a difference in dropout rate between treatments that included exposure techniques and those that did not; however, this was not subject to formal analysis. Hembree et al. (2003) found no evidence of differential dropout rates from different treatments. Bisson et al. (2007) did find that there was more dropout from TFCBT than from usual care, but this difference no longer held once lower quality studies were removed. Goetter et al. (2015) conducted a meta-analysis studies related to US veterans in particular, finding that there was no difference in dropout between those treatments that involved exposure and those that did not. Imel, Laska, Jakupcak, and Simpson (2013) found that most direct comparisons between active treatments did not demonstrate significantly different dropout rates, except where trauma-focused treatment was compared with Present Centred Therapy (PCT), with PCT having a reduced likelihood of dropout. Finally, Lewis, Roberts, Gibson, and Bisson (2020) found that there was a statistically significant relationship between dropout and treatments with a greater trauma focus than those without, although the difference was small and dropout rates were still comparatively low (18% and 14%, respectively,). Taken together, it remains far from clear whether there is definitive evidence to conclude that some treatments carry a greater risk of dropout. To the authors’ knowledge, there has not yet been a meta-analysis which has considered this important question in relation to children and young people. This is important if clinicians are to make informed decisions about which treatment approach to select to promote the retention of children and young people in treatment, giving them the best chance of benefitting from the intervention. The purpose of the current review is therefore to obtain an estimate of dropout rates for evidence-based PTSD treatments in children and young people and to ascertain whether there are different dropout rates across different treatment approaches (and in particular whether trauma-focused treatments are associated with increased rates of dropout among children and young people).

Methods

An overview of the proposed review was registered a priori with PROSPERO (CRD42019154257; 14 November 2019).

Search strategy

Three databases were systematically searched: PsycINFO, MEDLINE and Published International Literature on Traumatic Stress (PILOTS; now PTSDpubs). The following search terms were used: (Post-traumatic Stress OR ‘Posttraumatic Stress’ OR Trauma* OR PTSD OR ‘Post Traumatic Stress’ OR P.T.S.D.) AND (child* OR young OR adolescen* OR youth OR pupil OR student OR teenage*) AND (psychotherapy OR therapy OR treat* OR therap* OR cognitive OR CBT OR C.B.T. OR EMDR OR ‘Eye Movement’ OR E.M.D.R. OR Reprocess* OR Desensiti* OR ‘Narrative Exposure’ OR ‘Exposure Therapy’) AND (control* OR clinical trial OR randomized OR randomized or Randomized Controlled).

Eligibility criteria

Results were limited to those in the English language and those published since 1980. This reflects the inclusion of PTSD in the third edition of the Diagnostic and Statistical Manual of Mental Disorders (APA, 1980). Included studies were randomized controlled trials (RCTs) of evidence-based therapeutic interventions recommended by NICE, i.e. trauma-focused cognitive/behavioural or cognitive behavioural therapies or EMDR. Participants were required to have a diagnosis of PTSD (according to the DSM, the World Health Organization (WHO) International Classification of Diseases (ICD)) or clinically significant PTSD symptoms (baseline PTSD symptom scores above threshold on a validated scale). Studies had to have a mean age for participants that was 18 years old or younger. The event the symptoms relate to was required be a least 1 month prior to the start of treatment. To be included studies had to report sufficient data to compute dropout rates. Studies were excluded if none of the treatment arms constituted a NICE recommended intervention (e.g. play therapy, family therapy, child-parent psychotherapy, parent training (alone), or supportive counselling). Studies were excluded if the interventions under consideration were not primarily treating trauma symptoms or had been delivered to a whole group who had not been individually clinically assessed as having PTSD symptoms (e.g. to a whole class). Preventative studies were excluded on the basis that they occur in a different context (i.e. in close proximity to the trauma) to treatment studies and may therefore elicit a different response that found in the context of symptoms that may have been present for a sustained period of time. Moreover, there is currently less evidence to support the efficacy of preventative interventions than that for treatment interventions (Marsac, Donlon, & Berkowitz, 2014).

Study selection

Searches produced a total of 4076 results. Once duplicates had been removed, there were 2747 records. Excluding those studies not in the English language further reduced the number of results by 147, leaving 2600. These were then screened by title and abstract with reference to the eligibility criteria. This process removed 2339 records. The full text for the remaining 261 were then retrieved for detailed screening. Concerns about eligibility were resolved through consensus discussion between the first and third author. This process produced a selection of 40 studies. All 40 included studies were then separately assessed for eligibility by the third author. A PRISMA flowchart detailing the screening and selection process is presented in Figure 1.
Figure 1.

PRISMA flowchart of study identification process

PRISMA flowchart of study identification process

Study quality

Study quality was assessed with reference to a 10-point scale adapted from that which was used by Hoppen and Morina (2020) – itself an adaptation of that used by Cuijpers, van Straten, Bohlmeijer, Hollon, and Andersson (2010) – for their meta-analysis investigating study quality in the field of paediatric PTSD. One point was given for each of the following: (i) participants’ PTSD symptomology assessed personally via a clinical interview; (ii) the use of a treatment manual either published or specifically designed for the study; (iii) treatment delivered by therapists trained in the specific intervention either as part of the study or having had substantial prior experience; (iv) treatment integrity checked by, e.g., regular supervision, adherence checklists or recordings of treatment sessions being subjected to review; (v) intent-to-treat analysis; (vi) independent randomization process when allocating participants to different arms; and (vii) post-treatment assessment carried out by blind assessors. Three further criteria were added to reflect the focus on dropout in the current study: (i) presentation of a CONSORT diagram (Schulz, Altman, & Moher, 2010), (ii) defined and explicit criteria for distinguishing dropout and treatment completion, i.e. the minimum number of sessions required to be considered to have received the treatment, and (iii) inclusion of details of the stage and/or reasons for dropout or where there was no dropout, that this was clearly stated. Where there was insufficient information to determine whether the criterion was met, no point was awarded. All included studies were assessed for their quality by CS. A randomly generated subset of 50% of the studies was then assessed by HB. Cohen’s kappa was calculated to determine the degree of inter-rater reliability of the quality assessment as 0.72, suggesting substantial agreement (Landis & Koch, 1977). Differing scores were then resolved through discussion.

Data extraction

The following data were extracted from all included studies: authors, date and the country where study took place, whether the study concerned a specific event or category of trauma (e.g. an earthquake, or mass conflict); whether participants had experienced a single event trauma, or multiple trauma, or a mixture of the two; the age range and mean age of participants and the percentage of male and female participants, the treatment arms, including the number and length of sessions involved in each, the format (individual or group treatment), who delivered treatment, the proportion of participants who met diagnostic threshold for PTSD and the percentage of people who had dropped out from all arms in the study from the point of randomization.

Data analysis

The statistical analysis package Jamovi (Version 1.2) was used to carry out the analyses (The Jamovi Project, 2020. Retrieved from https://www.jamovi.org). Proportion meta-analyses were used to estimate the prevalence of dropout for all intervention arms and for subgroups of interventions. A random effects model was used in reflection of the anticipated heterogeneity between studies (Borenstein, Hedges, & Higgins, 2011). Estimates of prevalence of PTSD were arcsine square root transformed to prevent the confidence intervals of studies with low prevalence falling below zero (Barendregt, Doi, Lee, Norman, & Vos, 2013). Heterogeneity of effect sizes was assessed using Cochrane’s Q and Higgins’ I2. The first of these examines whether the variability of effect sizes is greater than would be expected by chance. The latter represents the proportion of the overall variability that is beyond sampling error (Borenstein et al., 2011). Odds ratios were used to determine whether there was a greater likelihood of dropout for different classes of intervention (e.g. trauma-focused cognitive behavioural therapies) and different types of control (i.e. active or inactive). Subgroup analyses (meta-regressions) were conducted to explore potential moderator variables: number of sessions, group or individual format, whether participants had experienced single incident or multiple traumas or a mixture of the two. Further meta-regressions were used to group interventions by modality (e.g. all TFCBT arms) and then compare them to all other intervention arms. The above analyses were repeated using only those studies that provided an explicit definition of what constituted dropout. In light of the finding by Bisson et al. (2007) that an apparent relationship between treatment and dropout disappeared once lower quality studies were removed, sensitivity analyses repeated the above analyses having removed the studies that scored six or fewer in the quality assessment (nine studies removed).

Results

Forty studies met the inclusion criteria. A summary of the included studies is presented in Table 1.
Table 1.

Characteristics of included studies

Authors, yearCountryTrauma typeSingle incident, multiple or mixedInterventionsNumberof participantsFormatMaximum duration weeks, sessions, (minutes)Delivered byAge range (mean)Met PTSD diagnostic threshold at pre-treatment (%)Male(%)/Female (%)Dropout (%)a
Ahmad, Larsson, & Sundelin-Wahlsten, 2007SwedenVariousMixedEMDR vs WL33Individual8, 8 (45)Therapists (authors)6–16(10)10041.2/58.89.1
Ahrens & Rexford, 2002USAViolenceMixedCPT vs WL38Group8, 8 (60)Experienced doctoral candidate and qualified psychologist15–18(16.4)100100/00
Barron, Abdallah, & Heltne, 2016PalestineMass ConflictMixedTRT vs WL154Groupn.r., 5 (60)School counsellors11–18 (13.5)10036.4/63.616.9
Catani et al., 2009Sri LankaCivil unrest, TsunamiMixedKidNET vs MED-RELAX31Individual2, 6 (60–90)Teachers trained as ‘master counsellors’8–14(11.9)n.r.54.8/45.20
Cohen et al., 2004USASexual abuseMixedTFCBT vs CCT229Individual(with parent involvement)12,12 (45)Experienced therapist (social workers and psychologists)8–14(10.7)8921.2/78.811.4
Cohen et al., 2011USAIntimate Partner ViolenceMixedTFCBT vs CCT124Individual(with parent involvement)8, 8 (45)Social workers7–14 (9.6)2549.2/50.839.5
Dawson et al., 2018IndonesiaCivil conflictMixedTFCBT vs PS64Individual(with care-giver involvement)6, 6 (60)Lay counsellors7–14 (10.4)7551.5/48.50
de Roos et al., 2011NetherlandsFirework Factory explosionSingle incidentTFCBT vs EMDR52Individual(with parent involvement)8, 4 individual plus 4 parent (60)Licenced therapists4–18(10.1)17.355.8/44.225.9
de Roos et al., 2017NetherlandsVariousSingle incidentCBWT vs EMDR vs WL103IndividualUp to 6, 6 (45)Clinical psychologists8–18 (13.1)61.242.7/57.33.9
Deblinger et al., 2011USAChild sexual abuseMixedTFCBT (with TN) vs TFCBT (without TN)210Individual(with caregiver involvement)Either 8 or 16, 8 or 16, (90)Graduates with 3+ years of clinical experience4–11 (7.7)n.r.39/6124.8
Diehle et al., 2015NetherlandsVariousMixedTFCBT vs EMDR48Individual(with parent involvement)8, 8 (60)Experienced therapists8–18 (13)3338/6225
Ertl et al., 2011UgandaFormer child soldiersMultipleKidNET vs Academic catchup with SC85Individual3, 8 (90–120)Lay counsellors12–25 (18)10044.7/55.37.6
Foa, McLean, Capaldi, & Rosenfield, 2013USAChild sexual abuseMixedPE vs SC61Individual14,14 (60 − 90)Masters-level counsellors13 − 18 (15.3)1000/10013.1
Ford et al., 2012USAVariousMixedTARGET vs ETAU59Individualn.r., 12 (50)Experienced therapists with professional qualifications13–17 (14.7)62.80/10027.1
Gilboa-Schechtman et al., 2010IsraelVariousSingle IncidentPE-A vs TLDP38IndividualPE-A: 15,15 (90)TLDP: n.r., 18 (50)‘MA level clinicians’12–18 (14.1)10037/6321.1
Goldbeck et al., 2016GermanyVariousMixedTFCBT vs WL159Individual(with parent involvement)12, 12 (90)Therapist with advanced clinical training7–17 (13.0)75.528.3/71.71.9
Jaberghaderi et al., 2004IranSexual abuseMixedTFCBT vs EMDR18Individual(with parent involvement)12, 12 (45)Clinical psychologist12–13 (n.r.)n.r.0/10021.1
Jaberghaderi et al., 2019IranDomestic ViolenceMultipleTFCBT vs EMDR40Individual(with parent involvement)12, 12 (60)Experienced therapists (including author)8–12 (n.r.)10050.4/49.623.8
Jensen et al., 2014NorwayVariousMixedTFCBT vs TAU156Individual(with parent involvement)n.r., 15 (45)Experienced therapist from mix of professions(15.1)66.720.5/79.525
Kemp et al., 2010AustraliaMotor vehicle accidentsSingle incidentEMDR vs WL27Individual6, 4 (60)Doctoral-level psychologist with advance training6–12(8.9)n.r.55.6/44.411.1
King et al., 2000AustraliaChild sexual abuseMultipleChild CBT vs Family CBT vs WL36Individual (child only)/Individual parent & child)20, 20 (50)Registered psychologist5–17 (11.5)69.431/6922.2
McMullen et al., 2013DR CongoWarMixedTFCBT vs WL50Groupn.r., 15 (45)Authors and experienced Congolese counsellors13–17 (15.8)n.r.100/04
Meiser-Stedman et al., 2017UKVariousSingle incidentCT-PTSD vs WL29Individual10, 10 (90)Clinical psychologists (including authors)8–17(13.3)10027.8/72.210.3
Murray et al., 2015ZambiaVariousMixedTFCBT vs TAU257Individual16, 16 (90)Lay counsellors5–18 (13.7)n.r.50.2/49.89.7
Nixon et al., 2012AustraliaVariousSingle incidentTFCBT vs Cognitive Therapy (no exposure)34Individual(with parent involvement)9, 9 (90)Trainee clinical psychologists7–17 (10.8)10063.3/36.738.2
O’Callaghan et al., 2013DR CongoWarMixedTFCBT vs WL52Group (plus x3 individual sessions & x3 caregiver sessions)5, 15, (120)Social workers12–17 (16.1)600/10011.5
O’Callaghan et al., 2015DR CongoWarMixedTFCBT vs CFS50Group3, 9 (90)Lay facilitators8–17 (14.8)9258/420
Peltonen & Kangaslampi, 2019FinlandVariousMixedNET vs TAU50Individual10, 10 (9)Experienced MH professionals9–17 (13.2)n.r.58/4214
Pityaratstian et al., 2015ThailandTsunamiMixedTRT (adapted) vs WL36Group0.4, 3 (120)bCertified child psychiatrists (incl. author)10–15 (12.3)10027.8/72.20
Robjant et al., 2019DR CongoFormer Child SoldiersMultipleFORNET vs TAU92Individual(plus x1 group session per week)6, 12 (120)Lay people16–25 (18)1000/1000
Rosner et al., 2019GermanyVariousMixedD-CPT vs WL/TA88Group20, 30 (50)Masters-level or postdoctoral therapists14–21 (18.1)10015/8521.6
Ruf et al., 2010GermanyRefugeesMultipleKidNET vs WL26Group8, 8 (120)Clinical psychologists7–16 (11.5)10054/463.9
Salloum & Overstreet, 2012USAVariousMixedGTI-CN vs GTI-C72Group (plus x1 individual & x1 parent session)10, 12 (60)Social workers, social work interns, psychology doctoral student6–12 (9.6)n.r.55.7/44.35.6
Santiago et al., 2014USACommunity ViolenceMixedCBITS vs CBITS + Family64Group (plus 1–3 individual & 1–2 group sessions for parents)n.r., 12 (50)Social workers10–14 (11.7)10041/590
Scheeringa et al., 2011USAVariousMixedTFCBT vs WL64Individual(with parent involvement)12, 12 (50)Social workers3–6 (5.3)2466.2/33.829.7
Schottelkorb et al., 2012USARefugeesMixedTFCBT vs CCPT31Individual(with parent involvement)TFCBT: 12, 20 (30)C0+: 12, 24 (30)Masters-level student counsellors6–13 (9.1)5854.8/45.216.1
Shein-Szydlo et al., 2016MexicoVariousMixedTFCBT vs WL100Individual12, 12 (60)Psychologists (Authors)12–19 (14.9)10044/561
Smith et al., 2007UKVariousSingle incidentTFCBT vs WL24Individual(with parent involvement)10, 12 (n.r.)Clinical psychologists8–18 (13.8)10050/500
Stein et al., 2003USAViolenceMixedCBITS vs WL126Group10, 10 (60)School cliniciansn.r. (11)n.r.43.7/56.39.5
Tol et al., 2008IndonesiaCivil conflictMixedCBT-CBI vs WL403Group5, 15 (n.r.)Local lay people(9.9)n.r.51.4/48.62.5

EMDR, eye movement desensitization and reprocessing; WL, waiting list; CPT , cognitive processing therapy; TRT, Teaching Recovery Techniques; KidNET, narrative exposure therapy for children; MED-RELAX, meditation and relaxation intervention; TFCBT, trauma-focused cognitive behaviour therapy; CCT, child-centred therapy; PS, problem solving intervention; CBWT, cognitive behavioural writing therapy; TN, trauma narrative; SC, supportive counselling; PE, prolonged exposure; TARGET, Trauma Affect Regulation: Guide for Education and Therapy; ETAU,enhanced treatment as usual (relationship supportive therapy); PE-A, prolonged exposure for adolescents; TLDP, time-limited psychodynamic therapy; TAU, treatment as usual; CBT,cognitive behavioural therapy; CT-PTSD, cognitive therapy for post-traumatic stress disorder; CFS,child friendly spaces; NET, narrative exposure therapy; FORNET, narrative exposure therapy adapted for offenders; WL/TA, waiting list with treatment advice; GTI-CN, grief and trauma intervention with coping skills and trauma narrative processing; GTI-C, grief and trauma intervention – coping skills only; CCPT, child centred play therapy; cbits=cognitive behavioural intervention for trauma in schools; CBT-CBI, cognitive behavioural therapy classroom-based Intervention; n.r. = not reported.

aDropout from all arms including waiting list. bIntervention delivered over three consecutive days followed by homework over the following month.

Characteristics of included studies EMDR, eye movement desensitization and reprocessing; WL, waiting list; CPT , cognitive processing therapy; TRT, Teaching Recovery Techniques; KidNET, narrative exposure therapy for children; MED-RELAX, meditation and relaxation intervention; TFCBT, trauma-focused cognitive behaviour therapy; CCT, child-centred therapy; PS, problem solving intervention; CBWT, cognitive behavioural writing therapy; TN, trauma narrative; SC, supportive counselling; PE, prolonged exposure; TARGET, Trauma Affect Regulation: Guide for Education and Therapy; ETAU,enhanced treatment as usual (relationship supportive therapy); PE-A, prolonged exposure for adolescents; TLDP, time-limited psychodynamic therapy; TAU, treatment as usual; CBT,cognitive behavioural therapy; CT-PTSD, cognitive therapy for post-traumatic stress disorder; CFS,child friendly spaces; NET, narrative exposure therapy; FORNET, narrative exposure therapy adapted for offenders; WL/TA, waiting list with treatment advice; GTI-CN, grief and trauma intervention with coping skills and trauma narrative processing; GTI-C, grief and trauma intervention – coping skills only; CCPT, child centred play therapy; cbits=cognitive behavioural intervention for trauma in schools; CBT-CBI, cognitive behavioural therapy classroom-based Intervention; n.r. = not reported. aDropout from all arms including waiting list. bIntervention delivered over three consecutive days followed by homework over the following month.

Sample characteristics

A total of 3413 children and young people were included in the identified studies, with sample sizes varying from 24 to 403. The approximate mean age of participants was 12.5 years, with the youngest age of eligibility being 3 years and the oldest being 25. An average 41.5% of participants were male and 58.5% were female. Seven studies included a single gender exclusively (two had only male participants and five had only female participants). Studies came from 18 different countries including the State of Palestine. Eleven studies were from the USA. Eight low- and middle-income Countries (LMIC; World Bank) and the State of Palestine, were represented accounting for 15 studies (37.5% of included studies). Seven studies (17.5%) looked at single incident trauma (e.g. motor vehicle accident, house fire, single event sexual or non-sexual assault). Five (12.5%) specifically only included participants who had experienced multiple traumas (e.g. child sexual abuse, domestic violence, former child soldiers), while the majority (n= 28; 70%) included participants with a mixture of multiple and single incident traumas.

Nature of interventions delivered

Twelve (30%) studies primarily reported interventions delivered in a group format, although three of these studies also included adjunctive individual child and/or parent sessions. Most interventions were delivered by professional therapists, social workers or trainees. Six studies (15%) involved interventions delivered by lay members of the community. The shortest intervention (Pityaratstian et al., 2015) took place over 3 consecutive days; however, this was then followed by daily homework to complete over the subsequent month. The longest interventions took place over 20 weeks (Rosner et al., 2019; King et al., 2000). The mean number of sessions was 11.8 (SD, 5.2). The intervention with the fewest number of sessions was three (again Pityaratstian et al., 2015 as noted above) the highest maximum number of sessions was 30 (Rosner et al., 2019). Considering all arms of each study, including waiting list, the mean dropout was 12.7%. The highest reported dropout was 39%. Eight studies reported that they did not have any dropout at all (i.e. a rate of 0%). The most frequently studied intervention was TFCBT, featuring in 21 RCTs (52.5%). NET was included in five studies (12.5%), PE, three (7.5%) and CPT two (5%). EMDR featured in seven trials (17.5%), four of which were a direct comparison between EMDR and TFCBT. Fourteen trials (35%) compared a trauma-focused treatment with an inactive, waiting list control arm alone. Fourteen trials (35%) compared a trauma-focused treatment with a non-trauma focused active control such as Child Centred Therapy, Supportive Counselling or Treatment as Usual. A further three studies compared two conditions, one of which contained explicit exposure or trauma narrative and one of which was the same but without this component (Deblinger et al., 2011; Nixon et al., 2012; Salloum & Overstreet, 2012). For the purposes of this analysis, these non-exposure or non-trauma narrative arms were treated as active control conditions. Although they would involve implicit exposure through the provision of, for example, psychoeducation about trauma reactions, they would not meet the criteria set out in the NICE Guidelines set about above (NICE Guideline NG116; 2018)

Definitions of dropout

Sixteen studies (40%) included a clear definition of dropout and/or the minimum number of attended sessions that would constitute treatment completion. These can be found in Table 2.
Table 2.

Studies with explicit definitions of dropout or completion

StudyDefinition of completion
Ahmad et al., 2007Three or more sessions of a possible eight
Cohen et al, 2004Three or more sessions of a possible 12
Cohen et al, 2011Completion of all eight sessions
Dawson et al, 2018Completion of all five sessions
de Roos et al, 2011Completion of four sessions unless asymptomatic
de Roos et al, 2017Completion of six sessions or fewer if units of distress reduced to zero
Deblinger et al, 2011Three or more sessions of a possible 8 or 16
Diehle et al, 2015Eight sessions but treatment could be concluded earlier if cured
Ertl et al, 2011Completion of all eight sessions
Foa et al., 2013Eight or more sessions of a possible 14
Ford et al, 2012Five or more sessions of a possible 12
Goldbeck et al, 2016Eight or more sessions
Jaberghaderi et al, 2004Ten or more sessions of TFCBTNo minimum for EMDR
Jaberghaderi et al, 2019Five or more sessions of a possible 12
Jensen et al, 2014Six or more sessions
Peltonen & Kangaslampi, 2019Seven or more sessions
Studies with explicit definitions of dropout or completion The quality of all studies was assessed with reference to the 10 criteria outlined above. A total quality score was calculated by summing the scores for each indicator. The average score was 7.8 (SD = 1.6). The scores for each criterion in each study are presented in Supplementary Figure S1.

Proportion meta-analyses

The results from the proportion meta-analyses are presented in Table 3. Heterogeneity was large (I2 > 59%) and significant in all instances. The estimated dropout across all treatment arms (any treatment or active control, excluding only waiting list conditions) was 11.7% (k = 66, 95% CI 9.0, 14.6). The forest plot (Supplementary Figure S2) shows dropout rates with 95% confidence intervals. A second proportion meta-analysis considered treatment or control arms from only those studies that had defined dropout (k = 32); this yielded an increase in dropout (15.9%; 95% CI 12.0, 20.2).
Table 3.

Results of proportion meta-analyses

    95% CI
Heterogeneity statistics
AnalysiskNPrevalence (%)LIULQdfpI2 (%)
Dropout from all treatment arms excluding WL66265811.79.014.6326.565<0.00179.0
Lower quality removed53238311.68.814.8286.752<0.00180.7
Defined dropout32138615.912.020.2132.031<0.00176.1
Dropout from all TFCBT arms41169610.67.514.2206.140<0.00179.3
Lower quality removed31145710.16.714.0166.830<0.00180.1
Defined dropout1677814.79.420.970.115<0.00178.7
Dropout from all TFCBT and EMDR arms48186911.28.214.6226.547<0.00177.6
Lower quality removed36160810.87.614.5186.735<0.00179.2
Defined dropout2289115.210.620.485.321<0.00174.9
Dropout from all EMDR arms717315.57.825.315.760.01559.0
Lower quality removed515116.26.928.514.740.00570.1
Defined dropout616016.78.027.815.150.01063.6
Dropout from all non-trauma focussed armsa1878912.87.619.190.117<0.00182.4
Lower quality removed1777513.47.920.087.816<0.00183.1
Defined dropout1049517.410.525.643.49<0.00179.2

WL, waiting list; TFCBT, trauma-focused cognitive behavioural therapies; EMDR, eye movement desensitization and reprocessing.

aAll active control arms, non-NICE recommended psychotherapies and the arms from component studies with exposure or trauma narrative elements removed.

Results of proportion meta-analyses WL, waiting list; TFCBT, trauma-focused cognitive behavioural therapies; EMDR, eye movement desensitization and reprocessing. aAll active control arms, non-NICE recommended psychotherapies and the arms from component studies with exposure or trauma narrative elements removed. A series of further proportion meta-analyses examined dropout for particular modalities of treatment, and when using only those studies which defined dropout and when removing studies rated to have low quality (see Table 3). Drop rates were low in each case (<18%), increasing slightly when restricting results to studies when defined dropout. There appeared to be little impact of removing low quality studies.

Odds ratios

Odds ratios were calculated to determine the relative likelihood of dropout between different classes of intervention and control arms. The results are presented in Table 4. There were no instances of statistically significant difference between experimental and control conditions. Moreover, these results were not accompanied by heterogeneity.
Table 4.

Odds ratios of dropout from different types of intervention

  
95% CI
 Heterogeneity statistics
AnalysiskNOdds ratioLLULpQdfpI2 (%)
TFCBT vs any active control2218480.890.681.170.39812.2210.9350
Lower quality removed2017990.870.661.140.3989.1190.9720
Defined dropout1513370.856.231.150.3988.0140.8890
EMDR vs any active control52831.030.541.930.9381.340.8700
Lower qualityremoved42651.030.531.990.9381.330.7410
Defined dropouta----------
TFCBT or EMDR vs WL1714171.010.502.040.97525.9160.05542.3
Lower quality removed1211531.220.332.030.97517.7110.08842.2
Defined dropoutb----------
TFCBT or EMDR vs active controlc1412990.880.631.210.4247.7130.8630
Lower quality removed1312680.850.611.180.4244.6120.9710
Defined dropout88000.830.571.210.4244.570.7200
Component studiesd43140.810.421.550.5182.030.5810
Lower dropout removeda----------
Defined dropoutb----------

LL, lower limit; UL, upper limit; CBT, cognitive behavioural therapies; EMDR, eye movement desensitization and reprocessing; WL, waiting list.

aAnalysis not conducted because there were too few eligible arms (k = 2). bSame as the analysis above. cExcludes component studies and EMDR vs TFCBT studies. dArms with exposure/trauma narrative component vs arms with those elements removed.

Odds ratios of dropout from different types of intervention LL, lower limit; UL, upper limit; CBT, cognitive behavioural therapies; EMDR, eye movement desensitization and reprocessing; WL, waiting list. aAnalysis not conducted because there were too few eligible arms (k = 2). bSame as the analysis above. cExcludes component studies and EMDR vs TFCBT studies. dArms with exposure/trauma narrative component vs arms with those elements removed.

Subgroup and moderator analyses

Proportion meta-analyses were conducted for subgroups and then meta-regressions were conducted in order to explore whether any predictor of dropout could be identified. Results are presented in Table 5. Two moderators produced statistically significant results. The first was individual versus group format: group interventions were associated with fewer dropouts. This continued to be the case once lower quality studies were removed. It was not possible to examine if this held true when considering only those studies that had defined dropout because doing this removed all of the group arms. The second statistically significant association related to whether the intervention was delivered by lay people from local communities or by professional therapists; interventions delivered by lay people were associated with significantly fewer participants dropping out. This continued to be the case when lower quality studies were removed, and when considering only those studies that defined dropout. No relationship was found between dropout rate and type of trauma (single vs multiple), intervention (TFCBT vs other, TFCBT & EMDR vs other) or number of sessions.
Table 5.

Proportion dropout meta-analyses for each active arm: subgroup and moderator analyses

    95% CI
Heterogeneity statistics
AnalysiskNDropout prevalence (%)LLULQdfpI2 (%)
Individual vs group         
Individual armsa53206714.211.017.6218.352<0.00176.9
Group armsa135914.01.87.134.912<0.00159.7
Test of moderation, p < .001; defined drop only, n/ab; lower quality studies removed, p = .005
Multiple vs single trauma        
Multiple/mixed trauma arms55241011.18.414.2286.054<0.00179.9
Single trauma arms1124815.17.624.738.910<0.00172.3
Test of moderation, p = .345; defined drop only, p = .322; lower quality studies removed, p = .269
Lay vs professional therapist         
Lay delivered arms136284.11.87.440.012<0.00164.3
Professional delivered arms53203014.011.017.4212.152<0.00176.2
Test of moderation, p = .003; defined drop only, p = .027; lower quality studies removed, p = .001
Number of sessions         
Test of moderation, p = .461; defined drop only, p = .434; lower quality studies removed, p = .914
CBT vs otherc         
Test of moderation, p = .317; defined drop only, p = .548; lower quality studies removed, p = .214
CBT or EMDR vs otherc         
Test of moderation, p = .612; defined drop only, p = .624; lower quality studies removed, p = .446

aExperimental or control arms. bNot applicable, as no eligible arms. cSubgroup data available in Table 2.

Proportion dropout meta-analyses for each active arm: subgroup and moderator analyses aExperimental or control arms. bNot applicable, as no eligible arms. cSubgroup data available in Table 2.

Publication bias

Visual inspection of the funnel plots related to the above analyses did not show evidence of publication bias (Page, Higgins, & Sterne, 2020).

Discussion

There has been well documented under-utilization of trauma-focused treatments and exposure techniques to treat PTSD despite their significant evidence-base. This has been linked to perceptions among clinicians about the potential adverse effects of these approaches, their potential for worsening symptoms and a consequent increased risk of dropout from treatment (e.g. Finch et al., 2020a). This study pooled data from 40 RCTs regarding PTSD treatment in this population. Results found that dropout from RCTs has tended to be relatively low, with all dropout estimates below 15.5%. These compare favourably with the mean dropout rate (28.4%) found by de Haan, Boon, de Jong, Hoeve, and Vermeiren (2013) in their meta-analysis of children and young people dropping out from treatment in psychotherapy efficacy studies, and are in a similar order to the recent meta-analytic findings of dropout among children and young people from psychotherapeutic interventions for depression (14.9%) (Wright, Mughal, Bowers, & Meiser-Stedman, 2021). They are also comparable to recent adult population meta-analyses that related specifically to PTSD: 16% (Lewis et al., 2020) and 18% (Imel et al., 2013). However, heterogeneity was large in all cases, suggesting that there was high degree of variability in dropout rates across studies. Odds ratios were used to examine whether there were differences in the likelihood of dropout from different conditions when directly compared. In these analyses, there was no evidence of significant heterogeneity across studies. No type of intervention or control condition was associated with significantly greater or lesser odds of dropout, including dropout from inactive control (waiting list) conditions. Different potential moderators of dropout were considered. Of these, group or individual format, and who delivered the intervention were significant. In contrast to adult population studies which have found group treatments to be either associated with higher dropout (Goetter et al., 2015; Imel et al., 2013) or not to be significant (Lewis et al., 2020), this review found that children and young people were less likely to dropout from group treatment. This finding was unexpected, and we can only offer speculative explanations for this effect. Children and young people may be more used to, and comfortable in, group settings, and there may be less pressure to discuss their own trauma experiences in detail. They often accessed group treatment by virtue of their participation in other systems and apparatus such as their school or via Non-Governmental Organizations established in local communities. LMIC were over-represented in the group interventions, making up 50% of group interventions but only 37.5% of the total sample. There may be additional factors in these contexts that promote attendance, such as access to other services and assistance or a paucity of alternative sources of support in situations of mass displacement, conflict or disaster. Alternatively, the peer-oriented support that may be available may through group intervention may be of particular value to children and adolescents; indeed, this would reflect the wider literature that speaks to the protective effects of peer support in youth (e.g. Yearwood, Vliegen, Chau, Corveleyn, & Luyten, 2019). It may be important to note that this finding is in contrast to the lack of difference between individual and group-based interventions observed for dropout from psychological treatments for depression in children and adolescents (Wright et al., 2021). Delivery of interventions by lay members of the community who had been trained to deliver the treatment was also associated with lower dropout. Lay-delivered interventions all took place in LMIC contexts. Lay people may bring cultural knowledge and credibility that enhances participation. This finding is promising in that it supports the vision espoused by the World Health Organization (WHO) of nonspecialised healthcare workers being critical in meeting the demand for mental health interventions around the world (mhGap Intervention Guide for mental, neurological and substance use disorders in non-specialized health settings; WHO, 2010). It is encouraging to note that while professionals have identified the need for additional training as a potential barrier to implementing trauma-focused treatments (Finch et al., 2020b), these needs may be met with relatively modest input given the success of these studies in utilizing lay facilitators. Study quality did not appear to affect the results. However, using only those studies which had explicitly defined dropout consistently yielded a higher dropout rate. One might expect that defining dropout could reduce the number of participants considered to have dropped out, as compared to inferring dropout rate from the difference between the number randomized and the number who participated in post-treatment assessment. In the first instance, someone could be considered to have completed treatment after only having taken part in a relatively fewer sessions and in the latter, someone could have attended all or almost all planned sessions but be absent only from post-assessment and still designated as having dropped out. Instead, our analysis found the reverse. If a lot of dropout occurs at the beginning of treatment, one might expect that there would be little difference between studies that defined dropout and those that did not, as early leavers from treatment would be captured in either instance. Therefore, these findings may imply that dropout tended to occur later in treatment, but this would require further research to explore. It may be that the fact dropout was considered a priori indicated a greater level of attention was given to the issue of dropout and therefore a more stringent approach to identifying dropouts was adopted.

Strengths and limitations

There are a number of limitations to this study. As noted above, inferring dropout from the numbers of participants that were randomized and at post-treatment assessment is imperfect. There may be people who were present at post-treatment assessment who had not attended all or most of the treatment sessions. Conversely, there may be people missing from post-treatment assessment who did attend the treatment sessions and were missing from post-assessment for some other reason. Dropout at an early stage might be associated with quite different factors to that which accompany dropout at a later stage in therapy, including that some later dropout might represent some ‘early responders’ (Szafranski, Smith, Gros, & Resick, 2017). Moreover, it has been consistently found that dropout from RCTs is less than in naturalistic settings (de Haan et al., 2013). This has been linked to the exclusion criteria for participation in RCTs, which is frequently seen to skew the sample away from comorbidity or complexity (Schottenbauer, Glass, Arnkoff, Tendick, & Gray, 2008). This may limit the applicability of these findings to other settings. Studies concerned with ‘real-world’ settings have found evidence of high rates of dropout from trauma-focused treatment, an outcome that is frequently found to be just as likely as the possibility of completing treatment (e.g. Steinberg et al., 2019, Murphy et al., 2014). One explanation for these differences would be that the samples enrolled into clinical trials are more homogeneous than those who utilize standard community services, with RCTs exclusion criteria tending to skew the sample away from comorbidity or complexity (Schottenbauer et al., 2008). There are methodological, practical and ethical reasons for this. Importantly, the more homogenous the sample, the easier it is to draw conclusions about treatment efficacy, which is rightfully the business of RCTs to address (see Schnurr (2007) for a more detailed discussion of this). However, it is important to recognize that the range of contexts and populations covered by the trials reviewed here does include diverse, complex and challenging contexts, including people who have encountered multiple and profound trauma on a mass scale or over long periods. Given what we understand about the impact of these experiences (Dorsey et al., 2017), one might suspect that comorbidity was high in some of these samples, whether or not there was a mental health infrastructure to identify it, or cultural schema to construe it, as such. The diversity of included studies may be a further limitation, in that the statistical heterogeneity between studies was high. This reflects the wide-ranging locations, treatments, format, duration and facilitators, and necessitates caution when pooling data in this way. The advantage of this pooling is that it allows for well-powered analysis in a context where there are often low numbers from individual studies. When it comes to retention, however, RCTs may have numerous advantages compared to usual care settings. There may be incentives to families to remain in the study, and there may be greater resources available to follow up absences or prompt attendance. Knowledge that one is involved in a trial may engender greater hope for change, motivating engagement. Other potential differences are greater fidelity to protocols and access to focused, timely supervision that supports this; differences in the skill, experience or confidence of those delivering interventions; differences in time and resources available or presence and promotion of explicit strategies to retain people in treatment; or differences in the profile of the people being treated (for example, symptom severity, co-morbidity, economic and social resources, attitudes and cultural identity). Encouragingly, there is some evidence to suggest that even quite modest retention strategies can be effective. For example, Dorsey et al. (2014) augmented TFCBT for children placed in foster homes, with an initial phone-call to foster carers which directly discussed potential barriers, caregiver concerns and problem solving around barriers; these matters were revisited with the family at the initial face-to-face appointment. This engagement strategy was not found to make a difference to the likelihood of first appointment attendance or to the number of cancelled sessions. However, families who received the additional engagement strategy phone call were more likely to receive four or more sessions than those who did not (96.0% vs 72.7%, respectively,) and a startling 80% of completed treatment, compared to 40.9% those in the standard condition. Research in this area would benefit from a consistent definition being adopted which would allow for greater confidence in drawing comparisons across studies. If trials are reported as standard, the definition used for treatment completion (whether expressed as a number of sessions or as the core components of the protocol that are required to have been delivered), and the known reasons for any dropout and the stage at which it occurred, the robustness of future analyses of this kind will much bolstered. This study designated interventions as either being trauma-focused and NICE consistent (i.e. involving explicit exposure) or not. It is likely that rather than dichotomous categories, the degree of exposure utilized by different trauma-focused approaches varies along a spectrum in a way that is not captured here. Reporting greater detail about the degree of explicit exposure contained within treatment conditions would also support further research in this area. Similarly, ‘catch-all’ categories for control conditions are also imperfect. ‘Treatment as usual’ controls often vary considerably, and these were then grouped with other active psychotherapeutic approaches. Categorizing studies in this way is likely to obscure real differences in the type and intensity of the interventions provided and therefore risks missing important information about the treatment experiences of these young people.

Conclusion

While it is difficult to be confident about the reasons for dropout, the picture found here overall is one of high levels of retention in psychological therapies for PTSD in children and young people, suggesting that these treatments are broadly well tolerated. Our absolute estimates of dropout were accompanied by a large degree of heterogeneity, limiting the generalizability of this conclusion. Nevertheless, our analyses of RCTs suggested that there was no evidence for different dropout rates when making comparison to control conditions. Click here for additional data file. Click here for additional data file.
  72 in total

Review 1.  Do patients drop out prematurely from exposure therapy for PTSD?

Authors:  Elizabeth A Hembree; Edna B Foa; Nicole M Dorfan; Gordon P Street; Jeanne Kowalski; Xin Tu
Journal:  J Trauma Stress       Date:  2003-12

2.  Effect of Developmentally Adapted Cognitive Processing Therapy for Youth With Symptoms of Posttraumatic Stress Disorder After Childhood Sexual and Physical Abuse: A Randomized Clinical Trial.

Authors:  Rita Rosner; Eline Rimane; Ulrich Frick; Jana Gutermann; Maria Hagl; Babette Renneberg; Franziska Schreiber; Anna Vogel; Regina Steil
Journal:  JAMA Psychiatry       Date:  2019-05-01       Impact factor: 21.596

3.  Factors related to clinician attitudes toward prolonged exposure therapy for PTSD.

Authors:  Josef I Ruzek; Afsoon Eftekhari; Craig S Rosen; Jill J Crowley; Eric Kuhn; Edna B Foa; Elizabeth A Hembree; Bradley E Karlin
Journal:  J Trauma Stress       Date:  2014-08

4.  The International Society for Traumatic Stress Studies New Guidelines for the Prevention and Treatment of Posttraumatic Stress Disorder: Methodology and Development Process.

Authors:  Jonathan I Bisson; Lucy Berliner; Marylene Cloitre; David Forbes; Tine K Jensen; Catrin Lewis; Candice M Monson; Miranda Olff; Stephen Pilling; David S Riggs; Neil P Roberts; Francine Shapiro
Journal:  J Trauma Stress       Date:  2019-07-08

5.  Treating sexually abused children with posttraumatic stress symptoms: a randomized clinical trial.

Authors:  N J King; B J Tonge; P Mullen; N Myerson; D Heyne; S Rollings; R Martin; T H Ollendick
Journal:  J Am Acad Child Adolesc Psychiatry       Date:  2000-11       Impact factor: 8.829

6.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials.

Authors:  Kenneth F Schulz; Douglas G Altman; David Moher
Journal:  BMC Med       Date:  2010-03-24       Impact factor: 8.775

7.  A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms.

Authors:  Judith A Cohen; Esther Deblinger; Anthony P Mannarino; Robert A Steer
Journal:  J Am Acad Child Adolesc Psychiatry       Date:  2004-04       Impact factor: 8.829

8.  A mental health intervention for schoolchildren exposed to violence: a randomized controlled trial.

Authors:  Bradley D Stein; Lisa H Jaycox; Sheryl H Kataoka; Marleen Wong; Wenli Tu; Marc N Elliott; Arlene Fink
Journal:  JAMA       Date:  2003-08-06       Impact factor: 56.272

9.  Effectiveness of Trauma-Focused Cognitive Behavioral Therapy Among Trauma-Affected Children in Lusaka, Zambia: A Randomized Clinical Trial.

Authors:  Laura K Murray; Stephanie Skavenski; Jeremy C Kane; John Mayeya; Shannon Dorsey; Judy A Cohen; Lynn T M Michalopoulos; Mwiya Imasiku; Paul A Bolton
Journal:  JAMA Pediatr       Date:  2015-08       Impact factor: 26.796

10.  Treating children traumatized by war and Tsunami: a comparison between exposure therapy and meditation-relaxation in North-East Sri Lanka.

Authors:  Claudia Catani; Mahendran Kohiladevy; Martina Ruf; Elisabeth Schauer; Thomas Elbert; Frank Neuner
Journal:  BMC Psychiatry       Date:  2009-05-13       Impact factor: 3.630

View more
  1 in total

1.  Predictors of nonresponse and drop-out among children and adolescents receiving TF-CBT: investigation of client-, therapist-, and implementation factors.

Authors:  Ane-Marthe Solheim Skar; Nora Braathu; Tine K Jensen; Silje Mørup Ormhaug
Journal:  BMC Health Serv Res       Date:  2022-09-29       Impact factor: 2.908

  1 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.