Investigating the critical elements and psychosocial outcomes of Youth Flexible Assertive Community Treatment: a study protocol for an observational prospective cohort study.

INTRODUCTION: When adolescents experience complex psychiatric and social problems, numerous healthcare services usually become involved. In these cases, fragmentation of care services is a risk that often results in both ineffective care and in patients disengaging from care services. To address these issues, Youth Flexible Assertive Community Treatment (Youth Flexible ACT) was developed in the Netherlands. This client-centred service delivery model aims to tackle the fragmented care system by providing psychiatric treatment and support in a flexible and integrated manner. While Youth Flexible ACT is gaining in popularity, the effectiveness of the care model remains largely unexamined. METHODS AND ANALYSIS: Here, we present an observational prospective cohort (2017-2021) in which a broad range of treatment outcomes will be monitored. The primary aim of the study is to examine change in treatment outcomes over the course of the Flexible ACT care. The secondary aim is to examine the association between (elements of) Youth Flexible ACT model fidelity and treatment outcomes. An estimated total number of 200 adolescents who receive care from one of the 16 participating Youth Flexible ACT teams will be included in the study. Participants will be asked to complete assessments at four time points in 6-month intervals, resulting in a study duration of 18 months. Latent growth curve analysis will be conducted to examine change in psychosocial functioning over time and its relation to model fidelity. ETHICS AND DISSEMINATION: This study received ethical approval from Trimbos Ethics Committee (201607_75-FACT2). This approval applies for all participating institutions. The results of the study will be reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology statement. Results will be disseminated via peer-reviewed academic journals and presentations at conferences. In addition, results will be made available for participating sites, funders and researchers. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first multicentre study in which the effectiveness of Youth Flexible Assertive Community Treatment (Youth Flexible ACT) will be investigated.

This study will provide a complete overview of Youth Flexible ACT in which model fidelity and both psychiatric and social functioning will be assessed.

A strength of the study is its observational and naturalistic character which improves external validity.

We examine changes in treatment outcomes in a longitudinal study design, with follow-up assessments up to 18 months.

Primary limitation: as no variables are directly manipulated, causal inference is impeded.

Introduction

About 5% of Dutch children and adolescents have a mental disorder that leads to functional impairment.1 2 Approximately 20%–30% of them have to deal with severe mental health issues that require a more intensive and integrated form of care.3 In addition to psychiatric problems, these adolescents experience various difficulties in everyday life, including problems with education, employment, peer relationships, family, housing, finances, health, substance abuse and issues with the criminal justice system. These difficulties hinder their development and limit their ability to function well in society.4 Adolescents and their relatives are often required to act as the central communicators and coordinators of the care they receive. This active role requires motivation and a fairly high level of knowledge about the healthcare system, which is often too challenging for adolescents with complex care needs. In addition, healthcare providers themselves often struggle to manage multiple healthcare issues of their patients due to limited communication and coordination between healthcare providers.5 6 As a result, treatment disengagement and dropout are common.7 8 Together, this calls for a model of care that provides longitudinal, comprehensive, flexible and assertive care. Youth Flexible Assertive Community Treatment (Youth Flexible ACT) is designed to meet those demands.

Youth Flexible ACT is a client-centred service delivery model for community mental healthcare that provides assertive outreach, psychiatric treatment and support with daily living, adapted to the individual needs. The primary focus is to set up a collaborative effort with adolescents, families and their (in)formal networks while working on shared goals aimed at improving their participation in the community and enhancing their quality of life.4 Youth Flexible ACT consists of a multidisciplinary team of professionals who deliver a complete range of services on a continuum of care.

Previous research into (Flexible) ACT and Youth (Flexible) ACT

Flexible ACT is a Dutch adaptation and elaboration of Assertive Community Treatment (ACT), which originated in the USA in the 1970s.9 10 ACT is a well-known approach for individuals with severe mental illness that has been studied extensively, spread widely throughout the world and became embedded in the Dutch Multidisciplinary Schizophrenia Guideline.11–13 Studies summarised in a Cochrane Database Systematic Review provide strong evidence that ACT can increase engagement with treatment, reduce hospitalisation and lead to improvements in social domains, including stable housing, employment and patient satisfaction compared with the care as usual.13 However, studies conducted after the initial Cochrane Review in 1998, mostly outside the USA, have shown mixed results.14 15 For example, the German ACCESS study revealed that ACT was associated with symptomatic and functional improvements and better service engagement in patients with schizophrenia spectrum disorders compared with standard care.16 A recent Chinese study showed positive results in terms of hospital readmission, symptoms and relapse, employment, social and occupational functioning, and quality of life of caregivers.17 However, the British Randomised Evaluation of Assertive Community Treatment (REACT) study found no effects on clinical and social outcomes and hospitalisation.18 Additionally, a Dutch study did not find a difference between ACT and standard care in reducing admission days and clinical outcomes.19 These inconsistent findings could be due to low ACT model fidelity or insufficient contrast between experimental and control conditions, since treatment as usual gradually incorporates elements of ACT.14 15 20

While ACT is indicated for the most vulnerable patients with severe mental illness (predominantly psychotic disorders) who have the greatest needs for care, Flexible ACT delivers care for a broader group of patients with severe mental illness.9 10 For stable patients, Flexible ACT provides multidisciplinary treatment and support through individual case management. For unstable patients, it provides intensive care offered by the same team.21 Flexible ACT allows for flexible delivery of different modes of care according to the stability of the patient, in turn enhancing continuity of care.10 The Flexible ACT model was developed in the Netherlands in 2003. Over the last 10 years, the model has been widely implemented in the Dutch mental healthcare system (roughly 300 teams21). Lately, the Flexible ACT model has gained considerable interest in England, Canada and Scandinavia.21–23 However, despite the enthusiasm of service providers, the evidence base for the effectiveness of the adult Flexible ACT model remains sparse.24–26 Preliminary results have shown increased symptomatic remission of psychotic symptoms in patients with severe mental illness compared with controls receiving standard treatment,27 higher levels of psychosocial functioning,28 fewer hospital admissions and reduction of inpatient bed use,23 29 and increased compliance with treatment, decrease in unmet needs and improved quality of life.30 Flexible ACT has been shown to be more cost-effective compared with assertive outreach teams in England due to reductions in bed use, face-to-face contacts and changes in staffing.23 Outcomes of Youth Flexible ACT, which was introduced in the Netherlands in 2011, are reported in two reports published in the Dutch literature. Both studies were uncontrolled pre-post studies, and they showed preliminary positive outcomes in that patients’ behavioural problems, problems with family life, hallucinations and delusions,8 attention problems, emotional symptoms, self-injury and peer problems improved.31 However, no improvement was found in the quality of life of the adolescents.31

The broader domain of Youth ACT is also limited to pre-post studies with small samples, which have reported similar positive outcomes, such as improvements in psychiatric condition,32–34 improved global functioning and increased life skills,35 and decreased number of days in hospital.35–37 Moreover, a recent Swiss study showed that Youth ACT results in improved daily functioning and clinical benefits.38

In the field of assertive outreach for adolescents, the largest controlled studies involved evaluations of early psychosis programmes. The Danish OPUS study demonstrated that patients receiving an assertive intervention for 2 years had fewer psychotic symptoms and decreased substance use, increased adherence to medication and increased treatment satisfaction.39 Furthermore, a randomised controlled trial conducted in England indicated that (a variant of) ACT was superior to standard care for maintaining contact with professionals and reducing readmissions.40 Finally, a Dutch quasiexperimental study showed positive effects of ACT on measures of psychopathology, psychosocial functioning and quality of life.41 Altogether, research of integrated outreach models for youth and adolescents supports the effectiveness of the Youth Flexible ACT model.

Research into (Flexible) ACT model fidelity

Studies on the effectiveness of (Flexible) ACT raised the issue of identifying essential elements of the model and investigated model fidelity, which reflects the degree to which different elements of the model are implemented in full accordance with the model.42 Studies have pointed out that higher ACT model fidelity is associated with improved outcomes for patients, such as level of daily functioning and less homeless days.43–46 Specifically, the typical team structure of ACT with shared case loads and daily team meetings has been found to be associated with better daily functioning45 and decreased hospitalisation rates.47 Furthermore, a positive association was found between the presence of consumer-providers, improvements in daily functioning and the number of homeless days.48 Nevertheless, no studies have examined critical elements related to effectiveness of Youth Flexible ACT.

Purpose of the present study

Although the Youth Flexible ACT model has been well received by mental health professionals, its popularity has been increasing and national Flexible ACT and (inter)national ACT studies have provided promising results, little evidence exists to support this particular model. Much research has focused only on adults, and the results have yielded coarse outcomes based merely on small study populations. Additionally, as the healthcare sector has been deinstitutionalised over the years, it is necessary to study Youth Flexible ACT in the current healthcare landscape.

The present study is an observational longitudinal study in which a broad range of psychosocial treatment outcomes will be monitored over time. The general objective of the present study is to give insight into outcomes of Youth Flexible ACT while exploring the Youth Flexible ACT population characteristics, content and process of care. The study addresses three research questions: (1) improvement in treatment outcomes over the course of the Flexible ACT care, (2) associations between Youth Flexible ACT model fidelity and treatment outcomes, and (3) associations between specific critical elements of the Youth Flexible ACT model and treatment outcomes. This study is of an explorative nature to the extent that no similar study has been conducted on the youth population. However, reasonable expectations based on the adult literature can be generated as follows: (1) psychosocial problems of adolescents will decrease when treated by a Youth Flexible ACT team and (2) high model fidelity will be associated with a decrease in psychosocial problems.

Methods and analysis

Design

The Youth Flexible ACT Study is an 18-month observational prospective cohort study that examines change in treatment outcomes over the course of Youth Flexible ACT care and examines the association between (elements of) Youth Flexible ACT model fidelity and treatment outcomes. A total of 16 (non-specific) Youth Flexible ACT teams across seven mental healthcare institutes in the Netherlands participate in the study. An estimated number of 200 adolescents receiving care from a Youth Flexible ACT team will be included in the study. Participants and their mental health workers will be asked to complete assessments at four time points in 6-month intervals. Also parents/carers were asked to participate. The data collection started in 2017 and is scheduled to conclude in February 2021. The results of the study will be reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.49

Patient involvement

This study is designed in collaboration with mental health workers. Before the start of the data collection, we have received input from all participating Youth Flexible ACT teams, a peer support worker and two Youth Flexible ACT patients. They have provided input about the recruitment process, information letter and (duration of) assessment battery. Results of the Youth Flexible ACT Study will be disseminated via scientific journals, presentations at conferences and will be made available at participating sites.

Study sample

The study sample comprises adolescents aged 12–24 years who receive Youth Flexible ACT care at one of the participating mental healthcare organisations throughout the Netherlands: Accare, GGZ Noord-Holland-Noord, GGZ Oost Brabant, Kenter Jeugdhulp, Lucertis, Mondriaan-Gastenhof and Triversum.

Participants receiving Youth Flexible ACT care will be included in the study if they meet the following research inclusion criteria:

Participants must be 12–24 years of age.

Participants must have sufficient knowledge of the Dutch language both spoken and written.

Participants and their parent/caregiver must provide a written informed consent.

Study population

Youth Flexible ACT provides treatment to young people with complex and severe mental health problems, who have difficulty engaging in regular mental healthcare. Practical experiences and literature indicate that these young people have a hard time accessing and remaining in regular outpatient mental healthcare.4 33 38 Several reasons for treatment disengagement in adolescents have been suggested, such as fragmented healthcare system,50 treatment discontinuity51 52 and difficulty to trust services.53 According the Youth Flexible ACT model description4 young people are eligible for Youth Flexible ACT care if:

They are diagnosed with a mental health disorder (or presumptive diagnosis).

They experience difficulties in multiple areas of daily life (eg, problems with education, employment, peer relationships, housing, finances, health, substance abuse and issues with the criminal justice system).

They face family system problems and/or parenting issues.

They live in the district of the Youth Flexible ACT team.

The pathway to Youth Flexible ACT care is straightforward and direct. Anyone (eg, patients’ parent, care workers) in the Netherlands can directly contact a Youth Flexible ACT team to suggest a potential referral when he/she thinks a patient is eligible. An intake coordinator of the Youth Flexible ACT team then determines if the inclusion criteria outlined above are met. If so, a referral from the general practitioner is requested before care can start.

Youth Flexible ACT

Youth Flexible ACT is a community-based mental health service in which integrated teams provide long-term assertive outreach care consisting of both treatment for psychiatric symptoms and practical assistance with daily living needs, rehabilitation and recovery support. Youth Flexible ACT teams consist of various professional disciplines, including psychiatrists, nurses, psychologists, employment specialists, psychiatric nurses, addiction specialists, peer support workers, social workers, and family and systemic therapists. Youth Flexible ACT encompasses a multiagency approach that coordinates collaboration with professionals from other services. Team members visit the patients at their home or at other preferred locations and provide assertive care when necessary. Time will be devoted to build and maintain trust between mental health workers and patients and to motivate patients to receive treatment and support. Mental health workers have a small case load (1:15) and deliver two modes of care: individual case management and intensive team care. Patients receiving individual care have a case manager and a head practitioner (psychiatrist, healthcare or clinical psychologist). Other team members can be added, as needed, for specific elements of treatment or support. A patient who is in need of extra care will receive intensive care from several team members. These patients are listed on a digital board, and the team discusses them every day to decide which form of care should be provided and by which team members. When the crisis or the need for intensive care is over, individual case management is resumed. Youth Flexible ACT, in which the family system plays a major part, has additional features beyond those that are part of Flexible ACT for adults. In particular, it is important to support the following four developmental tasks4 8: shaping changing relationships within the family (moving from dependence to autonomy), stimulating contact with peers (peers become more important as reference group as the influence of parents decreases), participating in education or work and filling leisure time. Furthermore, the possibilities for personal growth and the utilisation and cultivation of personal strength are emphasised instead of mental health disorder symptoms. Youth Flexible ACT is primarily a service delivery model. It describes the organisation of care for adolescents with complex care needs. The Flexible ACT model does not dictate the specific content of a treatment plan, although the use of evidence-based practices is advised. The degree to which teams implement these guidelines determines the level of model fidelity. With high Flexible ACT model fidelity, a complete multidisciplinary team provides the desirable treatment and support according to the guidelines. A more detailed portrayal of (Youth) Flexible ACT is outlined in the (Youth) Flexible ACT model description.4 21 54

Recruitment and assessments

During the intake process, team members of Youth Flexible ACT teams will ask eligible adolescents to participate in the study. After signing informed consent, participants will be asked to complete a baseline measurement within a 12-week margin. Participants will then be monitored every 6 months with questionnaires, up to four measurements. These questionnaires will be completed during a regular appointment by a familiar mental health worker or participants have the option to complete the questionnaires independently in their own time. It will take approximately 20–30 min for adolescents to complete the assessment battery. Adolescent participants receive a remuneration of €10. Both paper and online versions are available. Online versions are preferred to minimise missing data. Researchers are in close contact with mental health workers and ensure that participants complete the questionnaires timely. An online Dutch data system will be used to collect the data. Confidentiality of the data is guaranteed through a login procedure and each institution has its own digital environment. Participants who finish their treatment within 1.5 years will, with the adolescents’ permission, receive a link to the remaining questionnaires by email.

Training

To increase the reliability of assessments, all mental health workers involved in data collection were trained in administering the questionnaires before baseline assessments. In particular, mental health workers received a Health of the National Outcome Scales for Children and Adolescents (HoNOSCA) training based on the official training.55 The HoNOSCA56 is a global scale measuring daily functioning and mental health symptoms. To examine the inter-rater agreement of the HoNOSCA, original case vignettes were used. The training consisted of a HoNOSCA information lesson, completing a training vignette and discussion. Subsequently, 82 mental health workers of 13 participating teams completed the actual vignette of which scores were analysed.

Intraclass correlations (ICC) were calculated as a measure of agreement between raters of a single vignette based on absolute agreement using two-way mixed-effects model. Statistical analyses were performed using IBM SPSS Statistics V.22.0. The resulting ICC between raters ranged from moderate to good agreement: ICC average measures=0.99 (95% CI 0.98 to 1.00) and ICC single measures=0.57 (95% CI 0.40 to 0.78). The findings indicate a moderate degree of agreement between ratings within an item and between items. Since a single vignette was used, the results reflect the agreement between raters and do not reveal the reliability of scale items.

Study outcome measures

Table 1 displays an overview of outcome measures that will be reported by clinicians, patients and parents/carers at each time point. The employed set of questionnaires together reflect the multiple life domains in which Youth Flexible ACT operates. The combination of questionnaires assesses general psychological functioning, specific diagnostic characteristics and daily functioning of the participants.

Table 1

Overview of outcome assessments

Variable	Instrument	Time of assessment (months)
Clinician reported outcomes
Daily functioning	HoNOSCA	T0, T6, T12, T18
Sociodemographics	Questions concerning sociodemographics	T0
Content of care	Seven questions concerning content of care	T6, T12, T18
Patient reported outcomes
Psychosocial well-being	SDQ	T0, T6, T12, T18
Health-related quality of life	Kidscreen-10 + additional questions	T0, T6, T12, T18
Depressive symptoms	CDI-2	T0, T6, T12, T18
Social support	Scale ‘social support and peers’ from the Kidscreen-52	T0, T6, T12, T18
Empowerment	Subscale ‘interactional empowerment’ from the questionnaire EMPO 3.1	T0, T6, T12, T18
Psychosis risk screening	PQ-16	T0, T6, T12, T18
Treatment satisfaction	Four brief questions based on the Jeugdthermometer	T0, T6, T12, T18
Care utilisation	One question concerning care utilisation	T0
Care utilisation and coordination	Two questions concerning care utilisation and coordination	T6, T12, T18
Parent reported outcomes
Psychosocial well-being—child	SDQ-P	T0, T6, T12, T18
Health-related quality of life—child	Kidscreen-10 parent version	T0, T6, T12, T18
Psychological distress	MHI-5	T0, T6, T12, T18
Parenting stress	PSQ-S	T0, T6, T12, T18
Treatment satisfaction	Four questions based on the Jeugdthermometer parent version	T0, T6, T12, T18

CDI-2, Child Depression Inventory-2; EMPO, Dutch Empowerment Questionnaire; HoNOSCA, Health of the National Outcome Scales for Children and Adolescents; MHI-5, Mental Health Inventory; PQ-16, Prodromal Questionnaire; PSQ-S, Short version of the Parenting Stress Questionnaire; SDQ, Strengths and Difficulties Questionnaire; SDQ-P, Strengths and Difficulties Questionnaire for Parents.

Clinician reported outcomes

Daily functioning

The Dutch version of the HoNOSCA56 will be used to measure mental health symptoms and daily functioning. Items are rated on a 5-point scale indicating the severity of problems ranging from no problem (0) to severe problem (4). Studies demonstrated sufficient reliability and validity for use in clinical samples.56–58

Sociodemographics and general case load information

Mental health workers provide information about the patients’ level of completed education, referrer, admission duration and whether IQ tests have been conducted. Mental healthcare workers also complete questions about the size and composition of the case load as part of the audit procedure.

Content of care

Content of care encompasses seven multiple-choice questions that provides insight into type of treatment and support, frequency of visits and frequency of provided intensive ‘ACT’ care in the past 6 months.

Patient reported outcomes

Psychosocial well-being

The Strengths and Difficulties Questionnaire (SDQ) is a brief screening questionnaire for children and adolescents which screens for psychosocial well-being.59 60 It includes 20 questions measuring the subscales emotional, conduct, and hyperactivity/inattention symptoms and peer problems. The subscale prosocial behaviour is excluded because scores of this subscale are not necessary to compute a total difficulties score. Each item is scored on a 3-point scale (0=not true, 1=somewhat true and 2=certainly true). The subscales are ranging from 0 to 10 for each scale and are added together to generate a total difficulties score, ranging from 0 to 40. Scores above the cut-off of 16 (>90th percentile) are considered ‘abnormal’.59 61 In addition, if an adolescent experiences difficulty, the impact scale can be used to indicate the extent to which any problems interfere with daily functioning. The five items are scored on a 3-point scale (0=not at all/only a little, 1=quite a lot, 2=a great deal) and can be added to compute an impact score that ranges from 0 to 10. Scores above the cut-off of 2 are considered ‘abnormal’.59 61 The SDQ was found to have sufficient reliability and validity for assessment in clinical samples.59 60 62–64

Health-related quality of life

The Kidscreen-10 is a 10-item questionnaire measuring health-related quality of life.65 66 Items are rated on a 5-point Likert scale ranging from 0 (never) to 5 (always). A total score can be generated by summing the 10 items. Total scores will be converted into Rasch scores67 and translated into T values (M=50; SD=10), with higher values indicating higher health-related quality of life. In addition to these 10 brief questions, the adolescents will be asked to provide additional information concerning important areas of daily living needs, such as finance, education or housing. Research has shown adequate psychometric properties for the Kidscreen-10.66

Depressive symptoms

Depressive symptoms will be measured using the Dutch version of the Child Depression Inventory-2 (CDI-2).68 69 The CDI-2 consists of 28 items measured on a 3-point scale ranging from 0 (depressive symptom is absent) to 2 (depressive symptom is always present). Sum scores can be computed by adding together scores of all 28 items. A higher total score means more depressive symptoms. A total score of 14 or higher indicates clinical levels of depressive symptoms.68 The internal consistency and validity of the CDI-2 have shown to be good.68

Social support

To measure social support, the scale ‘Social support and peers’ from the Kidscreen-5267 will be used. This scale examines the quality of the social interaction between adolescents using six items, for example, ‘did you have fun with friends?’ The items are measured on a 5-point Likert scale ranging from 1 (never) to 5 (always). A total subscale score can be computed by adding the six items. The total score will then be converted into Rasch scores and translated into T values (M=50; SD=10), with higher values indicating higher quality of social interaction with peers.67 We added two additional questions about satisfaction with social support. The Kidscreen-52 has shown acceptable levels of reliability and validity.70 71

Empowerment

Empowerment will be measured with the subscale ‘interactional empowerment’ from the questionnaire Dutch Empowerment Questionnaire (EMPO 3.1),72 73 which assesses the willingness to change undesired situations, to look for solutions, to take control and to know how to access resources.73 The subscale comprises six questions measured on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). A total score can be computed by summing the six items and can then be translated into T values, with higher values indicating higher level of empowerment. According to the authors of the EMPO 3.172 the questionnaire shows good internal consistency in a large clinical sample (total scale α=0.89; subscale intrapersonal empowerment α=0.89; subscale interactional empowerment α=0.79).

Psychosis risk screening

The Prodromal Questionnaire (PQ-16)74 is a self-report questionnaire used to screen for subclinical psychotic symptoms that may indicate an increased risk of psychotic disorder in the future. It consists of 16 items that can be rated as true (1) or false (0), based on subjective experiences during the previous month. If true, the distress score will be measured on a 4-point Likert scale ranging from 0 (no distress) to 3 (severe distress). The total score on the PQ-16 can be calculated by adding up all items, ranging from 0 to 16. A cut-off score of ≥6 predicts a high-risk status with high sensitivity (87%) and specificity (87%).74 Research has shown good psychometric properties for the PQ-16.74–76

Treatment satisfaction

Satisfaction with treatment will be measured with four brief questions based on the Jeugdthermometer.77 Treatment satisfaction is rated on a scale from 1 (very bad) to 10 (very good).

Care utilisation

At baseline assessment, patients will report their previous care utilisation via a single multiple-choice question.

Care utilisation and coordination

At follow-up, patients will report which forms of care they receive in addition to Youth Flexible ACT. In addition, they will report their satisfaction between the cooperation with the Youth Flexible ACT team and other healthcare facilities on a scale from 1 (unsatisfactory) to 10 (very satisfactory).

Parent reported outcomes

The Strengths and Difficulties Questionnaire for Parents (SDQ-P) is the parent version of the SDQ, as described above.59 The 20-item total difficulties subscale is allocated to four domains: emotional symptoms, conduct problems, hyperactivity inattention and peer problems. Parents rate the items on a 3-point Likert scale, ranging from 0 (not true) to 2 (certainly true). Scale scores can be computed by summing the scores on the scale items (range 0–10). A total difficulty score can be computed by adding the four scale scores (range 0–40). Scores above the cut-off of 14 (>90th percentile) are considered as a raised level of psychosocial problems.59 78 Studies showed good psychometric properties of the SDQ-P.60 78 79

The Kidscreen-10 parent report is the parent version of the Kidscreen-10, a 10-item questionnaire on quality of life, as described above.66 67

Psychological distress

The Mental Health Inventory (MHI-5)80 is a brief questionnaire and will be used to assess parental psychological distress. The MHI-5 consists of five items scored on a 6-point Likert scale ranging from 0 (all of the time) to 5 (none of the time).

The total score will be obtained by summing up recoded scores, and transforming the scores to a scale ranging from 0 to 100. A higher score indicates better mental health and lower psychological stress levels. An MHI-5 cut-off score of ≤60 will be used to indicate psychological distress.81 The MHI-5 has shown good psychometric properties.82–85

Parenting stress

Parenting stress will be measured using the Short version of the Parenting Stress Questionnaire.86 The questionnaire focuses on the quality of the parent–child relationship measured using 10 items rated on a 4-point scale ranging from 1 (is completely applicable to me) to 4 (is not applicable to me). A total score will be calculated by summing all recoded items (ranging from 10 to 40) with higher total scores indicating higher levels of parental stress. Scores higher than 22 reflect raised level of parenting stress. Research has shown sufficient psychometric properties.73 86 87

Satisfaction with treatment will be measured using four brief questions based on the Jeugdthermometer parent version.77 The items are rated on a scale from 1 (very bad) to 10 (very good).

Youth Flexible ACT model fidelity

The association between psychosocial functioning of adolescents and (elements of) model fidelity will be studied. The level of implementation of the model will be measured with the Youth Flexible ACT scale (Youth FACTs) developed by the Centre for Certification ACT and Flexible ACT (CCAF) in 2014.42 The CCAF was established in the Netherlands in 2008 by Dutch mental healthcare professionals and researchers to ensure the quality of ACT and Flexible ACT. During the Youth Flexible ACT Study each team will be subjected to a single and official audit performed by the CCAF. These audits determine the degree to which each team complies with the Youth Flexible ACT model. All teams will be audited within a period of 1.5 years. The Youth FACTs consists of 62 items measuring seven main elements: team structure (15 items), team process (12 items), diagnostics, treatment and interventions (12 items), organisation of services (11 items), community care (5 items), monitoring (3 items) and level of professionalisation (4 items). Two independent raters from CCAF will score the Youth FACTs on a 5-point rating scale ranging from 1 (minimum implementation) to 5 (maximum implementation). The CCAF defines a total score on the Youth FACTs of 3.0 and lower as insufficient implementation while scores between 3.1 and 3.3 indicate a temporary certificate for 1 year, with improvements to be made to obtain a final certificate. Scores 3.4–4.0 are sufficient to receive the certificate while scores of 4.1 and higher are regarded as excellent. If a team has received a certificate, it remains valid for 3 years. During this period, the team is expected to report any major changes, including reorganisations and divisions, to the CCAF.

Administrative data

Sociodemographic and clinical characteristics (sex, age, psychiatric diagnose, treatment duration) will be collected via electronic patient records.

Statistical analysis

Latent growth curve analysis will be used to examine change in psychosocial functioning over time (T0, T6, T12, T18 in months) and its relation to model fidelity. We hypothesise that (1) psychosocial problems will decrease while treated by a Youth Flexible ACT team and (2) higher Youth Flexible ACT model fidelity will lead to greater improvements in psychosocial functioning over time. This change over time will be calculated with growth parameters (intercept, linear slope and possible quadratic factor) for each patient. For each of the study outcomes, the best fit function (model) will be assessed to measure change in outcomes. Control variables (age, gender and psychiatric diagnoses of patients) will be included as covariates in the growth model. In addition, the multiple elements of model fidelity will be used as predictors of the growth parameters. Regression coefficients will indicate the extent to which these elements contribute to these parameters. To deal with missing values, the full information maximum likelihood estimator, or the using-all-available information method, will be used. Furthermore, the data will be clustered within subjects and teams. Seven mental health institutions will participate in the study, with a total of 16 teams and an expected average of 20 patients per team. If the results of this study demonstrate substantial variation between the participating teams, analysis will be performed to account for team effects.

Sample size calculation

Given the practical challenges in conducting a multicentre study with a complex intervention, the aim of this study is to achieve maximum participation.88 A power analysis for a paired t-test with G*power indicated that a minimum of 156 patients will have to be recruited to achieve a power of 0.80, an alpha of 0.05 and an effect size of 0.20 (one tailed). Slightly more patients will be required (n=165) for a small slope effect of a linear latent growth model.89 When accounting for missing values, a target sample size of 200 participants should be sufficient. The past inflow data of all participating Youth Flexible ACT teams show that approximately 500 patients are included in care every year. This indicates that a sample size of 200 is attainable. Concerning follow-up measurements, the included teams indicated that most patients finish their Youth Flexible ACT treatment in 1–2 years. This suggests that most of the participants will remain in Flexible ACT care for the duration of the study (ie, 1.5 years). We expect being able to continue collecting data on most of these participants through follow-up assessments. The participating teams estimated that less than 5% of patients drop-out of Youth Flexible ACT care entirely. To ensure that the sample size is reached, a minimum number of inclusions are determined for each team. In addition, each team strives for maximum inclusion beyond this minimum bound.

Discussion

Preliminary reflection on the limitations and strengths of the design

The observational and naturalistic character of the study design is both its weakness and its strength. Conclusions are restricted to associations between Youth Flexible ACT care and treatment outcome with the obtained data. No causal relationships can be implied because a control group is absent. Realising a resembling control group and providing them treatment as usual is practically impossible to achieve, due to the complexity of the psychosocial problems, vulnerability to mental health crisis and an extensive avoidance of mental health services. Additionally, as it is difficult to find a resembling study population, it is not possible to match the research results with a data set of another comparable cohort study. Furthermore, as a consequence of a naturalistic study design, the data obtained in the practical field of mental healthcare are subject to transitions and developments during data collection (eg, changes in team structure). Nevertheless, an important strength of the current study is its strong external validity, as Youth Flexible ACT will be studied as it is used in daily practice.90 Another strength of the study is its longitudinal design with follow-up assessments up to 18 months, providing the opportunity to evaluate long-term effects. Furthermore, examining a broad set of outcomes (both psychiatric and social functioning) allows for a more complete view of Youth Flexible ACT, since the data on participant and service characteristics will be collected from a large sample of patients and mental health workers.

Implications for clinical practice

When adolescents experience complex problems affecting various aspects of their lives, fragmentation of care services can lead to inconsistent and ineffective care. Youth Flexible ACT actively engages adolescents in treatment and provides a flexible response to the needs in different stages of care that enhances continuity of care. By providing assertive and integrated treatment, Youth Flexible ACT aims to tackle this fragmented mental healthcare system. The present study will contribute to clinical practice by providing insights into the effectiveness of Youth Flexible ACT and the essential elements responsible for the effect. This will provide valuable information for mental healthcare organisations, funding organisations and policymakers on how to maximise the quality of care for a vulnerable group of adolescents for whom the existing regular outpatient mental healthcare is hardly suitable.

Ethics and dissemination

The medical ethics committee CMO Region Arnhem-Nijmegen in the Netherlands concluded that the present study (NL57443.091.16) does not require medical ethical approval. In addition, this study received ethical approval from Trimbos Ethics Committee (201607_75-FACT2). The Trimbos Institute is the Netherlands Institute of Mental Health and Addiction, a non-profit research and knowledge centre that reviews non-WMO (Medical Research Involving Human Subjects Act) research projects. This committee ruled that our protocol complies with all applicable regulations. This approval applies for all participating institutions. Written informed consent from adolescents and parents or legal guardians will be obtained. The results of the study will be reported in accordance with the STROBE statement. Results will be disseminated via peer-reviewed academic journals and presentations at conferences. In addition, results will be made available for participating sites, funders and researchers.