How well do children in the North East of England function after a mental health crisis during the COVID-19 pandemic: A service evaluation.

The COVID-19 pandemic has affected millions of people, and some researchers postulate that a mental health crisis will follow. The immediate effects of the COVID-19 pandemic on children's mental health are now starting to be published, and results appear to be mixed. There is no research, to the authors' knowledge, that empirically examines the functioning of young people following intervention from Child and Adolescent Mental Health Services (CAMHS) Crisis Teams in the UK during the COVID-19 pandemic. This service evaluation aims to do this using data from an NHS trust that supports 1.4 million people in the North East of England. We compared functioning, as measured by the Outcome Rating Scale (ORS), before and after treatment for young people discharged from the CAMHS Crisis Team between December 2019 and December 2020. ORS scores were significantly higher at the end of treatment (t(420) = -57.36, p < 0.001) with a large effect size (d = -1.56). Fifty eight percent of patients exhibited significant and reliable change (i.e. functioning improved to a 'healthy' level). No patients significantly deteriorated in functioning after accessing the crisis service.

Introduction

The COVID-19 pandemic has affected millions of people across the globe. Between January 2020 and March 2020, the disease spread to more than 110 countries (Al Omari et al., 2020). The World Health Organization declared COVID-19 a pandemic on 11 March, 2020 (World Health Organization, 2020). As of 24 May, 2021, there were 4,480,752 new cases of COVID-19 in the UK with 127,986 deaths. This is more than double that of 18 December, 2020 where there were 57,722 deaths reported in the UK. There are now over 160 million new cases globally, with over 3 million deaths (John Hopkins University & Medicine, 2021). In order to mitigate the effects of the outbreak, and limit the impact on the National Health Service (NHS), the UK government put a number of measures in place which included social distancing, restricted travel, remote learning for children and national lockdowns (Cabinet Office, 2021). Some researchers postulate that a mental health crisis will follow (Fofana et al., 2020; Heale & Grooms, 2020; Holmes et al., 2020). Research already highlights that individuals with mental health problems are more vulnerable to the negative impacts (on physical and mental health) of the pandemic (Minihan et al., 2020). The exact impact of the current pandemic on children’s mental health is largely unknown; however, experts assume a significant increase in the burden of mental illness, specifically the worsening of pre-existing mental health problems and the challenge of unmet treatment needs (Fegert et al., 2020; Loades et al., 2020).

The immediate effects of the COVID-19 pandemic on mental health are now starting to be published and results appear to be mixed. For example, a Canadian study with 1381 adult participants examined the impact of COVID-19 on psychological distress and found that even short-term social distancing practices in the first few weeks of the outbreak were associated with increased psychological distress (Best et al., 2020). As there are no validated scales to specifically measure psychological distress during global pandemics, Best et al. (2020) used an adapted questionnaire where reliability was high (α =.82) (Best et al., 2020). Similarly, a recent rapid review of the psychological impact of quarantine included 24 studies and found most reported negative psychological effects including post-traumatic stress symptoms, confusion and anger. Some researchers have also suggested long-lasting effects (Brooks et al., 2020). Whereas Pan et al. (2020) found, from distributing 1517 online questionnaires to three Dutch cohorts, that the COVID-19 pandemic does not seem to have further increased symptom severity of depression, anxiety or obsessive-compulsive disorders compared to pre-pandemic levels (Pan et al., 2020).

The results from child mental health and COVID-19 research are also mixed. Ford et al. (2021) examine the fluctuating patterns and presentations of children’s mental health across the pandemic year in 2020 (Ford et al., 2021). Some young people experienced an increase in mental health difficulties (Vizard et al., 2018) as well as increased behavioural problems (specifically amongst 4 to 11 year olds) (Waite et al., 2020). For other young people, they reported that lockdown made their life easier, and they felt happier during the summer of 2020 (Mansfield et al., 2020). Longitudinal data, which included 12 studies, found a moderate increase in adolescent depressive symptoms using data before and during the pandemic (Barendse et al., 2021). Some research suggests there has not been a substantial impact on children and young people’s mental health. For example, a North American study, with 202 participants (mean age of 13.6 years) compared pre-COVID (as defined by 1 January, 2020 to 28 February, 2020) and COVID-19 (as defined by 1 March, 2020 to 30 April, 2020) admissions for psychiatric emergencies. They found a significant decline in the volume of young people (however not in their adult sample) (Ferrando et al., 2020). Similarly, an Irish study examined self-harm data from the emergency department of a hospital by comparing admissions from the first 3 months of the COVID-19 pandemic in 2020 to the preceding 3 years. Young people aged 18 years and under made 15% of this hospital population in 2020 and 17.5% of the population in the preceding 3 years, which was not a significant difference during the pandemic (McIntyre et al., 2020). Similarly, a study examining UK data from the NHS Digital database and the National Commissioning Data Repository (NCDR) found a substantial decrease in the number of young people presenting to A&E with psychiatric symptoms in England in April 2020 compared to the previous year, as well as a reduction in psychiatric inpatient admissions (Ougrin, 2020). Furthermore, a study examining the effect of the first wave of the COVID-19 pandemic on suicide among children and adolescents during school closures in Japan, by comparing the school closure period (March to May 2020) with the same period the year before, found there was no significant change in suicide rates during school closures (Isumi et al., 2020).

However, some research does suggest that young people’s mental health has been negatively impacted by the COVID-19 pandemic. For example, another Irish study found an increase in hospital psychiatry presentations for the under 18 population during the initial 8 weeks of COVID-19 restrictions, compared to the same time period the year before (McAndrew et al., 2020). Leeb et al. (2020) report that throughout 2020, mental health admissions (for anxiety and panic) in US A&E departments increased by 24% for young children and 31% for adolescents compared to the previous year. New facilities for young people have opened to address the overcrowding (Leeb et al., 2020). A recent news article described children’s psychological reactions to the pandemic as a result of prolonged stress which impacts the brain’s ability to manage emotions. Where children may have been slightly anxious before the pandemic, they have become very anxious over the past year (Carey, 2021). One area which might make young people more vulnerable is their use of social media. Zhong et al. (2021) found, via an online survey in 558 adults living in Wuhan, that social media usage was related to depression and secondary trauma, and that excessive use of social media may lead to mental health issues (Zhong et al., 2021).

There is a need to examine the effects of school closure for children, as often school will be the first-place young people seek support. This is especially important for vulnerable children, such as those with special educational needs and developmental disorders (Holmes et al., 2020), as well as children from racial and ethnic minority groups, low socioeconomic backgrounds, or those without public health insurance in the US. These vulnerable groups are disproportionally likely to receive mental health support exclusively at school (Golberstein et al., 2020). There is also a need to examine the impact of grief for young people. Many children will have experienced sudden loss and grief for the first time if family or friends have died from COVID-19, which may impact on how safe and predictable they view the world. In addition, care givers may be overwhelmed and unable to modulate their child’s fear and sadness (Simon et al., 2020). It is also reported that children can experience grief with no single, distinct or specific cause which some psychologists call ‘ambiguous loss’ (Carey, 2021).

Examining the impact of the COVID-19 pandemic on children’s mental health and functioning has begun and yielded mixed results. One of the ways the NHS in the UK assesses a young person’s functioning is by monitoring patient outcomes, which is a central aim of the NHS in improving quality of healthcare (Department of Health, 2015). The Outcome Rating Scale (ORS) is a commonly used scale with moderate to high reliability which monitors a young person’s functioning by looking at symptom distress, interpersonal well-being, social role and overall well-being (Bringhurst et al., 2006). The ORS is recommended and licenced for use within NHS England (NHS Digital, 2020).

Before the onset of the global pandemic, the NHS identified that an increased number of young people need access to high quality mental health treatment in the community, which includes crisis services (NHS England, 2018). This need may be exacerbated by the current global pandemic; therefore, empirical evidence which assesses how well a young person functions after a crisis and treatment is essential. The first service evaluation to do this in the UK was published this year and examined data from 2018/2019. Staite et al. (2021) found a 16.2% increase in the mean ORS score (which measures a young person’s functioning) after intervention and 15% of young people exhibited both reliable change and clinically significant change (Staite et al., 2021). The service evaluation also reported that 97 (89%) patients were co-worked by another Child and Adolescent Mental Health Services (CAMHS) team and there were 29 (26%) multiple referrals within 2018/2019. This article replicated the analysis on data from 2019/2020 with the aim of assessing how well young people function following a mental health crisis and treatment during a global pandemic. Data were collected from the same CAMHS Crisis Team in Tees, Esk and Wear Valleys (TEWV) Foundation NHS Trust which supports 1.4 million people in the North East of England (Tees, Esk and Wear Valleys NHS Trust, 2008). The young person’s functioning was measured by analysing ORS data which is routinely administered to monitor a young person’s functioning.

Methods

Design

We conducted a service evaluation comparing treatment outcomes over time, replicating the methodology in Staite et al.’s (2021) original service evaluation. There was no randomisation to treatment group, as this was also a service evaluation of routine practice. Evaluation was completed before and after treatment for patients discharged from the CAMHS Crisis Team between December 2019 and December 2020. Full ORS information is available from Staite et al.’s (2021) original article.

Procedure

Following Staite et al.’s methodology, nurses from the CAMHS Crisis Team collected ORS data within 2 days of referral to the service. For 41 (10%) patients, pre-treatment ORS data were collected after this. The ORS was completed in a young person’s first session and last session with no long term follow up; however, patients could access the Crisis Team again at any point in the future (until aged 18 years) if needed. Some patients will also be discharged to other CAMHS services (e.g. generic CAMHS, Eating Disorder team, Autism Spectrum Disorder team, Learning Disability CAMHS, Early Intervention in Psychosis team, Liaison and Diversion team or Street Triage team) (Staite et al., 2021).

Description of service

The CAMHS Crisis Team offers a 24-hour service which is open 365 days a year. Open referrals are accepted for all young people in the catchment area under the age of 18 years who are experiencing a mental health crisis. Additionally, anyone who is concerned about a young person experiencing a mental health crisis can contact the service. The response time of the CAMHS Crisis Team is up to 4 hours, and there is no limit on how many times the service can be contacted.

A block of treatment can vary from an assessment and one follow up, to a referral to the intensive home treatment team (IHT) where the duration of treatment is up to 12 weeks. Typically, if there is no need for the extended period with the IHT team, the crisis intervention will be across 72 hours. However, this is flexible, and can be extended if needed. The provision of crisis services within the CAMHS Crisis Team include immediate stabilisation, safety planning with the young person and their family to help support the young person and family in the event of a future mental health crisis, as well as signposting to other services when needed.

Service delivery change during COVID-19

All crisis assessments were face to face; however, if the young person was indicated as low risk from the assessment, the follow up appointment would be offered via telephone. If a young person preferred a second face to face contact, that was provided and staff would be in full PPE (face mask, apron and gloves). To maintain social distancing during home visits, appointments included the young person and one other (e.g. parent). Where this was not possible, the family was invited into the clinic base. Before pandemic, the service was delivered by nurses, whereas during the pandemic, the team expanded to include a consultant psychiatrist and occupational therapist.

Participants

All patients who were discharged from the CAMHS Crisis Team between December 2019 and December 2020 were included in data collection. Patients without paired outcomes were not included in the analysis. Outcome data from 366 patients discharged from the CAMHS Crisis Team between December 2019 and December 2020 were evaluated. Patients were aged between 7 and 18 years and were able to complete the ORS without support from a caregiver.

Ethical Approval

The CAMHS Audit and Effectiveness Team of TEWV Foundation NHS Trust approved this project in November 2020. Other formal ethical approval was not required for this service evaluation. All data were stored securely at an NHS site, and no individual patient data are reported.

As with all research involving children and young people, patients aged 16 years and over have capacity to consent to treatment themselves. For those patients under 16 years, they were able to consent if Gillick competent. For patients not Gillick competent, someone with parental responsibility can consent to treatment for them (NHS, 2019). All patients gave verbal consent to treatment.

Measures

The Trust’s Section Head of Research & Statistics extracted all data from the NHS database. This included gender; whether the young person had multiple referrals to the CAMHS Crisis Team between December 2019 and December 2020; as well as whether the young person was co-worked by another CAMHS team (which includes generic CAMHS, Eating Disorder team, Autism Spectrum Disorder team, Learning Disability CAMHS, Early Intervention in Psychosis team, Liaison and Diversion team or Street Triage team).

As this is a replication of Staite et al.’s (2021) study, the Outcome Rating Scale was used again to measure a young person’s life functioning before and after treatment (Staite et al., 2021).

Analyses

The analysis plan is a replication of Staite et al. (2021). To compare whether there was a significant difference in pre- and post-treatment ORS scores, a paired sample t-test was used. All analyses were conducted using R (statistical computing programme).

The ORS data were further analysed using the Reliable Change Index (RCI) and clinically significant change (CSC) analysis. This refers to the extent to which patient change, in response to treatment, can be attributed to measurement variability of the instrument or is in fact ‘real’ change (Evans et al., 1998). An alpha coefficient of 0.97 was adopted from the normative data (Bringhurst et al., 2006). Calculation of CSC considers the extent to which change after treatment is meaningful for the patient (Evans et al., 1998).

Criterion ‘C’ was adopted, which provides an estimation of whether a patient has moved from a ‘clinical’ to ‘non-clinical’ distribution of scores following intervention (Evans et al., 1998). As normative data was available and there was a high overlap between clinical and non-clinical scores, Criteria ‘C’ was used (Jacobson & Truax, 1991). For the CSC analysis, population norms from Bringhurst et al. (2006) were used.

Statistical Power

For a medium effect size (d = 0.5) and an alpha of .05, an a priori sample calculation suggested 45 participants would be required for statistical power (Faul et al., 2007).

Results

Patient characteristics are displayed in Table 1 for 366 patients, of which 38 (10%) had more than one referral to the CAMHS Crisis Team within the year (December 2019–December 2020). 325 (89%) of the 366 patients were co-worked by another CAMHS Team (which included generic CAMHS, Eating Disorder team, Autism Spectrum Disorder team, Learning Disability CAMHS, Early Intervention in Psychosis team, Liaison and Diversion team or Street Triage team).

Table 1.

Patient characteristics.

Age (years), mean (SD)	15.55 (1.63)
Gender, n (%)
Female	272 (74)
Male	86 (23)
Gender neutral (birth sex male)	1 (<1)
Gender neutral (birth sex female)	7 (2)
Multiple referrals into the Child and Adolescent Mental Health Services Crisis Team*, n (%)
Yes	38 (10)
No	328 (90)
Co-worked by another team, n (%)
Yes	325 (89)
No	41 (11)

*Between December 2019 and December 2020.

Pre- and post-treatment ORS scores were collected by the CAMHS Crisis Team as part of this service evaluation for 366 patients. Data were normally distributed for both pre-ORS data (W = 0.99407, p = 0.1007) and post-ORS data (W = 0.99607, p = 0.384). Table 2 contains descriptive statistics which shows a 79.19% increase in the mean score after intervention. Patient ages ranged from 7 to 18 years and one patient was a Looked After Child (defined as a child who has been in care of their local authority for more than 24 hours (NSPCC, 2021)).

Table 2.

Descriptive statistics of pre- and post-ORS.

Measures (stage of treatment)	N	Mean	SD
ORS (pre-treatment)	421**	18.16	6.82
ORS (post-treatment)	421**	32.54	11.62

**This number is larger than 366 due to some patients having more than one referral to the Child and Adolescent Mental Health Services Crisis Team between December 2019 and December 2020.

Note: ORS = Outcome Rating Scale.

ORS scores were analysed using a paired t-test to compare whether there was a significant difference before and after intervention. ORS scores were significantly higher at the end of treatment (t(420) = −57.36, p < 0.001) with a large effect size (d=-1.56).

The data were subject to analysis using the RCI and CSC analysis using R. The number of participants exhibiting clinically significant and reliable changes in functioning is summarised in Table 3.

Table 3.

Reliable and clinically significant changes in Outcome Rating Scale score for patients.

	Reliable change, n (%)
Clinically significant, n (%)	Improve	No change
Not significant	86 (20)	7 (2)
Stable at baseline	85 (20)	0 (0)
Significant change	243 (58)	0 (0)

Table 3 indicates that 243 (58%) patients accessing the CAMHS Crisis service showed both reliable change and clinically significant change, suggesting improved functioning, as this was not due to measurement error. An additional 86 (20%) patients exhibited reliable improvement although this was not categorised as clinically significant using the adopted criteria. No patients significantly deteriorated in functioning after accessing the crisis service.

Discussion

Pre-pandemic research suggested that more young people in the UK require access to mental health services, which include CAMHS Crisis Teams. Preliminary research on the immediate impact of COVID-19 on children’s mental health is mixed. Some studies have found little impact, for example, a significant decline in the volume of admissions for psychiatric emergencies in young people (Ferrando et al., 2020); no significant increases in self-harm admissions to emergency departments (McIntyre et al., 2020); a substantial decrease in the number of young people in the UK presenting to A&E with psychiatric symptoms and a reduction in psychiatric inpatient admissions (Ougrin, 2020); and no significant change in suicide rates during school closures (Isumi et al., 2020). Whereas some research does suggest that young people’s mental health has been negatively impacted by the COVID-19 pandemic, by an increase in hospital psychiatry presentations for the under 18s in both Ireland and the US (Leeb et al., 2020; McAndrew et al., 2020), and an increase in mental health difficulties and behavioural problems (Vizard et al., 2018; Waite et al., 2020). The year 2020 was unusual due to the COVID-19 pandemic and subsequent repercussions which included national lockdowns, educational disruptions and limited social interactions for young people and, therefore, it is difficult to make generalised conclusions. However, as referrals are made to the Crisis Team when a young person is experiencing a mental health crisis, this may be a more direct estimate of a young person’s distress as it does not require presenting to A&E or an inpatient admission during the COVID-19 pandemic.

This service evaluation was a replication of Staite et al.’s (2021) study which was the first, to the authors’ knowledge, to empirically examine the functioning (as measured by the routinely collected ORS) of young people following a mental health crisis and treatment from a CAMHS Crisis Team in the UK. This first service evaluation supplied baseline data which provide some context to discuss our current findings, namely, the impact of the COVID-19 pandemic on a young person’s functioning. The original service evaluation using 2018/2019 data showed promising evidence that a CAMHS Crisis Team positively impacts a young person’s functioning following a mental health crisis (Staite et al., 2021). The service evaluation reported that 16 (15%) patients exhibited significant and reliable change, 97 (89%) patients were co-worked by another CAMHS Team and there were 29 (26%) multiple referrals within 2018/2019.

Overall, the findings of this article support the original service evaluation, in that the results continue to show promising evidence that a CAMHS Crisis Team positively impacts a young person’s functioning following a mental health crisis in a global pandemic. A key finding was a 79.19% increase in mean ORS scores after treatment (compared to 16.2% in the original service evaluation). Secondly, 243 (58%) patients accessing the CAMHS Crisis Team during a global pandemic displayed significant and reliable change (i.e. improved functioning) compared to 15% in the original service evaluation. Despite a conservative analysis and high benchmarks used, over half of the young people not only significantly improved functioning but were able to return to a ‘healthy’ level of functioning following a mental health crisis during a global pandemic. A further 86 (20%) patients exhibited reliable improvement although this was not categorised as clinically significant using the adopted criteria. One explanation for the improvement in ORS scores after treatment, as well as the increase in clinically significant and reliable change, between the first service evaluation and current service evaluation could be the expansion of the team to include multiple disciplines (consultant psychiatrist and occupational therapist) rather than the service being solely nurse led as it had been before pandemic. The initial ORS scores in the group in the first study were similar to those presenting in the current study, indicating that the same cohort of patients were looked at, as opposed to a group who were only accessing the Crisis service due to a lack of access to other services as a consequence of COVID-19. Further qualitative research would need to be conducted to gain further insight into this.

The same percentage (89%) of patients were co-worked by another CAMHS Team as in the first service evaluation. This suggests the majority of young people accessing the CAMHS Crisis Team consistently have complex difficulties requiring additional support from multiple mental health teams. Despite this, the majority of patients managed to return to a ‘healthy’ level of functioning following a mental health crisis and intervention during a global pandemic, demonstrating the positive impact of the service. This is further supported by the fact that 90% of patients did not require more than one referral to the CAMHS Crisis Team, suggesting that one block of treatment from the service was sufficient. This is an increase from the first service evaluation where 74% had a single referral. Additionally, 90% patients were seen within 2 days of referral in the current service evaluation (compared to 87% in the first service evaluation (Staite et al., 2021)). This does not support assumptions that the current pandemic will worsen pre-existing mental health problems and unmet treatment needs (Fegert et al., 2020; Loades et al., 2020).

Although some COVID-19 research has found a significant decline in the volume of young people presenting with a psychiatric emergency (Ferrando et al., 2020), we found a very similar number of referrals in our 2018/2019 ‘pre-pandemic’ dataset (N = 1319) and our 2019/2020 ‘pandemic’ dataset (N = 1311) although there are more paired outcomes to analyse in the latter dataset. The reasons for this are unclear; however, possible explanations include increased engagement with clinicians which improved ORS completion rates or longer periods of follow up. The results suggest from December 2019 to December 2020, which captures the COVID-19 pandemic, there has not been a significant increase in referrals to the CAMHS Crisis Team, but there has been an increase in the number of young people who exhibit significant and reliable change, enabling them to return to a ‘healthy’ level of functioning. These results are promising at a time where young people are experiencing extreme disruptions to their routine and education through the government’s national lockdown restrictions (Cabinet Office, 2021).

Limitations

Similarly to the original service evaluation (Staite et al., 2021), there is a possibility that the observed effects may be skewed as completer analysis was used for this service evaluation (i.e. only paired datasets were included). Additionally, as a self-report measure, the ORS has limitations such as social desirability bias. Other limitations for using the ORS as the sole outcome metric include the fact that it is symptom report which ignores functional improvements. Additionally, it is a broad band rather than a narrow instrument and therefore does not yield more molecular, fine grained analysis. We also recognise this dataset includes 3 months (December 2019 to February 2020) of pre-pandemic data. Lastly, there are limitations on how far the results can be taken as generalisable as this was a service evaluation and not a research project, and the sample was skewed towards female adolescents. Contextual variables, such as ethnicity and socioeconomic status, were not included in the data analysis to ensure anonymity. The majority of young people in the service were of the same ethnicity (white British), and therefore having access to ethnicity would have made patient identification possible for the researchers.

Clinical Implications

This is the second service evaluation, to the authors’ knowledge, that empirically examines the functioning of young people following intervention from a CAMHS Crisis Team in the UK. It supports the baseline data provided from the first service evaluation (Staite et al., 2021) in highlighting how intervention(s) from a CAMHS Crisis Team can contribute to a young person’s functioning, enabling the majority of young people to return to a ‘healthy’ level of functioning following a mental health crisis – despite being in a global pandemic. However, improvements still need to be made to reduce the 20% of patients whose functioning did not significantly improve following treatment from a CAMHS Crisis Team. Further exploration is needed for this, as the global pandemic may increase the acuity of referrals.

Prior to COVID-19, the CAMHS Crisis Team linked in with the local community and third sector colleagues to attend events and services (e.g. youth clubs) to provide a visible presence in the hope of increasing access for young people from under-served populations. This will resume in the near future once lockdown restrictions have been lifted.

Future evaluation or research

As the first service evaluation, to the authors’ knowledge, that analyses outcomes from a CAMHS Crisis Team in the UK during the COVID-19 pandemic, this article provides baseline data for future research looking at the impact the pandemic has on children’s mental health. There is a need for larger scale research studies looking at CAMHS Crisis Teams throughout different areas of the UK. Further qualitative research with those young people whose functioning did not significantly improve could be explored to consider possible expansion of clinical support. Future research could also include more in-depth studies of mediating and moderating variables.