Health System-Level Evaluation of Tele-Mental Health Services Among Children and Adolescents in Ontario, Canada.

OBJECTIVE: To describe the characteristics of children and adolescents receiving tele-mental health services in Ontario, Canada and examine access to a psychiatrist, in-person or via tele-mental health services, following a mental health and addictions (MHA)-related emergency department (ED) visit or hospitalization.
METHOD: Using linked health and administrative data, we described two cohorts: (1) children and adolescents (1-18 years) who used a provincial tele-mental health programme from January 1, 2013 to March 31, 2017, comparing their MHA-related service use (outpatient, ED, hospitalization) in the 1 year prior to and the 1 year following initial consultation; (2) children and adolescents with high mental health service needs, defined as those with an incident MHA-related ED visit or hospitalization between January 1, 2013 and December 31, 2016, examining their 1-year follow-up with telemedicine and other health care utilization.
RESULTS: In the first cohort, 7,216 children and adolescents (mean age 11.8 [±3.8] years) received tele-mental health services. The proportion of MHA-related ED visits [15.1% pre vs. 12.6% post (test statistic 23.57, P < 0.001)] or hospitalizations [10.2% pre vs. 8.7% post (test statistic 11.96, P < 0.001)] declined in the year following tele-mental health consultation, while local psychiatry visits increased [8.4% pre vs. 17.0% post (test statistic 298.69, P < 0.001)]. In the second cohort (n = 84,033), only 1.5% received tele-mental health services, 40.7% saw a psychiatrist in-person, and 32.5% received no MHA-related outpatient care in follow-up.
CONCLUSIONS: Tele-mental health services were rarely used in Ontario, even among high-needs children and adolescents, despite their association with increased access to care and less need for acute mental health care.

Introduction

Mental illness in young people is characterized by substantial morbidity, mortality, and negative economic impact. Between 15% and 25% of Canadian children and adolescents suffer from a mental disorder, however, only about one in five of those in need actually receive psychiatric care.[1-7] The last 10 years have seen a large increase in hospitalizations and emergency department (ED) visits for mental illness in children and adolescents, and the gap between mental health care needs and available resources is growing. Reasons for this include a lack of health human resources trained to effectively deliver needed mental health care; archaic mental health service silos operating in parallel to usual healthcare; and inadequate availability of effective and appropriate child and youth mental health care at the primary care level.[3,9-11] Rural and remote communities specifically suffer from a severe shortage of child and youth mental health practitioners and resources.[12,13] As a result, tele-mental health services have been implemented to bridge these geographical disparities and improve access to psychiatric care.

The TeleLink Mental Health Programme (TLMHP), a paediatric telepsychiatry service based in a large Canadian inner-city, follows a consultation model where the tele-psychiatrist provides a formulation and immediate recommendations to the referring provider who maintains ongoing patient care. Its mandate is to provide specialist care to rural, remote, and underserviced areas. Services are delivered through child psychiatrists, social workers, nurse practitioners and psychologists, and are free of charge to patients ≤18 years. Referrals come from directly primary care physicians (PCPs), child and youth mental health community agencies and school boards. Referring parties are responsible for the implementation of recommendations, ongoing patient care, and facilitating further medical and pharmacological interventions.[14,15]

While the evidence base in the field of tele-mental health services demonstrates a high degree of practitioner satisfaction, enhanced capacity of practitioners and families in rural communities to manage mental health in their practices, and overall therapeutic success, few studies have evaluated the extent of utilization at a broad health system-level. Recent Canadian data suggest tele-mental health services are underutilized for adult psychiatric populations, and there is no clearly defined systematic rationale for the organization of its delivery to match patient needs. Population-level healthcare utilization has not been examined in children and youth in Canada and was the focus of this study.

With the recent and rapid shift from in-person to virtual modes of mental health care service delivery precipitated by the coronavirus-2019 (COVID-19) pandemic, assessing how tele-mental health services has been previously used may help guide strategies to mitigate barriers to mental health service access. This study aimed to describe the characteristics and healthcare utilization patterns of children and adolescents who received tele-mental health services in Ontario, Canada between 2013 and 2017 through a provincially funded tele-mental health programme. In addition, we examined access to a psychiatrist, either in-person or by tele-mental health services, within one year following a MHA-related ED visit or hospitalization. Individuals who have MHA-related ED visits or hospitalizations represent a population with high mental health needs who, on balance, would benefit from access to mental health services. Our first objective describes the status of tele-mental health services; the second measures the extent to which tele-mental health services are meeting the needs of children and adolescents with high demand for psychiatrist services.

Methods

Study Design and Cohort

This was a retrospective population-based pre- post- study of all children and adolescents (age 1–18 years) receiving tele-mental health services through TLMHP in Ontario, Canada from January 1, 2013 to March 31, 2017. In 2017, the children and adolescent (age 0–19 years) population in Ontario, Canada was 3,095,381. Using linked administrative health data available at ICES Institute for Clinical Evaluative Sciences (ICES), an independent, non-profit research institute that evaluates universal health care services in the province of Ontario, Canada (population approximately 14 million), we examined MHA-related service use (outpatient visits, ED use, hospitalizations) one year prior to and following initial tele-mental health consultation. We excluded those who did not have publicly funded health insurance coverage for at least one year prior to the date of the initial tele-mental health consult. Children and adolescents without a valid health card number were also excluded as their records are not linkable to other health administrative databases.

In addition, we created a second cohort of paediatric patients with high mental health care needs to contextualize utilization patterns. It included children and adolescents (age 1–18 years) who had an incident MHA-related ED visit or hospitalization between January 1, 2013 and December 31, 2016, defined as no prior MHA-related ED visit or hospitalization in the year prior. We assessed follow-up with a psychiatrist; in-person or through tele-mental health; within one year. In addition to the exclusions noted above, we also excluded children and adolescents with an Ontario Health Insurance Plan (OHIP) telemedicine claim (B100 or B200) with a mental health outpatient diagnosis code in the year prior to the incident MHA event, as this care was provided outside of the provincial TLMHP.

The use of data in this project was authorized under section 45 of Ontario's Personal Health Information Protection Act, which does not require review by a Research Ethics Board and informed consent. STROBE reporting guidelines were used.

Data Sources

Population-based health administrative and clinical databases at ICES were linked using unique encoded identifiers. The legal status of ICES under Ontario's health information privacy law allows the research institute to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement.

The TLMHP database includes data on children and adolescents who received mental health services through a provincial tele-mental health services programme. Data elements include referral characteristics as well as clinical data collected at the time of consultation, including major mental health concerns at referral, diagnoses, medication and dosages, counseling history, and recommendations. The TLMHP dataset was used to identify all children and youth that received tele-mental health services between January 1, 2013 and March 31, 2017, using their initial consultation as a start date.

The OHIP database was used to capture all outpatient physician visits except those recorded in the TLMHP dataset. The National Ambulatory Care Reporting System and the Discharge Abstract Database were used to identify MHA-related ED discharge data and hospital admissions, respectively, using the primary discharge diagnosis fields. Diagnostic codes were determined using the International Classification of Diseases, 10th Revision (Supplement Material). Hospital admissions to Ontario psychiatric inpatient facilities were obtained using the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I diagnostic field in the Ontario Mental Health Reporting System (Supplement Material).

The Registered Persons Database (RPDB) provides demographic characteristics for all Ontario residents eligible for public health insurance and was used to obtain age and sex. Postal codes were linked to the 2016 Canadian Census to obtain area-based measures of neighbourhood income quintile and rurality.

Measures

A tele-mental health visit was measured as the first recorded visit in the TLMHP dataset. MHA-related ED visits and hospitalizations were identified using the primary discharge diagnosis, defined as any code from the Mental and Behavioural Disorders Chapter of the International Classification of Diseases 10th edition. Conditions were categorized as substance-related, schizophrenia, mood and affective disorders, anxiety disorders, and deliberate self-harm. Conditions including attention deficient hyperactivity disorder, conduct disorder and oppositional defiant disorder were all captured under the overall “any MHA diagnosis” category and not examined under separate strata like we did for other mental health conditions.

MHA-related outpatient visits to paediatricians and family physicians (FP) were determined using OHIP diagnostic and specialist codes based on a previously validated administrative data algorithm (Supplement Material), while all psychiatry visits were considered MHA-related. Tele-mental health consultations were defined as any record with a valid appointment start date in the TLMHP dataset.

The outcome for the high-needs cohort was a follow-up by a psychiatrist in the year following an MHA-related incident ED visit or hospitalization. We created mutually exclusive categories of outpatient follow-up to define appointment type, using the following hierarchy: (1) tele-mental health consultation and in-person psychiatry visit; (2) tele-mental health consultation only; (3) in-person psychiatry visit only; (4) MHA-related FP or paediatric follow-up visit; and (5) no MHA-related follow-up visit.

Data Analysis

Using the initial consultation date in the TLMHP database as the index for the tele-mental health cohort, we examined baseline demographic characteristics of children and adolescents that received a tele-mental health service consultation. MHA-related ED visits, hospitalizations, and outpatient visits were compared in the year prior and following the index tele-mental health consultation. Pre- and post-differences were tested using paired t-tests and McNemar's tests for continuous and binary variables, respectively.

For the second high-needs cohort, the date of the MHA-related event was used to define the index event. We created mutually exclusive categories to identify children and adolescents with an incident (no MHA-related event in the year prior) MHA-related ED visit that resulted in a discharge home, while those who were hospitalized, whether preceded by a transfer from the ED or not, were categorized as having an incident MHA-related hospitalization. MHA-related hospitalizations also included elective hospitalizations (i.e. not preceded by an ED visit). We then looked forward 1 year for MHA-related follow-up in an outpatient setting stratified by the type of MHA-related index event. Statistical analyses were performed using SAS 9.4

Results

TLMHP Cohort

Between January 1, 2013 and March 31, 2017, 7,216 children and adolescents had a tele-mental health consultation (Table 1). The mean age of tele-mental health patients was 11.8 ± 3.8 years; 6.1% were age one to five years; 43.2% were between six to 12 years; and 50.7% were between the ages of 13 and 18 years. Among all patients that received a tele-mental health consultation, slightly more than half were male (52.9%); with slightly higher representation from the lowest neighbourhood income quintile (24.6%). More tele-mental health consultations were for patients living in urban versus rural settings; 56.8% and 42.7%, respectively.

Table 1.

Baseline Characteristics of Youth That Received a Tele-Mental Health Consultation Between January 1, 2013 and March 31, 2017 (N = 7,216).

Characteristic
Unique youth that received a tele-mental health consultation, N	7,216
	Mean	SD
Age, years	11.82	3.81
Age group	N	%
1–5 years	438	6.1
6–12 years	3,117	43.2
13–18 years	3,661	50.7
Sex
Female	3,400	47.1
Male	3,816	52.9
Neighbourhood income quintile
Missing	61	0.8
Q1 (lowest)	1,778	24.6
Q2	1,580	21.9
Q3	1,451	20.1
Q4	1,230	17.0
Q5 (highest)	1,116	15.5
Rurality
Missing	31	0.4
Urban	4,101	56.8
Rural	3,084	42.7

SD: standard deviation.

In the year prior to receiving tele-mental health services, approximately 15.1% of children and adolescents had at least one MHA-related ED visit (Table 2). Overall, the proportion of individuals with an ED visit declined following an initial tele-mental health consultation, particularly for anxiety-related disorders [6.7%–5.1% (test statistic 20.42, P < 0.001)] and mood disorders [3.8%–3.2%, (test statistic 4.12, P=0.04)], which were among the most common reasons for ED visits. Approximately 10.2% had a MHA-related hospitalization prior to tele-mental health consultation, among which mood disorder was the most common reason for admission, compared with 8.7% following consultation (test statistic 11.96, P < 0.001).

Table 2.

Mental Health and Addictions-Related Health Service Use, Prior to and Following Initial Tele-Mental Health Consultation (N = 7,216).

Emergency department visits	Pre-TeleLink a		Post-TeleLink b		Test statistic b	P-value
	Mean	SD	Mean	SD
No. MHA-related ED visits	0.21	0.70	0.22	0.84	−0.11	0.913
	N	%	N	%	Test statistic	P-value
Any MHA-related ED visit	1,089	15.1	911	12.6	23.57	<0.001
Any substance-related ED visit	83	1.2	111	1.5	4.45	0.035
Any schizophrenia-related ED visit	19	0.3	30	0.4	3.10	0.078
Any mood-related ED visit	274	3.8	231	3.2	4.12	0.042
Any anxiety-related ED visit	487	6.7	366	5.1	20.42	<0.001
Any deliberate self-harm related ED visit	128	1.8	151	2.1	2.35	0.125
Hospitalizations	Mean	SD	Mean	SD	Test statistic	P-value
No. MHA-related hospitalizations	0.13	0.43	0.13	0.54	−1.02	0.309
	N	%	N	%	Test statistic	P-value
Any MHA-related hospitalization	733	10.2	626	8.7	11.96	<0.001
Any substance-related hospitalization	22	0.3	43	0.6	7.47	0.006
Any schizophrenia-related hospitalization	25	0.3	35	0.5	2.08	0.149
Any mood-related hospitalization	334	4.6	231	3.2	23.73	<0.001
Any anxiety-related hospitalization	190	2.6	171	2.4	1.16	0.281
Any deliberate self-harm related hospitalization	67	0.9	44	0.6	5.04	0.025
Outpatient visits	Mean	SD	Mean	SD	Test statistic	P-value
No. psychiatry visits	0.19	0.96	0.47	1.70	−13.40	<0.001
No. GP/FP visits	1.03	1.70	1.35	2.22	−12.35	<0.001
No. paediatric visits	0.85	1.72	1.02	1.90	−8.37	<0.001
	N	%	N	%	Test statistic	P-value
Any psychiatry visit	603	8.4	1,229	17.0	298.69	<0.001
Any GP/FP visit	3,268	45.3	3,294	45.6	0.30	0.585
Any paediatric visit	2,202	30.5	2,407	33.4	29.95	<0.001

ED: emergency department; FP: family physician; GP: general practitioner; MHA: mental health and addictions; SD: standard deviation.

TeleLink mental health programme.

Paired t-test/McNemar test.

The proportion of children and adolescents who had a psychiatrist visit nearly doubled [8.4%–17.0% (test statistic 298.69, P < 0.001)] in the year following tele-mental health consultation. There was a small increase in the proportion with a MHA-related visit to a paediatrician [30.5%–33.4% (test statistic 29.95, P < 0.001)] while no differences were seen for MHA-related visits to family physicians.

High Mental Health Care Needs Cohort

Among 84,033 children and adolescents with an incident MHA-related event between January 1, 2013 and December 31, 2016, 78.1% (N = 65,669) were ED visits and 21.8% (N = 18,364) were hospitalizations (Table 3). The mean age of children and adolescents in the high-needs cohort was 14.9 years ± 3.1. There was a higher proportion of females (58.5%) and youth from urban settings (85.2%).

Table 3.

Baseline Characteristics of Children and Youth in Ontario, Who Had an Incident MHA-Related Emergency Department Visit or Hospitalization Between January 1, 2013 and December 31, 2016 (N = 84,033).

	Incident MHA event				Total		P-value
	ED visit		Hospitalization
Children and youth with an incident MHA-related ED visit, N	65,669		18,364		84,033
	Mean	SD	Mean	SD	Mean	SD
Age, years	14.81	3.27	15.18	2.39	14.89	3.10	<0.001
	N	%	N	%	N	%
Age group
1–5 y	1,530	2.3	95	0.5	1,625	1.9	<.001
6–12 y	9,915	15.1	1,879	10.2	11,794	14.0
13–18 y	54,224	82.6	16,390	89.3	70,614	84.0
Sex
Missing	103	0.2	108	0.6	211	0.3	<.001
Female	37,493	57.1	11,696	63.7	49,189	58.5
Male	28,073	42.7	6,560	35.7	34,633	41.2
Neighbourhood income quintile
Missing	511	0.8	131	0.7	642	0.8	0.125
Q1 (lowest)	14,278	21.7	4,115	22.4	18,393	21.9
Q2	12,766	19.4	3,512	19.1	16,278	19.4
Q3	12,658	19.3	3,626	19.7	16,284	19.4
Q4	12,681	19.3	3,446	18.8	16,127	19.2
Q5 (highest)	12,775)	19.5	3,534	19.2	16,309	19.4
Rurality
Missing	380	0.6	63	0.3	443	0.5	<.001
Urban	55,581	84.6	16,003	87.1	71,584	85.2
Rural	9,708	14.8	2,298	12.5	12,006	14.3

ED: emergency department; MHA: mental health and addictions; SD: standard deviation.

Only 1.5% (N = 1,241) of children and adolescents with high mental health care needs received a tele-mental health consultation within one year following their incident MHA-related event (Table 4). Overall, 40.7% received care by a local psychiatrist in follow-up only; compared to 25.3% who only saw a FP or paediatrician. Approximately one-third (32.5%) of youth identified as having high mental health care needs had no MHA-related follow-up within one year. Children and adolescents with an incident MHA-related hospitalization were more likely than those with an incident ED visit to have a follow-up with a local psychiatrist (59.0% vs. 35.5%). Conversely, the proportion of children and adolescents that received MHA follow-up from family physicians or paediatricians, or had no follow-up, was higher among those with an incident ED visit compared to those hospitalized.

Table 4.

MHA-Related Follow-up Within 1 Year of Incident MHA-Related ED Visit or Hospitalization.

	Incident MHA-related event				Total		P-value
	ED visit		Hospitalization
Children and youth with an incident MHA-related ED visit, N	65,669		18,364		84,033
	N	%	N	%	N	%
Any tele-mental health services	916	1.4	325	1.8	1,241	1.5	<0.001
Access to psychiatry (mutually exclusive)
Local psychiatry and tele-mental health service	320	0.5	143	0.8	463	0.6	<0.001
Local psychiatry only	23,336	35.5	10,833	59.0	34,169	40.7
TeleLink consultation only	596	0.9	182	1.0	778	0.9
GP/FP or paediatric MHA visit only	17,105	26.0	4,179	22.8	21,284	25.3
No MHA-related outpatient care	24,312	37.0	3,027	16.5	27,339	32.5

ED: emergency department; FP: family physician; GP: general practitioner; MHA: mental health and addictions.

Discussion

The key findings emerging from our health system-level evaluation indicate that provincially delivered tele-mental health services are largely utilized by younger youth (mean age 11.8 ± 3.8 years) with a significant proportion from rural areas (42.7%), reflecting areas with low psychiatrist supply and need for access to care through virtual means. In the year following initial tele-mental health consultation, a modest decrease in acute mental health care service use and improved access to psychiatrists was observed in our cohort. Among children and adolescents with high mental health care needs, only 1.5% accessed care through tele-mental health services and 32.5% received no mental health care at all in the year following an incident MHA-related ED visit or hospitalization. These findings indicate that although tele-mental health care is meeting its mandate to deliver care to a number of children and adolescents across the province including rural settings, the volume of service is relatively low and it is not being used to support access to psychiatrist care for high-needs children and youth.

While tele-mental health services have been shown to be similarly effective as in-person care,[20,21] there are few macrosystem-level evaluations of utilization and accessibility. Findings from our study highlight opportunities for enhancing tele-mental health care and systematically aligning services with the needs of children and adolescents. The TLMHP is only one of several routes available to support children's and adolescents’ mental health in the province of Ontario. The programme's role is to enhance on-the-ground, local care – not be a substitute for it – and support primary care providers in the community, especially those in rural, remote and indigenous communities. While the programme does provide some urgent care consultations, it is not a front line, urgent care programme. The predominant reasons for ED visits and hospitalizations in the TLMHP cohort were for mood and anxiety disorders. This pattern is consistent with previously reported findings.[8,22] There is, unfortunately, a significant gap between the ED and community mental health follow-up for many children and adolescents presenting with acute mental health care needs. Unfortunately, this causes a barrier to accessing tele-mental health care services as many children and adolescents are not connecting with their local mental health centre following an acute care visit. The TLMHP programme does not accept referrals directly from the ED. In the future, this could be a potential area of focus for improvement initiatives.

As the COVID-19 pandemic evolves, access to mental health care for children and adolescents will become increasingly important and will stimulate the field to utilize innovations in mental health care delivery during the crisis and beyond.[24,25] While physical distancing measures have mandated the rapid uptake of tele-mental health services use during the pandemic,[26,27] an iterative approach is warranted for its implementation and evaluation with broad system-level considerations in order to ensure access matches the current and changing needs of the population.

Furthermore, child and youth mental health services have long been under-resourced, with many accessing the mental health care system for the first time in the ED. Risk factors for first contact mental health ED visit include social deprivation and living in a rural region. This partly reflects limited access to both specialist and primary care for mental health issues. The pandemic may further exacerbate these access issues and, thereby, hinder the detection of symptoms consistent with mental health disorders.

Our study fills a significant gap in the literature on health system-level evaluation of a child and adolescent tele-mental health programme by using administrative datasets to provide insight into a large population of children and adolescents with high mental health care needs. However, there are a number of inherent limitations including potential coding errors, deficiency of supporting clinical data, and lack of data on community mental health services provided by other clinicians including psychologists and social workers. Moreover, comparison of health service utilization prior to and following a tele-mental health contact shows associations between receipt of tele-mental health services and changes in acute mental health service use, but does not imply causation between tele-mental health services access and changes in acute mental health service use.

Conclusion

Tele-mental health services have been expanding in the past decade to improve access to psychiatric care and address critical shortages in the psychiatric workforce. However, our study indicates that these virtual services are vastly underutilized at a population-level in Ontario, Canada, especially among children and adolescents with high mental health care needs. In the context of growing mental health care demand, an under-resourced system with limited access to specialist care, and a need to physically distance during the COVID-19 pandemic, we must ensure tele-mental health care better aligns with patient need and does not further widen existing health care inequities. Tele-mental health services will continue to be a critical strategy for mental health care access and service delivery during and after the pandemic to support children and adolescents with pre-existing mental health problems, as well as those with new mental health care needs. Health system-level planning is needed for more effective implementation to optimize access to psychiatrist care for those who need it most.

Click here for additional data file.

Supplemental material, sj-docx-1-cpa-10.1177_07067437211043395 for Health System-Level Evaluation of Tele-Mental Health Services Among Children and Adolescents in Ontario, Canada by Alene Toulany, Paul Kurdyak, Sima Gandhi, Longdi Fu, Seena Grewal, Chetana Kulkarni, Natasha Saunders, Simone Vigod, Astrid Guttmann, Maria Chiu and Antonio Pignatiello in The Canadian Journal of Psychiatry

Click here for additional data file.

Supplemental material, sj-docx-2-cpa-10.1177_07067437211043395 for Health System-Level Evaluation of Tele-Mental Health Services Among Children and Adolescents in Ontario, Canada by Alene Toulany, Paul Kurdyak, Sima Gandhi, Longdi Fu, Seena Grewal, Chetana Kulkarni, Natasha Saunders, Simone Vigod, Astrid Guttmann, Maria Chiu and Antonio Pignatiello in The Canadian Journal of Psychiatry