Let's Talk about Children Evaluation (LTCE) study in northern Finland: a multiple group ecological study of children's health promotion activities with a municipal and time-trend design.

INTRODUCTION: Making change towards child and family-based and coordinated services is critical to improve quality, outcomes and value. The Let's Talk about Children (LTC) approach, which consists of brief psychoeducational discussions with parents of kindergarten-aged and school-aged children, has been launched as a municipality-specific programme in the Council of Oulu Region. The aim of this paper is to present a protocol of an ecological study evaluating the group-specific effects of an intervention about LTC activities in a geographically defined population. The programme is designed to promote children's socioemotional well-being. METHODS AND ANALYSIS: A quasi-experimental ecological study protocol is implemented to evaluate whether systematic LTC practices improve children's well-being. A multi-informant setting covers 30 municipalities in northern Finland and involves all the municipal teachers, social and healthcare workers. In each municipality, a Local Management Team is responsible for implementing the LTC programme and collecting the annual data of LTC discussions and network meetings. The outcome data are retrieved from child welfare statistics and hospital registers. The population data, child welfare statistics and referrals to hospitals was retrieved at baseline (2014), and will be retrieved annually. Furthermore, the annual data of LTC discussions and network meetings will be collected of the years 2015-2018. ETHICS AND DISSEMINATION: The study design has been approved by the management of the Oulu University Hospital in accordance with the guidelines given by The Regional Ethics Committee of the Northern Ostrobothnia Hospital District in Oulu, Finland. All data are treated and implemented according to national data security laws. Study findings will be disseminated to provincial and municipal partners, collaborative community groups and the research and development community. The Let's Talk about Children Evaluation study databases will guide future regional development action and policies. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The Let’s Talk about Children Evaluation study’s protocol is suitable when intervention is implemented annually in real-life municipalities. By creating a regional database for the Let’s Talk about Children (LTC) results, we can sample over 100 000 children and youths. The merit of such large-scale evaluations is high.

The LTC approach seeks to better link community services provided to families and to assist families in accessing the support they need.

Adequately measuring individual-level LTC intervention results is difficult, because the framework of strengths and vulnerabilities applies to both the capacity of individuals and their social and physical ecologies.

Introduction

Children’s mental health problems disrupt healthy development and are among the leading causes of child and youth disability.1 Opportunities exist, however, to improve mental capital through interventions: we can try to build the cognitive and emotional resources that influence how an individual is able to experience a high quality of life. Some problems nevertheless extend beyond the individual to family members and communities. Thus, system-wide changes and a settings approach are needed.1 2 The emergence of such an approach has been attributed to the Ottawa Charters (WHO, 1986) statement which claims that health is created and lived by people within the settings of their everyday life. More research into system interventions is needed to produce an evidence base to transform child and family services.

Many evidence-based interventions fail to take an ecological perspective for achieving a population-level impact. Children’s problems may develop because of problems within their families or communities, and may include parental mental health problems, food and housing insecurity or exposure to dangerous neighbourhoods or challenging schools.3 Intervention may be more effective when directed at the underlying issues and designed to fit the work flow and staffing of local child and family services.3 Integration efforts in Finland involve work with school-based and public health services. Health promotion activities are delivered in the context of co-occurring service conditions, and with a focus on both individual and family strengths and vulnerabilities.

A strategic approach to health promotion consists of both managerial and operational actions. This enables strong systems in local child and family services and universal delivery. Service systems are currently often expensive in terms of duplication, inefficiency and high procurement costs.4 The challenges to developing child and family services include the fragmentation of services, inter-sector borders, different work cultures and data transmission difficulties. Furthermore, families have a wide diversity of needs and varying degrees of access to assistance and advice.5

In planning the present development programme, we used the following framework to conceptualise the integration of child and family services. Activity integration involves the joint provision of intervention by different sectors of services and joint training sessions for professionals. At the regional level, policy integration includes the development of a harmonised incentive structure for operationalising actions, and the formation of a new partnership for municipal-based delivery of child and family services. Furthermore, capacity building strengthens the evaluation of health promotion activities among municipal child and family services.

The present development programme in the Council of Oulu Region is based on elements of brief psychoeducational discussions with parents (Let’s Talk about Children, LTC).5 LTC was developed for the Effective Family Programme, which provided methods for health and social services to support families and children of mentally ill parents. LTC has earlier been tested in mental health clinics to fit the real-life settings of multiprofessional child and family services.5–7 While the LTC development work was initiated in psychiatric services, the present programme will extend the LTC approach to all municipal child and family services.

According to Finnish School Health Promotion study, most children are happy with their lives.8 Not all of secondary school-aged children, however, get the best possible start in life: almost 10% of children live in jobless households, 4% see their parents drunk on a weekly basis, 7% report being bullied weekly at school and 8% have no close friends.8 In Finland, 6% of children aged 0–17 are subject to a child welfare notification, 5% take part in a child welfare intervention and 1% have placements outside the home.

The objective of this paper is to describe the protocol of a quasi-experimental ecological study with a municipal group and multiple time-series design. We will evaluate the effectiveness of a community-level intervention across all public child and family services in 30 municipalities in the Council of Oulu Region to promote the socioemotional well-being of children. The intervention will identify children’s needs and provide them with support in a broad spectrum of arenas in which it may be effective. The intervention is expected to produce positive aspects of child development and is impacted through changes to the children’s nearest social and physical ecologies. We intend to assess the associations between change in annual coverage of LTC actions and change in different outcome measures among different age groups residing in both urban and rural municipalities. The intervention is expected to improve children’s well-being after each municipality implements the LTC activities in all child and family services. We also anticipated changes in incidents of emergency placement or taking a child into care, and in incidents of new referrals to child or adolescent psychiatric units.

Methods and analysis

Design

The present study is an ongoing quasi-experimental ecological study, which is conducted in a naturalistic setting of public child and family services in the Council of Oulu Region, Finland. The study focuses on the population under the age of 18. We assess the ecological association between the average LTC intervention activity and the aggregated measurement of the rate of adverse incidents among multiple groups of children. The data sources used involve observations of multiple groups based on place and time (figure 1). The evaluation consists of data collection at five time points (baseline and four other time points).

Figure 1

Design of present study. Blocks of municipalities represent groups of intervention areas. Each time point (T0, T1, T2, T3 or T4) represents a data collection point. Each unit (control or intervention) represents one time period for one block of municipalities.

The rollout of intervention across municipalities is assigned without using randomisation. In each intervention area, the selection and timing is based on the agreement between the municipal town manager and the Council of Oulu Region.

Since 2012, LTC services have been applied in some municipalities. The training of service suppliers in LTC intervention was first carried out in the town of Raahe. During the study, the LTC intervention will gradually be conducted in up to 30 municipalities (see the Participants section), and its activities will branch out year by year (figure 2). The intervention will be fully implemented by the end of the study and all municipalities will be involved.

Figure 2

Data collection stages for the Let’s Talk about Children Evaluation study.

Participants

The ‘Change now. Let’s talk about the children in the Council of Oulu Region’ programme is coordinated by the Northern Ostrobothnia Hospital District. The programme was planned at both regional and local levels within regional offices, local public sector offices, non-governmental organisations and other regional partners. The programme coordinators guide the activities at the local level. At this local level, the implementers, including municipal employees and relevant community-based partners are all integrated. Interventions are carried out during early childhood education in primary schools, in secondary schools and in all health and social services.

The evaluation study is conducted by the Oulu University Hospital, in close liaison with the University of Oulu and the University of Lapland. The Oulu University Hospital received approximately €250 000 funding for the implementation of LTC services from member municipalities of the Northern Ostrobothnia Hospital District and the Council of Oulu Region (European Regional Development Fund). The hospital district consists of 30 distinct municipalities, of which 3 are urban (Kempele, Oulu and Raahe), 5 are rural close to urban areas (Ii, Liminka, Lumijoki, Siikajoki and Tyrnävä), 10 are core rural (Haapajärvi, Haapavesi, Kalajoki, Muhos, Nivala, Oulainen, Pyhäjoki, Reisjärvi, Sievi and Ylivieska) and 12 are sparsely populated rural (Alavieska, Hailuoto, Kuusamo, Kärsämäki, Merijärvi, Pudasjärvi, Pyhäjärvi, Pyhäntä, Siikalatva, Taivalkoski, Utajärvi and Vaala). The classification of municipalities will be described further in the context of statistical analyses.

Multiple children groups by citizenship

The complete numbers of children and young people are retrieved from the Population Statistics (Statistics Finland) and are sorted by the municipalities of the Council of Oulu Region. Population data by age group reveal the municipality's permanent resident population in each age group on the last day of the year. The multiple age groups evaluated in the study are 0–6 and 7–17 years. Table 1 presents a description of the groups and measures used in data collection.

Table 1

Description of study population and measures used in data collection, 2013

Area	Population*		Child welfare actions			Home help¶	Psychiatric referrals
			Notification†	Intervention‡	Placement§
	aged 0–9 (n)	10–19 (n)	aged 0–17 (n)	0–17 (n)	0–17 (n)		0–17 (n)	13–22 (n)
Alavieska	348	406	65	48	7	0		5
Haapajärvi	1020	1106	132	190	21	20	8	12
Haapavesi	976	1065	77	108	18	0	3	4
Hailuoto	80	87	19	11	0	0	1
Ii	1584	1311	161	209	7	19	11	5
Kalajoki	1647	1605	125	66	21	8	1	1
Kempele	2852	2496	428	264	32	54	19	9
Kuusamo	1649	1887	137	233	49	89	3	7
Kärsämäki	323	367	47	97	12	9	2	1
Liminka	2351	1708	248	120	10	31	11	7
Lumijoki	435	324	46	126	14		3
Merijärvi	178	179	13	8	0	0
Muhos	1550	1361	165	127	20	52	8	9
Nivala	1803	1606	284	275	34	0	6	6
Oulainen	999	1078	164	106	19	0	6	3
Oulu	26 476	23 337	4822	3719	672	860	110	105
Pudasjärvi	890	1036	103	121	21	0	4	3
Pyhäjoki	408	406	64	36	6	7
Pyhäjärvi	574	587	107	123	16	6	3	2
Pyhäntä	211	234	25	7	0	4
Raahe	3354	3183	570	321	80	60	19	12
Reisjärvi	380	404	76	59	8	9
Sievi	952	911	122	132	13	0	1	3
Siikajoki	848	790	136	83	20	10	8	3
Siikalatva	613	677	62	74	10	0	4	5
Taivalkoski	438	552	46	40	17	0	2	3
Tyrnävä	1562	1076	156	139	9	1	3	6
Utajärvi	319	360	48	82	10	0	5	2
Vaala	254	365	117	59	17	22	7	6
Ylivieska	2182	1869	290	326	39	0	7	12
Total	57 256	52 373	8855	7309	1182	1270	258	239

*Population by age group, year-end total.

†Number of child welfare notifications.

‡Number of child welfare interventions and in community care.

§Number of placements outside the home.

¶Home help, recipient families with children.

The study began in January 2014 and will be completed in December 2018. The different stages of data collection regarding both, summaries of observations derived from individuals in groups identified by citizenship (multigroup design) and by time (time-trend design) and data retrieved the national statistics (a Gantt chart shown in figure 3). Based on data by place and time, the interpretation of estimated effects is enhanced because two types of comparisons are made simultaneously: change overt ime within age groups and differences among municipalities with different phase of implementing the LTC activities in all municipal child and family services.

Figure 3

Gantt diagram of the Let’s Talk about Children Evaluation study. LTC, Let’s Talk about Children.

Intervention across all the municipal child and family services

The primary objective of the LTC intervention is to improve socioemotional well-being among children and adolescents. We aim to promote health by making changes to modifiable risk conditions and supportive changes through attributes that help children grow and develop successfully. At a system level, the LTC development programme also devotes resources to changing the regional child and family health promotion delivery system, building the capacity of this system to maintain LTC activities and to catching risk conditions before they get worse. A regional steering committee has been established by the Council of Oulu Region. The committee functions as ensuring coordinated actions and development. The committee will deliberate on regional establishment issues and challenges, engage with stakeholders, review and comment on progress made and exchange of experience between the municipalities. The steering committee will provide resources and support for initiating the local activities, ultimately aiming to make LTC discussions and network meetings self-sustaining overtime.

At the local level, the intervention is implemented and coordinated by a Local Management Team (LMT). The LMT includes managers of early childhood education, primary schools, health services and social services. Each LMT is responsible for implementing the LTC approach in the child and family services of the municipality. Each must have a different LMT due to the large number of service units (kindergartens, schools, health and social care centres), and because the units are spread across the whole county. No one LMT is big enough to cover the whole county.

Each municipality has a 4-month ‘municipal engagement phase’ prior to the local trainers’ training (figure 2). During this phase, the LMT engages in the LTC intervention activities with the elected member structures, schools and other units. The trainers are then trained in LTC discussion and network meetings. These training sessions are subsidised by programme resources and are free for the municipalities and for the participants. The LMT coordinates trainers’ training by finding participants having an avid interest in the subject. The LMT also advertises the LTC activities in local media (eg, newspapers, posters and leaflets). The local trainers then deliver LTC discussion training sessions to their colleagues in the municipal child and family services units.

Each LMT is given strategic support, as well as a clear framework and timescales for the assessment of the local activities and outcomes. The Oulu University Hospital annually coordinates the intervention activities led by the LMT, that consists of local managers. Furthermore, appropriate specialist support and mentoring is provided to help the LMT in collecting data and sustaining the local units in their LTC intervention activities. All the LMTs are responsible for the implementation and continuity of the universal LTC activities with parents.

All parents are offered the opportunity to take part in an LTC discussion and are shown the structure of the discussion themes. If they agree, they make an appointment with the professional. At the family level, the LTC practices includes universal LTC discussion for parents and an LTC network meeting for parents who need support.

LTC practices for children’s health promotion

LTC discussion

This parent-focused method has been described earlier.9 Briefly, the aim of the LTC discussion is to help parents recognise their children’s strengths and vulnerabilities and to inform them of ways in which to support their children, despite possible family problems. The LTC discussion consists of one or two sessions with the parent or both parents. The child’s own teacher, nurse or social worker is also present. The LTC manuals are available in Finnish on the internet for use in early childhood education, primary school, secondary school and all health and social services.9

When parents bring up a problem that the family is currently facing; for example, poverty, unemployment and housing problems, a further network meeting is offered. If the parents accept, the meeting is organised with the partners that are expected to be able to help the family.

LTC network meeting

The LTC network meeting is designed to respond to the different needs of the family on an ecological base. Depending on these needs, service representatives in addition to the family’s own network are invited to the meeting. The aim of the LTC network meeting is to activate all participants to provide the child and the family with support. The meeting offers a joint action forum for the aid and social support required.10

LTC activities must be measured in order to be implemented. In each municipality, the intervention includes measures on the operational data of LTC discussions and LTC network meetings. From 2015 onwards, aggregated counts of LTC discussions and network meetings have been obtained from each municipality of the Council of Oulu Region. We use a time-series design to compare different LTC intervention intensity among the age groups of 0–6 years and 7–17 years.

Outcome measurement

We assess children’s adverse outcomes on the basis of geographically and temporally defined populations. Outcome measurements are averaged for the populations in each geographical or temporal unit and then compared using standard statistical methods. The primary analysis compares the proportion of children referred to child welfare services through LTC activities of different intensities. The proportion of children referred to psychiatric clinics is also analysed. Secondary outcomes are the proportion of children who report being in moderate or poor health, experience inadequate parenting, have moderate or severe anxiety, have difficulties talking to their parents and feel that teachers are not interested in how they are doing.

Data sources

The follow-up of child and family service development includes items on commitment made by local organisations, local management groups, trainers’ training and local training of personnel. Each municipality has a contract in to the LTC approach together with the Council of Oulu Region. Programme maintenance and sustainability includes items on how LTC activities are adopted into the regular activity of all child and family services, and how LTC activities are maintained by local organisations. Problem solving ability includes how to tackle children’s health issues as an activity of its own, and how participants work together to plan actions.

Online questionnaires are sent out by email to each local management group once a year, in January, and the respondents fill in the questionnaire themselves. Additional requests are made until data from each participating municipality are received. Then the intervention phase (control phase, training and implementing phase and constant actions) is recorded for each municipality at baseline and after each of the four following intervention periods, that is, calendar year.

In Finland, municipalities are obliged to collect and report child welfare data. Each year, they receive a data request form that they must fill in within a specified time. They are instructed to log onto the Lasu-Netti website and check the provided list of current placements, record any changes to placements and record data on support interventions in community care. The child welfare data are retrieved in electronic form and updated on the national database. Child welfare statistics have been compiled and processed by the National Institute for Health and Welfare since 1991.

In Finland, the nationwide School Health Promotion study monitors the health and well-being of Finnish adolescents. The School Health Promotion study is carried out every other year in March/April. Since 1996, the study has surveyed eighth and ninth graders (ages 14–15) in comprehensive schools. For example, the questionnaire includes the measures that are shown in table 2.

Table 2

2013 School Health Promotion study survey of eighth and ninth graders in comprehensive schools in the municipalities of the Council of Oulu Region

Measure	2013%
Inadequate parenting (ind. 284)	17.6
In moderate or poor health (ind. 286)	16.2
Moderate or severe anxiety (ind. 328)	11.1
Difficulties talking to parents (ind. 329)	8.6
Teachers not interested in how pupil is doing (ind. 355)	58.0

The number represents the proportion of children who experienced the problem in relation to all those who responded to the survey (Sotkanet, Institute for Health and Welfare).

The data are gathered via an anonymous and voluntary classroom-administered questionnaire. Although the questionnaire is continuously being developed, most of the questions have remained the same for almost 20 years, to maintain comparability.

Child welfare notifications

The measures provides the numbers of child welfare notifications filed during a calendar year. The notification is filed in the municipal unit responsible for civic social services. According to the Finnish Child Welfare Act, a child welfare notification is when someone observes or reports circumstances relating to the care and upbringing of a child that may require an assessment regarding the need for child welfare measures. This can be made over phone, in writing or by visiting the municipal office in person.

Child welfare interventions in community care

The measures also depict the numbers of children receiving support via a community-based child welfare intervention during a calendar year: the community care support intervention comprises support for a child’s accommodation, livelihood, school attendance and hobbies. The measure also includes children receiving community care support before a placement. Before a child is placed outside the home, any opportunities for the child to live with relatives or other people they are close to must be investigated.

Placements outside the home

Finally, the measures show the numbers of children who have been placed outside the home during a calendar year. They include counts of children being placed outside the home through a child welfare intervention in community care, counts of emergency placements, counts of children taken into care involuntarily, and counts of children who receive after-care outside the home. The causes behind these placements may be related to parents or to the children themselves. Substance abuse is often an underlying factor.

Referrals to child and adolescent psychiatric clinics

Oulu University Hospital provides treatment for children in the Northern Ostrobothnia Hospital District and in northern Finland. Hospital referrals are made with the intention that the patient will be assessed and treated before responsibility is transferred back to the referring health professional or general practitioner. Any hospital that receives referrals will need to obtain information on these patients.

LTC discussion and network meeting data

In each local unit, each professional who is offering an LTC discussion keeps a record of the LTC discussions held. Furthermore, each professional who convenes an LTC network meeting also keeps a record. After each calendar year, all the LMTs that have passed the community engagement phase are contacted by email and online survey form. The aggregated counts of LTC services are reported for each municipal child and family service sector. Data collection are coordinated by the office of the Primary Health Care Unit of the Northern Ostrobothnia Hospital District.

Effect-measure modification

Home help rate, recipient families with children

This is the measure of the annual rate of families with children that receive home help from municipal welfare services. The denominator is the number of households with at least one person aged under 18 in the same year (Statistics Finland). Home help includes, for example, assistance with activities related to personal care, child care and other daily family activities. The data cover the services funded by municipalities, that is, services that the municipality has produced or paid for. Services funded by the clients themselves are not included.

LTC intervention rate

This is the measure of the annual rate of children who attend an LTC discussion in municipal child and family services. The denominator is the number of children in the same year.

Classification of municipalities

The areal division by Finnish Area Research11 reveals the differences in socioeconomic and endogenous development factors of the municipalities. It divides municipalities into four categories: urban areas (cities and towns), rural municipalities close to urban areas, core rural municipalities and sparsely populated rural municipalities. ‘Urban areas’ are those that form centres of high economic importance. In ‘rural municipalities close to urban areas’, residents have the option of working in nearby towns and cities with highly diverse businesses. These economically integrated rural municipalities have a high level of welfare. ‘Core rural municipalities’ are situated close to several medium-to-large centres, and most of the villages they contain are economically viable. ‘Sparsely populated rural municipalities’ are characterised by long distances from municipal centres and have rapidly declining populations. These municipalities face the threat of a cycle of poor development: not enough new jobs are available to replace the traditional jobs that are disappearing, young people move away, services disappear, and the capacity of municipalities for economic management is low.

Data analysis

Each calendar year, aggregated counts of child welfare notifications, child welfare and community care interventions and child placements outside the home are described for each municipality. Incidence rates and proportions for each event will then be computed by dividing the annual counts by the year-end total population. Incidence rates and proportions of referrals to child psychiatric clinics are also described each calendar year for each municipality. The LTC intervention data are calculated by dividing the number of LTC discussions by the population eligible for the intervention. Then, for each municipality, LTC intervention data collected across all study periods are classified into four categories according to the phase of the LTC implementation: control period, training in LTC services, first year with constant LTC discussions, second and following years with constant LTC discussions.

During 2014–2018, we will follow 500 000 person-years across the age group of 0–17 years. To summarise the relationship between LTC services and outcome variables, we use descriptive contingency tables. A categorised LTC variable is tabulated for each outcome variable. The table is created to display data for any outcome measure (number of events and incidence rate) and subcategories of LTC services (control period, training in LTC services, first year with constant LTC discussions, second and following years with constant LTC discussions). Most LTC services are rendered among families with school-aged children. Hence, similar analyses will be performed that separate data into subsamples according to age. Stratified analyses will include age groups of 0–6 and 7–17 years.

There are background differences between municipalities. To control for potential confounding, we use adjustment for municipal categories. For any given outcome, regression analysis involves the group-specific incidence rates and the group-specific coverage data of LTC discussions. We apply a log-linear model in the analysis of incidence rate-ratio estimation. The results of statistical analyses are presented as incidence rate ratios with 90% CIs and p values.

Ethics and dissemination

The study design has been approved by the management of Oulu University Hospital in accordance with the guidelines of The Regional Ethics Committee of the Northern Ostrobothnia Hospital District in Oulu, Finland. The Regional Ethics committee concluded that this study does not require ethics approval.

The two rationales behind the present study design are: individual-level LTC intervention is difficult to measure in real-life, because the framework of strengths and vulnerabilities applies to both the capacity of individuals and their social and physical ecologies. Moreover, in the Council of Oulu Region, all the municipal authorities autonomously make decisions on the implementation of LTC interventions. All data are treated and implemented according to national data security laws.

All parents are offered the opportunity to take part in an LTC discussion and are shown the structure of the discussion themes. If they agree, they make an appointment with the professional. The need for written consent was waived by The Regional Ethics Committee of the Northern Ostrobothnia Hospital District. This study is not research stipulated in the Finnish Medical Research Act (488/1999), which only applies to medical research involving intervention in the integrity of a person.

This paper outlines the LTC activities of the Council of Oulu Region’s evaluation study, its design and data collection and details on the implementation of the intervention. The LTC evaluation study uses a quasi-experimental trial design to evaluate the effectiveness of a community-level intervention that aims to improve children’s well-being. This design is suitable in cases when the intervention cannot be delivered to all intervention areas at the same time.

The strengths of the study include the large number of participating municipalities and the multiple data collection stages. An LTC intervention in local child and family services may reduce the need for highly specialised child protection and child psychiatric services in northern Finland. The results from the study will contribute to the limited research available on the effectiveness of promoting children’s well-being via an intervention programme that uses the child and family services approach.

This pragmatic evaluation of a community-led intervention in a real-life multiprofessional setting is expected to provide information on adapting evidence-based methods for diverse municipalities. Child and family services are often renewed and directed in such ways that leave little, if any, resources for programme evaluation and outcome monitoring. In this study, independent regional investigators work together with local professionals to assess, in real-life settings, the association between the change in the local LTC intervention rate and the change in ecological measures of children's health.

The LTC evaluation study in the Council of Oulu Region aims to expand our knowledge on the effectiveness of promoting children’s health via an intervention programme that uses a multisectorial approach. At the same time, it improves cooperation between researchers and practitioners. The results can be useful for both the researchers and the managers leading the local reform of public child and family services. If the LTC intervention proves to be effective, the intervention programme can be distributed throughout Finland, as well as in other high-income countries.