Is the Families First Home Visiting Program Effective in Reducing Child Maltreatment and Improving Child Development?

While home visiting programs are among the most widespread interventions to support at-risk families, there is a paucity of research investigating these programs under real-world conditions. The effectiveness of Families First home visiting (FFHV) was examined for decreasing rates of being in care of child welfare, decreasing hospitalizations for maltreatment-related injuries, and improving child development at school entry. Data for 4,562 children from home visiting and 5,184 comparison children were linked to deidentified administrative health, social services, and education data. FFHV was associated with lower rates of being in care by child's first, second, and third birthday (adjusted risk ratio [aRR] = 0.75, 0.79, and 0.81, respectively) and lower rates of hospitalization for maltreatment-related injuries by third birthday (aRR = 0.59). No differences were found in child development at kindergarten. FFHV should be offered to at-risk families to decrease child maltreatment. Program enhancements are required to improve child development at school entry.

Early childhood is a critical period for brain and biological development and has a strong influence on lifelong health. Birth weight and various physical and mental health conditions in childhood are associated with long-term outcomes such as educational attainment, adult employment, crime, and self-reported health (Currie & Rossin-Slater, 2015). Healthy child development depends upon nurturing, supportive, and stimulating environments (Maggi, Irwin, Siddiqi, & Hertzman, 2010), as children need both enrichment and protection from excessive stress. In order to promote these early childhood environments, it is crucial to support the adults who are caring for the children (Shonkoff, 2016).

Child maltreatment, on the other hand, is associated with short-term and long-term negative consequences for health and social trajectories, which may include mental health problems, drug and alcohol use, risky sexual behavior, obesity, and criminal behavior (Gilbert et al., 2009). The Adverse Childhood Event study found a relationship between seven categories of adverse childhood events (including three categories of maltreatment) and adult health risk behaviors and health status (Felitti et al., 1998). Recent genetic studies show that the biological embedding of childhood maltreatment may play a role in these long-term effects (Champagne, 2010). It is imperative that evidence-based interventions, aimed at preventing child maltreatment and improving healthy child development, be available for at-risk families who face many challenges such as poverty, single parenting, or mental health problems.

Service providers from all sectors who work with at-risk children and their families are increasingly aware of the importance of early childhood development and the detrimental effects of childhood maltreatment. These providers can offer referrals to programs and services that support families in providing nurturing environments for children. Home visiting programs are one of the most widespread interventions that target families at risk for neglecting or abusing their children. These programs consist of regular home visits by nurses, other professionals, or trained paraprofessionals. Objectives vary across programs, but generally the aim is to improve parent–child relationships, support parents (decrease stress, increase self-efficacy, and improve education and employment), and connect families with their communities.

In studies conducted under highly structured and supervised conditions, researchers have found that some home visiting programs are effective in reducing child maltreatment and improving child development. For example, children from the Nurse Family Partnership home visitation program had fewer health-care encounters and hospitalizations for injuries or ingestions, and in the longer term, lower mortality rates, compared to their counterparts in the comparison group (Kitzman et al., 1997; Olds et al., 2014). However, many reviews on home visiting studies report that these results are inconsistent across programs (Avellar & Supplee, 2013; Olds, Sadler, & Kitzman, 2007; Peacock, Konrad, Watson, Nickel, & Muhajarine, 2013; Sweet & Appelbaum, 2004). In a recent review (Selph, Bougatsos, Blazina, & Nelson, 2013), only two of the five randomized controlled trials that examined Child Protection Services (CPS) reports found that home visiting programs were associated with lower rates of children being removed from their homes (Barlow et al., 2007; Duggan et al., 2004), and one found that home visiting was associated with less CPS involvement (Lowell, Carter, Godoy, Paulicin, & Briggs-Gowan, 2011). Of three studies that evaluated hospital emergency department visits, one study found that children in home visiting programs had lower rates of hospital visits for injuries or ingestions compared to control children (Fergusson, Grant, Horwood, & Ridder, 2005), while the two others found no evidence of program effectiveness for improving emergency department visits (Duggan et al., 2007; Koniak-Griffin et al., 2003).

There remains a paucity of research investigating home visiting programs when disseminated or scaled up on a population level under real-world conditions, although some examples exist (Chaffin, Hecht, Bard, Silvosky, & Beasley, 2012; Green et al., 2014; Matone, O’Reilly, Luan, Localio, & Rubin, 2012). This type of research examines the transition of evidence-based interventions from research into routine use (Colditz, 2012; Rubenstein & Pugh, 2006). It provides policy makers with an understanding of program effects on their population and how to make adjustments if necessary. Determining the effectiveness of home visiting in child maltreatment outcomes has been particularly challenging because most trials lack the sample size required to detect rare outcomes of reported abuse over reasonable follow-up period (Rubin, Curtis, & Matone, 2014).

Results from research settings may differ from those obtained in community settings because compromises are often made to the original program when it is scaled up and implemented in the community. Evidence-based interventions should be conducted with fidelity (i.e., implemented as intended and as conceived by the developers) to obtain the desired outcomes (Allen, Linnan, & Emmons, 2012). Examples of compromises to fidelity include families not receiving the prescribed number of visits, the curriculum not being followed as intended, or the home visitors not receiving the required supervision. Given that home visiting is implemented across a variety of health and social agencies and geographical areas, evaluating the services under a variety of real-world circumstances would improve our understanding of its effects.

In the Canadian province of Manitoba, we saw an opportunity to address this knowledge gap by examining a home visiting program delivered through the publically funded public health system. The Families First Home Visiting (FFHV) Program offers home visiting services to at-risk families with children from the prenatal period to 5 years of age. Such a program was deemed critical in Manitoba because out-of-home placements were found to be higher than other developed countries, affecting 3% of infants every year (Gilbert et al., 2012). In Manitoba, the FFHV was implemented province wide in 1999 and was originally modeled after the Hawaii Healthy Start Program, a program showing promise to reduce child maltreatment and improve child health and development (Duggan et al., 1999).

The FFHV is a strength-based, family-centered program and is built on the premise that parents with strong attachments to their infants are at lower risk for child abuse and neglect. Public health nurses (PHNs) screen and assess families for program eligibility, provide clinical expertise and support to families as required, and provide training and weekly supervision to all home visitors. Paraprofessional home visitors have a high school education and either some postsecondary education in health, education, or child development or personal experience with parenting under difficult circumstances. Under PHN supervision, home visitors develop a trusting relationship with 10–18 families and focus on parent–child relationships, healthy child development, and connecting families with services (Healthy Child Manitoba, 2010). Depending on the family’s needs, the frequency of home visiting varies from once a week to once a month and lasts between 1 and 2 hr per visit. Families may participate in the program for up to 3 years, although according to program records maintained at health regions levels, the average time that families spend in the program is about 18 months.

Both home visitors and the supervising PHNs attend a weeklong core training session to learn the basic principles of the home visiting program and are oriented to the Growing Great Kids Curriculum (Growing Great Kids Inc., 2015), a curriculum used by Healthy Families America and added to the FFHV in 2001. This curriculum includes child development, parenting and health information, and parental and child activities to guide the home visitor. Shorter additional training sessions are offered to home visitors subsequent to core training. The decision to hire paraprofessionals rather than nursing professionals was based on prohibitive staffing costs and nursing shortages. PHNs administer the two-stage screening process (not hospital staff as in the original Hawaii Healthy Start program), as this provided a better fit in the existing health-care system in Manitoba (Healthy Child Manitoba, 2010). The introduction of the program and paraprofessional home visitors to the public health system required adjustments to public health nursing practice. Issues related to delivering the FFHV, factors contributing to its successful implementation, and the nature of relationships between the families, home visitors, and PHNs within the program have been studied (Heaman, Chalmers, Woodgate, & Brown, 2006; Heaman, Chalmers, Woodgate, & Brown, 2007; Woodgate, Heaman, Chalmers, & Brown, 2007).

A provincial evaluation of the FFHV that used multilevel modeling to test for differences in family measures over time found improvements in positive parenting, parental psychological well-being, and social cohesion; however, no program effects were found for maternal depression, early child development, or literacy activities (Healthy Child Manitoba, 2010). A recent observational study found that the FFHV was associated with improvements in immunization rates (Isaac et al., 2015). A comprehensive mental health promotion strategy was added to the program as a result of the program evaluation (Chartier et al., 2015). Although the program evaluation showed improvement in parenting, it was still not known if these effects translated into fewer children being taken into care by child welfare. In addition, the measurements used to evaluate early child development were conducted at age 1, before most developmental delays would be apparent. We wondered whether the effect of the program might be observed at school entry when kindergarten teachers provide a child development measure.

Given the influences that parenting factors have on child development and child maltreatment (O’Donnell et al., 2015; Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009), we undertook this study to determine the effectiveness of the FFHV in decreasing rates of children taken into care of child welfare, decreasing rates of hospitalization for maltreatment-related injuries, and increasing child development scores at school entry. Given that in practice, programs do not reach all eligible families, we also sought to determine the potential program effect for the children from families who did not actually receive the home visiting program. Families may not receive home visiting services for reasons related to the program or to family characteristics. Some of these reasons may preclude them from participating in randomized controlled trials. To our knowledge, no studies have attempted to estimate home visiting program effectiveness for children from families who did not receive the program. We hypothesize that the rates of children in care at ages 1, 2, and 3, and hospitalization for maltreatment-related injuries for children at age 3, will be lower among children in the FFHV compared to those not in the program. We further hypothesize that child development scores at school entry will be higher for children in the FFHV compared to those not in the program. Finally, we hypothesize that the potential program effect will be attenuated for children from families who have not received the program.

Method

Study Design

In this population-based retrospective cohort study, we examined child maltreatment and child development for at-risk children born between 2003 and 2009 who were eligible for the FFHV. Deidentified home visiting program data were linked to administrative health, social services, and education data from January 2003 to March 2013. This study received approval from the Health Research Ethics Board at the University of Manitoba.

Study Cohort

Eligibility for the FFHV is determined by a two-stage process where PHNs first screen all births in the postpartum period. The universal newborn screen consists of 39 biological, social, and demographic factors such as congenital anomalies, birth weight, multiple births, mother’s age, education, marital status, mental health, family social isolation, and relationship distress. Then, families with three risk factors or more on the universal newborn screen are assessed with the 10-item parent survey based on the Kempe Family Stress Checklist (Korfmacher, 2000). A parent survey score of 25 or more is required for program eligibility. Program records maintained at health regions levels during the study period (2003–2009) recorded the following reasons for eligible families not receiving the FFHV: program was full (11%), parent declined participation (37%), or program not offered because the parent survey was not administered (52%; M. Ross, personal communication, July 16, 2014).

In our study, we included children from at-risk families who were eligible for the program, some of whom had participated in the program and some of whom had not, and followed them from 1–5 years after initial exposure to the program. The process for selecting the study cohort is illustrated in Figure 1. Our database contained 16,153 families whose scores were in the at-risk range on the universal newborn screen (three or more risk factors) and thus would be eligible for the second-stage assessment called the parent survey. One third of these families (n = 5,369) were missing the parent survey score, and so we conducted an imputation process to estimate these scores (details below). After imputations, a total of 10,649 children met the eligibility criteria of scoring 25 or more on the parent survey. Once we removed 889 children who were missing the program assignment variable and another 15 who were missing other key variables, there were 4,562 program children and 5,184 eligible children who had not received the program. The sample size to examine child development at age 5 was 1,491 program children and 1,688 nonprogram children because the child development measure (described below) is only collected biannually by government agencies and also because not all children in our cohort had reached age 5 at the time of the study.

Figure 1.

Development of study cohort.

Data Sources and Variables

The data for this study came from the PATHS Data Resource, which was derived from holdings within the Manitoba Population Research Data Repository at the Manitoba Centre for Health Policy (MCHP) (Nickel et al., 2014). The PATHS Data Resource contains no personal identifying information, such as names and addresses, but a scrambled numeric identifier is attached prior to the data being deposited in the Repository. Because the identifier is scrambled in the same way for each file, these data sets are linkable across files and over time. The data in the Repository have been studied extensively and validated for research purposes (Roos, Gupta, Soodeen, & Jebamani, 2005; Roos, Menec, & Currie, 2004).

Variables for this study are described in Table 1. The outcome variables “in care of child welfare” and “hospitalizations for maltreatment-related injuries” were chosen as proxies for child maltreatment. Based on the child welfare system’s administrative database, children are taken into care by child welfare when evidence of child maltreatment is found. From these databases, we were able to ascertain that the child was removed from their family’s home and taken into care. The data fields on unsubstantiated reports and specific reasons for being taken into care are not consistently coded and were not used here. Child maltreatment can also be defined in the hospital records through the International Statistical Classification of Diseases and Related Health Problems, 10th Edition (ICD-10) codes for maltreatment-related injuries assigned by medical staff. These codes included maltreatment syndrome (code directly related to abuse or neglect), assault, undetermined cause, and codes about adverse social circumstances that are the indicators of neglect or broader welfare concerns (social environment, family support, and lifestyle). These codes are described in greater detail elsewhere (Gilbert et al., 2012). Developmental vulnerability at school entry was measured with the Early Development Instrument (EDI), a 103-item survey completed by kindergarten teachers when children are 5 or 6 years old. The EDI measures five domains of child development, namely, physical health and well-being, social competence, emotional maturity, language and thinking skills, and communication skills and general knowledge. Teachers are given 20 min per student to complete the EDI in the middle of the school year when the teacher is familiar with the students. Children with scores in the 10th lowest percentile as determined by Canadian averages are deemed developmentally vulnerable (Janus, Brick, & Duku, 2011). To our knowledge, this instrument has not been previously used as an outcome measure for home visiting programs.

Table 1.

Study Variables.

Main Outcome Variables	Variable Descriptions
In care of child welfare at 1 year	Taken into care for at least 1 day between birth and first birthday
In care of child welfare at 2 years	Taken into care for at least 1 day between birth and second birthday
In care of child welfare at 3 years	Taken into care for at least 1 day between birth and third birthday
Hospitalization for maltreatment-related injuries	Hospitalized for injury due to maltreatment (determined by ICD-codes)
Developmentally vulnerable in physical health and well-beinga	Low EDI scores for physical readiness for school, physical independence, and gross and fine motor skills
Developmentally vulnerable in social competencea	Low EDI scores for social competence, showing responsibility and respect, and readiness to try new things
Developmentally vulnerable in emotional maturitya	Low EDI scores for emotional development, prosocial, and helping behaviors; exhibits anxious, aggressive, or inattentive behaviors
Developmentally vulnerable in language and thinking skillsa	Low EDI scores in basic literacy and numeracy skills and use of memory; demonstrates lack of interest in numeracy and literacy
Developmentally vulnerable in communication skills and general knowledgea	Low EDI scores in ability to communicate with others and in level of general knowledge
Confounding variables accounted for in the statistical model
Mother’s age at first birth	Mother’s age at birth of first child
Socioeconomic factor index-2	A Canadian census data-based index that reflects nonmedical social determinants of health (higher scores mean higher risk)
Parent survey score	10-Item assessment by PHNs—based on the Kempe Family Stress Checklist
Maternal low education	Mother’s highest level of education is lower than Grade 12
Family social assistance	Family on social assistance/income support
Single-parent family	Mother not currently in common-law relationship or married
No prenatal care before 6 months	Mother did not receive prenatal care before 6 months gestation
Nurse screened family prenatally	Family screened prenatally for risk factors rather than postnatally
Alcohol and/or drug use during pregnancy	Any alcohol and/or drug use by the mother during pregnancy
Smoking during pregnancy	Maternal smoking during pregnancy
Family history of disability	Family history of a disability not detectable at birth that could affect development (e.g., deafness, mentally disabled/challenged)
Maternal mental disability	Mother has mental or intellectual disability
Maternal depression and/or anxiety	Mother has depression (including postpartum) and/or anxiety disorder
Maternal schizophrenia	Mother has schizophrenia
Maternal substance abuse	Current substance abuse by mother
Paternal antisocial behavior	Father has antisocial behavior
Maternal antisocial behavior	Mother has antisocial behavior
Family social isolation	Lack of social support or isolation related to culture, language, or geography
Parental relationship distress	Mother reports relationship distress with parenting partner
Violence between parents	Current or history of violence between parenting partners
Maternal history of child abuse	Mother has a history of child abuse or neglect

Note. EDI = Early Development Instrument; PHNs = public health nurses; ICD = International Statistical Classification of Diseases and Related Health Problems.

aDerived from the EDI completed by kindergarten teachers and collected by Healthy Child Manitoba office.

Data Analyses

All analyses were performed using SAS Sotware Version 9.2. Multiple imputations were conducted to address the missing parent survey scores by utilizing data from the universal newborn screen form and shown in Table 1. Markov Chain Monte Carlo methods were used to fill in missing values 10 times, generating 10 complete data sets using the SAS procedure MIANALYZE. We then combined the results from the 10 complete data sets using this SAS procedure to reflect the fact that uncertainty remains with respect to the unknown values and to provide statistically valid inferential results (Rubin, 1987).

We used inverse probability of treatment weights (IPTWs) to adjust for measured confounders which would be related to both entry into a program such as the FFHV and the outcomes under study (Rosenbaum, 2010). Using logistic regression, propensity scores were calculated for each family to determine the probability that the child’s family would participate in the home visiting program. The scores were based on 21 family characteristics that could influence program participation such as alcohol use, mental illness, and violence between parents (see Table 1).

Three sets of IPTWs were created with the propensity scores, each corresponding to the treatment effect of interest: the average treatment effect (ATE), the average treatment effect on the treated (ATT), and the average treatment effect on the untreated (ATU). The ATE is the average program effect on child maltreatment and child development scores across the entire eligible population. The ATT is the average effect among children who actually received the FFHV, and the ATU is the potential program effect among children in the nonprogram group if they had received the home visiting program (Rosenbaum, 2010).

Separate multiplicative models were conducted to estimate adjusted risk ratios (aRRs) and risk differences for each outcome (Wacholder, 1986). Measures of precision (e.g., 95% confidence intervals and standard errors) came from the models, and an a priori significance level of p < .05 was used. Generalized linear modeling was used to test for an association between program participation and child outcomes generating the three treatment effects. With the available data (up to 2012/2013), we conducted analyses on child maltreatment outcomes until age 3 using the entire birth cohort (2003–2009). Our analyses for the child development outcomes, which were only available at age 5, were conducted on a subsample (2003–2007).

While it is not possible to test directly whether the IPTWs controlled for all unmeasured confounding, gamma sensitivity analyses provide an assessment of how sensitive the results were to unmeasured confounders. Gamma values were, therefore, calculated for each statistically significant result to determine the strength of unmeasured confounding required to invalidate these results (Rosenbaum, 2010).

Results

Family Characteristics

The columns under “before weighting” in Table 2 show the study cohort characteristics and illustrate the high rates of risk factors found for children in both the FFHV program and nonprogram groups. The FFHV families had a higher percentage of maternal depression or anxiety, of being on social assistance, and of parental relationship distress and maternal history of child abuse, while nonprogram families had a higher socioeconomic factor index score (made up of living in areas of higher unemployment, lower income, and lower education), lower maternal age of birth of first child, higher percentage of single-parent families, and higher percentage of maternal smoking during pregnancy. The columns under “after weighting” demonstrate that weighting was effective in balancing the risk factors for families who received the program and those who did not. We calculated standardized differences of the risk factors before and after weighting (data not shown). More than half of the risk factors were balanced before applying the weights (standardized difference of less than 10%), and all were balanced after weighting.

Table 2.

Study Cohort Characteristics for Families Before and After Weighting.

Characteristics	Before Weighting		After Weightinga
	FFHV	Non- FFHV	FFHV	Non- FFHV
Mother’s age at first birth (mean)	21.09	20.75	20.94	20.94
SEFI-2 (mean)	0.53	0.73	0.63	0.63
Parents survey score (mean)	38.45	36.66	37.38	37.35
Maternal low education (%)	52.18	53.17	53.13	52.95
Family social assistance (%)	66.44	69.07	67.65	67.69
Single-parent family (%)	42.55	47.73	45.64	45.36
No prenatal care before 6 months (%)	6.78	9.73	8.38	8.36
Nurse screened family prenatally (%)	17.49	7.72	12.28	12.29
Alcohol and/or drug use during pregnancy (%)	37.96	38.80	38.32	38.31
Smoking during pregnancy (%)	49.79	56.01	52.86	52.86
Family history of disability (%)	5.34	4.69	4.98	4.94
Maternal mental disability (%)	1.69	1.00	1.34	1.43
Maternal depression and/or anxiety (%)	36.23	31.57	34.20	34.06
Maternal schizophrenia (%)	1.65	1.55	1.65	1.64
Maternal substance abuse (%)	3.30	2.73	3.00	2.95
Paternal antisocial behavior (%)	4.98	4.22	4.77	4.76
Maternal antisocial behavior (%)	2.19	1.80	1.95	1.93
Family social isolation (%)	16.50	9.46	12.81	12.89
Parental relationship distress (%)	26.00	19.03	22.51	22.51
Violence between parents (%)	9.50	8.13	8.72	8.82
Maternal history of child abuse (%)	26.40	18.61	22.20	22.30

Note. FFHV = Families First Home Visiting; SEFI-2 = Socioeconomic Factor Index (version 2).

aUsing weight for average treatment effect.

In Care of Child Welfare

Table 3 shows the effect of the FFHV on being taken into care of child welfare up to the children’s third birthdays. In program families, the probability of children being taken into care in the first year of life was lower (7%) than in nonprogram families (10%). A similar pattern occurred at children’s second and third birthdays. The adjusted risk difference and risk ratio in their first year of life showed significant differences when comparing program children and children who were eligible but not in the program. The risk ratios were attenuated (but were still significant) for children at their second and third birthday. Effect estimates for children in the program (ATT) and those not in the program (ATU) were similar to the ATE reported above, although contrary to our hypothesis, the program effect was slightly larger in the nonprogram group than the program group. This means that stronger program effects could be expected among nonprogram children (if they had received the program) than children who actually received the program. The γ value (.15) indicates that these results are moderately robust to unmeasured or hidden confounders. In other words, after adjusting for the confounding variables in the propensity score, there would need to be unmeasured confounding that would both perfectly predict enrolment in the FFHV and account for 15% of the relationship between the program and being taken into care. It is unlikely that such confounding exists, given that we have controlled for many of the factors that are criteria for admission into the program (Table 2).

Table 3.

Child Maltreatment and Child Development Outcomes for FFHV Families and Non-FFHV Families.

Outcomes	Predicted Probability		Risk Difference	95% CI	Risk Ratio	95% CI
	FFHV	Non-FFHV
In care of child welfare at 1 year
Unweighted	.074	.101	−.027*	[−.038, −.016]	0.73*	[0.64, 0.84]
ATE	.075	.099	−.025*	[−.037, −.012]	0.75*	[0.66, 0.86]
ATT	.075	.097	−.022*	[−.036, −.009]	0.77*	[0.67, 0.88]
ATU	.075	.102	−.027*	[−.040, −.014]	0.74*	[0.65, 0.84]
In care of child welfare at 2 year
Unweighted	.108	.140	−.032*	[−.046, −.018]	0.77*	[0.69, 0.87]
ATE	.106	.136	−.030*	[−.044, −.015]	0.79*	[0.70, 0.88]
ATT	.108	.131	−.024*	[−.039, −.008]	0.82*	[0.74, 0.92]
ATU	.104	.140	−.035*	[−.051, −.020]	0.76*	[0.68, 0.85]
In care of child welfare at 3 years
Unweighted	.130	.167	−.037*	[−.054, −.020]	0.78*	[0.70, 0.87]
ATE	.131	.161	−.030*	[−.047, −.014]	0.81*	[0.73, 0.91]
ATT	.129	.156	−.026*	[−.043, −.010]	0.83*	[0.74, 0.93]
ATU	.132	.166	−.034*	[−.051, −.017]	0.80*	[0.71, 0.89]
Hospitalization for maltreatment-related injuries
Unweighted	.006	.008	−.002	[−.006, .001]	0.72	[0.43, 1.19]
ATE	.005	.009	−.004*	[−.007, .000]	0.59*	[0.35, 0.99]
ATT	.006	.009	−.004*	[−.007, .000]	0.60*	[0.37, 1.00]
ATU	.005	.008	−.003*	[−.007, .000]	0.57*	[0.33, 0.99]
Vulnerable in physical health and well-being
Unweighted	.26	.26	.00	[−.04, .03]	0.98	[0.87, 1.11]
ATE	.26	.25	.01	[−.02, .04]	1.05	[0.93, 1.19]
ATT	.26	.24	.02	[−.01, .05]	1.08	[0.95, 1.23]
ATU	.27	.26	.01	[−.02, .04]	1.03	[0.91, 1.16]
Vulnerable in social competence
Unweighted	.22	.22	.00	[−.03, .03]	1.00	[0.87, 1.14]
ATE	.21	.21	.00	[−.03, .03]	1.01	[0.88, 1.16]
ATT	.22	.20	.02	[−.01, .05]	1.08	[0.94, 1.24]
ATU	.21	.22	−.01	[−.04, .02]	0.95	[0.83, 1.09]
Vulnerable in emotional maturity
Unweighted	.20	.19	.02	[−.01, .05]	1.10	[0.95, 1.27]
ATE	.20	.18	.02	[−.01, .05]	1.11	[0.96, 1.29]
ATT	.20	.17	.03	[.00, .06]	1.19*	[1.03, 1.39]
ATU	.19	.19	.01	[−.02, .04]	1.04	[0.90, 1.21]
Vulnerable in language and thinking skills
Unweighted	.23	.24	−.01	[−.04, .02]	0.97	[0.86, 1.10]
ATE	.24	.23	.01	[−.02, .04]	1.04	[0.91, 1.18]
ATT	.23	.22	.01	[−.02, .04]	1.06	[0.93, 1.21]
ATU	.25	.24	.00	[−.03, .03]	1.02	[0.90, 1.15]
Vulnerable in communication skills and general knowledge
Unweighted	.18	.19	−.01	[−.04, .02]	0.94	[0.81, 1.10]
ATE	.19	.19	.00	[−.03, .03]	0.98	[0.84, 1.15]
ATT	.18	.19	−.01	[−.03, .02]	0.97	[0.83, 1.13]
ATU	.19	.19	.00	[−.03, .03]	0.99	[0.85, 1.16]

Note. The risk difference is the predicted probability for FFHV families minus the predicted probability for non-FFHV families; the risk ratio is the predicted probability for FFHV families divided by the predicted probability for non-FFHV families. FFHV = Families First Home Visiting; CI = confidence interval; ATE = average treatment effect; ATT = average treatment effect for treated group; ATU = average treatment effect for untreated group.

*Statistically significant at p < .05.

Hospitalizations for Maltreatment-Related Injury

The probability of hospitalizations for maltreatment-related injuries during the 3-year period after birth was low for both groups of children, but the probability was lower among children in the program (0.5%) than children not in the program (0.9%; Table 3). The statistically significant risk difference and risk ratio for the ATE were 0.004 and 0.59, respectively. Estimates for program and nonprogram children were similar to the ATE reported above. The γ value was calculated to be .01, meaning that we cannot rule out the possibility of unmeasured confounding.

EDI

The probability of being developmentally vulnerable at school entry on the five domains of child development as measured by the EDI did not differ significantly between children from program and children from nonprogram families. The risk differences and risk ratios for the ATE, ATT, and ATU were not statistically significant (Table 3). Furthermore, the modeled probability of children who were developmentally vulnerable in at least one domain of the EDI was not significantly different between children from program families (46.5%) and nonprogram families (45.2%) at school entry (not shown in table).

Discussion

This study supports the hypothesis that home visiting as implemented through the FFHV in Manitoba was associated with decreases in key child maltreatment indicators. These findings are consistent with previous research conducted under controlled conditions (Avellar & Supplee, 2013; Casillas, Fauchier, Derkash, & Garrido, 2016; DuMont et al., 2008), and different from other postimplementation research that found no program effects under real-world conditions (Rubin et al., 2014). The FFHV had been previously associated with improving protective factors related to child maltreatment such as positive parenting, psychological well-being, and social cohesion (Healthy Child Manitoba, 2010).

The decreases in child maltreatment associated with the FFHV may be attributed to how the program is implemented. A recent meta-analysis showed that staff training, home visitor supervision, and program fidelity monitoring were associated with higher effect sizes and that home visitors’ educational background and the sector (school, social services, mental health, or medical) where the home visiting program is delivered had no influence on the effect size (Casillas et al., 2016). In the FFHV, paraprofessional home visitors receive intensive training, weekly reflective supervision is provided by PHNs with supervision training, and program fidelity is monitored through the supervision (Healthy Child Manitoba, 2010).

The findings have important implications for the well-being of children. It is well recognized that maltreated children are far more likely to experience difficulty in school, be arrested, and have more emotional, developmental, or acute health conditions (Gilbert et al., 2009). Furthermore, substantiated reports of child maltreatment and statistics on children in foster care likely represent only the “tip of the iceberg” of child abuse and neglect (Trocme, Fallon, MacLaurin, & Neves, 2005). Supporting families with parenting challenges through home visiting, before neglect and abuse come to the attention of child welfare, is a promising preventative public health intervention. It provides parents and caregivers the opportunities to develop strong parent–child relationships and decrease all types of maltreatment, reported and unreported. Notwithstanding the traumatic experiences for children, the economic impact of child maltreatment to society is substantial (Fang, Brown, Florence, & Mercy, 2012). In jurisdictions like Manitoba, where rates of children in care are among the highest in the world, these types of interventions are crucial (Gilbert et al., 2012).

No differences in child development scores across program groups were found even though the home visiting curriculum included many child developmental activities. This finding is consistent with previous research where no FFHV program effects were observed for improving the child development scores as measured by the Ages and Stages Questionnaire at 1 and 2 years (Healthy Child Manitoba, 2010), nor in increasing activities associated with child development, such as reading aloud to children. In this study, the modeled probability of being developmentally vulnerable in each domain was between 19% and 26%, which was considerably poorer than rates found at the population level in this jurisdiction (between 10.7% and 12.6%; Healthy Child Manitoba, 2014). Given the high percentage of developmentally vulnerable children, curriculum enhancements and connecting families to early learning centers are warranted to improve development scores of these at-risk children.

A substantial number of families eligible for FFHV (n = 5,184) did not receive home visiting services and are, therefore, not benefiting from the program. By calculating the potential program effect in children whose families did not actually receive the program (ATU), we estimate that a strong effect could be expected among these families—if they had had the opportunity to receive home visiting. This is motivating to service providers who strive to engage families to participate. Unfortunately, problems with engaging and retaining families in programs are common and interfere with improving child outcomes on a population level (Daro, McCurdy, Falconnier, & Stojanovic, 2003; Durlak & DuPre, 2008; O’Brien et al., 2012).

The strengths of this study include the large, real-world sample, the ability to adjust for a wide range of factors (by using information from the universal newborn screen), and the ability to link program data to child welfare, hospitalization, and education databases. Our population-based data have information collected at a population level for baseline variables and child maltreatment and developmental outcomes. Utilizing existing administrative and population-wide data is an important innovative practice in early childhood home visiting and child maltreatment prevention, as we strive to understand programs that are scaled up across geographic areas and delivery systems. With the richness of the available data, we were able to construct a strong comparison group and to conduct sensitivity analyses to ensure our results were not exceedingly susceptible to confounding factors. The use of IPTWs enabled us to estimate the potential program effect on children who did not receive the program, which is particularly useful since we have included almost all eligible families who did not receive the program. It provides policy makers with the reassurance that investing in better engagement will improve outcomes for children.

There were study limitations that warrant discussion. Despite our efforts to balance children from program families and nonprogram families in this observational study, we acknowledge that we could not control all potential confounders. For example, we were not able to control for family participation in other programs offered by health, social, and community services. A selection bias may have been present, given that about a third of eligible families refused to enroll in the program. We attempted to measure the robustness of our findings to unmeasured confounding. The γ sensitivity analyses indicated that the finding on children in care was robust to unmeasured confounders, but that the finding on hospitalizations for maltreatment-related injuries was not. Another limitation is the substantial number of missing parent survey scores used to select families eligible for the program. To enable us to generalize to all at-risk families in Manitoba, we strove to include as many families as possible. Fortunately, we were able to impute these scores with rich data provided in the universal newborn screen.

The magnitude of our results is likely underestimated for the following reasons: First, we could not account for surveillance bias, meaning that families receiving home visiting services are closely watched by home visitors, and child maltreatment could be reported more frequently than families without this surveillance (Chaffin & Bard, 2006). Child maltreatment in nonprogram families is more likely to go unnoticed. Second, we used an intent-to-treat approach, meaning that all families who accepted the program were assigned to the FFHV group. We acknowledge that families were exposed to the program for varying lengths of time and we could not assess how this influenced program effectiveness due to limitations in the program implementation data. We speculate that the program effect may have been stronger if we had excluded the families with low program exposure.

Conclusions

This study found that the province-wide FFHV for at-risk families was associated with substantially lower rates of children being taken into care of child welfare and with fewer hospitalizations for maltreatment-related injuries by their third birthday. No evidence was found that the program improved child development scores at school entry. Future research should explore whether home visiting programs are effective at decreasing inequities in child maltreatment and child development at a population level. The findings presented here may be useful for program planners in providing insights for program enhancement, for example, ensuring that at-risk children are linked to early learning programs could lead to improved child development at school entry. Given the long-term sequelae of child maltreatment, greater efforts are required to improve the reach of home visiting programs to all at-risk families.

Authors’ Note: The results and conclusions are those of the authors, and no official endorsement by the data providers is intended or should be inferred.