Are Content-Specific Curricula Differentially Effective in Head Start or State Prekindergarten Classrooms?

Head Start and state prekindergarten (pre-K) programs can boost the school readiness of low-income children through the use of effective preschool curricula. Encouraging results from some studies suggest that children who receive targeted or content-specific curricular supplements (e.g., literacy or math) during preschool show moderate to large improvements in that targeted content domain, but recent research also suggests differences in children's school readiness among different preschool program settings. We examine whether children in Head Start or public pre-K classrooms differentially benefit from the use of randomly assigned classroom curricula targeting specific academic domains. Our results indicate that children in both Head Start and public pre-K classrooms benefit from targeted, content-specific curricula. Future research is needed to examine the specific mechanisms and classroom processes through which curricula help improve children's outcomes.

RCT Entities:   Population Interventions Outcomes

Selecting curricula is an important component in structuring children’s experiences in early childhood education programs. All Head Start and many state prekindergarten (pre-K) programs are required to use some form of a child development curriculum, and research demonstrates that curricula are a promising avenue for improving the effectiveness of publicly funded preschool programs at scale (e.g., Weiland & Yoshikawa, 2013). Particularly encouraging are the results from recent studies suggesting that children who receive targeted or content-specific curricular supplements (e.g., literacy or math) during preschool show moderate to large improvements in that targeted content domain (Clements & Sarama, 2008; Clements, Sarama, Spitler, Lange, & Wolfe, 2011; Clements, Sarama, Wolfe, & Spitler, 2013; Duncan et al., 2016; Lonigan, Farver, Phillips, & Clancy-Menchetti, 2011; Morris et al., 2014; Weiland & Yoshikawa, 2013). Unclear from the prior research, however, is whether the impacts of content-specific curricula implementation vary across different types of preschool programs with respect to classroom processes and children’s school readiness outcomes.

Across the United States, the two largest preschool programs serving 4-year-old children are Head Start and state-funded and developed pre-K programs, which some studies found are differentially associated with children’s outcomes (Gormley, Phillips, Adelstein, & Shaw, 2010; Henry, Gordon, & Rickman, 2006; Zhai, Brooks-Gunn, & Waldfogel, 2011). Although there is substantial between-state variation (Karoly, Ghosh-Dastidar, Zellman, Perlman, & Fernyhough, 2008; Jenkins, 2014) as well as variation across centers and classrooms within the same program type (Bloom & Weiland, 2015; Dotterer, Burchinal, Bryant, Early, & Pianta, 2009; Walters, 2015), there are important differences between these two early childhood program models, such as their goals, populations served, staff qualifications, enrollment, program size, and quality standards and regulations. Head Start provides comprehensive child and family supports—including services such as health and dental care and nutritious meals—to promote development across academic, socioemotional, and physical domains and targets children in poverty. State pre-K programs focus primarily on children’s academic and socioemotional development to prepare children for kindergarten and, depending on the state, may be available to any age-eligible child regardless of income or need. Such differences in program features and in the populations served might mean that different types of instructional or curricular interventions may not work in the same way. Given current efforts at the state and local levels to expand early childhood education programs and the associated tax dollars invested each year on curricula for these programs (with per-classroom costs averaging $2,000), understanding the circumstances under which content-specific curricular packages do and do not influence classroom processes and children’s outcomes is essential.

Our study aims to answer two key questions: (1) Does the use of a content-specific curricular supplement differentially influence classroom processes across Head Start and pre-K classrooms? (2) Does the use of a content-specific curricular supplement differentially benefit children across Head Start and pre-K classrooms? Furthermore, there may be specific identifiable program features involved in the moderation of content-specific curricula. Thus, to more thoroughly examine the mechanisms through which differences in classroom processes may influence curricular impacts on children’s school readiness, we also test for moderation of curricula by two policy-relevant classroom factors—teachers’ experience and education—across curriculum type.

We use data from a multisite experimental study where a sample of preschool classrooms or centers were randomly assigned to either supplement their regular instruction with a new content-specific curriculum or to continue using their “business as usual” curriculum. In four study sites, Head Start and pre-K program classrooms were included in both treatment and control conditions. Therefore, our study is uniquely positioned to examine whether there exists heterogeneity by preschool program type in the impact that content-specific curricula have on classrooms processes and the school readiness skills of low-income children. In answering these questions, our study combines two key areas of emerging research—differences between preschool program type and curriculum type. This work, at the intersection of program type and curricula type, is important in the current early learning policy context where researchers are actively examining the best way to scale up early learning interventions and the conditions under which preschool is most effective for different populations of children (e.g., Bloom & Weiland, 2015; Duncan & Vandell, 2011).

Background

Structural Differences Between Head Start and State Pre-K

Marked shifts in the number of families relying on child care, welfare policy, and subsequent federal, state, and local investments over the past decade have increased the number and type of early childhood programs available to low-income children (Laughlin, 2013; Loeb, Fuller, Kagan, & Carrol, 2004; Magnuson & Shager, 2010; Pianta, Barnett, Burchinal, & Thornburg, 2009). Although Head Start and state pre-K both provide educational care for vulnerable young children, there exist several differences between the program models. In the sections that follow, we begin with a description of the two program models, including eligibility for services, funding streams and regulations, goals and objectives, and teacher education requirements. We present a comparison of the two program models in Table 1, followed by a detailed description of their curricular differences.

TABLE 1

Comparing Head Start and State-Funded Prekindergarten

	Head Start	State pre-Ka
Funding stream	Federally funded	Locally funded, typically by the state
Target population	Very low-income families and children at risk for early school failure	Varies by state: California, Texas, and Virginia target low-income and at-risk children; New York has a universal pre-K program
Program focus	Comprehensive program that provides children with preschool education, health exams, nutritious meals, opportunities for socioemotional development, and family support services	Varies by state: each state creates its pre-K programs independently, primarily focused on early academic skills to prepare children for subject matter learning in kindergarten
Teacher education	Teachers must have at a minimum an associate degree or a Child Development Associate permit	California requires a Child Development Associate permit; New York, a master degree; Texas and Virginia, a bachelor degree

We include information only about states with prekindergarten (pre-K) programs in our analysis sample: California, New York, Texas, and Virginia. These are comparisons of Head Start and state pre-K during the 2003–2004 academic year to reflect the data.

Head Start

Head Start was conceived as the federal government’s primary tool to address child poverty in 1965 during the “war on poverty.” Thus, its central purpose is to provide early care and education for the nation’s poorest preschool-age children who are at risk of entering school unprepared. Federal funding is channeled to local grantees at private and nonprofit agencies (Magnuson & Shager, 2010). Most Head Start programs are mixed age, combining children aged 3 to 5 years in most classrooms. Head Start is characterized by its “whole child,” or “global,” programmatic approach to child development, which includes providing a comprehensive range of services for children and their families in addition to center-based early education. Programs are mandated to provide children with nutritious meals and health and dental care, to refer families to providers in the community for health and social services (e.g., educational advancement opportunities, job training), and to involve parents in programming. Head Start is also highly regulated, involving monthly reporting to the federal government with strict program quality standards and regular audits of all grantees. Historically, Head Start teachers were not required to have rigorous educational preparation (Saracho & Spodek, 2007), although new mandates now require all Head Start lead teachers to have at least an associate degree (U.S. Department of Health & Human Services, 2011). Data from the 2003 cohort of the Head Start Family and Child Experiences Survey indicate that about a third of Head Start teachers had a bachelor’s degree and slightly more than half had an associate degree or training in early childhood education.

State pre-K

Similar to Head Start, state pre-K programs offer center-based early education programs prior to kindergarten entry. However, unlike Head Start, which has the same standards for every program, these programs are state developed and not uniformly regulated by the federal government, so substantial variation exists between (and within) states in how pre-K programs are designed and implemented (Jenkins, 2014). As such, numerous important differences exist in pre-K programs across states in terms of population served, program availability, classroom quality standards, and teacher qualifications (Phillips et al., 2017). State pre-K programs can range from those made available to only a subset of the most vulnerable children (e.g., below the family poverty line) to statewide universal provision where programs are available to any child in a given state, regardless of family income, child abilities, or other factors. Whereas Head Start programs serve children across multiple age groups, most pre-K programs are targeted primarily toward 4-year-olds. State requirements and regulations for pre-K providers also vary substantially. The annual census of state pre-K programs conducted by the National Institute for Early Education Research document that the quality standards, monitoring, funding, and technical assistance for programs range from minimal in some states to very high in others (Barnett, Hustedt, Robin, & Schulman, 2004). Regarding teacher qualifications, in 2003 almost half of all state pre-K programs required lead teachers to have a bachelor degree, and a little more than half required that the assistant teacher have a Child Development Associate credential (Barnett et al., 2004). In a majority of states, pre-K teachers were required to have some form of specialized training in early childhood education. In some contexts, state pre-K programs employ teachers with greater education and training and offer teachers higher pay (Bellm, Burton, Whitebrook, Broatch, & Young, 2002).

Variations in Program Features

Of the differences between Head Start and state pre-K highlighted so far, the most important that would lead to differential effects from content-specific curricula on classrooms processes and children’s outcomes are likely the programming regulations and policies and the populations served. For instance, as compared with Head Start, state pre-K programs often have different teacher education or credentialing requirements. Most state pre-K programs are also not mandated to provide the comprehensive Head Start model in terms of additional services for children and their families (e.g., health, community services referrals), which may allow pre-K programs to focus more on classroom instruction and children’s academic skills. Head Start programs specifically target very low-income families and children with disabilities, whereas state preschool programs range from serving only children from low-income families to non–means tested universal programs. Moreover, previous research showed that children come into state pre-K settings with higher levels of skills than children in Head Start settings (Gormley et al., 2010; Henry et al., 2006; Zhai, Waldfogel, & Brooks-Gunn, 2013), which could influence the impact of curricula in these settings. These programmatic, policy, and population differences may lead to variation in curricular implementation or classroom activities.

Some research examined whether pre-K and Head Start are differentially effective at improving children’s school readiness. These studies compared the outcomes of children attending pre-K and Head Start classrooms at the end of the preschool year and found pre-K attendance to be more strongly related with children’s academic skills than Head Start (Coley, Votruba-Drzal, Collins, & Cook, 2016; Gormley et al., 2010; Henry et al., 2006; Jenkins, Farkas, Duncan, Burchinal, & Vandell, 2016; Magnuson, Meyers, Ruhm, & Waldfogel, 2004; Zhai et al., 2013). Moreover, very little research has examined the program processes or components of preschool classrooms that may be driving these differences in children’s outcomes. Classroom curricula, which frame and organize classroom experiences, are a prime candidate source of such differences between the pre-school program settings. Our study tests whether there exists heterogeneity in the impacts of one type of curricular supplement (content-specific curricula) by program model (i.e., Head Start or pre-K) on classroom processes and children’s school readiness outcomes. Given that the variation within these different preschool program models also exists, our study tests whether the impacts of content-specific curricula differ by teacher characteristics, including years of teaching experience and degree attainment. Understanding variations in program features with respect to curricula is critical for facilitating the most efficient use of limited resources, informing decisions about how best to target specific curricular programs, and suggesting ways to improve the design or implementation of the programs for high-quality early education settings.

Curricula Use in Head Start and State Pre-K

If the differences highlighted here between the Head Start and pre-K program models, goals, and populations generate differences in children’s experiences and outcomes in the program, an important question is whether a given curricular model will operate differently across the two programs. Motivating our study is whether the use of content-specific curricular supplements promote similar changes in classroom processes and children’s outcomes in Head Start and pre-K programs. Researchers and practitioners consider the use of curriculum an important indicator of quality, and quality rating and improvement systems incorporate curriculum into their tiered rankings (Auger, Karoly, & Schwartz, 2015; Tout et al., 2010). Programs have myriad curricular choices, but curricula generally fall under two broad categories: whole child (global) and content specific (targeted).

Whole-child (global) curricula

Performance standards for Head Start require that curricula educating the “whole child” be developmentally appropriate and encourage classroom interactions that promote children’s development in multiple domains (U.S. Department of Health & Human Services, 2016; Zigler & Bishop-Josef, 2006). Whole-child or global curricula typically take a constructivst approach to learning. These curricula are not domain specific but rather promote children’s learning through a developmental-interaction approach. As a result, most preschool programs—including Head Start, pre-K, and private centers—use whole-child curricula. Survey data from the 2009 Head Start Family and Child Experiences Survey show that the majority of Head Start teachers report using Creative Curriculum for Pre-school (53%) and HighScope Curriculum (15%) as their primary curricula (Moiduddin, Aikens, Tarullo, West, & Xue, 2012), with the same patterns found in the 2012 National Survey of Early Care and Education (55% and 17%, respectively; Jenkins & Duncan, 2017). Apart from these two widely used curricular packages, teachers have reported using other published whole-child curricula (e.g., Scholastic, HighReach), and some use a curriculum designed locally at the teacher or school level.

In contrast with Head Start programs, curricular mandates for state pre-K programs vary extensively. States such as Georgia and North Carolina and accrediting agencies such as the National Association for the Education of Young Children often provide program directors with a preapproved assortment of curricula from which to choose, including whole-child and content-specific curricula. Other states ask providers to submit their curricula for approval on a case-by-case basis (e.g., Illinois, Florida). Still, most state pre-K classrooms also use published whole-child curricula. Among pre-K centers sampled in the National Survey of Early Care and Education, 41% use a whole-child curriculum (75% of these centers use Creative Curriculum; Jenkins & Duncan, 2017). Data from the National Center for Early Learning and Development Multi-State Study of Pre-Kindergarten (Early et al., 2005) also indicate Creative Curriculum as the most widely used curriculum package across pre-K programs in the six states studied. Despite being the most commonly used curriculum in Head Start and state pre-K settings, little empirical support exists on the effectiveness of Creative Curriculum and most whole-child curricula for promoting children’s school readiness outcomes (National Center on Quality Teaching and Learning, 2014; U.S. Department of Education, 2013).

Content-specific (targeted) curricula

The development and evaluation of content- or skill-specific preschool curricula or curricular supplements have increased drastically in recent years. Indeed, 20% of Head Start and 25% of pre-K classrooms use a skill-specific curriculum (focusing on math or literacy) as either the sole curriculum or a supplement to a more general comprehensive or whole-child curriculum (Jenkins & Duncan, 2017). The growth in content-specific curricula stems from an increasing focus on developing children’s early academic and social skills, as well as from substantial evidence that exposure to explicit learning opportunities in specific content domains contributes to the effectiveness of early childhood programs (Clements & Sarama, 2007; Hamre, Downer, Kilday, & McGuire, 2008; Preschool Curriculum Evaluation Research [PCER] Consortium, 2008). Supporters of content-specific curricula argue that young children benefit most from sequenced instruction, where learning activities build on existing skills and focus on specific school readiness skills (Clements & Sarama, 2007, 2008; Wasik & Hindman, 2011).

A number of recent experimental evaluations of skill-targeted curricular models in Head Start and state pre-K programs found positive impacts on low-income children’s early reading and math skills (Assel, Landry, Swank, & Gunnweig, 2007; Clements & Sarama, 2007, 2008; Diamond, Barnett, Thomas, & Munro, 2007; Fantuzzo, Gadsden, & McDermott, 2011; Klein, Starkey, Clements, Sarama, & Iyer, 2008), highlighting the importance of structured literacy and math learning activities during preschool. For example, children who receive the Building Blocks curriculum—a preschool math curriculum—scored significantly higher on assessments of early mathematics skills than children who attended preschool not using the curriculum (Clements & Sarama, 2007, 2008). These effects are indeed targeted; in a later study, Clements et al. (2011) found that the curriculum had significant effects on mathematics skills. These effects also targeted domains that were not a primary focus of the curriculum. Sarama, Lange, Clements, and Wolfe (2012) found that the curriculum had an effect on children’s language and literacy skills. Similarly, children who received curricula focused on developing language and literacy skills demonstrated larger gains in the targeted domain when compared with children who received a general whole-child curriculum (Justice, Mashburn, Pence, & Wiggins, 2008; Longian et al., 2013). Results are comparable for curricula aimed at promoting children’s reading and behavior management. The PATHS curriculum for children in Head Start was designed to improve social competence and reduce problem behavior (Domitrovich, Cortes, & Greenberg, 2007). After experimental assignment to the PATHS curriculum, children in the intervention group showed small to moderate improvements in the targeted domains of knowledge and understanding of emotion, social problem-solving skills, and social behaviors (Bierman, Domitrovich, et al., 2008; Morris et al., 2014) and improvements in nontargeted domains of executive function (Bierman, Nix, Greenberg, Blair, & Domitrovich, 2008).

This evidence suggests that content-specific curricula are likely an important tool for early childhood education programs to children’s school readiness skills in targeted academic and socioemotional domains. In fact, the Head Start advisory committee recommended that grantees use scientifically validated content-specific curricula or curricular supplements, given the preponderance of evidence of their effects, but it noted that more evidence is needed concerning the effects of different curricula on children’s outcomes (Advisory Committee on Head Start Research and Evaluation, 2012). Furthermore, Boston’s widely hailed public pre-K program uses a unique curricular approach that combines two content-specific curricula—Building Blocks and the literacy and language curriculum Opening the World of Learning (Schickedanz & Dickinson, 2005)—which may have driven its strong program impacts on children’s learning (Weiland & Yoshikawa, 2013). Thus, it is of considerable interest to practitioners and policy makers to determine whether content-specific curricula are effective across different preschool settings, to understand how best to support children’s academic and socioemotional development in public preschool programs.

Examining heterogeneity of treatment effects across different settings is important for understanding the modest and sometimes null effects of early childhood interventions. In one study, Assel et al. (2007) considered how content-specific curricula differ across preschool program settings. As part of a national study, they evaluated the effectiveness of two curricula targeting children’s literacy and language skills—Let’s Begin With the Letter People and Doors to Discovery—within multiple preschool settings that included Head Start, Title 1, and universal pre-K classrooms. The curricula were found to be effective across all three settings, but the greatest gains were made for children in Head Start classrooms that used literacy and language–specific curricula, as compared with children in the Title 1 and universal pre-K classrooms. The current study differs from that of Assel et al. in that we include classroom-level processes as intermediary outcomes and consider the different characteristics of Head Start and state pre-K programs, including the training and educational background of teachers, in moderating the impacts of curricula type. Approximately half the children enrolled in their study were also enrolled in the national evaluation component of our study, from which we draw our data. Furthermore, unlike Assel et al., our study is not an evaluation of the effects of individual curricula on children’s school readiness, nor does it include treatment variation with and without teacher mentoring; rather, it is an examination of the impacts of content-specific curricula more generally, including three language and literacy–focused curricula as well as a mathematics-focused curriculum.

Present Study

The purpose of our study is to test whether content-specific curricula operate differently in Head Start and pre-K programs as measured by classroom processes (e.g., quality) and children’s outcomes, given differences in the salient features of the programs and the populations served by these programs. Based on prior research and the documented variation in quality within both programs, predictions about which program may benefit more from such curricula are unclear. It is possible that because pre-K programs do not offer the comprehensive services included in the Head Start model, teachers and administrators are better able to focus more exclusively on children’s academic skills, which may mean that teachers are better primed to facilitate quality content-specific instruction, thereby providing a stronger advantage for children attending these programs. However, content-specific curricula may provide a greater value added to the comprehensive nature of Head Start programs in terms of developmentally appropriate academic instruction, benefiting Head Start children more than pre-K children. Another possibility is that content-specific curricula are robust to different early learning environments and are similarly effective at improving children’s skills in both programs.

We take advantage of a multisite experiment of curricular effectiveness to examine whether differences exist in the impacts of randomly assigned, content-specific curricula or curricular supplements in Head Start and state pre-K settings on classroom processes and children’s school readiness skills. Our analyses are based on a subset of the PCER study (PCER Consortium, 2008); specifically, we use four study sites where Head Start and state pre-K program classrooms were included in both treatment and control conditions, allowing us to test whether the effects of a curriculum on children’s outcomes varies by program type and features. Curricula were randomly assigned, but program type was not; as such, we use matching methods to create balance across child participants across program type. We provide details about the PCER study, our study subsample, and analyses in the following section.

Method

Data

This study uses secondary data from the PCER initiative (PCER Consortium, 2008), which involved 12 grantees in various locations across the United States to implement independent experimental evaluations of the efficacy of 14 preschool curricula. The study included 2,911 children, 315 preschool classrooms, and 208 preschools across 13 states. Each grantee individually recruited early childhood education centers, conducted random assignment, and managed his or her own evaluation with assistance from large research firms. Mathematica Policy Research (Princeton, NJ) and Research Triangle Institute (Rockville, MD) conducted classroom and child assessments to ensure that instruments and measurement were consistent across research sites. The research site was a collection of preschool programs or classrooms in a specific geographic location that were recruited by each grantee. Level of random assignment differed across grantees, with most randomly assigning whole preschools to the treatment or business-as-usual control condition and a few randomly assigned classrooms within pre-school centers to treatment or business-as-usual curriculum. For our subsample, which we describe later, classrooms within preschool centers were randomly assigned to treatment and control curricula. The centers included in the PCER study were public preschools, Head Start programs, and private child care centers. The majority of the centers served children from low-income families. Data were collected on the children attending preschool programs and their parents during the fall and spring of the 2003–2004 school year.

We use these data to answer our research questions, which are based on randomly assigning classrooms to content-specific curricula and sampling classrooms from Head Start and state pre-K programs. Our analytic sample was restricted to sites that implemented content-specific curricula and included Head Start and pre-K classrooms only, omitting private child care centers. This led us to drop approximately 2,140 children from the original PCER study sample (PCER Consortium, 2008). Our sample is limited to three research sites, for a total of approximately 770 children and 98 centers (42 Head Start and 56 state pre-K). Our analytic sample was further limited to children within our three focal sites for whom there was at least one academic outcome at the end of preschool, which excluded another 50 children who were missing on all outcomes. In our analytic sample, the majority of the children were from low-income families and had mothers with less than a high school degree. Children were on average 4.5 years old at baseline (Head Start = 54.25 months, pre-K = 54.08 months), and about half were from families with at least one parent employed (Head Start = 53%, pre-K = 55%). Approximately 34% of the children in Head Start and 23% in pre-K were Black (Table 2). Note that children were not randomly assigned to preschool programs in the PCER. We address the possibility of selection into preschool environments in our analyses here.

TABLE 2

Child and Family Backgrounds and Demographics and Child School Readiness Skills by Program Type at Baseline (Fall of Preschool 2003)

	Observed group means		Propensity score– weighted group means		Mean difference postmatching
	Pre-K	Head Start	Pre-K	Head Start
Baseline covariates
Female	0.48	0.49	0.46	0.48	0.02
Black	0.23	0.34	0.30	0.30	0.00
Hispanic	0.20	0.33	0.28	0.26	0.02
Asian	0.03	0.01	0.02	0.03	0.01
Other	0.10	0.06	0.08	0.07	0.01
Age, months	54.08	54.25	53.99	53.84	0.15
Parent education, years	13.45	12.29	12.75	12.67	0.08
Parent working	0.55	0.53	0.51	0.53	0.02
Parent’s age, years	32.95	31.33	31.71	32.00	0.29
Log of income	3.47	2.92	3.10	3.14	0.04
Receiving welfare	0.10	0.14	0.17	0.14	0.03
Missing
Parent education	0.11	0.08	0.09	0.1	0.01
Parent working	0.11	0.08	0.09	0.1	0.01
Parent age	0.11	0.09	0.10	0.11	0.01
Log of income	0.21	0.25	0.21	0.26	0.05
Receiving welfare	0.12	0.09	0.10	0.12	0.02
Child outcomes—fall 2003
Standard score
PPVT	94.03	86.15	89.96	90.85	0.89
WJLW	101.67	97.84	100.2	99.81	0.39
WJS	94.27	92.00	91.67	93.00	1.33
WJAP	98.51	94.59	96.36	96.71	0.35
CMAAa	0.45	0.40	0.44	0.43	0.01
Social skills	105.24	102.93	104.2	105.7	1.50
Behavior problems	98.02	99.02	98.22	96.66	1.56
Teacher characteristics—fall 2003
Black	0.11	0.43	0.23	0.24	0.01
Hispanic	0.05	0.11	0.11	0.13	0.02
Asian	0.06	0.05	0.05	0.05	0.00
Other race	0.03	0.02	0.01	0.02	0.01
No college	0.00	0.25	0.21	0.22	0.01
Associate	0.05	0.22	0.20	0.23	0.03
Bachelor	0.61	0.44	0.35	0.33	0.02
Master	0.34	0.09	0.24	0.22	0.02
Log of annual salary	4.00	3.71	3.76	3.68	0.08
Female	0.98	1.00	0.99	1.00	0.01
Age, years	45.74	40.26	45.24	44.15	1.09
Teaching experience, years	17.83	12.31	16.49	14.28	2.21
Classroom quality—spring 2004, standardized
ECERS
Total score	0.13	0.04	0.08	0.06	0.02
Factor 1: Language/Interactions	0.14	0.02	0.09	0.04	0.05
Factor 2: Provisions for Learning	0.09	0.05	0.06	0.02	0.04
Arnett Caregiver Interaction total	0.14	0.08	0.11	0.10	0.01
TBRS
Math instructional
Quality	0.26	−0.27	0.12	−0.03	0.15
Quantity	0.27	−0.31	0.13	−0.05	0.18
Literacy instructional
Quality	0.25	−0.38	0.06	−0.01	0.07
Quantity	0.18	−0.39	0.07	−0.14	0.21
Observations, n
Child	450	320	450	320
Classrooms	60	40	60	40

Note. All observations are rounded to the nearest 10 in accordance with National Center for Education Statistics data policies. CMAA = Childhood Mathematics Assessment–Abbreviated; ECERS = Early Childhood Environment Rating Scale–Revised; PPVT = Peabody Picture Vocabulary Test; pre-K = prekindergarten; TBRS = Teacher Behavior Rating Scale; WJAP = Woodcock-Johnson Applied Problems; WJLW = Woodcock-Johnson Letter Word; WJS = Woodcock-Johnson Spelling.

Composite score.

Measures

Curricula

Content-specific curricula were randomly assigned to classrooms at the three sites that we include in our analytic sample. In some cases, teachers in the treatment classrooms received additional curricular supports to implement the treatment curricula. Control group classrooms implemented their business-as-usual curriculum, which was either a whole-child curriculum (e.g., Creative Curriculum) or a locally or teacher-developed curriculum. Thus, the whole-child curricula included in our study represent the curricula as they are typically implemented in large-scale preschool programs, because they did not receive any intervention services from the PCER study (PCER Consortium, 2008).

Overall ratings of implementation from observers trained for each curriculum, experimental and control, were reported in the PCER report (PCER Consortium, 2008). Fidelity ratings ranged from 0 (not at all) to 3 (high), and implementation of most curricula, including the control curricula, was judged medium, with a score in the range of 2.0 to 2.5. We describe each site and its treatment curricula in further detail.

Pre-K Mathematics

Researchers from the University of California, Berkeley, and the State University of New York at Buffalo implemented the math curriculum Pre-K Mathematics supplemented with DLM Early Childhood Express Math Software (Klein et al., 2008) in preschool classrooms in California and New York. Several curricula were implemented in the control condition, including Creative Curriculum, HighScope, and local school district and teacher-developed curricula. The Pre-K Mathematics curriculum consisted of structured small-group mathematics activities as well as concrete manipulatives for use by teachers and children in preschool classrooms. Teachers were also provided with a manual that linked the classroom activities to activities in the home. Similar materials and activities were available to children during free play. The DLM Early Childhood Express Math Software included numerical, quantitative, geometric, and spatial activities. The software program provided individualized preschool math instructional activities for children to use approximately twice a week.

Doors to Discovery and Let’s Begin With the Letter People

Researchers at the University of Texas Health Science Center at Houston implemented the literacy curricula Doors to Discovery and Let’s Begin With the Letter People (Assel et al., 2007). Classrooms within preschools were assigned to one of the two treatment curricula or the business-as-usual curriculum (locally developed). The Doors to Discovery curriculum is a preschool program based on the areas identified as being important for literacy success: oral language, phonological awareness, and concepts of print, alphabet knowledge, writing, and comprehension. The program focuses on the use of learning centers and shared literacy activities in the preschool classroom. Teachers were provided with initial training and ongoing professional development support. Let’s Begin With the Letter People emphasizes early language and literacy development through play. In addition to classroom teaching, the program has a strong home/parent component. These programs were separately compared with a control group, implementing teacher-developed curricula.

Language-Focused Curriculum

Researchers from the University of Virginia implemented the Language-Focused Curriculum (Bunce, 1995; Justice et al., 2008). Individual classrooms were assigned to the treatment or business-as-usual curriculum (HighScope). The curriculum has a thematic organization and focuses on the use of daily dramatic play to teach and use linguistic concepts. There are teacher- and child-led activities with explicit attention to oral language development that is enhanced by high-quality teacher-child conversations. The control teachers reported using the HighScope Curriculum.

Child school readiness skills

We use several literacy, mathematics, and socioemotional outcomes assessed during the fall and spring of children’s preschool year. All measures are considered valid and reliable and are widely used within the field of child development. To assess multiple outcome domains and to mitigate chance findings, we created composite scores for literacy, mathematics, and socioemotional outcomes by combining measures within each outcome domain (described later). Each measure was standardized within the sample to have a mean of 0 and a SD of 1 and then averaged with the other measures within that domain. We then restandardized each composite to have a mean of 0 and a SD of 1. We also constructed an academic composite score combining the math and literacy composites and then restandardized the composite variable. Literacy, language, and mathematics skills were assessed by trained administrators from the Mathematica research team, and socioemotional skills were assessed by preschool teachers.

Literacy and language skills

The literacy composite was drawn from three literacy outcomes. The Peabody Picture Vocabulary Test (PPVT; Dunn & Dunn, 1997) assesses children’s vocabulary by having children point to the picture that represents the word spoken to them by the assessor. Words increase in difficulty, and scores are standardized for the age of the child. The second and third literacy measures are the Letter Word and Spelling subtests from the Woodcock-Johnson Psycho-Educational Battery–Revised III (Woodcock, McGrew, & Mather, 2001). The Letter Word subtest is similar to the PPVT in that it asks children to identify the letter or word spoken to them, and the test gradually increases in difficulty to require the child to read words out of context. The Spelling subtest requires children to write and spell words presented to them. Scores from both assessments were also standardized by the age of the child and nationally normed. The individual assessments (PPVT; Letter Word and Spelling subtests) were averaged for a composite score (α =.66).

Mathematics skills

The math composite comprised two math measures. The Woodcock-Johnson Applied Problems (Woodcock et al., 2001) subtest assesses children’s basic skills (e.g., number recognition) and requires children to solve increasingly difficult math problems. Similar to the literacy and language measures, the Applied Problems subtest is standardized for the child’s age and nationally normed. The second math assessment, the Childhood Mathematics Assessment–Abbreviated (Klein & Starkey, 2002), was designed for the study. It assesses children’s math competencies in numbers, operations, geometry, patterns, and measurement. Our analyses use the composite score from the assessment. We then averaged the two mathematics skills measures to form a composite score (α = .67).

Socioemotional skills and behaviors

The socioemotional composite was created from a scale constructed by the Social Skills Rating System (Gresham & Elliot, 2008), which is a teacher-administered assessment of children’s social skills and problem behaviors. The Social Skills Rating System preschool edition contains 30 items related to social skills and 10 items related to problem behaviors. Each item is rated on a 3-point scale (0 = never, 2 = very often). The social skills and problem behaviors scales are also nationally normed. The problem behaviors scale was reverse coded (i.e., higher scores indicate better behavior), averaged (α = . 76), and then restandardized for the analyses.

Classroom processes and quality

Classroom processes and quality of care were measured with several instruments, each considered to be valid and reliable and each widely used within the field of child development. Each measure was standardized within the sample to have a mean of 0 and a SD of 1. We discuss individual measures and their scales in greater detail.

Early Childhood Environment Rating Scale–Revised

The Early Childhood Environment Rating Scale–Revised (ECERS-R; Harms, Clifford, & Cryer, 1998) is a widely used observer-rated measure of global classroom quality, designed for use in classrooms serving children between 2.5 and 5 years of age, and it was used in each study. Scores on the ECERS-R range from 1 to 7, with 1 indicating inadequate quality; 3, minimal quality; 5, good quality; and 7, excellent quality. The scale’s authors report an internal consistency of .92 for the total scale. We report the total ECERS-R score and the Provisions for Learning and Interactions factor scores in Table 2.

Arnett Caregiver Interaction Scale

This observational measure consists of 26 items reflecting teacher sensitivity, harshness, and detachment, which are rated on a 1–4 scale indicating how characteristic they are of the teacher (1 = not at all, 4 = very much; Arnett, 1989). Psychometric analyses suggest that the items load onto a single factor (Cronbach’s α= .93).

Teacher Behavior Rating Scale

The Teacher Behavior Rating Scale uses trained observers to rate the quality and quantity of academic activities present in a classroom (Landry, Smith, Swank, Assel, & Vellet, 2001). This assessment measures two content areas: math and literacy. Literacy is composed of five subdomains—written expression, print and letter knowledge, book reading, oral language, and phonological awareness. Quality of activities were rated from 0 to 3 (0 = activity not present, 3 = activity high quality). Quantity of activities was similarly rated from 0 to 3 (0 = activity not present, 3 = activity happened often or many times). For both scales, we use the average score across each rated activity. Cronbach’s α is .94 for the math scale and .87 for the literacy scale.

Covariates

The child- and parent-level controls included in the analyses are as follows: baseline achievement scores (fall of preschool year), child gender (1 = female), child race (White as the reference category; Black, Asian, Hispanic, and other), number of years of education of mother or primary caregiver, household income in thousands of dollars, and indicators for whether receiving welfare aid, mother employment status, and marital status. We used the following teacher and classroom covariates in our classroom process models: proportion of classroom female; Hispanic, Black, Asian, or other race; mother’s education; and teacher characteristics, including hourly wage, education, experience.

Analysis

Our analysis proceeded in the following steps. First, we examined whether randomly assigned curricula had different impacts on classroom processes in Head Start and state pre-K. Next, we addressed the fact that children were not randomly assigned to preschool programs, by adjusting the sample with propensity score weighting. We then used these weights in our final impact analyses where we tested for heterogeneity in the impact of content-specific curricula on children’s outcomes in Head Start and pre-K.

Classroom process regression models

We used ordinary least squares regression to test for differential effects between targeted preschool curricula and classroom process outcomes for Head Start and pre-K classrooms. Because curricula were randomly assigned to both programs, results from these models can be considered causal effects of targeted curricula on pre-K and Head start classrooms. Our model is as follows: where Y is the classroom process outcome of interest (ECERS-R, Arnett, Teacher Behavior Rating Scale math and literacy quantity) for classroom c in research site j. Pre-K represents whether the classroom was Head Start or pre-K, with Head Start as the reference group. Targeted is a dichotomous indicator of assignment to a literacy or math curriculum. Targeted × Pre-K is an interaction between receiving a math or literacy curriculum and the pre-K classroom indicator. This is our coefficient of interest because it indicates whether pre-K classrooms are different on the classroom process outcomes from the use of a targeted curriculum as compared with Head Start classrooms at the same research site. Covariates are teacher and classroom covariates; a is a constant; γ represents research site fixed effects; and e is an error term. We also included baseline classroom process scores. Standard errors were clustered at the center level.

Propensity score weighting

Although children were similar on baseline characteristics (since they were drawn from a low-income sample), we applied propensity score weights to adjust for any sample-specific differences among children to isolate curricular impacts. For example, past research suggests that children who enter Head Start often differ from children who attend other types of early childhood programs, demonstrating more behavioral problems and having more disadvantaged parents (Lee, Zhai, Brooks-Gunn, Han, & Waldfogel, 2014; Zhai et al., 2011). We then modeled treatment (1 = Head Start, 0 = pre-K) using all the covariates listed in Table 2 and predicted each student’s propensity for treatment from this model. After calculating the weights as the inverse of the propensity score, we assessed comparability in covariate means across preschool program type. Our balance checking involved regressing each covariate on the Head Start indicator using propensity score weights (Rosenbaum & Rubin, 1985). The results presented here report the propensity score–weighted group means for Head Start and public pre-K as compared with unweighted group means. The two groups become very similar with respect to observed covariates after weighting, and there are no remaining significant differences between Head Start or pre-K and the covariates. We also ran ordinary least squares regression results for these models without applying the propensity score weights and found that the results were comparable.

Child outcome regression models

We applied the propensity score weight to our outcome analyses. We use multivariate regression to examine the differential associations between targeted preschool curricula and children’s cognitive and socioemotional outcomes for Head Start and pre-K classrooms. Our analyses include research site fixed effects, comparing children in the treatment and control groups within the same research site. For each outcome composite score, our model is as follows: where Y is the individual outcome of interest (literacy, math, academic, socioemotional composite) observed for child i in classroom c in research site j. Pre-K represents whether the child was in Head Start or pre-K classroom, with Head Start as the reference group. Targeted is a dichotomous indicator of assignment to a literacy or math curriculum. Targeted × Pre-K is an interaction between receiving a math or literacy curriculum and the pre-K indicator. This is our coefficient of interest because it indicates whether children in pre-K classrooms differentially benefit from the use of a targeted curriculum during preschool as compared with similar children in Head Start classrooms (at the same research site). Covariates are child and family covariates for child i. We included all academic baseline scores in each academic outcome model (literacy, math, and academic composite scores), including only baseline socioemotional scores for the social skills and problem behaviors models. Because these socioemotional outcomes were teacher reported, our models would have been subject to measurement bias had we included them in the academic outcome models. γ are research site fixed effects; a is a constant; and e is an error term. Standard errors were clustered at the classroom level for all models.

We estimated additional models that test for moderation of program- and curriculum-type differences from the training and educational background to examine the mechanisms through which differences in classroom processes may influence curricular impacts. This adds to the specification above the focal teacher variable (e.g., education level) and a version of b3 that expresses this as a three-way interaction (e.g., Targeted × Pre-K × TeacherEd).

We addressed missing data using dummy variable adjustments. Variables were created for the baseline scores and covariates indicating if the value was missing, and the missing value on the variable of interest was set to zero. This method is particularly effective for data analysis from random assignment studies (Puma, Bell, Cook, & Heid, 2010).

Results

Table 2 summarizes the weighted and unweighted group means of child and family demographic characteristics and all child outcomes by preschool program type for each curriculum and site included in our study, as well as other study information. We present results from the unweighted group means, revealing that, in general, children served in these two preschool program types are indeed different. Children attending state pre-K score higher on every measure at baseline. Furthermore, classrooms across the two program types are also different, with pre-K programs scoring higher on the ECERS-R, Arnett Caregiver Interaction Scale, and Teacher Behavior Rating Scale measures. Descriptive analyses of the ECERS-R subscales by program type indicate that pre-K classrooms were stronger in language, communication, and interactions, whereas Head Start classrooms were stronger in supervision of children and in providing opportunities for art and sand/water activities (see online Supplemental Table S1). For the purposes of our models in estimating the impact of content-specific curricula, we find that child, family, and classroom characteristics are very similar across preschool program type after applying the propensity score weights.

Our classroom process models (Table 3) indicate overall differences in the ECERS-R and Arnett Caregiver Interaction Scale (Models 1 and 2) between pre-K and Head Start classrooms, with pre-K coefficients of .23 and .29 SD, respectively. The coefficients for targeted curricula in Models 3 through 6 show that targeted curricula affected classroom processes in the targeted content domain; classrooms with targeted curricular supplements have more frequent literacy (.25 SD) and math (.27 SD) activities and are higher in instructional quality in literacy (.27 SD) and math (.30 SD). Recall that targeted curricula were randomly assigned to either classrooms or centers, so these results are experimental impacts. The interaction term between pre-K and using a targeted curriculum tells us whether using a targeted curriculum differentially affected classroom processes in either pre-K or Head Start classrooms. The interaction was not significant in any model, indicating that across all classroom processes measured in the study, targeted curricula affect classroom processes in the same way in both Head Start and pre-K programs.

TABLE 3

Experimental Estimates of PCER Content-Specific Treatment Curricula on Classroom Outcomes Interacted With Program Type

	(1) ECERS	(2) ACI	TBRS math		TBRS literacy
			(3) Instructional quality	(4) Instructional quantity	(5) Instructional quality	(6) Instructional quantity
Pre-K	0.23**(0.10)	0.29**(0.12)	0.17(0.10)	0.13(0.09)	0.15(0.10)	0.12(0.09)
Targeted curricula	0.12(0.08)	0.12(0.08)	0.25*(0.09)	0.27**(0.10)	0.20*(0.09)	0.30**(0.09)
Pre-K × Targeted curricula	0.16+(0.09)	−0.06(0.12)	0.05(0.12)	−0.02(0.14)	0.10(0.12)	−0.05(0.13)
Observations (classroom), n	100	100	90	90	90	90

Note. All outcomes are in SD units. Standard errors clustered at the center level (in parentheses). All models include proportion of classroom female; Hispanic, Black, Asian, or other race; mother’s education; teacher characteristics (hourly wage, education, experience). Baseline measures and teacher/classroom characteristics are included in each model Classroom n rounded to nearest 10 per National Center for Education Statistics data security policy. ACI = Arnett Caregiver Interaction; ECERS = Early Childhood Environment Rating Scale–Revised; PCER = Preschool Curriculum Evaluation Research; pre-K = prekindergarten; TBRS = Teacher Behavior Rating Scale.

p < .10.

p < .05.

p < .01.

Table 4 displays results for children’s academic and socioemotional outcomes. All dependent variables are standardized to have a mean of 0 and SD of 1 such that the coefficients should be interpreted as effect sizes. These figures are propensity score–weighted to address family-level selection into Head Start and pre-K programs. The coefficient of interest in all models is the interaction term between being in a pre-K classroom and having a targeted curriculum. Overall, content-specific curricula are positively and significantly associated with the focal academic domain, and these associations do not differ by preschool program type. Model 1 displays results for the effect of content-specific curricula on children’s literacy composite score. The main effect of content-specific curricula is .08 (p > .10), and the interaction term testing for a differential effect of content-specific curricula in pre-K classrooms is not significant (effect size = .05). Model 2 displays outcomes for children’s math composite scores. The main effect of content-specific curricula is significant, .21 (p < .05), but the differential effect of content-specific curricula by program type is not (effect size = −.18). Model 3 displays results for the academic composite scores. The main effect of content-specific curricula is .23 (p < .05). The effect size for the difference is not significant (effect size = −.09). Last, Model 4 displays outcomes for children’s social skills composite. The main effect of content-specific curricula is –.18 and not statistically significant, and neither is the interaction term (effect size = .09). We conducted joint F tests for the difference in effect sizes for each outcome and found that none is significant. In addition to these primary models for children’s outcomes, we estimated our models using each child assessment as a separate outcome. These estimates show the same pattern of results and are presented in online Supplemental Table S2.

TABLE 4

Propensity Score–Weighted Estimates of PCER Content-Specific Curricula on Child Outcomes Interacted With Program Type Predicting Child School Readiness Skills at the End of Preschool

	Composite
	(1) Literacy	(2) Math	(3) Academic	(4) Social skills
Pre-K	0.36*(0.09)	0.31*(0.09)	0.33*(0.08)	−0.06(0.17)
Targeted curricula	0.08(0.10)	0.21*(0.10)	0.23*(0.09)	−0.18(0.19)
Pre-K × Targeted curricula	0.05(0.13)	−0.18(0.14)	−0.09(0.12)	0.09(0.25)
Observations (child), n	760	760	760	720

Note. Standard errors are in parentheses and clustered at the classroom level. Prior achievement (included in all models) refers to baseline achievement scores measured in the fall of pre-K. Child family background characteristics (included in all models) include child’s gender, race, and age in months. Family background characteristics (included in all models) include parent or primary caregiver’s highest level of education in years, annual household income in thousands, whether receiving welfare. Teacher characteristics (included in all models) include gender, race, level of education (with no degree as the reference group), salary in thousands, age in years, and teaching experience in years. All models include site/grantee fixed effects and missing dummy variables. Number of child observations were rounded to the nearest 10 in accordance with National Center for Education Statistics data policies. PCER = Preschool Curriculum Evaluation Research; pre-K = prekindergarten.

p < .05.

We also attempt to unpack how variations in one of the more salient program features—teacher’s level of education and years of experience—influence children’s outcomes by curricular type through a set of interaction analyses (Table 5). Across all models, we did not find any evidence of teaching experience or teacher’s education level moderating a differential association between content-specific curricula and children’s outcomes.

TABLE 5

Propensity Score–Weighted Estimates of PCER Content-Specific Treatment Curricula on Child Outcomes Interacted With Program Type and Teacher Experience and Education

	Composite
	(1) Literacy	(2) Math	(3) Academic	(4) Social skills
Pre-K	0.34*(0.09)	0.28*(0.09)	0.30*(0.09)	0.06(0.18)
Targeted curricula	0.10(0.10)	0.29*(0.10)	0.22*(0.10)	−0.11(0.20)
Pre-K × Targeted curricula	0.07(0.14)	−0.12(0.16)	−0.05(0.14)	−0.11(0.15)
Teacher years of experience	0.01(0.01)	0.01(0.01)	0.01(0.01)	0.00(0.01)
Teacher has associate degree	0.17(0.17)	0.04(0.21)	0.10(0.17)	0.10(0.28)
Teacher has bachelor degree	0.21(0.27)	0.13(0.18)	0.10(0.15)	0.14(0.18)
Teacher has master degree	0.07(0.22)	0.15(0.17)	0.13(0.19)	0.34(0.32)
Pre-K × Targeted curricula × Teacher years of experience	0.00(0.01)	0.01(0.01)	0.02(0.01)	0.01(0.02)
Pre-K × Targeted curricula × Teacher has associate degree	0.27(0.33)	0.10(0.31)	0.36(0.26)	0.32(0.33)
Pre-K × Targeted curricula × Teacher has bachelor degree	0.45(0.31)	0.43(0.34)	0.47(0.32)	0.24(0.45)
Pre-K × Targeted curricula × Teacher has master degree	0.33(0.33)	0.17(0.27)	0.26(0.28)	0.16(0.38)
Observations (child), n	760	760	760	720

Note. Standard errors are in parentheses and clustered at the classroom level. Prior achievement (included in all models) refers to baseline achievement scores measured in the fall of prekindergarten. Child family background characteristics (included in all models) include child’s gender, race, and age in months. Family background characteristics (included in all models) include parent or primary caregiver’s highest level of education in years, annual household income in thousands, whether receiving welfare. Teacher characteristics (included in all models) include gender, race, level of education (with no degree as the reference group), salary in thousands, age in years, and teaching experience in years. All models include site/grantee fixed effects and missing dummy variables. Number of child observations were rounded to the nearest 10 in accordance with National Center for Education Statistics data policies. PCER = Preschool Curriculum Evaluation Research; Pre-K = prekindergarten.

p < .05.

Discussion

The aim of our study was to understand whether the implementation of a targeted math or literacy curriculum might differentially affect classroom processes and children’s outcomes by preschool program type. Existing research examined the effects of individual preschool curriculum interventions, but almost no studies compared across different types of curricula and preschool programs to determine their differential relationships to children’s school readiness skills. The degree to which key components of children’s classroom experiences in center-based preschool programs (e.g., curricula) contribute to children’s skill development is a key area for research, given the national focus on school readiness and federal and state preschool program initiatives and the mounting accountability pressure on such programs.

Our results from the classroom process models indicate that targeted curricula increased classroom literacy and math activities in both preschool programs. The results from the child outcome models indicate that Head Start and pre-K programs equally benefit from a content-specific curriculum, where targeted curricula increased children’s outcomes in the content domain identified by the curriculum in both settings. To unpack the specific mechanisms through which curricula help improve children’s outcomes, we examined the training and educational backgrounds of teachers as process-type factors but did not find any evidence of teacher’s characteristics moderating the association between targeted curricula and children’s outcomes.

These findings are consistent with other studies showing that content-specific curricula boost children’s outcomes in the targeted content domain (Clements & Sarama, 2008; Diamond et al., 2007; Duncan et al., 2016; Fantuzzo et al., 2011; Klein et al., 2008). These results provide support for efforts to include content-specific preacademic curricula in Head Start and state-funded pre-K. The evidence suggests that curricular alternatives or supplements focused on academic skills are more successful at increasing children’s literacy and math skills than widely used whole-child curricula alone. We also find that children attending Head Start enter preschool with lower academic skills than do children enrolled in public preschool programs but that these children show greater gains over the year when compared with children in state pre-K. Both programs clearly provide an important value added to young children from low-income families.

Our findings stand in contrast to those reported by Assel et al. (2007), who found that children in Head Start classrooms implementing literacy and language curricula—Let’s Begin With the Letter People and Doors to Discovery—made the stronger gains versus children in Title 1 and universal pre-K classrooms. An important distinction between our study and that of Assel et al. is that we use the original PCER study (PCER Consortium, 2008) data to pool samples from multiple evaluation sites into one analysis, and our study includes language/ literacy- and math-focused curricula. Assel and colleagues’ study represented a different sample size, utilized different outcome measures, and assessed the curricular impacts of the mentoring version of the two literacy and language curricula that the original PCER study did not. We also took into account classroom process variables as important intermediary outcomes.

In the original PCER report (PCER Consortium, 2008), only two of the 14 PCER projects found positive effects on a few measures of children’s school readiness by the end of preschool. The math curriculum in the PCER study was one of the few that produced positive and significant effects on children’s outcomes. The findings of the PCER study were largely null, although several analytic issues, such as low statistical power, were cited to explain the lack of significant effects. We pooled curricula across the individual evaluations into broader categories, which allowed us to increase our sample size to provide more power to investigate secondary hypotheses. Our current investigation attempts to understand whether there was any variation in the impacts of content-specific curricula—considered more generally as a curricula type—across preschool program type. The novel contribution of our study is that we have put these two areas of research together—differences between preschool program type and type of curriculum being implemented. Combining these two lines of research is important in the current early childhood education policy context, where researchers are actively examining what works best for pre-school scale-up and under what conditions. More research at the intersection of program type and curricula type can help to guide the millions of dollars invested each year in curricula for public preschool programs to improve the outcomes for the children whom they serve.

With respect to curricular effectiveness more generally, other mechanisms might include the type of supports offered to teachers to implement the curricula with fidelity, such as staff training and procedures for monitoring adherence and dosage (Dane & Schneider, 1998). Professional development opportunities for teachers to implement curricular activities is paramount (Fixsen, Blase, Naoom, & Wallace, 2009; Neuman & Cunningham, 2009; Pianta et al., 2009; Powell, Diamond, Burchinal, & Koehler, 2010). In the original study, researchers provided support in preschool classrooms to help implement the curricula. In light of our results, this suggests that the instruction that preschool teachers receive for curricula and the overall level of fidelity of implementation are also important for curricular effectiveness. Unfortunately, the PCER study (PCER Consortium, 2008) did not collect data on the fidelity of implementation at the classroom level, but the original PCER report did provide overall implementation ratings from independent observers that average across all study classrooms using the same curriculum package. The report indicates that each curriculum in our sample was implemented with adequate fidelity. Ideally, we would have more information on fidelity to fully understand the intersections among program type, curricula type, and children’s outcomes that we aimed to examine in our study. However, the conditions captured in the PCER study may in this way better reflect the de facto implementation of preschool curricula for programs operating at scale—such as pre-K and Head Start—and so our results do provide useful information for the research literature on curricular use in large-scale public programs. The study and measurement of implementation fidelity in curriculum interventions is a critical area for future studies to help researchers explain why an intervention did or did not produce the expected results and what factors relate to adequate implementation and lead to effective classroom practices and child outcomes.

Our study has a few other limitations. First, we recognize that children select into preschool environments as well, and we addressed this to the best of our ability with propensity score weights; however, propensity score methods are subject to the assumption of ignorable treatment, meaning that all confounding covariates are assumed to have been observed and included in the models, which cannot be tested (Rosenbaum & Rubin, 1985). Because we had a low sample size in many research sites, controlling for site fixed effects greatly increased our standard errors, which may have limited our ability to detect differential effects by preschool program type. These data are also not nationally representative, even though the sample included children from diverse locations across the United States. Finally, another limitation of this study is that the whole-child curricula have been updated to more recent versions since the final PCER report (PCER Consortium, 2008) was released. Because there has been an increased focused on academic school readiness, these curricula often now include more specific language and literacy and math content. We acknowledge that the versions of the whole-child curricula included in our study may not reflect the most current classroom practices and activities prescribed by the curricula.

The hypotheses tested here should be tested in future research, ideally with a larger, nationally representative sample of pre-K and Head Start classrooms. To further unpack differences in curricula and program type, future studies could examine differences across pre-K auspices or, in other words, the type of site where a pre-K program is located (e.g., public school, private child care centers, community-based organization). Unfortunately, in our current study, we do not have enough pre-K centers across treatment and control conditions with different program types to do a more detailed analysis of this nature. Still, from a policy maker’s perspective, pre-K is a meaningful characterization of early childhood education programs. Therefore, our results, within the constraints of the data, are still useful for research and policy but should be followed with future research that closely examines important differences within program type. Specifically, future research should explore classroom-level differences within and between program types to identify key areas for professional development, quality improvement, and opportunities to enhance children’s learning experiences.

Heightened public and policy attention to expanding pre-school opportunities for low-income children and increasing preschool quality requires an understanding of whether different types of preschool program settings are more or less effective in promoting the school readiness skills of low-income children through the use of curricula. This is crucial for considering program funding and ensuring quality and consistency for different preschool programs. Findings from this study demonstrate that content-specific curricula are effective in different preschool settings and should be leveraged to improve children’s educational outcomes.