Effects of higher-order cognitive strategy training on gist-reasoning and fact-learning in adolescents.

Improving the reasoning skills of adolescents across the United States has become a major concern for educators and scientists who are dedicated to identifying evidence-based protocols to improve student outcome. This small sample randomized, control pilot study sought to determine the efficacy of higher-order cognitive training on gist-reasoning and fact-learning in an inner-city public middle school. The study compared gist-reasoning and fact-learning performances after training in a smaller sample when tested in Spanish, many of the students' native language, versus English. The 54 eighth grade students who participated in this pilot study were enroled in an urban middle school, predominantly from lower socio-economic status families, and were primarily of minority descent. The students were randomized into one of three groups, one that learned cognitive strategies promoting abstraction of meaning, a group that learned rote memory strategies, or a control group to ascertain the impact of each program on gist-reasoning and fact-learning from text-based information. We found that the students who had cognitive strategy instruction that entailed abstraction of meaning significantly improved their gist-reasoning and fact-learning ability. The students who learned rote memory strategies significantly improved their fact-learning scores from a text but not gist-reasoning ability. The control group showed no significant change in either gist-reasoning or fact-learning ability. A trend toward significant improvement in overall reading scores for the group that learned to abstract meaning as well as a significant correlation between gist-reasoning ability and the critical thinking on a state-mandated standardized reading test was also found. There were no significant differences between English and Spanish performance of gist-reasoning and fact-learning. Our findings suggest that teaching higher-order cognitive strategies facilitates gist-reasoning ability and student learning.

RCT Entities:   Population Interventions Outcomes

Introduction

The ability for the United States to maintain a competitive edge in the global economy is dependent on the reasoning and critical thinking skills of this and future generations of students (Ravitch, 2010). Results from the Program for International Student Assessment (PISA), which provides international testing of reading, math and science literacy in 15 year old students, indicate that students in the United States are not performing as well as their peers in a majority of developed countries (U.S. Department of Education, 2010). Although there are multiple contributing factors, one commonly espoused cause is the over-emphasis on high-stakes standardized testing that relies on “information-in/information-out” processes (i.e., fact-learning or rote memorization) rather than fostering top-down cognitive processes such as reasoning (Ravitch, 2010). Students are primarily tested on how well they learn rote facts rather than evaluating how effectively they assimilate new knowledge with world knowledge to abstract meaning (gist meaning) and apply these meanings and concepts in novel ways to different contexts (Alberts, 2009; Chapman et al., in press). This emphasis on learning rote facts may actually motivate teachers to emphasize strategies to improve the recall of isolated information rather than the application of advanced reasoning principles when reading texts. Students in turn, come to rely on these narrow strategies for comprehension and predictably have difficulties in abstracting and generalizing what has been read. As McNamara et al. (1996) point out, everyday observation confirms the results of laboratory experiments that, in general, students do not like to expend the necessary effort for learning, and are all too easily satisfied with superficial understanding. In short, the current educational system focuses on teaching students “what to learn” rather than “how to learn.”

A failure to develop adequate reasoning skills during adolescence may have a profound and lasting effect on the individual in college and throughout adulthood (Willingham, 2009). Research in cognitive neuroscience has identified adolescence as a pivotal developmental stage and critical window for acquiring reasoning and critical thinking skills in terms of both cognitive expansion and brain remodeling (Giedd et al., 2006). In particular, adolescence is the period of life when advanced reasoning skills should be developing and expanding, with continued sophistication and refinement in adulthood (Blakemore and Choudhury, 2006). The underlying neural substrates that support reasoning are undergoing dramatic growth during adolescence. Longitudinal neuroimaging research reveals extensive brain development and remodeling, particularly in the frontal lobe networks, throughout adolescence and into early adulthood (Gogtay et al., 2004).The complex frontal neural connections which subserve reasoning skills also support higher-order cognitive functions such as problem solving, decision-making, reasoning, judgment, and planning, and are often referred to as “executive control functions” (Sowell et al., 1999; Bunge et al., 2005).

In an effort to advance reasoning and higher-order cognitive skills, research suggests explicit instruction may be beneficial (Pearson and Dole, 1987; Rosenshine and Meister, 1994; Alfasi, 1998; Alvermann, 2002; Phelps, 2005; Fletcher et al., 2006; Deshler et al., 2007; Alberts, 2009). Reduced reasoning competence is elevated in students from low socioeconomic status (SES) families (McNeil, 2005). Low SES has been associated with increased vulnerability of frontal lobe development (Kishiyama et al., 2009), as well as greater risk for academic failure and school dropout, especially for students of Hispanic descent (McNeil, 2005). The alarming dropout rates have motivated educators and cognitive neuroscientists to seek evidence-based studies driven by theoretical models to better evaluate practices that purportedly enhance reasoning and learning potential in the classroom.

In the present pilot study, we focus on gist-reasoning, a form of developmentally advanced reasoning that is pivotal to new learning as defined by Brainerd and Reyna (1990)andReyna and Brainerd (1995) in their fuzzy trace model. According to fuzzy trace theory, new learning results in two forms of memory: verbatim and gist (Reyna, 2008). Memory at a verbatim level is represented by the explicit facts or concrete details. In contrast, gist-memory involves assimilating and interpreting incoming information at a generalized level of meaning. For example, in explaining what a student learned from a lesson, he or she could respond in one of two ways (Lloyd and Reyna, 2009). At a rote fact-learning level, the student conveys a listing or retelling of literal information, reproducing predominately the surface level meaning. At a gist-level, new information is integrated with previous knowledge to construct and abstract meaning, involving a process called gist-based reasoning (Chapman et al., in press).

Reyna and Brainerd (1995)and Brainerd and Reyna (1990) have synthesized extant empirical evidence supporting a theoretical basis that these two levels of memory, i.e., verbatim and gist, are encoded separately. According to their dual process fuzzy trace model, gist-memory, and precise memory for explicit facts operate independently. Regarding real-life school performance, the verbatim–gist distinction indicates there is not a direct correspondence between the two types of memory, such that an individual who has high memory for factual information may not necessarily have strong gist-reasoning skills. This dissociation between verbatim and gist appears to be counter to previously held notions derived from information processing theory which would predict that increased memory and higher-order cognitive skills, such as gist-reasoning would be linked (LaBerge and Samuels, 1974).

More recently, Reyna (2008) has proposed that when a relationship does exist between these two types of memory, it is more likely that gist-reasoning will shape the content of verbatim memory than vice versa, taking a more top-down processing, constructivism view. The empirical evidence suggests that an individual with higher gist-reasoning skills may demonstrate increased memory for details than an individual with lower gist-reasoning. In this model, verbatim memory is subordinated to gist-reasoning.

Despite educators’ recognition of the importance of gist-based reasoning skills, there is little published evidence regarding objective, informative ways to assess these skills within school and classroom contexts (Ablin, 2008). Because of these limitations, our group developed the Test of Strategic Learning (TOSL©, Chapman et al., submitted) to assess gist-reasoning ability in the summarization of texts as well as the ability to remember facts presented in texts through probes. The task of summarization provides an informative way to characterize whether a student spontaneously uses verbatim fact-learning or gist-reasoning to condense information by abstracting meaning (Brown and Day, 1983; Brown et al., 1983; Kintsch, 1998, 2004; Chapman et al., 2006; Gamino and Chapman, 2009; Gamino et al., 2009a). For example, when asked to summarize a text, a student may spontaneously: (1) produce a summary by constructing global gist concepts, that condense and abstract meaning indicative of a top-down process or (2) produce a summary in a condensed verbatim fact-based version indicative of a bottom-up approach reflective of rote surface level learning (Chapman et al., 2006; Gamino et al., 2009a). The condensed, verbatim form of summary may reflect an ability to select the most important facts in a hierarchical manner, but not the ability to go beyond the surface level meaning (Chapman et al., 2006). The distinction between these two summary forms is relevant to previous research focused on training summarization through hierarchical information structure (Taylor and Beach, 1984) or concept mapping (i.e., the latter summary-types described below;Chang et al., 2002) versus the current approach targeting summaries comprised of abstracted meaning.

Whereas the fuzzy trace model states that gist-reasoning may involve various levels of abstraction (Reyna, 1998, 2008), we focus on a specific form of gist, namely gist-reasoning that evokes a deep level of meaning. Thus, we extend the construct of gist-reasoning as described by Brainerd and Reyna (1990, 1995)andReyna and Brainerd (1995) to focus on a deeper level of meaning derived from complex information. As such, we conceptualize gist-reasoning at an abstracted level of meaning whereby ideas are combined over large sections of text through top-down cognitive processes (Chapman et al., in press). We propose that in order to produce abstracted gist meanings, one must first synthesize pertinent facts within the context of world knowledge and deduce the deeper implication of the information. The heightened capacity to integrate and consolidate information into more generalized/abstracted meanings is a remarkable ability in human cognitive development (Brown and Day, 1983; van Dijk, 1995a,b; Gabrieli, 2004).

In addition to Brainerd and Reyna (1990, 1995), Reyna and Brainerd (1995) and Gabrieli (2004), Chapman et al. (2004, 2006, in press) also suggest that gist meanings are more robustly stored and retrieved as compared to a rapid decline of memory for specific, isolated, concrete details, whether important or unimportant. Corroborating a distinction between fact-learning and gist-reasoning, we found empirical evidence of a disparity between fact-learning and gist-reasoning of text-based information in pediatric populations using the TOSL (Chapman et al., 2004, 2006, in press; Gamino et al., 2008, 2009a,b,2009c). We briefly summarize some of our findings regarding gist-reasoning and fact-learning that motivate the present study.

Children and adolescents with traumatic brain injury (TBI; Chapman et al., 2006; Gamino et al., 2009a) and those with attention deficit hyperactivity disorder (ADHD; Gamino et al., 2008, 2009a,c) showed comparable ability in basic fact-learning but significantly reduced abilities in gist-reasoning when compared to typically developing youth. In separate empirical studies, we found a dissociation between fact-learning and gist-reasoning skills in both students with TBI and those with ADHD (Chapman et al., 2001, 2005, 2006; Gamino et al., 2008, 2009a,b; Gamino and Chapman, 2009). The TBI populations across studies showed comparable performance on fact-learning tasks as compared to typically developing control groups. In contrast, we found a significantly lower performance on gist-reasoning as compared to typically developing control groups (Chapman et al., 2006; Gamino et al., 2009a,b; Cook and Chapman, in press). The pattern of relatively intact fact-learning indicates that youth with TBI recover the ability to encode details at a level comparable to typically developing adolescents. Moreover, in a longitudinal study of recovery from TBI, we found the ability to abstract gist meanings from text stalls and fails to show improvement when measured at intervals three years and longer after TBI. We described this delay as a neurocognitive stall in developing gist-reasoning (Chapman et al., 2006; Gamino et al., 2009a; Cook and Chapman, in press). Disruptions in frontal neural networks after TBI supporting higher-order reasoning have been implicated as a contributing factor (Levin et al., 1993).

Similarly, we found students with ADHD demonstrated impaired ability to produce gist meanings, whereas memory for facts was comparable to typical students (Gamino et al., 2008, 2009b,c). We postulated that the discrepancy between reduced performance of gist-reasoning and comparable performance of fact-learning is likely a result of the vulnerability of executive control that entails top-down processes found to be compromised in disorders such as ADHD. These findings motivated the question regarding the potential to mitigate or prevent impaired gist-reasoning in different groups through short-term intensive cognitive strategy training. For the purposes of this article we define top-down processing as the ability to spontaneously synthesize numerous details with prior knowledge to facilitate gist-reasoning. We define bottom-up processing as verbatim or paraphrased retell or recall of information without evidence of gist-reasoning.

To attempt to mitigate impaired gist-reasoning in these populations, we developed the Strategic Memory and Reasoning Training© (SMART©) program, at raining program designed to improve top-down reasoning skills (Chapman and Gamino, 2008). SMART is an intensive, 9–10 session program conducted over a 4-week period that specifically trains hierarchical cognitive strategies that support higher-order abstraction of meaning from incoming details and world knowledge. Thus, the training does not teach specific content for a test, but rather trains students to efficiently apply strategies to extract synthesized meanings from a wide variety of texts. The learning strategies are applicable to the content conveyed in a classroom course/textbook, a movie, on the Internet, or song lyrics. As such, SMART goes beyond basic reading and literacy programs in order to teach students how to think about information, such that the student learns to process information at a deeper level rather than surface level, “information in/information out.” In short, the SMART program teaches students “how to learn” rather than “what to learn.”

In a recent randomized study, we found that the SMART program improved gist-reasoning in youth with ADHD (Gamino et al., 2009b) compared to a control group of students with ADHD who received behavioral, attention training. At baseline testing, both groups performed similarly on gist-reasoning measures. After the training sessions concluded, children who participated in the SMART program significantly improved their gist-reasoning performance, whereas participants from the behavioral attention training did not. We gathered additional evidence for the efficacy of SMART in two summer “camp” sessions of SMART (Gamino et al., 2009c). Students with ADHD were found to improve their gist-reasoning ability after 10 sessions of SMART training in a small group setting.

Previous investigations have explored the effects of teaching higher-order thinking skills through summarization-type activities to improve learning in typically developing students (Brown et al., 1981; Palincsar and Brown, 1984; Taylor and Beach, 1984; Pearson and Dole, 1987; Malone and Mastropieri, 1992; Rosenshine and Meister, 1997; Chang et al., 2002; Kintsch, 2004). The existing evidence supports a view that strategy-based training improves learning, but the best metric remains unclear for measuring change. In fact, in Rosenshine and Meister's (1994) review of 16 studies addressing reciprocal teaching, they reported that significant differences between students receiving reciprocal teaching only and those who received explicit instruction prior to reciprocal teaching varied as a function of the type of assessment used. The results were typically significant when experimenter tests were administered and non-significant when standardized tests were used. While a number of more recent cognitive strategy studies have shown similar results using both types of measures to assess the impact of different reasoning strategies on student's ability to summarize information, predict outcomes; monitor comprehension, and apply gist-reasoning, the measures do not allow for the isolation of reasoning difficulties due to a lack of memory for specific events, difficulties due to an impairment in extracting synthesized meaning from complex text (gist), or both (see Alfasi, 1998; McMaster et al., 2005). This consistent finding may be explained by the fact that experimenter-designed assessments are typically based on tests and/or tasks that are similar to those used in the instruction (Alfasi, 1998).

It is unclear from previous studies if gains are greater for top-down compared to a bottom-up training protocol focused on fact-learning. Taylor and Beach (1984) compared the effect of training summarization skills based upon hierarchical information structure (i.e., identifying superordinate versus subordinate ideas) versus answering questions about main ideas and details on the ability to recall facts from written texts. The effect of the training protocols was measured, through responses to short answer probes, and an overall rating of quality in written expression. Results of their study revealed that both types of training (i.e., hierarchical summarization training and question–answer practice) significantly improved ability to correctly respond to short answer probes as compared to a control group. Also, both training types showed gains in the ability to recall information when writing as much as one could remember from a text about a relatively familiar topic. The hierarchical summary training group showed greater gains when compared to the control group on the recall of facts from an unfamiliar text and on quality of written expression, but not when compared to the question-answer practice group. The researchers concluded that text structure training can improve both recall of unfamiliar texts and quality of writing; however limited learning benefits were found between training hierarchical summarization and training question answering. The failure to find differences between the two training protocols may be due to the fact that both focused on the explicit important and supporting details, with one elicited through summarization and the other through probe questions.

In a related but different approach from that of Taylor and Beach (1984), Chang et al. (2002) compared the benefits of three modified concept-mapping training protocols (i.e., concept map-correction, scaffold-fading, and concept map-generation; see Chang et al., 2002 for descriptions) on performance measures of fact-learning and summarization in fifth graders. Concept mapping involves constructing a graphic representation of the information according to its importance/hierarchy, which is related to the Taylor and Beach (1984) text structure approach. Fact-learning was measured via multiple-choice questions and summarization efficiency with the latter measured by dividing the number of major idea units by the total word count. The concept map-correction group showed significant improvements in fact-learning when compared to the concept map-generation and control groups. The scaffold-fading group showed higher performance on the summarization efficiency measure than the other training groups. Neither of the concept-mapping protocols appeared to mutually benefit both fact-learning and summarization skills on the employed metrics. Additionally, it is not clear if the metric of summarization efficiency corresponds to abstraction of meaning and higher-order thinking (Ulatowska and Chapman, 1994).

Palincsar and Brown (1984) published one of the first investigations of reciprocal teaching. In an elaborate study, the investigators incorporated summarizing strategies amidst reciprocal teaching for seventh and eighth grade students over the course of 20 training sessions. Reciprocal teaching involves trainers and students taking turns leading a discussion on the text's meaning. The students were asked to summarize what they had learned from a reading, and the teachers used the students’ summaries to monitor comprehension levels. As above, the summarization training in this study entailed condensing the information to include predominantly explicitly stated information with a topic sentence that was either overtly stated or, if not present, constructed by the student with the inclusion of supporting facts. Beyond summarization, reciprocal teaching also incorporates questioning, clarifying, and predicting activities. Not only did Palincsar and Brown examine delivery of the training in student dyads, but they also tested the ability to train classroom teachers to deliver the reciprocal teaching intervention in a classroom setting. The researchers found the summarizing task the most helpful activity, demonstrating benefits in comprehension, maintenance of trained skills up to eight weeks post-training, and transfer to classroom learning. The investigators note that it was unclear whether the benefits were due largely to the reciprocal teaching approach, a single summarization strategy, or all were required to attain these benefits. Other researchers have conducted additional investigations of the efficacy of reciprocal teaching finding mostly significant results on experimental measures, with standardized measures showing less sensitivity to improvement (see Rosenstine and Meister, 1994 for review).

Additional evidence that summarization training may enhance learning is reported for students with learning disabilities (Malone and Mastropieri, 1992). In this study, summarization was trained in a method parallel to that described above by Taylor and Beach (1984) wherein the students were trained to write a summary sentence that represented who or what each paragraph was about by including the supporting core facts. An additional training condition consisted of summarization combined with self-monitoring through a check list of questions. Both of these training conditions were compared to a traditional instruction group where participants read the texts and answered fact-based questions. Both summarization training conditions achieved significantly higher levels of recall for specific information when immediately probed as compared to the traditional approach. The summarization training results also generalized to untrained material from a social studies text. However, the impact of the training protocols on higher-order thinking or reasoning was not directly addressed.

The previous theoretical and empirical data (Brown et al., 1981; Taylor and Beach, 1984; Palincsar and Brown, 1984, 1988; Chang et al., 2002) motivated the current study. The aforementioned studies examining or training summarization skills as well as others, indicate promise for enhancing learning ability (Brown et al., 1981; Taylor and Beach, 1984; Pearson and Dole, 1987; Palinscar and Brown, 1984, 1988; Chang et al., 2002; Ladewski et al., 2007). The current study expands previous studies by investigating the efficacy of top-down meaning abstraction training versus bottom-up rote memory training on enhanced performance of gist-reasoning and fact-learning in a group of inner-city eighth grade public school students, most of whom spoke English as their second language. Our study encompassed average students from low socio-economic families of largely Hispanic descent who were randomized into either a training group or a control group. The control group provided evidence for the efficacy of increased, affirmative adult interaction on performance. While summarizing skills were not directly taught to our training groups, we used summarization and directed probe questions to determine the efficacy of our cognitive training. Although previous studies have used summarization as a metric for improved cognition, it is not known if the metrics were based primarily on fact retrieval/recall or gist-reasoning abilities (Palincsar and Brown, 1984, 1988; Taylor and Beach, 1984; Rosenstine and Meister, 1994; Chang et al., 2002).

The current pilot study elaborates previous work by proposing and testing a metric of summarization using the construct of gist-reasoning that measures abstracted ideas that are not explicitly stated. To restate, gist-reasoning is the ability to derive global meaning from explicit details, entailing frontally mediated, top-down cognitive control processes. We propose that gist-reasoning operates independently from and is superior to rote fact-learning. Although gist-reasoning can operate independently from fact-learning one goal in the current study was to explore whether training strategies to abstract meaning benefit both recall of facts and gist-reasoning ability, as set forth in Reyna's fuzzy trace theory of a superiority of gist over fact memory (1998, 2008).

The present study was a randomized, controlled cognitive training trial. The primary goal was to compare the effects of two forms of training in a group of public middle school eighth grade students from low SES families of predominantly Hispanic descent: (1) Strategic Memory and Reasoning Training and (2) rote memory training. In addition, we employed a control group to mitigate the influence of adult attention and interaction. We evaluated the effect of the two training protocols that used the same content and a control group on three tasks; (a) a measure of gist-reasoning ability, (b) a measure of direct recall of information from a text, and (c) a standardized measure of reading achievement. Another important goal of the study was to test the bidirectional impact of the two forms of training as compared with the control group. Specifically, we examined the impact of SMART and rote memory training on the ability to improve gist-reasoning and recall of facts. A final goal was to compare gist-reasoning versus fact-based recall ability after training between Spanish and English in a subset of students for whom Spanish was the primary language spoken at home.

Methods and Measures

Participants

Participants in the pilot study included 54 students (30 girls and 24 boys) in three sections of an eighth grade “advancement via individual determination (AVID)” class in an inner-city urban Texas public middle school. The students were recruited for this study by their teacher and through a letter to parents. The parents of the adolescents in the study signed informed consent and the students signed informed assent agreements as required by the University of Texas at Dallas Institutional Review Board for the protection of human participants. Parents of the students provided background information regarding household income, the adolescent's health status, and any diagnosis of learning disabilities. Based on the health status questionnaires, no student was reported to have sustained a brain injury, a diagnosis of ADHD, or other learning deficits. Six students had repeated a grade, and one child was reported as having been in a special education class previously. All but five students received free or reduced lunches, indicating low SES status for over 90% of the class. Additionally, the ethnicity of the class consisted of 93% Hispanic, 6% African American, and 1% Caucasian. The participants ranged in age from 13 to 15 years (see Table 1).

Table 1

Demographic information by group.

Characteristics	Memory	SMART	Teen brain/control
N	18	18	18
Gender	Male: 8	Male: 8	Male: 8
	Female: 10	Female: 10	Female: 10
Age at baseline range	13–14	13–15	13–15
Mean (SD)	13.8 (0.73)	13.6 (0.70)	13.8 (0.51)
Socio-economic status	Free/Red. Lunch 17	Free/Red. Lunch 17	Free/Red. Lunch 15
Ethnicity	Hispanic 16	Hispanic 18	Hispanic 16
	African-Am. 2	African-Am. 0	African-Am. 1
	Caucasian 0	Caucasian 0	Caucasian 1
Cognitive abilities test
Standard age score
Verbal range	78–99	67–98	79–105
Mean (SD)	83.82 (7.39)	87.18 (7.34)	87.39 (6.81)
Cognitive abilities test
Standard age score
Non-verbal range	79–114	74–118	89–119
Mean (SD)	95.81 (8.67)	99.29 (13.3)	103.67(9.31)

The Cognitive Abilities Tests (COGAT ®; Lohman and Hagen, 2001) was administered to determine the verbal and non-verbal reasoning abilities of the students prior to the commencement of the training programs. The COGAT is a norm-referenced standardized measure of cognitive abilities that are purported to be acquired through school and other environments. The cognitive abilities measured by the COGAT are related to successful academic achievement and were most recently normed in 2005. The COGAT has been validated against other standardized achievement tests and is group administered. Two reasoning measures (verbal and non-verbal) were used to determine whether the students who participated in the study fell within the typical range of cognitive development. The standardized scaled scores for the two administered subsections were used as an indicator of general cognitive ability and to control for the confound of atypical development.

The COGAT was administered and scored according to the instructions in the manual provided by the publisher. Standardized age scores were used to determine that the students were all within the normal range of development (see Table 1). The median percentile rank for the verbal and non-verbal composite was 40.

Measures

Baseline measures were assessed 1 week prior to the commencement of the training programs. Outcome measures were assessed from 2 to 3 weeks after the conclusion of training (See Table 2).

Table 2

Baseline and post-training outcomes by group.

Group	Rote memory	SMART	Teen brain/control
N	18	18	18
Gist-reasoning range pre (age criterion 16)+	2–15	6–19	3–16
Mean (SD)	9.06 (3.95)	11.5 (4.15)	9.17 (4.0)
Gist-reasoning range post (age criterion 16)+	2–16	9–27	7–19
Mean (SD)	10.89 (3.72)	14.61 (5.33)*	10.89 (3.77)
Fact-learning range pre (out of 48 possible)	12–41	19–42	17–43
Mean (SD)	34.33 (8.39)	33.61 (6.08)	33.22 (6.91)
Fact-learning range post (out of 48 possible)	23–48	35–45	29–44
Mean (SD)	41.61 (4.35)*	40.61 (3.29)*	38.61 (3.6)
TAKS reading 2008 (seventh grade) total standardized range	1877–2438	1994–2532	1994–2400
Mean (SD)	2165 (145)	2230 (141.6)	2214 (101.18)
TAKS reading 2009 (eighth grade) total standardized range	1827–2467	2101–2734	2067–2579
Mean (SD)	2244.89 (185.6)	2347.5 (175)	2201.39 (157.46)
TAKS critical thinking Objective (eighth grade)++
Range (out of 16 possible)	6–16	11–16	10–16
Mean (SD)	13 (2.97)	14 (2)	14 (1.86)

+Age criterion based upon previous control studies.

++Used for correlation analysis.

*p < 0.05.

Test of strategic learning

The Test of Strategic Learning (TOSL©; Chapman et al., submitted) was administered prior to training after randomization of the students into the three groups and 6 weeks later after the commencement of the training. The TOSL assessment tool provides a systematic method to evaluate developmental reasoning skills, in terms of both fact-learning and higher-order gist-reasoning of lengthy text-based information much like that encountered in a classroom setting (Chapman et al., 2001, 2006; Gamino et al., 2009a, b). The validity of the TOSL to measure higher-order thinking skills in general, and gist-reasoning ability in particular, has been established in prior studies (Brookshire et al., 2000; Chapman et al., 2004, in press; Anand et al., 2010; Vas et al., 2010). Gist-reasoning ability as measured by the TOSL has been associated with frontally mediated measures of executive function such as working memory, cognitive switching, and fluid reasoning (Brookshire et al., 2000; Chapman et al., 2006; Anand et al., 2010; Vas et al., 2010). No known standardized measure of abstraction of meaning for complex texts exists.

The TOSL provides two major scores relevant to measuring the ability to construct meaning from complex information. One score examines gist-reasoning ability through spontaneous production of gist concepts/abstracted meanings in a written summary, and the other measuring fact-learning through the ability to retrieve and convey important information from the texts. Gist-reasoning is measured through coding abstracted concepts conveyed through summarization of text-based information. Fact-learning is measured through recall via probe questions that require short answers regarding detail information from the text. The TOSL consists of three texts of increasing length and complexity (see Summary Examples from the Test of Strategic Learning (TOSL in Appendix for examples of students’ summaries).

Administration

The students were assessed as a group in their classrooms using the TOSL. The assessment was completed during a 50-min class period. The TOSL consists of three texts to be summarized through written response. Following the summarization task, probe questions regarding information from the original text were administered.

The students were instructed regarding the qualities of a good summary. Specifically, they were told that a summary (a) is a shortened version of the original text, (b) conveys high level ideas while omitting unimportant details (c) is well-organized, and (d) contains enough information so that someone who had not read the information would have a good understanding of the global meanings conveyed in the original text. Subsequently, an example of a well-written summary comprised of gist-based ideas of a common fairy tale, “Little Red Riding Hood,” was presented.

Following the instructions and example of a good summary, the first of three texts were presented orally and in written form for the students. Thus, each text was read out loud by the examiner while the text was simultaneously shown on a screen at the front of the classroom through an LCD projector to allow the students to read along as the examiner read. After reading, the displayed text was removed and the students were reminded that they did not have to give all the details but rather they needed to give a generalized summary that included high level ideas to show they were able to interpret the overall meanings conveyed through the text. The students’ written summaries for the first text was collected after 10 min; the students were then given a form with eight written probe questions regarding important information from the text. The written answers to the questions were collected after 5 min. The same methodology was followed for three texts and the corresponding questions. The time allotment was based on our prior objective measurement that when writing summaries for the TOSL, more than 95% of students complete a summary within 6–8 min, and finish the written answers to the probe questions within 3 min. The controlled time for each portion of the TOSL allowed comparable time for each student and completion of the assessment during one classroom period. The students were reassessed with the TOSL in the same manner 2 weeks after the conclusion of the 4-week training programs.

Scoring

For the summary gist-reasoning score, a checklist scoring system was established from a normative sample in which one point is awarded for each accurate gist-based concept produced during summarization. Gist-based concepts represent abstracted/higher-order ideas and meanings that were not explicitly stated in the text but were derived through synthesizing the text information with world knowledge. The cumulative total score possible for gist-based reasoning across the three texts was 35. The rubric of 35 abstracted ideas across the three texts provides more accurate reliability between scorers, and was developed from data collected from previous studies. For the fact-learning scores, the written answers to the probe questions regarding the important information from the text were awarded 0, 1, or 2 points depending upon correctness and completeness of the answer. The cumulative score possible for fact-learning questions across the three texts was 48.

Reliability

Two trained raters scored each summary independently and were blinded to the group. Point-by-point reliability between the two raters was 92.7%, with all disagreements resolved through discussion and consensus. The corresponding questions had a maximum possible score of 16 for each text. Point-by-point reliability between the two raters was 98.7% for the correctness of answers to the probes.

Spanish TOSL

A Spanish version of the TOSL was translated and administered by a native Spanish speaker/interpreter 2 weeks after the post-training English version of the TOSL had been administered. We used an identical method of administration and scoring for a subset of 25 students in the study whose families spoke predominantly Spanish at home. The Spanish-version TOSL assessment was given on a different day than the English-version TOSL post-assessment. The scoring method for the Spanish version of the TOSL was the same as stated previously for the English version. The students who agreed to take the additional TOSL in Spanish had participated in each of the three training groups. Six students from the SMART group, 12 students from the rote memory strategies group, and eight students from the control group participated in this secondary testing. The Spanish TOSL was administered one time after the conclusion of the training programs.

The texas assessment of knowledge and skills – critical thinking objective

The Texas Assessment of Knowledge and Skills (TAKS; Pearson Educational Measurement, 2003) is a state-mandated standardized test used in Texas primary and secondary schools to assess basic skill attainment in reading, writing, math, science, and social studies. The TAKS reading test is administered in grades three through nine and consists of four objectives: Basic Understanding, Applying Knowledge of Literary Elements, Using Strategies to Analyze, and Applying Critical Thinking Skills.

We used the TAKS reading test for two purposes; first, we used the students’ reading TAKS scaled score performance to determine if the training had an effect on score improvement from seventh to eighth grade. Second, we used the students’ Applying Critical Thinking Skills objective raw scores to ascertain if there was a relation between the objective and TOSL gist-reasoning scores. The Applying Critical Thinking Skills objective is the subset of 16 questions throughout the reading TAKS that, according to the Texas Education Agency (TEA) and the TAKS publisher, Pearson Educational Measurement, required complex understanding, inferential interpretation, and abstraction of deeper meaning. The Applying Critical Thinking Skills objective questions are interspersed throughout the test and consist of approximately 30% of the reading TAKS.

Administration and scoring

The TAKS reading test was administered by classroom teachers, as required by Texas law, and coincided with the conclusion of the training programs. The TAKS Reading test consists of multiple-choice questions scored by a computer. An overall scaled score of 2100 is considered passing and a scaled score of 2400 earns a “commended” rating.

Training programs

Prior to the TOSL administration, the students in three sections of eighth grade classes were randomized into one of three groups (n = 18), to determine the effect of the training conditions on gist-reasoning and fact-learning performance. The groups included (1) the SMART program, (2) a rote memory strategy program, and (3) a control group that learned information about the teen brain. There were 10 girls and 8 boys in each group. Neither the educator of the classes nor the students knew which group was one of the two training groups versus the control group. All three programs were developed to look similar to each other, with student manuals that provided information and pen and paper exercises to practice the training/information provided. The three programs were conducted by three trained researchers in separate classrooms to avoid cross-contamination between the groups of the content of the programs.

All three programs (SMART, Rote Memory Training, and the control group) encompassed nine, 45 min periods across 4 weeks of instruction. Two weeks after the conclusion of the training programs, the students were reassessed with the TOSL, using the same procedure for assessment as mentioned above. The students also completed the state-mandated TAKS reading test three weeks after the training programs.

Strategic memory and reasoning training program

The SMART© (Chapman and Gamino, 2008) program was developed to train individuals to derive a deeper level of understanding by abstracting meaning from texts. The differential aspect of the SMART©program is that it focuses primarily on constructing abstracted meanings through reasoning. The middle school students were trained using this strategy-based program. The strategies include inhibition of extraneous information, inferencing, paraphrasing, and abstracting ideas through reasoning (Brown et al., 1981; Palincsar and Brown, 1984; Mayer, 1989; Chapman et al., 2004, 2006; Kane et al., 2004). Through practice, the students learned to strategically select relevant information in order to abstract and construct meaning through top-down processing extensively during the last four sessions of the training. Thus, the students learned to interpret and abstract meanings in the context of their own world knowledge (see Strategic Memory and Reasoning Training Stages and Sequence in Appendix for description).

The texts used in the program were similar to content that is typically encountered in English, literature, social studies, history, and science texts. The SMART© program is based upon cognitive neuroscience research of higher-order, top-down cognitive skills (Brown and Day, 1983; van Dijk and Kintsch, 1983; Mayer, 1984, 1989; Ulatowska and Chapman, 1994; Luck and Vogel, 1997; Chapman et al., 2004, 2006) and consists of hierarchical strategies that are explained and practiced through group exercises and pen and paper activities in a student instructional manual. The strategies taught over the course of 4 weeks are postulated to bolster cognitive processes that underpin reasoning and higher-order abstraction of meaning (Brown et al., 1981; van Dijk and Kintsch, 1983; Mayer, 1989; Kane et al., 2004; Tenenbaum et al., 2006; Beller and Kuhnmunch, 2007; Garcia-Madruga et al., 2007). The SMART program was validated previously with students with ADHD to teach specific cognitive strategies to enhance comprehension, interpretation, and abstraction of meaning (Gamino and Hull, 2009; Gamino et al., 2009a,b,c).

Rote memory training program

The rote memory training program is based on cognitive neuroscience research related to the basic properties of bottom-up memory processes. The materials used for the rote memory training imitated the SMART program in the use of the same texts and the presentation of activities in a student manual. The program was conducted using the same parameters as the SMART program, utilizing nine classroom sessions over the course of 4 weeks. The rote memory training program institutes direct instruction regarding basic memory strategies as well as the opportunity to practice the processes that extant research has established as important for improvement of memorization techniques. The memory strategies presented and practiced with pen and paper tasks include rehearsal (Cox et al., 1989), retrieval practice (Bjork, 1989), method of loci (Verhaeghen and Marcoen, 1996), and association (Graf and Schacter, 1985). Students practice using memory aids such as mnemonics, visualization (Marschark and Surian, 1989), and flash cards. Through practice, the students were expected to learn to use rote memorization strategies for verbatim recall of facts.

Teen brain information program

The teen brain information program is based on cognitive neuroscience research related to adolescent brain development (Giedd et al., 2006). The informational program is similar in presentation to the treatment programs with the incorporation of pen and paper exercises and activities included in a student manual. Likewise, the teen brain program encompasses nine classroom sessions over a 4-week period. The program includes adolescent-appropriate subjects regarding aspects of healthy brain development such as nutrition, adequate sleep, and physical fitness; as well as dealing with peer pressure; the negative effects of drugs, such as tobacco, alcohol, and illicit pharmaceuticals; and the risk of brain injury. Additionally, the program teaches the importance of managing stress, engaging in mental exercise, and prioritizing important life aspects (i.e., schoolwork and relationships with family and friends). The students learn the names and functions of various lobes of the brain and their functions. In addition, the students discuss various problems encountered in adolescence such as balancing parental and school expectations with peer relationships, as well as other salient subjects such as stress reduction.

Analyses and Results

We used SPSS and SAS (Version 9.2, SAS Institute, Inc., NC, USA) to analyze the data between groups with a single factor analysis of variance (ANOVA) and Chi Square. Post-training we used within group t-tests and post hoc Bonferroni adjustments to account for multiple tests to determine the significance of change scores within each group. Repeated measures ANOVA was preformed to determine the effect of group, Tukey comparisons were used to ascertain differences post-training, and Pearson correlations to determine the relation between TOSL scores and TAKS Applying Critical Thinking Skills scores. Alpha was set at 0.05. Baseline and post-training assessment outcomes are listed in Table 2.

Baseline analyses

After the students had been randomized into one of three groups (n = 18, 10 girls and 8 boys for each group), the baseline TOSLs were scored by trained raters who were blinded to group randomization. The scores were then analyzed to determine if there were statistically significant group differences prior to training in the ability to produce gist-based concepts or answer probe questions regarding important information from the text. An ANOVA failed to reveal significant differences across means of the three groups on the baseline TOSL measures of gist-reasoning ability (F(2,51) = 1.72, p = 0.20) and correct answers for probe questions regarding the text (F(2,51) = 4.26, p = 0.54, see Table 1). Chi-Square analysis failed to reveal differences in performance due to gender χ2(2) = 0.45, p = 0.80), and an ANOVA failed to reveal significant differences in age (F(2,33) = 0.71, p = 0.41) across the three groups. Demographic information for the participants is listed in Table 1, above.

An ANOVA failed to reveal significant differences between the groups on the COGAT verbal standardized age scores (F(2,49) = 1.33, p = 0.27) or the COGAT non-verbal standardized age scores (F(2,47) = 2.22, p = 0.12; see Table 1). Thus, the three groups were found to be equally matched for age, gender, gist-reasoning, fact-learning, and cognitive abilities prior to training. Pearson correlations were performed to determine the relation between baseline gist-reasoning scores from the TOSL and the COGAT composite verbal and non-verbal scores. We found a moderate but significant relation between baseline gist-reasoning score and COGAT verbal and non-verbal composite scores (r(54) = 0.28, p = 0.05).

Post-training analyses

To test the hypotheses that gist-reasoning scores would change between baseline and post-intervention assessments, a paired t-test for each group was performed, and post hoc Bonferroni adjustments were made; adjusted p-values are reported. The results of these statistical tests indicated that there was a statistically significant increase in gist-reasoning scores for the SMART group (N = 18, M = 3.11, SD = 3.00, t(17) = 4.39, p = 0.001, d = 0.73). In contrast, the analyses failed to reveal a significant change in gist-reasoning scores for the fact-based rote memory group and for the control group (N = 18, M = 1.8, SD = 3.09, t(17) = 2.52, p = 0.09, d = 0.33, and N = 17, M = 1.4, SD = 4.18, t(17) = 1.39, p = 0.55, d = 0.43, respectively; see Figure 1). Only the English TOSL scores were used for the paired t-tests, as a small number of students from each group participated in the Spanish TOSL assessment administered post-training.

Figure 1

Changes in gist-reasoning scores after training programs for each group. Groups: Memory, Rote Memory Group; SMART, Strategic Memory and Reasoning Training Group; Control, Teen Brain Information Group.

To determine the differences between the groups on gist-reasoning ability post-training, a repeated measures ANOVA and Tukey multiple comparisons were performed. The results indicate that there was a significant effect of group (F(2,51) = 4.24, p = 0.02). The multiple comparisons indicated that gist-reasoning scores for the SMART group were significantly higher than the rote memory group (p = 0.03) and the control group (p = 0.04) and the control group (p = 0.04). There were no significant differences found between the rote memory and the control group on the gist-reasoning measure (p = 1). The analysis failed to find a significant interaction between group and time of assessment (i.e., baseline versus post-training F(2,51) = 1.23, p = 0.30).

To test the hypothesis that fact-learning scores would change from baseline to post-intervention in the SMART group and the rote memory training group, we performed paired t-tests for each group with post hoc Bonferroni adjusted p-values reported. A significant difference was found in change scores between baseline and post-training for both the SMART Group (N = 18, M = 7.89, SD = 4.84, t(17) = 6.92, p = 0.002 and the memory training group (N = 18, M = 5.2, SD = 4.05, t(17) = 5.47, p = 0.03). We failed to find a significant difference between fact-learning ability at baseline and post-intervention for the teen brain information group (N = 17, M = 4.29, SD = 8.51, 7(17) = 2.08, p = 0.99; see Figure 2). As mentioned above, only the English TOSL post-assessment scores were used for the paired t-tests, as a small number of students from each group took the Spanish TOSL post-training.

Figure 2

Changes in fact-learning scores after training for each group. Groups: Memory, Rote Memory Training Group; SMART, Strategic Memory and Reasoning Training Group; Control, Teen Brain Information Group.

To determine the effect of group in fact-learning ability post-training, a repeated measures ANOVA and Tukey multiple comparisons were performed. The results failed to indicate a significant effect of group (F(2,51) = 1.03, p = 0.36). The Tukey multiple comparisons indicated no significant differences between any of the groups.

Texas assessment of knowledge and skills

We used two measures from the TAKS reading test to: (1) ascertain the effect of training groups on total TAKS reading scaled score outcome from seventh grade (2008) to eighth grade (2009), and (2) the raw scores from the “Applying Critical Thinking Skills” objective to help validate the TOSL as a measure of higher-order critical thinking. Analysis of the data sample failed to indicate a significant difference from seventh (pre-training) to eighth grade (post-training) on TAKS reading scaled scores across the three training groups. A trend, however, was found in the expected direction for the SMART group, and a power analysis indicated that given a larger sample size, the results would be statistically significant (See Figure 3).

Figure 3

A positive trend for the SMART group of higher scaled scores on TAKS reading test.

Pearson correlations were performed to determine the relation between gist-reasoning scores from the TOSL and the TAKS reading objective “Applying Critical Thinking Skills.” We found a moderate but significant relation between gist-reasoning score on the post-training TOSL and the post-training TAKS “Applying Critical-Thinking Skills” raw score for all participants (r(51) = 0.28, p = 0.04; see Figure 4).

Figure 4

Positive correlation between gist-reasoning scores and TAKS critical thinking objective.

Spanish versus english test of strategic learning

To evaluate whether reasoning ability in English would be comparable to reasoning ability in Spanish, 25 of the students who primarily spoke predominately Spanish at home were assessed using the Spanish TOSL. The results failed to find a significant difference in performance between Spanish and English versions on gist-reasoning ability (F(2,22) = 0.39, p = 0.28) or the ability to answer probe questions from the text (F(2,22) = 0.51, p = 0.61).

Discussion

This randomized pilot study conducted in an inner-city public middle school examined the potential benefits of a training program focused on instruction of higher-order cognitive strategies that support abstraction of meaning versus a rote memory training program. Few studies have specifically investigated gist-reasoning skills and the efficacy of subsequent training of a top-down cognitive strategy program compared against a bottom-up, rote memory training program in a randomized group of minority public school adolescents. In the present experimental study, we examined whether we could improve eighth grade students’ ability to engage in gist-reasoning after short-term intensive training programs that consisted of less than 9 h delivered over a 1-month period. The effects of the top-down cognitive strategy program, SMART were compared against a bottom-up, rote memory training program and an equally engaging program that provided educational information about adolescent brain health. Training was equated across conditions such that all groups had the same number of sessions, participants, curriculum appearance, and were informed that the activities would help them with their school performance.

Four important contributions of this training study are discussed, followed by the study's limitations which motivate and refine future investigations. The contributions of this research relate to (1) the potential advantage of teaching eighth grade students higher-order cognitive strategies that are theorized to support top-down processing to abstract meaning (van Dijk and Kintsch, 1983; Reyna and Brainerd, 1995), (2) the transfer effects of reasoning training to untrained areas, including within group fact-learning scores and a school-based measure of reading aptitude, (3) the ability to effect change in one of the most vulnerable demographic groups of middle school students in terms of risk for school failure and drop-out, i.e., a group of students from minority low SES status families (McNeil, 2005), and (4) the preliminary evidence that students who spoke predominately Spanish at home demonstrated similar reasoning ability in both English and Spanish languages after training.

The present findings suggest that cognitive strategies that support abstraction of meaning can be taught in a typical public school eighth grade classroom to improve students’ ability to utilize gist-reasoning to convey the meaning of texts. The adolescents who were trained in SMART demonstrated significantly higher post-training scores as compared to baseline performances on the measure of constructing higher-order, gist-based meanings. The benefit was demonstrated after less than 9 h of training in a typical classroom setting. In contrast, neither the rote memory training group nor the control group demonstrated a significant improvement in gist-reasoning.

Our findings suggest that a program that trains higher-level cognitive strategies has the potential to improve gist-reasoning ability and may be a promising way to promote deeper level understanding and transfer of knowledge than one focused on how many facts one learns by rote. Post-training, we found a correspondence between higher gist-reasoning scores and higher performances on two other measures: (1) the ability to recall key facts when answering probe questions regarding textual information, and (2) a skill objective from the Texas Assessment of Knowledge Skills (TAKS) for reading, “Applying Critical Thinking Skills,” a state mandated test used as an index of academic achievement. As stated earlier, this latter test was the key objective described by the Texas Education Agency (TEA) as a measure of higher-order verbal reasoning skills in the TAKS reading test.

The present results confirm and extend earlier work by other research teams (Brown et al., 1981; Palincsar and Brown, 1984, 1987, 1988; Taylor and Beach, 1984; Pearson and Dole, 1987; Malone and Mastropieri, 1992; Rosenshine and Meister, 1997; Chang et al., 2002; Kintsch, 1998, 2004; Ladewski et al., 2007). Taylor and Beach (1984), in particular, focused on instructing typically developing seventh graders to organize explicitly stated information according to superordinate and subordinate ideas as compared to learning facts without hierarchical parsing of the information. Similar to our results, these researchers used summarization as a metric to show enhanced learning gains. Our paradigm extended their approach by exploring the ability to train students to derive meaning that went beyond the most important stated facts. In particular, the SMART program trained the students to abstract novel meanings that were not explicitly stated in the text. The capacity to abstract meaning, is illustrated symbolically by a + b = c, where “a” and “b” are explicit important facts and “c” represents novel, unstated, generalized meanings (Chapman et al., in press). The ability to process abstracted meaning has been associated with cognitive measures of executive control and frontally activated neural networks (Chapman et al., 2006; Anand, 2008; Vas et al., 2009, 2010). In contrast to Taylor and Beach's method, we found that training abstraction of meaning was superior to training rote memory as it benefited both gist-reasoning and fact-learning within the SMART group.

Our finding that the meaning abstraction training demonstrated a within group transfer effect to the ability to recall important information is similar to (Brainerd and Reyna's 1995; Reyna, 1998) theory that gist-reasoning shapes memory for details. Specifically, the transfer effect found within the SMART group provides empirical evidence that gist-reasoning ability may influence the ability to learn facts from a text, a crucial aspect of academic success. In contrast, the fact-learning group showed significant improvement within group only for retention of information from a text, as indicated by increased scores for correctly answering probe questions, but showed little improvement of gist-reasoning ability. This finding corroborates evidence of a positive effect of explicit instruction for learning important facts especially when identifying the most important ideas (Brown et al., 1981; Taylor and Beach, 1984). However, in our study the bottom-up approach to learning did not significantly improve spontaneous gist-reasoning production during summarization. Moreover, the control group failed to demonstrate significant changes within the group in either gist-reasoning ability or recall of important information. For this latter group, the focused adult interaction and discussion/learning activities did not have a significant effect on within group change scores for measures of higher-order cognitive processing or basic memory ability.

In recent years, there have been renewed calls to train students to think about information thoroughly in order to produce deeper understanding and abstraction of meaning to support superior critical thinking rather than train students to primarily learn facts by rote (Kaminski et al., 2008; Schwartz et al., 2008; Ramsay et al., 2009).The current findings corroborate this belief. Earlier studies did not examine a superiority of training abstraction of meaning as a window to enhance fact-learning. Nonetheless, they laid the foundation for the current study by revealing improved performances when strategies such as hierarchically parsing information (Taylor and Beach, 1984), concept mapping (Chang et al., 2002), and reciprocal teaching (Palincsar and Brown, 1984; Pearson and Dole, 1987; Rosenshine and Meister, 1997; Ladewski et al., 2007) are compared to more typical classroom learning focused on rote learning. Thus, our current study employed a new metric from those previously used to measure summarization skills, namely gist-reasoning through abstraction of meaning.

The present results of a generalization of gist-reasoning skills suggest an advantage to teaching students strategies that support abstraction of meaning and corroborate a similar pattern found in teaching math to college students (Kaminski et al., 2008). Specifically, Kaminski et al. (2008) showed that students benefited more from learning abstract mathematical concepts as compared to learning multiple concrete math examples in isolation. They also reported greater transfer of knowledge to novel and complex contexts when students were taught to understand math conceptually. Similarly, Schwartz et al. (2008) found that college students who received in-depth conceptual science instruction during high school retained information longer and generalized concepts better than students who received predominately fact-based science instruction in high school.

The finding of a significant correlation between the TAKS “Applying Critical Thinking Skills” objective scores and post-training gist-reasoning scores is promising and suggests that SMART may improve performance on a high-stakes state mandated test of achievement. We postulate that this finding provides evidence that teaching cognitive strategies that support higher-order processing may benefit academic performance. While the TAKS reading scaled scores increased between seventh and eighth grades for our cohort of students, there are potentially many factors that contributed to the improvement. Nevertheless, the students in the SMART group all passed the reading portion of the TAKS and 7 of those students received commended scores in the eighth grade, compared to 3 students who received commended students in the previous year.

The present findings have implications that go beyond the promising results of achieving significant cognitive effects after less than nine hours of training over a one-month interval. More encouraging is the attainment of these results in one of the most vulnerable demographic adolescent groups, that is, a group of economically disadvantaged students (Table 1). Hispanic and African American students living in poverty are at greater risk of school failure, with Hispanic students at greatest risk in Texas, the origin of the current study (Good et al., 2003; McNeil, 2005; Lofstrom, 2007). Eighth grade may be an opportune time to implement specialized training to improve learning in this vulnerable population, as longitudinal evidence indicates that dropout rates for at risk populations escalate in ninth grade (Lofstrom, 2007). Thus, the predominately minority, low-income adolescent students in this study, after participating in the SMART program, showed significant gains in skills necessary for academic success, specifically, gist-reasoning and the ability to correctly answer probes regarding important text-based information (Brown and Day, 1983). It is important to note that the students who participated in the study were all members of AVID classes. AVID students in general are not considered high achievers or gifted, but are average students who show promise; many of whom will become their family's first generation of high school graduates. Perhaps as cognitive training programs are found to improve gist-reasoning and higher-order learning, the potential for these students to succeed will be elevated. This relationship needs to be investigated in a longitudinal study.

In addition to poverty, lack of English proficiency is a key factor associated with high academic failure and dropout rates among Hispanic students (McNeil, 2005; Lofstrom, 2007). However, while the students in the present study were not recent immigrants, English was their second language and many predominately spoke Spanish at home. Interestingly, we failed to find significant differences on gist-reasoning ability or fact-learning from texts when comparing responses in English and Spanish. Thus, preliminary evidence suggests that Hispanic students’ ability to utilize reasoning in their first language was no better than their ability in English, their second language, at least by the time they were in eighth grade. These findings indicate that a deficit in higher-order cognitive skills may not be due solely to language differences.

Our findings of improved outcome in the SMART group argue against enhancing reasoning through bottom-up strategy instruction or focused attention from educators, since the later groups did not show a significant improvement in spontaneous gist-reasoning ability. Further, our findings support the efficacy of teaching high-level cognitive strategies to a vulnerable population, Hispanic eighth grade students for whom English is a second language. Subjectively, the AVID instructor noted that the students seemed more focused on their schoolwork after SMART training, with a noticeable organization to their thought processes. Future studies should consider inclusion of teacher and student self-report of perception of training gains.

Limitations

The current study has a number of promising findings that motivate future studies to address several limitations. One of the primary limitations of this is pilot study is the non-significant interaction between group and time. The small number of students in each group may have contributed to this finding and we will test this hypothesis in a planned study with a larger number of students. Whereas we did not find pre-existing differences between groups in gist-reasoning scores at baseline, the scores obtained by students in the SMART group were relatively higher than those obtained by students in the memory and control groups. Despite these limitations we are encouraged by the significant improvement in gist-reasoning performance among students in the SMART group in contrast to marginal improvements for the memory strategy and control groups and conclude that these findings reflect support for the potential efficacy of the SMART program in the development of reasoning abilities. That said, the findings must be interpreted with caution and warrant further research to address the potential confounds described above.

Additionally, future research will need to determine the feasibility of training a larger number of classroom students at one time. In the present study, each section of the eighth grade was randomized into the three groups, making the training groups small across the three classes. Another factor that needs consideration is the impact of a greater number of training sessions. It will be important in future studies to establish who benefits from short-term training versus who needs more intensive training to receive the same benefits, as well as who fails to benefit at all. Furthermore, all the students were members of an AVID class, a motivated group who are specifically chosen as students with promise. It is unknown from the present study if a larger group of students including those with less motivation to achieve would be as amenable to training. In addition, students need to be followed longitudinally to determine if the SMART program produces lasting and generalized effects that may be measured through improved grades and other measures of academic achievement. We plan to re-evaluate the students to determine the maintenance of reasoning gains. Students should also be queried regarding their ability to use the learning strategies for assignments and/or homework and the impact of using the strategies in various core classes.

Further research should address the efficacy of training programs implemented by educators in the classroom to maximize the potential use and application of the SMART program. Results, such as reciprocal teaching implemented by Palincsar and Brown (1984) and others (See review in Rosenshine and Meister, 1994), offer the promise that teachers could deliver an effective training program, given proper training. Whereas positive results were found in low SES AVID students, the impact of SMART on students not enroled in AVID classes as well as those from average to high SES levels would be informative to consider. In addition, brain imaging studies should be conducted to determine the impact of reasoning training on brain development and changes in activation levels and/or neural regions.

Conclusion

Building a mature reasoning mind is a pivotal goal of education; as such, the evidence for improved gist-reasoning through abstraction of meaning training is highly relevant for educational best practices. That is, theoretical and empirical evidence indicates that educational practices focused predominately on rote fact-learning may fail to reach the goal of promoting gist-reasoning skills. The failure to stimulate development of gist-reasoning during adolescence ignores recent evidence from neuroscience that suggests that the adolescent brain is primed to acquire abstract thinking. Our evidence indicating improved gist-reasoning following training in students with ADHD within a laboratory environment (Gamino et al., 2009a,b) prompted a larger question regarding the efficacy of implementing the SMART program in a typical public school classroom. This paper represents an important first step in elucidating the differential impact of fact-learning versus gist-reasoning to an educational setting of typically developing adolescents.

The evidence provided herein supports the importance of formal cognitive strategy instruction that enhances abstraction of meaning and top-down processing of information within an urban public middle school of minority, low socio-economic status students. This study provides evidence that students who are taught to use higher-order cognitive strategies to enhance abstraction of meaning benefit not only from the ability to spontaneously utilize gist-reasoning when summarizing complex information but also benefit in their ability to recall important information from a text. The evidence from this study also suggests that students who learn to abstract meaning perform better on high-stakes standardized reading comprehension measures, purported to measure critical thinking abilities. Thus, training students to engage higher-order cognitive strategies during learning has the potential to transfer to standardized measurements of critical thinking, purported to be indicative of academic performance.

The current study corroborates and expands the promising outcomes that can be derived when cognitive neuroscience and education partner. Given the importance of reasoning and critical thinking skills across the United States (U.S. Dept of Education, 2004) and the high dropout rate among minority students (Lofstrom, 2007), this work provides hope that viable solutions for these escalating problems may eventually be obtained through evidence-based cognitive training.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Strategic Memory and Reasoning Training Stages and Sequence.

Stages of training	Strategy	Session number(s)
1. Inhibit/select and organize	To delete/inhibit unimportant details and prioritize important information	One
	To organize important information into chunks
2. Inference	To use inferencing to extract the deeper/abstracted meaning of information	Two
3. Paraphrase	To convey information in own words	Three
4. Combine and connect	To combine details together into gist based concepts, using inferencing and paraphrasing	Four
5. Integrate	To integrate previous knowledge with new information to formulate high-level gist concepts	Five
6. Generalize	To abstract ideas through gist-reasoning from supporting key points, generalizing to other contexts and situations	Six–nine