The School for Science and Math at Vanderbilt: An Innovative Research-Based Program for High School Students.

The School for Science and Math at Vanderbilt (SSMV) is an innovative partnership program between a Research I private university and a large urban public school system. The SSMV was started in 2007 and currently has 101 students enrolled in the program, with a total of 60 students who have completed the 4-yr sequential program. Students attend the SSMV for one full day per week during the school year and 3-6 wk in the summers following their ninth- to 11th-grade years, with each grade of 26 students coming to the Vanderbilt campus on a separate day. The research-based curriculum focuses on guiding students through the process of learning to develop questions and hypotheses, designing projects and performing analyses, and communicating results of these projects. The SSMV program has elevated the learning outcomes of students as evidenced by increased achievement scores relative to a comparison group of students; has provided a rigorous research-based science, technology, engineering, and mathematics elective curriculum that culminates in a Summer research internship; has produced 27 Intel and Siemens semifinalists and regional finalists over the past 4 yr; and has supported the development of writing and communication skills resulting in regional and national oral presentations and publications in scientific journals.

INTRODUCTION

The School for Science and Math at Vanderbilt (SSMV) was started in 2007 as a part-time program on the Vanderbilt campus that provides an interdisciplinary research-based science, technology, engineering, and mathematics (STEM) curriculum for highly motivated and talented high school students. Twenty-six eighth-grade students are selected each year to participate in this 4-yr program in partnership with the local public school district (Metropolitan Nashville Public Schools [MNPS]). The goals of the program are to elevate the scientific research and critical-thinking skills of highly motivated students to promote their entry into STEM careers. The research-based curriculum focuses on guiding students through the process of learning to develop questions and hypotheses, designing projects and performing analyses, and communicating results of these projects.

The SSMV can best be classified as a selective STEM school, a category of schools that have the potential to reform U.S. STEM education (National Research Council [NRC], 2011). Specialized STEM schools generally enroll a relatively small number of highly motivated and talented high school students with an interest in and aptitude for STEM studies as in the SSMV. The goal of many of these schools is to prepare students for careers in STEM fields, supporting the national call for a workforce with enhanced science and technological skills (U.S. Congress Joint Economic Committee, 2012). However, recent reports have shown that U.S. achievement in STEM is not improving (National Center for Education Statistics [NCES], 2011). Furthermore, fewer than four of every 100 ninth graders in the United States will go on to graduate from college with a STEM degree, while other countries report that up to 45% of their high school students choose to enter STEM careers (NCES, 2009). Indicators of student achievement underscore this rising and urgent concern. The most recent Program for International Student Assessment study shows that U.S. 15-yr-olds rank 21st out of 34 countries in general science knowledge and 26th in math literacy (Organization for Economic Cooperation and Development, 2012). U.S. high school students rank 29th out of 40 in applying problem-solving skills to real-world situations, and 18% fall below the basic level (NCES, 2012).

A number of reports have emphasized the need for universities to be involved in science education reform at all levels to enhance the critical-thinking and problem-solving skills of K–12 students. The NRC recently published a report entitled Research Universities and the Future of America in which they stated that “research universities have an obligation to play a key role in reforming and improving education in the United States … as we seek to bolster our global competitiveness” (NRC, 2012, p. 159). Fuhrman and Streim (2008) reported that universities have the breadth of expertise to help public schools across many disciplines, they have access to the highest-quality research and knowledge to design and implement new educational models, they know the imperatives and challenges of quality education, and they are a part of the communities at large. As noted by the executive director of the University of Chicago's Center for Urban School Improvement, “America has some of the best universities … in the world, and … some of the most problematic K–12 schools serving poor kids” (Robelen, 2007, p. 20). Colwell and Kelly (1999) have called for university scientists to assist in reform of K–12 math and science education by participating in “effective equal partnerships with K–12 schools” (p. 237). In a recent meeting of the Committee on Science and Technology, Congress reaffirmed the commitment to STEM reform in the America Competes Act, emphasizing “the importance of deeply engaged partnerships among all of the stakeholders—school systems, institutions of higher education, informal science education centers, science based institutions, museums, local governments, and the business community” (Committee on Science, Space and Technology, 2010).

In response to the call for universities to form effective partnerships with local K–12 school districts, the Center for Science Outreach (CSO) at Vanderbilt University designed and implemented an innovative 1-d research-based program for high school students in partnership with MNPS. MNPS is a large, high-poverty, and high-needs urban school district of ∼75,000 students. Scores in science and math are well below state and national achievement levels (Hanuschek ; Tennessee Department of Education, 2012). The SSMV opened its doors to the first class in the Fall of 2007 and has since graduated 60 students. This unique “pull-out” program has seen remarkable successes and is now integrated into the academic cultures of both the university and the MNPS K–12 community. In the current paper, the SSMV is described and discussed as an innovative model of a university and K–12 partnership that provides a pipeline program for highly motivated and talented public school students.

METHODS

Program Description

The SSMV program was initiated in 2007 to provide a research-intensive enrichment program for Nashville public school students. MNPS agreed to contribute to the operating budget, assist with student recruitment, provide support for students missing classes at their home schools while attending the SSMV, and provide computers for participating students who do not have access to computer resources at home. Vanderbilt University supplied additional funding as well as dedicated laboratory, office, and classroom space on the medical school campus. A Science Education Partnership Award grant, funded by the National Institutes of Health in 2007, provided 3 yr of support for the start-up of the SSMV program. At the end of the grant period, the school district and university shared operational expenses, with the university continuing to provide space on campus.

The CSO and Vanderbilt University have full governance over the direction of the SSMV. All decisions regarding student admissions and dismissal, evaluation and assessment of student work and achievement, hiring of instructors, development and implementation of curriculum, and scheduling of in- and out-of-class activities are made by the SSMV staff. This agreement has allowed the program to run autonomously and outside the normal restrictions associated with a large school district, and provides an environment for instructors to focus on developing a rigorous academic program of study geared to each student's needs and/or interests. Primary features of the SSMV model are summarized in Table 1 and discussed below.

Table 1.

Summary of the primary features of the SSMV

Feature	Description
Programmatic structure	• Part-time specialized STEM school hosted on a university campus
		• Sequential 4-yr research-based curriculum for 26 students per grade (grades nine to 12)
		• Students leave their zoned high school and attend the SSMV for 1 d per week
		• Classes are held in a dedicated laboratory classroom, with field trips on and off campus
	• Funding shared by MNPS and Vanderbilt, with space provided by the university
Admission requirements	• Student admission is based on seventh-grade achievement test scores, science and math grades, teacher recommendations, student essays, and in-person interviews
Instructors	• Four full-time PhD scientists
		• More than 50 volunteer university faculty, fellows, and students participate per year
	• Graduate and undergraduate student tutoring support
Courses	• Seven state-approved elective credits: Interdisciplinary Science I-IV; Research I-II; Advanced Research
Curriculum	• Ninth grade: introduction to STEM research through an multidisciplinary focus using varied instructional approaches, including lectures, demonstrations, field trips, and class-wide research projects
		• 10th grade: student-driven small-group research projects, class-wide journal clubs, and reading and discussing scientific literature
		• 11th grade: immersion in a laboratory setting through university faculty–mentored individual research projects, small-group journal clubs and discussion forums
	• 12th grade: submission of research results to national science competitions and scientific journals, communication of research results to scientific and general audiences, and student-driven small-group community research projects

Twenty-six students from each of the high school grades (ninth to 12th) leave their zoned school and attend the SSMV for one full day (7 h) per week, with each grade attending on a different day of the week. Students also attend SSMV Summer sessions after their ninth-grade (3 wk), 10th-grade (3 wk), and 11th-grade (6 wk) years. The complete 4-yr program consists of a total of seven science elective courses, all of which have been approved by the State of Tennessee as honors electives. Classes meet from 8 am to 3 pm during the academic year, with an optional 2-h study hall on all of the regular class days. SSMV students may request tutoring from volunteer Vanderbilt undergraduate and graduate students or postdoctoral fellows during this study hall on any subject in which they need assistance. All SSMV students are responsible for their own transportation, although Vanderbilt provides students with a Vanderbilt ID badge that allows them free transportation to and from the university on the city bus system and access to all university libraries.

Program Participants

SSMV Students.

Students are selected for admission to the SSMV program based on seventh-grade science and math course grades and scores on the Tennessee Comprehensive Assessment Program (TCAP) tests, teacher recommendations, student essays, and in-person interviews. A total of 917 applications were received for the graduating classes of 2011–2017, with 191 students admitted (Table 2). The gender ratio among applicants and accepted students is ∼46% male and 54% female. Fourteen percent of the total of 191 admitted students were black, 9% Hispanic, 50% white, and 27% Asian, with 15% on free or reduced lunch. Although not representative of the overall MNPS district ethnic or socioeconomic composition (45% black/17% Hispanic/33% white/4% Asian/72% free/reduced lunch), the diversity of accepted students reflects the diversity within the pool of applicants. Recruiting efforts continue to encourage more diversity overall and to ensure that the SSMV is accessible to students from all backgrounds.

Table 2.

Student demographic and admissions dataa

Graduating class	Total number of applications	Number admitted/number graduated or enrolled	# H.S.	M:F	W	Black	His	Asian	F/RL
2011	139	26/20	8	10:16	15	3	2	6	2
2012	131	27/21	5	12:15	11	5	3	8	6
2013	126	27/19	7	13:14	17	1	2	7	2
2014	125	30/24	4	11:19	13	5	3	9	3
2015	118	28/25	6	16:12	16	4	4	4	6
2016	121	27/26	7	16:11	12	6	3	6	6
2017	157	26/26	5	10:16	11	3	0	12	3
Totals	917	191/161		88:103	95	27	17	52	28

aTwenty-six students were selected each year from an average of 131 applications from eighth graders in all 38 MNPS middle schools. Student selection was based on achievement test scores, class grades, essays, teacher recommendations, and in-person interviews. Numbers in the table refer to the number of students admitted vs. the number of students who graduated or are currently enrolled (191 vs. 161), resulting in a retention rate of 84%. H.S. = high schools; W = white; His = Hispanic; F/RL = free/reduced-lunch program.

Of the 191 admitted students, 30 left the program for an overall retention rate of 84%. The majority of students (16 of the 30) dropped during the first year, most of these during the first semester. The remaining 14 students left the program during the second or third year, and no students left in year 4. The primary reasons for students electing to withdraw included inability to balance the academic and/or extracurricular demands required in their zoned school plus the rigorous course work in the SSMV, lack of interest in the SSMV research-based curriculum (primarily year 1), or a change in their focus of studies (years 2 and 3).

The 101 students currently enrolled in the SSMV (classes of 2014–2017) and the 60 who have graduated (classes of 2011–2013) have come from four magnet and eight comprehensive high schools (representing 75% of the total of 16 MNPS high schools). Approximately two-thirds of the SSMV students attend one of the two academic magnet schools reflecting admission to the SSMV based on academic achievement. The remaining third, although qualified by test scores and grades to attend one of these two magnets, either chose to stay at their zoned school or were not selected in the lottery process for admission to the magnet schools.

SSMV Instructors.

Four full-time PhD scientists serve as the primary instructors for the SSMV. These instructors hold PhD degrees in various STEM disciplines, including molecular biology, chemistry, biomedical engineering, and geophysics. The four instructors are responsible for curriculum development and all primary instruction, as well as assistance in program evaluation. One of these instructors serves as the director of the SSMV, providing leadership for the faculty and staff and overseeing the daily operation of the program and curriculum design. The SSMV director reports to the director of the Vanderbilt CSO.

Vanderbilt Faculty, Graduate Students, and Postdoctoral Fellows.

The CSO has a 20-yr history of involvement with faculty and PhD trainees in all departments at Vanderbilt. Major activities of faculty have included hosting students in laboratories for Summer research experiences and providing guest lectures for CSO-sponsored student and teacher programs. In addition, graduate students and postdoctoral fellows have participated for 14 yr in one of the longest-running GK–12 programs run by the CSO (Ufnar ). Using these well-established contacts with departments, centers, and institutes, SSMV instructors recruit faculty and trainees to host SSMV juniors and seniors in scientific and community outreach projects; to lead discussions during SSMV classes; and to host students for laboratory visits and demonstrations. Many faculty members are formally involved with the SSMV program through broader-impact projects on federal grants.

Curriculum

The SSMV curriculum can best be described as “dynamic,” with a basic framework that provides the underlying foundation for each year of the program. An overview of the curriculum and examples of “A Day in the Life …” for each grade are included in the Supplemental Material. The overarching goal of the curriculum is to help students understand how to ask a good question, how to answer that question, how to ask the next question(s), and how to communicate the results. The progression in learning from freshman year to senior year is based on benchmark criteria derived from the ACT critical-thinking skill progression (Supplemental Material; ACT, 2012). For example, in the scientific-thinking category, students begin in ninth grade by learning to think beyond scientific facts and ask scientific questions, to ultimately using facts to predict results, and finally moving on to formulate the next questions by 12th grade. Each grade moves progressively from class-wide/large-group research projects (ninth grade), to small-group projects (10th grade), to independent research in a Vanderbilt laboratory (11th and 12th grade, first semester), and finally to small-group community projects (12th grade, second semester). The overall research approach and research outcomes for each grade are summarized in Figure 1. The overall curricular components are described in more detail below.

Figure 1.

Schematic representation of the SSMV curriculum, grades nine to 12. The SSMV curriculum focuses on research projects and research outcomes during all 4 yr of the program. As students move through each grade, the projects involve changes in mentoring, number of students per project, and student-versus faculty-driven development of hypotheses. Research outcomes move progressively from group presentations to sophisticated posters and reports.

Beginning in the ninth grade, each instructional day is structured around a question that students then research through class discussion, presentations by SSMV and Vanderbilt faculty, and group projects. The instruction integrates all STEM disciplines, with guiding questions ranging from “Are viruses alive?” to “Do animals compete for bandwidth?” In the Summer following their freshman year, students participate in Research I, where they conduct two group research projects, including a long-term ecological assessment of an urban nature park and a laboratory-based forensics project.

In the 10th grade, students begin to develop scientific collaborations and build skills for conducting independent research. In semester one, small groups of students conduct class-wide projects that follow trends in scientific discoveries and are based on instructor expertise. In semester two, students work collaboratively with an SSMV instructor and additional Vanderbilt faculty mentors to develop small-group research projects. Each group searches the scientific literature for a specific paper related to their project and begins work on their hypothesis and research approach. Students start data collection during the last half of the Spring semester and complete their research in the Summer. In the first semester of their junior year students write and submit applications to the Tennessee Junior Academy of Science (TJAS) competition.

The primary focus of the 11th-grade curriculum is to prepare students for their upcoming Summer research internship in a Vanderbilt laboratory. In the Fall semester, students participate in a series of modules focused on learning more about the research process and the role of science in society. In the Spring semester, students spend a half day in the research laboratory of a university investigator, acclimating to the laboratory environment and preparing for their Summer research experiences. The remainder of their time is spent in small-group journal clubs and additional discussion forums entitled Advances in Technology, Famous Scientists, and Ethics in Research. In the Summer Research III course, students conduct their 6-wk research internship under the direction of Vanderbilt faculty, fellows, and graduate students. The project results are presented at a poster symposium at the end of the Summer and are submitted to both the Siemens Competition in Math, Science and Technology and the Intel Science Talent Search in the Fall semester of their senior year.

In the Fall semester of the 12th grade, students complete their research work and write up their results for submission to national science competitions and to the Young Scientist, a journal started under the auspices of the CSO for high school research projects. In the Spring semester, students design and implement a community research project with a goal of enhancing the students’ skills in communicating the importance of science in the real world. Results of these projects are presented at community events or to community organizations.

Although the primary focus of the SSMV curriculum is on research-based interdisciplinary science, students also spend significant time on developing their science-related and/or technical reading, writing, and communication skills throughout the 4-yr program. Book discussions are a part of the ninth- to 11th-grade curriculum, and students are required to complete written essays as well as original art-focused projects related to the specific book. Significant emphasis is also placed on developing oral communication skills through incorporation of small-group scientific journal clubs, individual journal clubs, scientific presentations within and outside class, poster presentations, and research presentations at laboratory meetings. Technical writing skills are developed through laboratory journals, online discussions, written in- and out-of-class assignments, and preparation of manuscripts for publication.

Assessment of Program Goals and Student Satisfaction

Surveys consisting of a combination of rating and open-ended questions were administered to SSMV students during the 2007–2011 program years to assess major program goals, instructional techniques, student satisfaction with the program, gains in student learning, and the ability of SSMV students to balance work at the SSMV and responsibilities at their regular schools. These short, anonymous surveys were administered online four times during the school year and once during the Summer. A total of 50 anonymous surveys were administered from the Fall of 2007 through the Summer of 2011. Each survey began with a core set of Likert-scale rating questions followed by a series of three to five open-ended questions. Rating questions were tabulated and analyzed to compare means and SDs. Open-ended questions were analyzed for thematic responses to assist in understanding the student experience for formative evaluations. The average response rate for the online anonymous surveys was ∼75%, ranging from 50–100% for various surveys. The SSMV faculty members had real-time access to tabulated summary results from the online surveys. Although the surveys were anonymous, responses assisted the faculty in determining the effect of instructional strategies and served as early indications of any possible issues that needed to be addressed by the faculty.

Assessment of College Readiness

A second survey was developed and administered to students to measure self-reported gains in science content, literacy, research, and outreach. A 10-item survey based on national science education standards, the assessment rubric for the science ACT test (ACT, 2012), and scientific careers interests, with questions that increased in depth for each of the 10 items, was developed by program evaluators (Supplemental Material). Each item was ranked on a 10-point scale from “I know little to nothing” (1) to “I have developing skills” (5) to “I am an expert” (10). The instrument was given to participants in September and May of each program year. To match each student pretest to the corresponding posttest and to preserve confidentiality, the SSMW staff assigned an ID number sticker to each test. The paper survey results were then entered by ID number into a spreadsheet for analysis. The questionnaire given at the end of each year also asked the participants to retroactively rank their knowledge at the beginning of the program using the technique of “retrospective pretest (Nimon ). This technique allows participants to reflect back on their self-assessments at the beginning of the year with the realization of how much they did not know at that time compared with what they had learned by the end of the year. In total, each year of the program yielded three self-report measures: knowledge on the true pretest, knowledge on the retrospective pretest, and knowledge on the posttest. Reliability of the instrument was tested for the true pretest, the “retrospective pretest,” and the posttest by looking at the interitem reliability (or internal consistency) of the measure using interitem correlations, item-to-total correlations, and Cronbach's alpha, a measure of internal consistency. Cronbach's alpha was greater than 0.81 for all three time points, indicating high internal consistency. Paired-samples t tests were calculated to assess whether differences between pre, retrospective pre, and post responses were statistically significantly different.

Assessment of the Rising Senior Summer Research Experience

A key experience in the SSMV curriculum is a Summer internship in a Vanderbilt research laboratory. To gain an understanding of both the mentors’ and students’ experience, we designed surveys based on the literature about undergraduate research experiences (Kardash, 2000). Ten expected outcomes were selected based on conversations with the evaluators and the SSMV instructors. The surveys for the SSMV students and their mentors contained an identical set of items for the students to rate themselves and for the mentors to rate the students. Other items on the surveys were designed to gather information about logistics, the student breakout sessions, the influence on student career choices, time spent with the student by laboratory mentors and assistants, and the optimal length of the research experience. Questions were scored on a Likert scale, and answers were tabulated and analyzed to compare means and SDs.

Quantitative Assessment of Student Achievement

A quasi-experimental study was designed at the beginning of the program to investigate academic achievement for SSMV students compared with a matched group of students who did not attend the SSMV. For each SSMV class, a comparison group of ∼100 students was selected by the MNPS Office of Assessment. After considering several criterion tests to select the comparison group, the MNPS researchers suggested that the strongest match was based on the seventh-grade science and math TCAP scores that were used in the selection criteria of SSMV students. Students were assigned a code by the Research Office (deidentified), and data were collected in the following categories: EXPLORE, PLAN, and ACT scores; end-of-course (EOC) scores in algebra I and biology; overall grade point average (GPA), and scores on Advanced Placement (AP) tests in biology and chemistry. Data were collected over 4 yr, utilizing student information from at least four different databases. The study included data from 92 SSMV participants and 417 comparison-group participants. A comparison of demographics for the SSMV students and the comparison group is shown in Table 3, together with statistical data for TCAP scores for the SSMV and comparison groups.

Table 3.

Seventh-grade achievement data for SSMV and comparison studentsa and demographic data for SSMV and comparison studentsb

	SSMV students, class of 2011					Comparison students, class of 2011
	Max	Min	Mean	SD	n	Max	Min	Mean	SD	n
Seventh-grade science TCAP	248	211	230	8	22	271	213	234	11	116
Seventh-grade math TCAP	730	553	630	52	23	730	553	623	38	117
		SSMV group numbers (%)	Comparison-group numbers (%)
Gender	Male	38 (41.3)	253 (60.7)
	Female	54 (58.7)	164 (39.3)
Ethnicity	White	45 (48.9)	304 (72.9)
		Black	12 (13.0)	47 (11.3)
		Hispanic	10 (10.9)	11 (2.6)
		Asian	25 (27.2)	53 (12.7)
	Other	0	2 (0.5)

aA comparison group of students was selected by the research team for each SSMV graduating class in the first 4 yr of the program (2007–2011). There were a total of 92 SSMV students and 417 comparison students. The comparison group was selected based on seventh-grade math and science scores on the TCAP. Scores were not statistically different between the two groups. The data for the 2011 graduation class are shown as an example.

bThe gender and ethnic breakdown for the SSMV and comparison groups for the classes of 2011–2014, expressed as numbers of students and percent of the total.

Differences in outcomes between the SSMV group and the comparison group were examined using SPSS 16.0 software and GLM UNIANOVA statistical analyses. When analyzing each outcome, the general approach was to include the SSMV group as the fixed effect, with a set of related covariates (seventh-grade TCAP math scaled score and/or seventh-grade TCAP science scaled score, free/reduced lunch, and gender). When sample size allowed, the analyses were repeated after splitting the file by ethnicity in order to identify significant differences between SSMV and comparison groups within each subgroup (Asian, black, Hispanic, and white). For the Spring 2011 GPA outcome measure, differences were also examined by cohort, since exposure to the intervention would increase with years in high school, and unlike the other outcome measures that have set administration time lines (e.g., PLAN is taken in 10th grade), GPA represents freshman, sophomore, junior, or senior GPA depending on the cohort. Data analysis used TCAP scores as a covariate, suggesting that the differences between the SSMV and comparison group on the outcome measures (GPA, ACT scores, EOC scores, etc.) are beyond those that can be accounted for by just TCAP alone.

Protection of Human Subjects

This research study was approved by the Vanderbilt University Institutional Review Board, study number 070624.

RESULTS AND FINDINGS

A variety of strategies were used to assess the success of the SSMV program. First, anonymous online surveys were designed using a core set of Likert-scale rating questions to measure the SSMV program goals as reflected in the specific objectives for each year's curriculum. The set of core questions also included items regarding communication with SSMV faculty, logistics of the schedule, and the ability of students to maintain their responsibilities at their zoned schools. A comparison of the ratings of core questions for each class over time is shown in Table 4. For example, to measure the goal to “elevate the scientific insights and research capabilities of highly competent and motivated MNPS students” the core questions were “I am learning new concepts and skills in science” (freshmen); “I feel confident in my ability to pose scientific questions” (sophomores); “SSMV has taught me to approach problems like a scientist or mathematician” (juniors); and “I feel confident in my ability to formulate a research hypothesis from an interdisciplinary perspective” (seniors). Average ratings for questions that pertained to increasing scientific and research capabilities and gaining new concepts and skills in science were consistently high, ranging from 4.05–4.72 across all classes during the first 4 yr, suggesting that the SSMV is meeting its goals of providing students with an intense learning experience in science and math and the development of research capabilities. Other categories that were consistently rated above four throughout all 4 yr included questions related to specific programmatic features such as understanding core assignments and the goals of each class day. Lowest ratings (freshmen through juniors) were related to keeping up with assignments and scheduling, which aligns with the issue of students missing regular school classes 1 d per week (ratings of 3.49–4.23).

Table 4.

Anonymous online survey analysis: measurement of ACT core conceptsa

		Class of 2011 (n = 105)	Class of 2012 (n = 90)	Class of 2013 (n = 86)	Class of 2014 (n = 90)
Graduating class	Core questions	Mean ratings: strongly disagree (1) to strongly agree (5)
Freshman	I am learning new concepts and skills in science	4.69 ± 0.54	4.61 ± 0.67	4.72 ± 0.48	4.58 ± 0.67
		I am learning to apply science, mathematics, and/or technology in real-world settings	4.39 ± 0.66	4.43 ± 0.81	4.27 ± 0.68	4.31 ± 0.92
		The instructors clearly explained my responsibilities, challenges, and assignments	4.44 ± 0.69	4.19 ± 0.78	4.17 ± 0.80	4.02 ± 0.92
	I am able to keep up with my regular high school assignments	4.10 ± 0.87	3.83 ± 0.74	4.23 ± 0.68	3.93 ± 0.85
Sophomore	This year's curriculum builds on what I learned last year at the SSMV	4.45 ± 0.08	4.16 ± 0.62	4.24 ± 0.83	4.05 ± 0.78
		I feel confident in my ability to pose scientific questions	4.37 ± 0.69	4.06 ± 0.65	4.36 ± 0.61	3.96 ± 0.71
		I understand the main focus and goals of each class day	4.55 ± 0.68	4.11 ± 0.82	4.53 ± 0.60	4.20 ± 0.80
	I am able to keep up with my regular high school assignments	4.07 ± 0.86	3.51 ± 0.84	4.10 ± 0.67	4.09 ± 0.51
Junior	The SSMV has taught me to approach problems like a scientist or mathematician	4.52 ± 0.50	4.45 ± 0.53	4.63 ± 0.49	4.50 ± 0.71
		I feel confident in my ability to solve complex problems	4.11 ± 0.58	4.05 ± 0.51	4/24 ± 0.58	4.17 ± 0.62
		Brainstorming with other students in our collaborative group work has increased my problem-solving ability	4.56 ± 0.56	4.16 ± 0.74	4.28 ± 0.61	4.00 ± 1.13
	SSMV assignments enhance my critical-thinking skills	4.34 ± 0.73	3.74 ± 0.85	4.43 ± 0.71	4.04 ± 0.89
Senior	I feel confident in my ability to formulate a research hypothesis from an interdisciplinary perspective	4.41 ± 0.57	4.20 ± 0.41	4.60 ± 0.63
		I feel confident in my ability to be a contributing member of a research laboratory	4.54 ± 0.61	4.26 ± 0.61	4.52 ± 0.80
		I feel confident in my ability to explain my research to a scientific audience	4.13 ± 0.75	4.09 ± 0.51	4.59 ± 0.67
		I feel confident in my ability to explain my research to the general public	4.40 ± 0.57	4.14 ± 0.65	4.43 ± 0.74
	I feel confident in my ability to write scientific manuscripts that meet the guidelines for peer-reviewed publications	4.09 ± 0.68	3.79 ± 0.59	4.07 ± 0.84

aA series of Likert-scale questions were developed based on the ACT core concepts (Supplemental Material). Students completed the survey anonymously online each year of participation at the beginning and end of each semester (four times per year). The analysis included the classes of 2011–2014 (the year refers to the year of graduation), with three complete surveys of freshman through senior years. Shown in the table are representative questions for each category. Values represent the mean ± SD.

The 2011 class as seniors ranked all survey items above 4; these questions were focused on specific issues related to laboratory research, such as formulating a research hypothesis, preparing scientific papers, and presenting results to the scientific and public communities. These are critical results that support the success of the SSMV in preparing these students to continue their independent and group research as they enter colleges. These high ratings from seniors about laboratory research correlate with the positive ratings from students and mentors about the Summer research internship between junior and senior year described below.

The anonymous online surveys also included three to five open-ended survey questions and ended with space for additional comments. Strong positive themes emerged from student comments, including their overall appreciation for the SSMV; the importance of the research skills they were learning; the positive contribution of the SSMV program to their academic performance at their regular schools; and the value of laboratory (hands-on) learning, independent research, and a sense of “belonging” in a university research environment. Negative themes also emerged for some students, including the scheduling of SSMV assignments and class hours; communication with their regular school teachers; and high levels of stress. Importantly, these open-ended responses gave the SSMV instructors a regular perspective on the students’ experiences, enabling mid-program corrections.

College Readiness Pre/Post Assessment

The 10-item college readiness survey measured three time points: a true pretest, a retrospective pretest, and a true posttest (Nimon ). To highlight where students felt they made the greatest gains, the mean differences between the retrospective pretest scores and the true posttest scores were analyzed in rank order. Results in Table 5 show the top three largest learning gains for each class as measured by the retrospective pretest to posttest. The corresponding mean gains on the true pretest to posttest are also listed to compare the degree that students changed their opinion about how much they had learned at the end of the year compared with the beginning. Results indicate that students’ opinions about how much they learned during the SSMV school year showed significant gains between the retrospective pretest and posttest scores as measured using a series of paired t tests. Even where results were not statistically different, retrospective pretest responses were often rated lower than the original pretest responses, suggesting that students had overrated themselves on pretests in light of their new understanding in each learning domain. The finding that many of these differences are statistically significant suggests that students made true gains within the areas assessed during the school year.

Table 5.

Top three response items with largest mean gain on retrospective pretest and posttest by classa

Class	Rank	Item	Mean gain true pre to true post	Mean gain pre retro to true post
Freshman	1	Communication: communicating with School for Science and Math scientists and staff	1.50	4.32
		2	Use of technology: using technology such as data analysis software and Internet resources	1.63	4.13
	3	Scientific careers: understanding of how to pursue careers in science	1.07	3.91
Sophomore	1	Use of technology: gathering and organizing data through the use of scientific tools and technology	0.00	2.80
		2	Problem solving: identifying an alternate method for testing a hypothesis	0.45	2.55
	3	Communication: communicating with scientists using scientific vocabulary	0.92	2.52
Junior	1	Use of technology: using software to create scientific models to make predictions	0.78	2.84
		2	Communication: communicating a critical thesis that clearly establishes the focus of my position on an issue	0.76	2.61
	3	Science content: making use of primary research literature (e.g., journal articles) to understand current advances in a scientific field	0.73	2.50
Senior	1	Problem solving: conducting community-engaged research to identify a local health or wellness problem	−0.05	2.55
		2	Communication: communicating research findings to a scientific audience that meet professional standards for written or oral communication	0.70	2.45
	3	Analysis of data: critically analyzing the quality of data generated in my own research	0.50	2.17

aA 10-item pre/post survey was administered to students at the beginning and end of each school year based on the ACT college readiness standards for grades nine to 11 and interest in STEM careers (included in the Supplemental Material). Each question was ranked by students on a 10-point scale from “I know little to nothing” (1) to “I have developing skills” (5) to “I am an expert” (10). The survey given at the end of the year also asked students to retroactively rank their knowledge at the beginning of the program. The difference or mean gain between the true pretest and true posttest, and the mean gain between the retrospective pretest and true posttest are shown in the table.

The largest gains were reported by the freshman class (mean gain of 3.91–4.32 on a 10-point scale), suggesting that the highly accomplished and motivated entering freshmen had a relatively high opinion about the extent of their skills and knowledge. By the end of the year, the large differences on the retrospective pretest to posttest scores suggest the expanded scope of their learning experiences at the SSMV. Gains reported by sophomores, juniors, and seniors were similar (mean gain of 2.17–2.80), suggesting they developed a more realistic assessment of their knowledge and skills at the beginning of each year. The use of technology was one of the top three learning gains for freshmen, sophomores, and juniors, reflecting the sophisticated and varied use of technology in the SSMV curriculum. Examples include the use of sophisticated laboratory equipment, such as several types of microscopy, thermocyclers, spectrophotometers, magnetic resonance imaging, geographic information systems software, and advanced statistical packages such as JMP. The largest gains for each class corresponded to the focus of that year's curriculum. For freshmen, understanding how to pursue careers in science was one of the largest gains. Sophomores reported gains in alternate methods for testing hypotheses and scientific vocabulary. Juniors reported gains in communicating a critical thesis and making use of primary research literature. Seniors reported gains in conducting community-based research, communicating research findings to a scientific audience, and critically analyzing the quality of data generated in their own research.

Rising Senior Research Experience

Results from the surveys administered in the Summers of 2010 and 2011 to assess students’ and mentors’ evaluation of the Summer research experience are shown in Table 6. Laboratory mentors rated students higher than student self-ratings in both years on their ability to orally communicate the results of research projects. The highest mentor ratings were students’ ability to reformulate an original research hypothesis in 2010 and understand the importance of controls in research in 2011. The highest student self-ratings in both 2010 and 2011 were understanding the importance of controls in research. More than 90% of laboratory mentors surveyed indicated that they would recommend the Summer research program to a colleague and greater than 80% would host an SSMV student in their laboratory next year. The findings from this survey also demonstrate that high school students can succeed in the intense environment of a research laboratory at a Research 1 university. Eighty-five percent of mentors answering the 2011 survey felt that the students made a positive contribution to their laboratories.

Table 6.

Analysis of rising senior Summer research laboratory surveys and SSMV mentor surveysa

	Class of 2011				Class of 2012
	Student self-rating		Mentor rating		Student self-rating		Mentor rating
	Mean ± SD	n	Mean ± SD	n	Mean ± SD	n	Mean ± SD	n
Reformulate an original research hypothesis (as appropriate)	4.07 ± 0.46	15	4.43 ± 0.79	7	4.17 ± 0.71	19	3.64 ± 1.34	14
Interpret data by relating results to the original hypothesis	4.27 ± 0.59	15	4.40 ± 0.70	10	4.11 ± 0.68	19	3.77 ± 1.09	14
Understand the importance of “controls” in research	4.50 ± 0.76	14	4.36 ± 0.81	11	4.68 ± 0.48	19	4.23 ± 0.93	14
Orally communicate the results of research projects	4.00 ± 0.85	15	4.27 ± 1.01	12	4.00 ± 0.71	19	4.14 ± 1.23	14
Relate results to the “bigger picture” in your field	4.33 ± 0.72	15	4.18 ± 0.98	11	4.42 ± 0.69	19	3.79 ± 1.19	14
Think independently	4.43 ± 0.51	14	4.08 ± 1.00	12	4.05 ± 0.71	19	3.57 ± 1.28	14
Identify a specific question for investigation based on the research in your field	4.13 ± 0.64	15	3.91 ± 1.04	11	3.95 ± 0.78	19	3.43 ± 1.40	14
Make use of primary scientific research literature (e.g. journal articles)	4.47 ± 0.52	15	3.83 ± 0.83	12	4.16 ± 0.90	19	3.64 ± 1.01	14
Formulate a research hypothesis based on a specific question	4.20 ± 0.56	15	3.64 ± 1.03	11	4.00 ± 0.58	19	3.50 ± 1.22	14
Design an experiment or theoretical test of the hypothesis	4.07 ± 0.46	15	3.60 ± 0.97	10	3.74 ± 0.56	19	3.57 ± 1.40	14

aSummer 2010 and 2011 Likert-scale surveys were administered to SSMV students and their research mentors following the junior-year Summer research internship. Results from students in the graduating classes of 2011 and 2012 are shown.

Student Achievement

For assessing the impact of the SSMV program on student academic achievement, ∼100 MNPS students were selected as a comparison group for each of the four class cohorts between 2007 and 2011. These students were chosen based on seventh-grade math and science TCAP scores. A summary of the achievement scores during their high school years for both the SSMV and comparison groups of students is shown in Table 7. Overall, even though the comparison group was selected based on equivalent/similar achievement scores in seventh grade, SSMV students outperformed the comparison students in all categories except ACT math. Examining the specific measures, the mean GPA for the SSMV students in the class of 2011 was three points higher than the comparison group. SSMV students scored 0.62 and 0.78 points higher on AP biology and chemistry tests, respectively. On EOC tests, there was an approximate 19-point difference between SSMV students and the comparison group. On the PLAN test, science and math scores were significantly higher for SSMV students than for the comparison students, and SSMV students had significantly higher ACT science scores than comparison-group students. When repeating these analyses by ethnicity, significant differences between the SSMV and comparison group were found for both black and white SSMV students for GPAs, biology EOC test scores, and PLAN science scores (Figure 2).

Table 7.

SSMV and comparison student performance analysisa

	SSMV		Comparison
Measure	Score ± SE	n	Score ± SE	n	p
GPA	96.87 ± 0.51	86	93.87 ± 0.26	293	0.001**
Biology EOC	630.83 ± 5.56	39	611.73 ± 2.47	196	0.002**
PLAN Math	27.01 ± 0.55	64	25.53 ± 0.25	268	0.016**
PLAN Science	25.45 ± 0.54	64	23.84 ± 0.25	266	0.007**
ACT Math	28.03 ± 1.02	14	26.69 ± 0.46	69	0.235
ACT Science	28.77 ± 1.09	14	25.77 ± 0.5	69	0.015**
AP biology	4.10 ± 0.28	16	3.48 ± 0.19	34	0.073*
AP chemistry	4.38 ± 0.34	14	3.60 ± 0.24	28	0.070*

*Marginally significant: p = < 0.10.

**Significant: p ≤ 0.05.

aA quasi-experimental approach was used to examine the academic achievement of SSMV students compared with a group of students who did not attend the SSMV. The comparison group (∼100 students for each SSMV class) was selected on the basis of matching seventh-grade science and math scores on the end-of-year TCAP tests. Differences in scores were examined using SPSS 16.0 software and GLM UNIANOVA statistical analyses.

Figure 2.

Achievement scores for SSMV and comparison students analyzed by ethnicity. A comparison group of students was selected by the research team for each SSMV graduating class in the first 4 yr of the program (2007–2011). The comparison group was selected based on seventh-grade math and science scores on the TCAP. There were a total of 92 SSMV students and 417 comparison students, with 13% black SSMV students, and 11% black students in the comparison group. A comparison of GPAs and biology EOC and PLAN science scores for all students, white students, and black students in shown in the figure. In each case, the SSMV student scores were significantly higher (p < 0.05) than the comparison student groups.

Student Awards and Achievements

Three competitions are of particular importance to the SSMV curricular requirements. First, in the Fall semester of their junior year, all SSMV students are required to submit written results from their sophomore projects to the TJAS, the primary state-wide STEM competition for high school students. Based on these reports, students are invited to orally present their projects at the annual TJAS meeting, and students are recognized for their work with awards and publication in the TJAS journal. To date, a total of 89 students have submitted reports of their group projects, with 78% invited to give oral presentations at the annual symposium, and 63% recognized with first- or second-place or honorable mention awards (Table 8). In all 3 yr, more than half of the TJAS awardees have been SSMV students, a significantly higher representation than from any other high school in the state.

Table 8.

Student awards and recognitionsa

Competition	Class of:	Number of students submitting	Number of student winners
Siemens	2011	17	4 semifinalists
		2012	18	2 regional finalists (1 project); 2 semifinalists
	2013	16	4 semifinalists
Intel	2011	20	2 semifinalists
		2012	19	3 semifinalists
	2013	19	7 semifinalists
TJAS	2011	16 (6 projects)	Second place: 11 students (4 projects); honorable mention: 5 students (2 projects)
		2012	16 (5 projects)	Second place: 11 students (3 projects); honorable mention: 5 students (2 projects)
		2013	15 (5 projects)	Second place: 10 students (3 projects)
	2014	22 (6 projects)	First place: 4 students (1 project); second place: 10 students (3 projects)

aSSMV students were required and/or encouraged to compete in local, state, regional, and national science competitions. Results of awards won by students since 2007 in the three required competitions—the Siemens Competition in Math, Science and Technology (Siemens); the Intel Science Talent Search (Intel); and the Tennessee Junior Academy of Science (TJAS)—are shown. Student research projects for both Siemens and Intel were conducted over the course of junior-year second semester through senior-year first semester and were submitted for the competitions in the Winter of the senior year. Projects for the TJAS competition were completed as sophomores and submitted in the Fall of the junior year.

Second, SSMV seniors who have completed their laboratory research projects spend the first semester of their 12th-grade year preparing submissions for both the Siemens and Intel competitions, which are generally regarded as the top science competitions in the United States, with more than 2000 students applying annually from hundreds of high schools. Each year, Siemens recognizes ∼300 semifinalists and 90 regional finalists, and Intel names 300 semifinalists and 40 finalists. Three SSMV classes have now competed in these competitions with a phenomenal success rate (Table 8). Competing against much larger schools with enrollments ranging from 200 to 800 per high school grade, the SSMV was tied for 11th place in the United States in 2013, with a total of 24 Intel and Siemens semifinalists and regional finalists out of 102 submissions (Figure 3).

Figure 3.

Comparison of the number of SSMV semifinalists and regional finalists in the national Siemens and Intel science competitions to other top U.S. high schools. SSMV seniors are required to submit results of their junior/senior-year research projects to the annual Siemens Competition in Math, Science and Technology and the Intel Science Talent Search. Over the past 3 yr, the SSMV has had a total of 24 semifinalists and regional finalists in both contests, placing this program 11th among all high schools submitting to these competitions. TAMS: Texas Academy of Math and Science; NCSSM: North Carolina School of Science and Math; IMSA: Illinois Math and Science Academy. (Note: SSMV student winners in the Intel contest are listed by their zoned school, not the SSMV.)

Third, one of the main curricular focuses during the Advanced Research course is the importance of scientific writing. This component has led to the development of a novel university-based high school research journal called Young Scientist, a publication that recognizes the research and achievements of high school students who have participated in research internship across campus. In each article, the abstract and introduction targets a broad audience, allowing the general public to understand the focus and outline of the research, and the methods and discussion sections are written more technically so that details of the experiment can be understood by scientific peers. Submitted articles are anonymously reviewed by two to three graduate students or postdoctoral fellows before publication. Young Scientist is available in both print (ISSN: 2163–7822) and online (ISSN: 2163–7830, http://www.youngscientistjournal.org). Young Scientist has published three issues with a total of 59 articles. The goal is to expand access to this journal to all high school researchers at Vanderbilt and potentially establish Young Scientist as a national high school journal. Importantly, SSMV students have also coauthored publications in peer-reviewed scientific journals, with five publications to date (Biswas ; Mandel ; Moitra ; Nyman ; Pekala ).

College Attendance and STEM Majors

The SSMV has now graduated three classes, with a college matriculation rate of 98%. Approximately 60% of SSMV students are attending top 50 colleges and universities, compared with < 20% for students from MNPS academic magnet schools. In addition, 12% of SSMV student are attending Tennessee public institutions compared with 45% from the magnet schools. Longitudinal studies are also underway to track STEM majors and, ultimately, entry into professional schools and STEM careers. Early data suggest that a high number of graduates (79%) are continuing in STEM disciplines compared with 10% nationally and 53% from specialized schools focusing on science and math instruction (Thomas, 2000).

DISCUSSION

In the current paper, we have described the development and implementation of an innovative partnership program between a Research I private university and a large urban public school system that provides a rigorous part-time academic program for highly talented students with an interest in future careers in STEM disciplines. As the program has expanded from the initial class of 25 ninth graders to its current capacity of 101 students representing all four grades, the demonstrated outcomes include higher achievement on standardized tests compared with a matched control group, high college matriculation rates and selection of STEM majors, recognition in national science competitions, and coauthorship on scientific research publications. In the current study of the SSMV program, a number of key features have emerged that have been highlighted by other researchers as factors promoting entry of students into STEM careers.

At the heart of the SSMV program is the concept of research-based science throughout all 4 yr, culminating in an independent research project in the junior and senior years in the laboratory of a Vanderbilt faculty member. This type of instruction at the high school level not only introduces the students to the world of the research scientist but provides the students with opportunities to conduct their own authentic research project, potentially resulting in a scientific publication before entering college. In their article entitled “Becoming a Scientist,” Hunter ) reported that participation in STEM research provided students with the confidence to conduct independent research and increased their self-concept and interest in the field. In a recent study of a high school research internship program at the University of Rochester, students reported that the program had a positive influence on their performance in advanced science courses, enhanced their interest in pursuing a career in science, and increased their confidence in applying sophisticated laboratory skills (Knox ). In the recent study of specialized STEM schools, Subotnik reported that students who had a research experience and whose teachers connected the content across STEM disciplines were almost twice as likely to complete a STEM major than their peers (Subotnik , 2011). These characteristics apply to the SSMV and suggest that the graduates of the SSMV are more likely to enter STEM careers.

The SSMV course work is composed of seven honors electives that focus on rigorous, research-based curricula. More rigorous course work has been linked directly to increased overall achievement by students, leading to higher selection of STEM majors in college (Adelman, 1999). In addition, taking eight or more semesters of science has been positively correlated with an increase in science proficiency for students who started at the top science proficiency level in eighth grade (NCES, 1997), suggesting that the additional seven credits of science for SSMV students should contribute positively to their movement along the STEM career trajectory. Using data from the National Education Longitudinal Study of 1988 (NELS: 88) to study the effects of taking science courses, Madigan found that for students who started at the lower end of the science proficiency spectrum, taking advanced-level science courses and a higher number of science courses was related to increased science proficiency (NCES, 1997). For higher achievers, taking advanced science courses seemed more crucial than the number of science courses. In a separate study using the High School Effectiveness Supplement of the NELS: 88 data set, Lee ) found that completing high-level mathematics courses is strongly related to achievement, and Leow ) reported that taking advanced courses is consistently associated with higher achievement.

The SSMV curriculum includes a variety of instructional strategies that researchers have underscored as critical for enhancing positive attitudes toward STEM learning. Oakes reported that active learning and group work enhance attitudes as well as increase persistence in continuing studies beyond high school (Oakes, 1990). Piburn and Baker reported that students preferred group work, hands-on activities, fewer lectures, and in-depth discussions (Piburn and Baker, 1993). Maltese and Tai reported that students who were taught by teachers who used textbooks or lectured were less likely to go on to STEM degrees (Maltese and Tai, 2011).

Within the SSMV classroom, students are routinely involved in activities that require interaction between instructors and students, as well as small-group activities that promote peer relationships. Students feel comfortable in asking questions and in participating in challenging discussions with SSMV instructors and visiting faculty. These types of activities promote a classroom community of students and their instructors, resulting in enhanced self-concept and interest in STEM (Beier and Rittmayer, 2008). The SSMV curriculum also provides forums such as small-group work for students to develop relationships with one another, promoting a sense of “belongingness” while also providing peer support (Beier and Rittmayer, 2008). As reported by Subotnik ), this feeling of belonging shows a positive association with completion of a STEM-related course of study in college.

Finally, the SSMV has provided an important opportunity for graduate and postdoctoral trainees to gain mentoring and teaching experience as well as involvement in K–12 outreach activities which may stimulate continuation of these activities in their future careers (Demb and Wade, 2012). One Vanderbilt faculty member recently commented that hosting SSMV students in his laboratory actually renewed the enthusiasm for science discovery for his graduate and postdoctoral trainees. As Cacciatore and Sevian have suggested, this type of involvement of scientists in K–12 education can reinforce their own excitement about science as they contribute to improving K–12 students’ readiness for college (Cacciatore and Sevian, 2011).

Conclusion and Recommendations

The national reform mandates and the demand by federal funding agencies to involve university scientists in K–12 education have prompted a dramatic rise in the commitment of university STEM departments to assist in enhancing the preparation of K–12 students for entry into colleges in general and STEM careers specifically. In the 1990s, a number of reports encouraged universities to become “engaged campuses” (Ramalay, 1997). As pointed out by Sanders, colleges and universities “have a special responsibility and opportunity to build alliances with external constituencies and to develop stronger public support for learning at every level by every sector of society” (Sanders, 2004, p. 8). Universities can have a major impact on student achievement by implementing and supporting precollege activities on their campuses (Harvey, 2008). Major professional scientific societies have called on their members to become involved in K–12 outreach efforts and have encouraged institutions of higher education to give appropriate credit to faculty who participate in K–12 STEM outreach activities (American Society of Human Genetics, 2012; American Physiological Society, 2009).

The SSMV has emerged as a highly successful, effective, and unique model for engaging university scientists in the education of precollege students. It fulfills many of the recommended features of a university and K–12 partnership by clearly defining its participation opportunities, fulfilling the “service to community” mission of most universities, and demonstrating an impact on promoting students into continuing scientific research in college as STEM majors (Andrews ; Demb and Wade, 2012). What started out as an unusual 1-d pull-out enrichment program has evolved into a stable, highly sought after, rigorous, research-intensive curriculum for highly motivated high school students. All partners have realized benefits from this program. A great number of faculty members regularly participate in the program through lectures and classroom discussions, direction of small-group projects, and mentoring of students in independent research projects. Graduate students and postdoctoral fellows gain teaching, mentoring, and communication skills through interaction with SSMV students. SSMV students gain higher university aspirations, choose STEM majors in college, and continue their participation in research as undergraduates. The model has matured to the point where waiting lists for admission are increasing, university faculty members are requesting students for internships in their laboratories, and outcomes can now be reported.

With the increasing calls for universities to become involved in K–12 education, new partnership models are needed to better prepare our students for the challenges of the 21st century. While a number of full-time STEM secondary schools operate in collaboration with a college or university, in most cases the cost of these stand-alone schools is prohibitive when considering the increasing budgetary challenges faced by universities and local school districts (Thomas and Williams, 2009). To circumvent this challenge and continue specialized STEM schools, other university and K–12 models that are more cost effective must be considered. The SSMV is one example of a successful cost-effective model that provides high school students with the necessary skills to continue toward a STEM career path.