How Do You Like Your Science, Wet or Dry? How Two Lab Experiences Influence Student Understanding of Science Concepts and Perceptions of Authentic Scientific Practice.

This study examines how two kinds of authentic research experiences related to smoking behavior-genotyping human DNA (wet lab) and using a database to test hypotheses about factors that affect smoking behavior (dry lab)-influence students' perceptions and understanding of scientific research and related science concepts. The study used pre and post surveys and a focus group protocol to compare students who conducted the research experiences in one of two sequences: genotyping before database and database before genotyping. Students rated the genotyping experiment to be more like real science than the database experiment, in spite of the fact that they associated more scientific tasks with the database experience than genotyping. Independent of the order of completing the labs, students showed gains in their understanding of science concepts after completion of the two experiences. There was little change in students' attitudes toward science pre to post, as measured by the Scientific Attitude Inventory II. However, on the basis of their responses during focus groups, students developed more sophisticated views about the practices and nature of science after they had completed both research experiences, independent of the order in which they experienced them.

INTRODUCTION

The contributions of authentic research experiences to students’ recognition and comprehension of a wide range of science practices have been widely established in the science education community (Driver ) and have been called out in A Framework for K–12 Science Education (hereafter referred to as the Framework; National Research Council, 2012), which underlies the Next Generation Science Standards (NGSS; NGSS Lead States, 2013). The science practices highlighted in the NGSS include, among others, planning and carrying out investigations, analyzing and interpreting data, constructing explanations, and engaging in argumentation from evidence.

Closely allied with the practices of science woven throughout the NGSS is the importance of providing students a means of understanding science as both a way of knowing and a body of knowledge—an understanding of the nature of science (NOS). “Engaging in the practices of science helps students understand how scientific knowledge develops” (National Research Council, 2012, p. 42). The NOS has been defined and discussed extensively in the science education research literature and generally includes the following aspects: 1) science is both empirical and inferential; 2) scientific knowledge is reliable but tentative; 3) scientific investigation requires creativity; 4) scientific knowledge is partly subjective (theory-laden) because of the beliefs, prior knowledge, training, and currently accepted theories of scientists; 5) the laws of science are different from scientific theories; 6) science reflects society and culture; and 7) scientists use many methods and practices to develop scientific knowledge (for descriptions of the NOS, see Lederman and Lederman, 2004; Abd-El-Khalick, 2012).

The science education research community is divided on the usefulness of instruction in the NOS at the K–12 level, in part due to lack of a unified view of the NOS among scientists (Wong and Hodson, 2009). In addition, some argue that the NOS described above does not comprise a useful framework to guide K–12 students as they develop their understanding of science concepts and practices (Elby and Hammer, 2001). In the research described in this paper, we did not set out to teach the NOS, although several questions asked during the discussion groups probed student understanding of aspects of the NOS as well as the practices of science as defined in the Framework and the NGSS. Many of the responses made by students during the focus groups reflected a growing understanding of the NOS, especially after completing both research experiences. For this reason we will include discussion of the NOS throughout this paper and point out examples of student comments related to their understanding of the NOS.

As discussed by Wong and Hodson (2009, p. 110), “the goal of improving NOS understanding is often prejudiced by stereotyped images of science and scientists consciously or unconsciously built into school curricula and perpetuated by science textbooks.” In contrast to the controlled experiment typically taught in precollege classrooms, Windschitl and colleagues point out,

Scientists in astronomy, genetics, field biology, oceanography, geology, and meteorology routinely create models of phenomena not by controlling conditions, but rather by selecting naturally occurring observations and looking for descriptive, correlative, or causal trends in those observations… Indeed, these researchers may be looking for cause and effect relationships through differences between two sets of observations, but these observations do not arise from controlled situations per se. (Windschitl , pp. 383–384)

Thus, as educators strive to expand students’ images of science, it is critical to immerse the students in authentic research experiences that more closely resemble the practices conducted by scientists, while identifying and expanding upon student perceptions of how science is carried out.

In many biology courses, teachers have relied on molecular wet lab experiments to provide context and hands-on experiences for high school students. Though effective, these wet lab experiments often require expensive laboratory reagents and equipment and a substantial time commitment from teachers, limiting access to these experiences. Increasingly, scientists carry out “dry lab” research using publicly accessible computer tools and databases, and these resources have potential for immersing secondary-level science students in authentic research practices as an alternative or addition to traditional molecular biology. This approach has been used successfully in a few innovative biology units at the high school, undergraduate, and graduate levels (Campbell, 2003; Rice ; Kumar, 2005; Shaffer ; Kovarik ). However, the use of authentic online tools and databases in high school science courses does not necessarily support students in developing an expanded understanding of inquiry or the NOS, as described in a study by Waight and Abd-El-Khalick (2011). For students to optimize their ability to conduct meaningful research using scientific tools and databases, they need to understand related scientific concepts; the context, purpose and history of the tools and/or database; and the process and limitations for their use.

The gold standard for engaging students in authentic research has been through multiweek internships in science labs, which have proven to be a powerful tool for promoting student science, technology, engineering, and mathematics (STEM) interest and learning (Seymour ; Hunter ; Lopatto, 2007, 2010; Sadler and McKinney, 2010). However, relatively few such research opportunities exist for high school students. As an alternative approach, we have developed two linked science education programs that enabled students to participate in authentic research as part of their biology courses under the direction of their teacher. These two multiyear projects involved students in contributing to an ongoing epidemiological study of smoking behavior developed specifically for classroom research. This approach allowed hundreds of students to participate in research each year through a classroom experience that lasted approximately 2 weeks.

Through the foundational project in which the smoking behavior study was designed and implemented, we worked with high school teachers, scientists, biostatisticians, and bioethicists to design a research investigation of smoking behavior and develop a curriculum that enabled high school students to contribute as scientists to aspects of the research (Munn ). The smoking investigation used a case control study design to compare ∼150 case subjects who were adult regular smokers and ∼150 control subjects who were adult nonsmokers who had tried smoking but never became regular smokers. The investigation included the use of a questionnaire to collect information about a wide range of environmental factors to which research subjects may have been exposed when they first tried smoking and the collection of a small blood sample used for genotyping three candidate gene loci that were selected based on association with smoking in the literature.

At the start of the foundational project, high school students contributed to the study design by writing questions related to environmental factors that they thought might influence becoming a smoker, and their questions were integrated into the research questionnaire. Later, after the study design had been approved by the University of Washington (UW) Institutional Review Board (IRB) and subject recruitment had begun, students from subsequent classes contributed by genotyping subject DNA through an in-class wet lab. Aspects of the smoking study that were not appropriate for students to conduct, including subject recruitment, consenting, blood draws, questionnaire collection, and DNA isolation, were conducted by professionals at the UW. The questionnaire and genotyping data were entered into the queriable smoking behavior database. The completed smoking behavior database became the centerpiece for a new classroom-based research program called Exploring Databases (Munn ) in which students could propose and test their own research questions using the genetic and environmental data for ∼300 research subjects contained in the database.

In the study described in this paper, we examined how two different research experiences influence students’ perceptions and understanding of the practices of scientists, how scientific research is conducted, and related scientific concepts. Students carried out 1) the wet lab used to genotype human DNA at three loci associated with smoking behavior, as conducted in the foundational project; and 2) a dry lab in which they used the smoking behavior database to test their own hypotheses about factors that affect smoking behavior. Classrooms were assigned to two possible sequences to complete the lessons and activities so the influence of each research activity could be evaluated. Through their participation with the project, students learned about genetics and neuroscience, developed skills in formatting and testing hypotheses and applying scientific reasoning, and used information and communication technology. The two research experiences do not follow the experimental design traditionally taught in high school textbooks, thus helping to broaden student understanding of how research is conducted, what kinds of research questions scientists ask, and the multiplicity of approaches used by scientists to pursue the research questions that interest them.

This study addresses the following questions:

To what extent do students learn science concepts as a result of the two research experiences?

Does this curriculum influence student attitudes toward ­science?

Do students perceive wet lab genotyping and dry lab database experiences to be “authentic science” to the same extent?

METHODS

Overview of Study

The Exploring Databases curriculum used for this study is described in Table 1 and is provided in the Supplemental Material. In lesson 1, students were exposed through videotaped interviews to the many methods used by scientists from different fields to conduct their research (as opposed to a single “scientific method”). Lessons 2–5 guided students in developing their own theoretical and disciplinary knowledge related to smoking behavior and case control studies through examination of smoker profiles and lessons on genetic influences on smoking behavior, the case control study design, and relevant statistical analyses. Through instructions in lesson 6, students used the database to test their own research questions. Two lessons from the curriculum of the foundational project were used for the genotyping experiment, activity D3, in which students modeled genotyping using a paper and scissors activity, and activity D4, which provided the genotyping protocol (Munn ; provided in the Supplemental Material).

TABLE 1.

Exploring Databases lessons and genotyping activitya

Lesson 1. Why and how do people do science?	Through several interviews with scientists, students learn about the many approaches scientists take to conduct their research.
Lesson 2. Why do some people become smokers and others do not?	Students explore the wide range of smoking behavior and discover factors that influence people’s smoking through profiles of actual smokers and nonsmokers.
Lesson 3. How do genes influence smoking behavior?	Students learn how nicotine interacts with the body, discuss what genes might influence variation in smoking behavior, and learn about the genes included in the smoking behavior study.
Lesson 4. How can we study genetic and environmental influences on smoking behavior?	Students learn the characteristics of one commonly used epidemiological study design, the case control study, and they learn the details of the smoking behavior case control study.
Lesson 5. Analysis of data in case control studies: the odds ratio	Students learn how to calculate the odds ratio and determine its statistical significance using the 95% confidence interval. They learn the difference between association and causality and how to apply the criteria for causality in their research.
Lesson 6. Database research: What can we learn from the smoking behavior data?	Students test their research hypotheses using the smoking behavior database. They develop research presentations and present their research to their class.
Wet labb activity D3: Modeling the Reverse Dot-Blot Assay	Students learn about the genotyping process using a paper model
Wet labb activity D4: Reverse Dot-Blot Assay	Students use a reverse dot-blot assay—similar to a microarray, but on a macroscopic scale—to genotype human subject DNA in their classroom labs. Their teacher submits their results to program staff for inclusion in database.

aThe current version of Exploring Databases curriculum consists of seven lessons. In the version used for this study, students were allowed to view the data before forming their hypotheses, so they could learn from examining the patterns in the data. In the revised curriculum, students form their hypotheses before entering the database, so they can make more appropriate scientific inferences from the data. In the final lesson of the revised curriculum, “Hypothesis Generation,” they can examine patterns in the data before generating new hypotheses that could be tested in a future investigation.

bFrom curriculum of foundational project, Investigating the Effects of Genes and the Environment on Smoking Behavior.

The curriculum was implemented by teachers in western Washington during the 2010–2011 school year. Implementation from start to finish took 2–3 weeks of classroom time. All teachers completed the lessons as described herein, although they each made some modifications to customize for the learning needs of their students.

The overall structure of the study is shown in Figure 1. All classrooms involved in this study completed the six lessons of the Exploring Databases curriculum, including the database research and the genotyping experiment. Classrooms were assigned one of two different orders to present the curriculum so we could measure the impact of each type of research experience independently and after students had completed both. In sequence 1, referred to as GTDB (genotyping → database), teachers presented lessons 1–4 of the database curriculum, modeled and performed the genotyping experiment, and then followed with lessons 5 and 6. In sequence 2, called DBGT (database → genotyping), the six lessons of the curriculum were presented sequentially, followed by the genotyping modeling activity and experiment. Thus, all students experienced the same lessons and research experiences, but in a different order. Students completed pre and post surveys at the beginning of the curriculum and after completing both lab experiences. The survey included 10 science content questions to probe student understanding of science concepts and the Scientific Attitude Inventory II (SAI II; Moore and Foy, 1997; see Table 3 later in this article) to measure their attitudes toward science. A subset of randomly selected students participated in a focus group after each of the two research experiences to probe students’ understanding of the practices of science as it pertained to the two research experiences.

FIGURE 1.

Overview of research study design. All students completed the pretest and lessons 1–4, which provided background and context. Students in the GTDB condition completed the genotyping experience first and then the database experience (green boxes), and students in the DBGT condition completed the database experience and then the genotyping (blue boxes). For each condition, focus groups of five to eight students were held after each research experience. After completion of both research experiences, all students completed the posttest.

TABLE 3.

Position statements and subscales of SAI II

Subscale	+	−	Position statement	Corresponding NOS
SAI 1	A		The laws and/or theories of science are approximations of truth and are subject to change.	2, 5
		B	The laws and/or theories of science represent unchangeable truths discovered through science.
SAI 2	A		Observation of natural phenomena and experimentation is the basis of scientific explanation. Science is limited in that it can only answer questions about natural phenomena and sometimes it is not able to do that.	1, 7
		B	The basis of scientific explanation is in authority. Science deals with all problems and it can provide correct answers to all questions.
SAI 3	A		To operate in a scientific manner, one must display such traits as intellectual honesty, dependence upon objective observation of natural events, and willingness to alter one’s position on the basis of sufficient evidence.	1, 4
		B	To operate in a scientific manner one needs to know what other scientists think; one needs to know all the scientific truths and to be able to take the side of other scientists.
SAI 4	A		Science is an idea-generating activity. It is devoted to providing explanations of natural phenomena. Its value lies in its theoretical aspects.	3
		B	Science is a technology-developing activity. It is devoted to serving mankind. Its value lies in its practical uses.
SAI 5	A		Progress in science requires public support in this age of science; therefore, the public should be made aware of the nature of science and what it attempts to do. The public can understand science and it ultimately benefits from scientific work.	6
		B	Public understanding of science would contribute nothing to the advancement of science or to human welfare; therefore, the public has no need to understand the nature of science. They cannot understand it and it does not affect them.
SAI 6	A		Being a scientist or working in a job requiring scientific knowledge and thinking would be a very interesting and rewarding life’s work. I would like to do scientific work.
		B	Being a scientist or working in a job requiring scientific knowledge and thinking would be dull and uninteresting; it is only for highly intelligent people who are willing to spend most of their time at work. I would not like to do scientific work.

Wet Lab Research Experience.

Students worked in teams to genotype DNA from research subjects from the original epidemiology study (two to three class periods to complete activities D3 and D4, as described in Table 1). They analyzed polymorphic sites in the three candidate genes for smoking behavior already included in the smoking behavior database, and their data verified the previous data for those subjects. The three genetic loci had been selected by the project design team based on their association with aspects of smoking behavior published in previous scientific publications (Gelernter ; Duan ; Yu ). Teachers guided their students in using the protocol developed in the foundational project (Munn ; provided in the Supplemental Material) and equipment and reagents provided by this program. Students followed the reverse dot-blot protocol (analogous to microarray technology), first binding PCR-amplified DNA from a single subject to DNA probes bound to a nylon membrane and then staining the membrane so they could visualize the positions where subject DNA had bound. After conducting the experimental process, student groups interpreted their own results and compared their findings with those of other groups to determine the most accurate genotype for each subject. Their teachers submitted their data to program staff for final verification and entry into the database.

Database Research Experience.

Working in teams, students developed a research question and related hypothesis about how specific genetic or environmental factors might influence becoming a persistent smoker (two to three class periods to complete lessons 5 and 6, as described in Table 1). Through the database, students were able to query the responses of case and control subjects to many of the 86 questions included in the research questionnaire, as well as subjects’ genotypes for the three genetic loci included in the study. Students selected up to four items in the database that were related to their research question, with each item being a question from the research questionnaire or one of the three genetic loci. Typically, case control studies use a statistic called an odds ratio to compare the odds that case subjects were exposed to a particular factor with the odds of control subjects being exposed. For each odds ratio, a 95% confidence interval is also calculated to determine statistical significance. Guided by directions embedded in the database and curriculum, students set up an odds-ratio calculation for each item of interest by defining the parameters for each query, used the database software to calculate the odds ratio and confidence interval, and then applied the criteria typically used by epidemiologists to determine whether the association was likely to be causal (Bradford Hill, 1965). Student teams presented their results and analyses through a research poster or PowerPoint presentation to their classmates, who provided feedback and critiques.

Participants

IRB Approval for Research Study.

Before beginning this research, we applied for and received IRB approval for research conducted with teachers and students through the UW Human Subjects Division (#38651).

Teacher Participants.

Teachers consenting to participate in the study were assigned to one of the two orders of curriculum implementation, such that similar student numbers, grade levels, and courses were represented in the two groups. At the end of the study, the classrooms of six teachers (out of 11 who ­initially signed up for the study) had completed both research experiences, had valid data for pre and post surveys, and allowed us to collect focus group data after each research ­experience.

Table 2 shows the teachers (numbered to maintain anonymity) and courses for the two study conditions and the number of students by grade level in each course. Teachers received training on the genotyping process during a weeklong teacher ­professional development (PD) workshop of the foundational project or during a 1-day genotyping workshop offered at the start of this study. The six teachers also participated in a 3- or 4-day PD workshop focused on implementation of the Exploring Databases curriculum. During the workshop, teachers experienced the lessons as their students would, discussed strategies for implementing each lesson, and participated in informal discussions with invited speakers on topics such as “Learning and Equity,” “The Use of Databases in Genome Research,” and “Research on Complex Traits: Ethical and Scientific Issues.” They also participated in a 1-day review workshop on the database research before the start of this study.

TABLE 2.

Teachers, courses, and number of students per grade level in each course for each conditiona

Condition	Teacher	Course	9th	10th	11th	12th	Total
GTDB	1	Genetics	20 (20)	80 (74)	10 (10)		110 (104)
	2	Genetics			2 (2)	15 (12)	17 (14)
	3a	Biotechnology			15 (11)	19 (15)	34 (26)
	3b	Advanced Biotechnology			4 (3)	15 (15)	19 (18)
	4a	Human Anatomy			6 (3)	3 (1)	9 (4)
	5a	General Biology	14 (13)	43 (39)	2 (2)		59 (54)
	5b	AP Biology		2 (0)	2 (0)	3 (0)	7 (0)
DBGT	4b	Pre-AP Biology	23 (23)				23 (23)
	6	AP Biology			1 (1)	15 (15)	16 (16)
Total			57 (56)	125 (113)	42 (32)	70 (58)	294 (259)

aNumbers in parentheses indicate students who completed pre and post assessments.

Student Participants Providing Pre–Post Survey Data.

At the end of data collection, the majority of students (87%), including all the 10th graders, were in the GTDB condition, as shown in Table 2. Because the distribution was so uneven, we did not attempt to make comparisons of student performance pre to post based on sequence of delivery. Instead, data from students in both sequences were combined for analysis of overall changes pre to post on content questions and SAI II items. To be eligible for inclusion in the survey analysis, subjects were required to have both pre and post SAI and content assessment data and to have a score for each SAI subtest and an answer to all 10 content assessment items. The number of students in each class that met the above criteria is given in parentheses in Table 2.

Student Participants Providing Focus Group Data.

The six teachers randomly selected five to eight students from each class to participate in the focus group. Focus groups were held midcurriculum after completion of the first research experience and postcurriculum after completion of the second research experience.

A total of 86 mid- and 85 postcurriculum focus group worksheets were collected from students participating in focus groups, including students in grades 9, 10, 11, and 12. Seventy students participated in both the mid- and postcurriculum focus groups and provided worksheets for each. Sixteen students participated only in the mid focus group but not the post focus group, while 15 students participated only in the post but not the mid focus group. Thus, focus group data came from a total of 101 unique students (70 + 16 + 15).

Data Collection for Pre–Post Survey

Before the curriculum implementation began, students completed a pre survey (see the Supplemental Material) that included 10 scientific content items and a vetted scientific attitude survey. The same survey was administered after implementation of the second lab experience.

Pre–Post Survey Scientific Content Items.

The content portion of the survey consisted of 10 items developed by the curriculum designer and evaluator. Of these, four were multiple choice and the remaining were open-ended. To simplify scoring, a value of 1 was assigned to each correct response, with no partial points given. The open-ended items were scored by two external content experts who were trained by the lead evaluator in the use of the scoring rubric. After 10 tests were scored as a group and discussed, a sample of 32 tests were pulled and scored independently by the scorers. Scores for each item were compared across all 32 tests. For each item a “1” was indicated if both scorers agreed and a “0” if they disagreed. The total agreement score for each test was computed by summing for each item and then dividing by the number of items. Agreements ranged from 50% to 100% across all tests. The percent agreement (82.7%) was calculated by averaging the percent agreement across all tests. The Cronbach’s alpha for reliability for the pre and post content items were 0.701 and 0.793, respectively. Content items had a high degree of face validity, because they were drawn from the curriculum.

Pre–Post Survey Scientific Attitude Inventory.

We included the SAI II (Moore and Foy, 1997; see Table 3) in the pre and post survey to test whether there was any change in students’ attitudes toward science as a result of the curriculum. The SAI II was integrated into a post survey administered to teachers during the 2011 Exploring Databases summer workshop and was also administered informally to 17 UW scientists (unpublished data). SAI scores were computed for each of the 6 SAI subtests and converted to a percentage for each item.

The SAI II is based on the SAI (Moore and Sutman, 1970) but was revised to include fewer attitude statements and simpler, gender-neutral language. The SAI II includes the same 12 position statements as the original inventory. These were divided into six subscales, each with one positive attitude toward science and a matching statement expressing the opposing view. For each position statement, there are either three or five attitude statements (40 total). Responses are given using a five-point Likert scale ranging from “strongly agree” to “strongly disagree.” Attitude statements that best supported each position statement were determined by a panel of 10 expert judges for the original SAI, and this scoring was maintained for the revised inventory.

The first five subscales of the SAI II encompass many aspects of the NOS, including the tentative and empirical nature of scientific knowledge, the theory-laden aspect of conducting scientific research, the need for creativity/imagination in generating scientific ideas, and the reflection of society and culture in science. The last column of Table 3 cross-references the NOS items described in the Introduction to each of the first five subscales. The final subscale is focused on students’ personal reflections on whether they would want to be a scientist. This instrument was selected for this study because it probed a wide range of students’ attitudes about the NOS as well as their personal interest in being scientists, was easily comprehended and completed by students, and was easy to score. In previewing the 40 questions included in the survey, we anticipated that we might see a change in student attitudes as a result of the enacted curriculum and research experiences.

As shown in Table 3, position statements marked with an “A” are meant to be positive, while those with a “B” are negative (position statements quoted from Moore and Foy, 1997). Item choices associated with the positive subscales are scored 5–1 while item choices associated with negative items are scored 1–5 for the five possible responses ranging from “strongly agree” to strongly disagree.” Total scores for each subscale were calculated by adding total items of both positive and negative items. Results were analyzed at both the subscale and individual item levels to identify changes from pre to post administration. Moore and Foy (1997) calculated a reliability statistic (Cronbach’s alpha = 0.805) using data from 557 subjects in the development of the SAI II, and our reliability calculations for alpha (pre: 0.792; post: 0.826) yielded similar results.

Data Collection for Focus Groups

A focus group protocol was developed to probe more deeply into the impact of the genotyping and database experiences on student perceptions about science. As described earlier, focus groups of five to eight students were held twice during the curriculum implementation after each research experience.

During each visit, the researcher observed the classroom and then held a focus group either after class, at lunch, or after school. Each focus group consisted of two parts: 1) a focus group worksheet and 2) a facilitated discussion using a set of guiding questions.

Focus Group Worksheet.

Students completed the worksheet individually at the start of the focus group to collect quantitative data and to focus student attention on the discussion issues (provided in the Supplemental Material). Components of the worksheet are summarized below:

The first item asked students to list words, concepts, and images of a “typical” scientist at work. Using this mental image as a point of reference, students then placed a rating on a seven-point semantic differential scale to indicate how the lab that they just completed was similar (or not) to their ideal image of what scientists do.

Students were then presented with 13 task descriptions typically associated with scientific inquiry and were asked which of these tasks they had performed in the lab activity just completed.

Facilitated Focus Group Discussion.

After students completed the focus group worksheet, the discussion was started by the researcher. The responses to the worksheet items were discussed in detail and analyzed for common themes and unique perspectives. As time permitted the researcher used several discussion questions (provided in the Supplemental Material) to elicit more detail from the students about how each type of lab influenced their thinking about the scientific enterprise.

RESULTS

Results for Pre–Post Survey

Content Items (Items 14–24 of the Survey).

Figure 2 shows the percentage of students scoring correctly on each of 10 content items. The questions corresponding to each item are provided in the Supplemental Material. Overall, increased percentages of students from pre to post were able to correctly answer each content item, as shown in Figure 2. The increase was significant (p < 0.05) for all questions except 15 (purpose of controls) and 21 (explanation for their hypothesis in previous question). Item 24 (Explain your answer using math) had the lowest number of correct responses on both the pre- and posttests compared with the other questions, although 6.5 times as many students answered correctly after the intervention than before.

FIGURE 2.

Percentage of students answering correctly on each of 10 content items on the pre- and posttests (N = 259). In brief, content items were 14. The reward pathway is 15. Purpose of controls in a scientific research study; 16. Purpose of doing same three controls during genotyping; 17. Why is each subject genotyped more than once?; 18. How do scientists decide which gene regions to test?; 19. Give one example of a multifactorial trait; 20. Create a hypothesis based on data provided; 21. Why did you construct hypothesis this way?; 23. Explain your answer in words; 24. Explain your answer using math. A significant increase in correct responses (p < 0.05) occurred for all except items 15 and 21.

Scientific Attitude Inventory.

A high score on the SAI II reflects a positive attitude toward science, as defined by the panel of 10 expert judges who developed the scoring guide for the original SAI (Moore and Sutman, 1970). As shown in Figure 3, from pre to post there were no significant differences in the paired means for any of the six SAI subscales for students (p > 0.05). Teachers scored higher than students on all six of the subscales. Teachers and students scored lowest and nearly the same on SAI 4, which has items dealing with the purpose of science: knowledge building (positive subscale) versus technology development for practical applications that serve mankind (negative subscale).

FIGURE 3.

Comparison of student scores on the six SAI II items pre to post (N = 259). There was no significant change in student scores pre to post at the p < 0.05 level. Scores from 11 teachers who completed the SAI II at the end of a workshop are also shown.

Results for Focus Groups

Focus Group Worksheet Analysis

Which Research Experience Did Students Rate as Being More Like “Real” Science?

Regardless of the order in which they had completed the research experiences, students rated the genotyping activity higher in terms of being more like “real” science than the database activity (means = 5.3 and 4.94, respectively, on a seven-point scale). This result (t test for independent samples) is significant at the p < 0.05 level (p = 0.026). While the total effect size is relatively small, it nonetheless supports the general stereotyped view of science held by many students that science is more about manipulation of equipment and tools to generate data than about analysis of data to generate ideas (see Focus Group Discussion Analysis below).

As part of the worksheet, students were given a list of tasks associated with scientific research and asked to indicate whether or not they had the opportunity to engage in each task for both the database and genotyping activities. The results are presented in Table 4. Students reported that, on average, they engaged in 9.5 of the 13 scientific tasks presented on the worksheet during the database activity and 8.3 tasks during genotyping. This difference is significant at the p < 0.01 level (p = 0.001; t test for independent samples). The magnitude of the difference is small (∼1 additional task indicated for the database activity) but illustrates that students recognize the scientific tasks that they engaged in during the database activity, even though they generally felt that genotyping was more like real science.

TABLE 4.

Percentage of students indicating that they performed each stated inquiry task when engaging the either the genotyping or database labsa

Inquiry taskb	Which lab?	Percent indicating they did this task	χ2(1)
Came up with a research question to answerc	Database	96	28.8
	Genotyping	63
Came up with at least one hypothesis to testc	Database	98	48.1
	Genotyping	53
Designed a way to test your hypothesisc	Database	68	4.7
	Genotyping	52
Collected data in some way	Database	91	0.4
	Genotyping	93
Used the same kind of “tools” and “equipment” that scientists used	Database	75	13.3
	Genotyping	95
Used the same kind of ‘techniques’ that scientists use	Database	85	1.5
	Genotyping	91
Analyzed data using qualitative methods such as describing results using wordsc	Database	85	5.0
	Genotyping	71
Analyzed data using quantitative methods such as numbers and statisticsc	Database	92	31.4
	Genotyping	55
Represented data in some way (e.g., chart)c	Database	76	5.1
	Genotyping	60
Came up with a conclusion based on your data analysisc	Database	83	5.1
	Genotyping	68
Came up with an answer to your questionc	Database	78	11.5
	Genotyping	54
Shared your results with others	Database	75	1.8
	Genotyping	66
Justified your conclusions to othersc	Database	64	4.6
	Genotyping	48

aThe 171 focus group worksheets were divided into two piles containing similar numbers from the first and second focus groups. One pile (N = 91) was analyzed for responses related to genotyping, and the other (N = 88) was analyzed for responses related to database.

bAll items in bold were statistically significant at p < 0.05 using a 2 × 2 chi-square procedure. Chi-square statistics (df = 1) are given in the rightmost column.

cPercent of students indicating that they did this task was greater for the database unit compared with the genotyping unit.

dPercent of students indicating that they did this task was greater for the genotyping unit compared with the database unit.

As shown in Table 4, a significantly higher percentage of students (p < 0.05 level) associated the following inquiry tasks with the database activity compared with the genotyping activity:

Came up with a research question to answer

Came up with at least one hypothesis to test

Designed a way to test your hypothesis

Analyzed data using qualitative methods such as describing results using words

Analyzed data using quantitative methods such as numbers and statistics

Came up with a conclusion based on your data analysis

Came up with an answer to your question

Justified your conclusions to others

One task, “Used the same kind of ‘tools’ and ‘equipment’ that scientists use,” was rated higher for the genotyping activity by a significantly larger percentage of students than the database activity. The focus group discussion analysis confirmed that students are more likely to perceive the tools and equipment of scientists to consist of items such as lab coats, beakers, and test tubes than computers and databases.

How Did Students Rate Their Understanding and Interest in the Two Research Experiences?

While there were no significant differences between how students rated their understanding of each lab type (nearly 90% indicated that they understood each totally or somewhat), a higher percentage of students indicated that they “totally” understood the database lab (34.7%) as compared with genotyping (26%). Similarly, a larger proportion of students indicated that they understood genotyping not very much or not at all (12.5%) as compared with the database activity (8.9%). In general, there was more understanding of what students were doing in the database lab than in the genotyping lab, although the percentages are high for both types of labs. A nearly equal proportion of students indicated that they were interested in both types of labs. A quarter reported that they were totally interested, and more than half were somewhat interested in the labs. About a fifth rated their interest as not at all or not very much. The type of lab does not appear to have much influence on the interest level of students toward the lab.

Focus Group Discussion Analysis.

Each focus group discussion was recorded and transcribed. The researchers read each transcript and identified common and unique responses to each question. Responses were then compiled and organized by focus group occurrence (mid or end of curriculum) and by curriculum sequence condition (GTDB or DBGT). The results for three questions are discussed below.

Did Completing Both Research Experiences Influence Student Images of Scientists and What They Do?

A summary of GTDB and DBGT responses based on both the worksheets and discussions are found in Tables 5 and 6, respectively.

TABLE 5.

Summary of initial and post images of scientists and science: GTDB conditiona

First focus groups (after genotyping)	Second focus groups (after database)
SETTINGS
• Lab or lab station (22)	Similar to FG1
• Not necessarily a lab (1)
MATERIALS
• Lab coat/aprons (26)	Similar to FG1, but more students mentioned:
• Chemicals/solutions (20)	• Databases (9)
• Test tubes/vials/flasks/beakers (19)	• Computers (4)
• Goggles/glasses (16)	• Research subjects/case studies (4)
• Notebooks/chalkboards (5)	In addition:
• Pipettes (4)	• Surveys (3)
• Microscopes (4)
• High-tech equipment (4)
• Research animals/plants/organisms (4)
• Computers (3)
• Databases (1)
• Research subjects/case studies (1)
PHYSICAL DESCRIPTION
• Einstein-esque, crazy hair (3)	Similar to FG1
• Middle-aged adults (2)
• Graduate students (1)
• Look like regular people (1)
COGNITIVE PROFILE
• Smart (15)	Similar to FG1, but in addition, students asked:
• Collaborative/work in teams (8)	• What do scientists consider as they do their work?
• Motivated/hard-working (5)
• Thoughtful/full of ideas (5)
• Nerds/boring (4)
• Curious (2)
• Well educated (2)
SCIENCE PROCESSES
• Conduct research (34)	Similar to FG1, but in addition:
• Make hypotheses (16)	• Not only discover with new ideas, but also examine and test existing knowledge
• Design experiments (47)	• Do scientists have a broad choice of subjects in which to investigate, or are they given topics to research?
• Make observations/collect data (12)	• Scientists explore a variety of factors in their data-collection process, including environmental, before making hypotheses and experimenting.
• Analyze data/discovery/develop models (9)	• Scientists experiment, discover, and learn (are more than a cartoonish caricature with crazy hair and glasses).
• Draw conclusions/find solutions (11)
• Present results in reports/scientific journals/at conferences (6)
• Use math or statistics (3)
TOPICS INVESTIGATED
• Diseases/cancer/malaria/cures (7)	• Similar to FG1
• Genetic information/DNA/mutations (10)
• Astronomy/space travel (2)

aStudent responses from focus group worksheets and follow-up discussions after genotyping (FG1; N = 71) and database (FG2; N = 73) research experiences. Number of students giving each response shown in parentheses.

TABLE 6.

Summary of initial and post images of scientists and science: DBGT conditiona

First focus groups (after database)	Second focus groups (after genotyping)
SETTINGS
• Lab or lab station (9)	Similar to FG1
• At a desk (1)
• The Amazon (1)
MATERIALS
• Lab coat/aprons (8)	Similar to FG1, but in addition:
• Chemicals/solutions (8)	• Computers are used more in science than [I] previously realized.
• Test tubes/vials/flasks/beakers (6)	• Computers and database enabled students to do higher-level work than they might do otherwise.
• Goggles/glasses (3)	• Micropipettes and goggles
• Pipettes (1)
• Microscopes (3)
• High-tech equipment (2)
• Research animals/plants/cells (4)
• Computers (2)
• Databases (1)
• Research subjects/case studies (1)
PHYSICAL DESCRIPTION
• Middle-aged adults (2)	Similar to FG1
• Graduate students (1)
• Wear formal clothing/clean (2)
• Do not take showers for several days (1)
COGNITIVE PROFILE
• Smart (3)	Similar to FG1
• Collaborative/work in teams (1)
• Motivated/hard-working (7)
• Thoughtful/full of ideas (5)
• Serious (2)
• Well educated (1)
• Logical/analytical (1)
SCIENCE PROCESSES
• Conduct research (5)	Similar to FG1, but in addition:
• Make hypotheses (3)	• Use math or statistics (1)
• Design experiments (9)	• Variety of methods/settings to answer similar questions
• Make observations/collect data (4)
• Analyze data/discovery/develop models (4)
• Draw conclusions/find solutions (2)
TOPICS INVESTIGATED
• Chemistry (4)	Similar to FG1, but in addition:
• Physics (2)	• Technological science
• Biology (2)	• Psychologist
• Diseases/cancer/malaria/cures (2)
• Genetic information/DNA/mutations (1)
• Epidemiology (1)
• Engineering (e.g., rocket science) (2)
• Astronomy/space travel (1)

aStudent responses from focus group worksheets and follow-up discussions after genotyping (FG1; N = 15) and database (FG2; N = 12) research experiences. Number of students giving each response shown in parentheses.

Students in the GTDB condition were asked to describe their mental images of a scientist on two occasions: just after the genotyping lab and just after they had completed the database activity. Responses fell into six general categories: setting, materials, physical description, cognitive profile, science processes, and topics investigated. After the genotyping lab but before the database activity (Table 5, column 1), GTDB students generated a stereotypical view of scientists in action: in a lab wearing a white coat and goggles, using pipettes, and mixing chemicals in test tubes. The scientists they described generally looked like Einstein and were smart, curious, thoughtful, and concerned about the future. The activities students attributed to scientists included creating hypotheses, conducting experiments, dissecting animals, recording data, and making new discoveries.

During the second focus group, just after the database activity (Table 5, column 2), GTDB students held onto the first focus group visions of the materials, setting, and physical descriptions of scientists. However, some students reported a broader understanding of science, reflecting that scientists not only discover new knowledge but examine and test existing knowledge; scientists explore a variety of factors and variables to generate hypotheses; and scientists are learners. A question about what scientists consider as they do their work was introduced and discussed.

Students in the DBGT group were also asked what their mental images of a scientist were on two occasions: just after the database lab and just after they had completed the genotyping activity. The first focus group, described in Table 6, column 1, yielded responses similar to those of the GTDB group. After the second focus group, summarized in Table 6, column 2, DBGT students added that computers were used more than they had previously thought, and one group added that, through the use of computers and the database, they were able to do more high-level research than might otherwise have been possible. Another group discussed the variety of methods used by ­scientists to answer similar questions. They also introduced technological science and psychology for the first time in the discussions.

When asked what influenced their images of scientists, student responses typically reflected their own experiences rather than the immediate influence of either of the two research experiences they conducted for this study. Common responses included their parents or other adults they know, the school lab, and depictions of scientists in movies and on television, including Bill Nye the Science Guy.

Regardless of the order of completion, students provided more insightful descriptions of scientists after they had completed both research experiences compared with only one. They also demonstrated a deeper appreciation for the roles of computers and databases in scientific research after both experiences, independent of the order in which they completed them.

How Did Students Perceive the Similarities and Differences between the Genotyping Lab and What “Real Scientists” Do?

This question was asked of both groups right after they had finished the genotyping activity. Students gave a semantic differential rating on the focus group worksheet and then were asked to explain why they gave that rating. Table 7 presents a summary of responses. These responses came from both first and second focus groups, depending on when students had just completed the genotyping activity. No students rated or explained how genotyping was not like their ideas of what scientists do. Students equated the activities of science with doing something “hands-on.” Genotyping was similar, because they followed a process (like scientists do), took on the role of a scientist, participated in a “real” study, and explored an open-ended question. Scientists would actually do this lab, although on a larger scale. Reasons why genotyping was not as similar included the reduced time students had to solve the problem, that scientists might deal with different subject matter, that genotyping was a recipe lab (not open-ended or driven by student questions), and that it was only a small window into what scientists do. Overall, students indicated that the genotyping lab was mostly similar to real science.

TABLE 7.

Similarities and differences between the genotyping lab and what “real scientists” do: asked after students had completed the genotyping activity (data from both focus groups)a

Why GT is not similar	Why GT is somewhat similar	Why GT is completely similar
(None given)	Scientists, as compared with students, are able to take their time with experiments and investigations.	Just like scientists, students followed a process in the genotyping activity to a desired outcome.
	Scientists deal with different subject matter than what students are dealing with (“rockets vs. micropipettes”).	Scientists would actually do genotyping.
	Genotyping activity was essentially a recipe lab; students were not investigating their own questions.	Students took on the role of scientists, confirming generated data.
	Classroom is a noisier environment than a science lab.	Genotyping activity is part of a real smoking study; students working on something similar to scientists’ actual work.
	Genotyping activity is just a window into the “bigger picture” of what scientists do.	Genotyping was consistent with how the teacher characterized genotyping as a scientific activity throughout the school year in AP Biology.
		There was no confirmational aspect; it felt like testing a question with an unknown answer.
		It seemed like a scaled-down version, in the number of subjects and time allotment of what scientists would do.

aN = 39 for FG1; N = 12 for FG2.

How Did Students Perceive the Similarities and Differences between the Database Lab and What “Real Scientists” Do?

Again, this question was asked just after the database activity, and their responses are summarized in Table 8. Many students saw the database activity as just computer work, not hands-on, as group work (not individual, in the way they imagined scientists would work), and not consistent with their stereotypical views of what scientists look like or do. The responses for why the database activity was somewhat similar to what scientists do reflected a view that it was similar in some respects (had to follow a procedure, used the same equipment) but it was not “stressful” enough, not a complete experiment, and did not allow students to obtain and work with their own data. Surprisingly, students held these views even though they had seen videos in which scientists discussed the highly collaborative and multifaceted aspects of their research, including using computers and databases containing data collected by others. The student comments that the database research was “just computer work” and not “stressful” enough may reflect the design of the database query, in which students set up the odds-ratio calculation by defining exposures for cases and controls, but the database software calculated the odds ratio and 95% confidence interval. This design was intended to focus students on the analysis of their queries rather than the potentially confusing mathematical aspects, but it may have had the unintended consequence of oversimplifying the process to the extent that students did not equate what they were doing as being real science. Responses in column 3 of Table 8, “Why DB Is Completely Similar,” were provided entirely by students in the GTDB group after they had completed both experiences. Many of these students were able to provide reasons for why the database activity was completely similar to what scientists do. Students suggested that it was real science, but a scaled-down piece of it with fewer variables and not as complex. It was completely similar because it involved trying to prove a hypothesis using research and statistics, and it consisted of looking for effects of interactions on human characteristics, like psychology research.

TABLE 8.

Similarities and differences between the database lab and what “real scientists” do: asked after students had completed the database activity (data from both focus groups)a

Why DB is not similar	Why DB is somewhat similar	Why DB is completely similarb
It is just computer-related work, and not a hands-on lab using materials and solutions.	Like scientists, students had to follow some procedures, such as going online for some information and trying to find odds ratios; however, they were not working with hands-on materials.	Database activity was really similar to what scientists do, but was just a small piece, meaning fewer variables and not as complex, sample for students.
Scientists work individually, and not in teams like the students did.	Real scientists are busy and stressed in their work, but the database activity was not stressful and somewhat easy; database activity was a research and poster creation exercise, and not a complete experiment, which would have involved collecting data from subjects.	Incorporating research, finding, and then trying to prove, or prove wrong, hypotheses using calculations and analyses of odds ratios is like what real scientists do.
Database activity was not consistent with participant’s self-described “cartoonish” view of scientists (i.e., crazy hair; goggles).	Classroom lab equipment was “pretty similar” to what scientists would use in a real lab.	Work is related to what scientists would do in terms of looking at effects of interactions on human personality and characteristics.
Not consistent with the view of scientists as wearing lab coats, working with DNA, and using high-tech equipment to run lab tests.	Case studies and class research seem similar to what real scientists would do; database activity did not seem similar to what scientists would do.	It was similar to psychology research.
In previous science classes this year, the discussions and work had been about procedures and labs, and this database activity felt qualitatively different.	Database activity was similar in that scientists are viewed as people sitting at desks and working, which is what the students did in the activity; however, …scientists would be allowed to go and take their own data. They wouldn’t be confined to the data presented in the database.

aN = 15 for FG1; N = 40 for FG2.

bResponses entirely from students in FG2.

What Were Student Views on the NOS after Completing Both Research Experiences?

During each focus group discussion, students were asked several questions that probed their understanding of the NOS as time allowed. Their responses are summarized in Table 9. Overall, students indicated that scientific knowledge is reliable but tentative and partly subjective, dependent on the background of the scientists, and that scientific investigation requires creativity. In response to questions about the purpose of science and how scientists achieve that purpose, most discussions focused on using the scientific method or testing to answer questions we do not know, making discoveries, or proving something. Most groups supported the view that the purpose of science is to help society or find cures. Only one student disputed this view.

TABLE 9.

Student reflections on NOS questions asked during focus groupsa

What is science, and what is its essential purpose? (NOS1) (asked in seven focus groups)

• To find answers to things we don’t know. (6)

• To prove something by testing. (4)

• To discover new things. (2)

• To help society/find cures/save lives. (5)

• NOT to save lives. (1)

How do scientists go about achieving this purpose? (NOS7) (asked in seven focus groups)

• Make a theory and hypothesis and go by the (scientific) method to try and figure it out and to have evidence back it up. (3)

• Do research/getting background information. (2)

• How they do their research depends on what question they are trying to answer (e. g. may use computers or chemicals). (2)

• Have to figure out how to explore things—a lot of innovation. (1)

• By asking questions. (1)

After scientific theories are developed do you think they change? (NOS2) (asked in four focus groups)

• Most times it evolves over time. It changes…as they get new information. (4)

To what extent are scientists creative, if at all? (NOS3) (asked in six focus groups)

• They have to be really creative/have an open mind and think outside of the box/figure out what to do. (4)

• They have to “ think of different possibilities that might affect the experiment itself and the results. (1)

• Not really, because “they have to think out of the box, but it’s in a certain area of things… they have to use the tools they have, and they don’t have to think of something new. They’re not artistic creative.” (1)

• “I feel like you have to earn the creativity because you have to learn so much to be able to grasp whatever you’re studying before you can even be creative in what you’re going to look at specifically.” (1)

To what extent are scientists certain about the results of their studies? (NOS2) (asked in four focus groups)

• Some uncertainty because of potential effects of other variables that cannot be controlled/new area of research/possibility of human error/small study size. (4)

• Need for multiple trials/evidence to support claim. (2)

• Need to be open to failure and to learn from failure. (1)

Given some results from a study to what extent would a group of scientists agree on their meaning or interpretation? (NOS4) (asked in five focus groups)

• Even when they trust the results, they might have different theories about why they came about, depending on their perspective or field of study. (5)

• Even if two research groups got similar results, they may have different interpretations because of differences in the factors/circumstances related to the experiment. (1)

• It is important to keep an open mind about interpretation because new things/information can come up. (1)

aNumber in parentheses is number of focus groups in which a similar response was given.

What Common Themes Emerged during Focus Group Discussions?

The following ideas were discussed during several focus groups:

Students valued both types of research because the answer was not known and they were contributing to a real research study.

Many students equated genotyping with “doing science,” while the database portion was “doing research” similar to what they might do in an English or history class.

Students in several groups reflected on the role of databases in science, both in formulating research questions as part of developing a project and as part of data analysis. One student commented on the fact that data analysis is the part of science that is often not depicted in the news or popular media.

Some students considered the database research to be “just an observational study.” However, other students discussed the necessity of conducting an observational study instead of a randomized control study because of the ethical implications of assigning human subjects to a condition such as smoking that is known to be harmful to human health.

Students had different reactions to the database research. In one focus group, there was a consensus reaction that the causes of smoking were already well understood, so the ­students were not discovering anything new. These students were generally not interested in the topic of smoking. In contrast, students in other focus groups indicated genuine interest in and surprise about their database analyses, whether their results showed strong association or were “right in the middle.” One student remarked that “all results tell a story.”

Students discussed their frustration with several aspects of the database, including the small sample size, lack of diversity, and the fact that they could not contribute their own questions to the questionnaire. One group expressed a desire to combine questions in their query (i.e., to be able to calculate the odds of having two variables, X and Y). This feature was deliberately not included in the data analysis, because there would be insufficient power for two questions to be combined given the sample size.

Students expressed some frustration with the genotyping experiment, including lack of understanding of what was happening at each step, the high level of precision needed, not being able to design their own protocol (although several students pointed out that they did not have the background to do so), and not knowing more about the research subject whose DNA they were analyzing, such as whether they were a case or a control in the study.

In several classes, students indicated that they felt rushed when doing each of the research experiences and wished they had more time.

DISCUSSION

This study examined how two kinds of authentic research experiences, a wet lab genotyping experience and a dry lab database experience, influence students’ perceptions and understanding of scientific research. This discussion addresses our initial questions.

To What Extent Do Students Learn Science Concepts as a Result of This Curriculum?

Students showed significant learning gains (p < 0.05) on most of the 10 questions in the content survey (except items 15 and 21) after they had completed the curriculum. Almost 70% of students scored correctly on item 15 before and after completing the curriculum, with no significant increase. This item asked about the role of controls in scientific research, a topic most high school students would have studied throughout their ­science courses, explaining the high score in the pretest. The small gain made pre to post on item 21 was not statistically significant at the p < 0.05 level. In this question, students were asked to explain the hypotheses they had created for the previous question, essentially to provide explanations for the claims they had made. In our own experience, students often have difficulty articulating how data support their claims, which may explain the small gain for this question.

The lowest percentage of correct responses occurred for item 24, in which students were asked to use math to explain their interpretations of some study data (and were expected to use an odds ratio in their responses). It is not surprising that very few students scored correctly on the pretest, because the odds ratio is not normally taught in high school. However, only 20% of students were able to provide the correct response after the curriculum. This may reflect the fact that students were not required to calculate the odds ratios for their own queries—this was done by the database software to avoid calculation errors—although they did calculate an odds ratio in an earlier lesson. For this curriculum, the emphasis was on interpretation of the odds ratio (and the 95% confidence interval) rather than setting up the calculations. If making these calculations is an important part of a teacher’s instructional design, then more emphasis would need to be placed on this aspect.

Nearly 90% of all students reported that they partially or totally understood the content of the curriculum as it pertained to both the database and genotyping content. This suggests that students were confident in their learning gains, which is consistent with the results of the post survey.

Does This Curriculum Influence Student Attitudes Toward Science?

There was no significant difference in student attitudes toward science before and after the curriculum as measured by their responses to the SAI II included in the pre and post surveys. Even on the pretest, students had relatively high scores for each SAI item (the lowest mean score was 60% and the highest was greater than 80%). Within the context of this 2-week intervention, there may have been insufficient time and specific instruction to change students’ broad perceptions of science. On all items, teachers scored higher than students, and the scientists who were informally surveyed had scores very similar to the teachers.

Both teachers and students scored lower on SAI 4 than other position statements. Disaggregation of this statement into ­positive and negative components showed that students and teachers (as well as scientists) tended to agree with the positive statement, “Science is an idea-generating activity. It is devoted to providing explanations of natural phenomena. Its value lies in its theoretical aspects.” However, they also tended to agree with the negative statement, “Science is a technology-developing activity. It is devoted to serving mankind. Its value lies in its practical uses,” which resulted in a lowering of the overall score for this statement. The fact that students, teachers, and scientists all tended to agree with the negative statement may reflect a shift in the way our society views the role of science compared with when this test was originally designed in 1970. For example, we now group science with technology, engineering, and mathematics in the common educational acronym, STEM. Furthermore, there has been a change in how science is conducted in academic settings, which is promoted by federal funding agencies, with more emphasis placed on solving problems related to technical and medical issues. Student responses during focus groups also supported a frequently expressed view that the purpose of science is to help society (see Table 9).

In contrast to the results of the SAI II, analysis of the focus group discussions indicated that students developed more nuanced views of the nature of scientific research after completing both of the lab activities, demonstrated by their reflections on the theory-laden aspects of scientific research and the multifaceted approach that scientists use (column 2 of both Tables 5 and 6). Student responses to specific questions related to the NOS (Table 9) demonstrate a general grasp of the features of the NOS among students who participated in the focus groups. Their level of understanding may in part be attributed to participation in this study as well as other experiences through the science class they were taking. Many aspects of the NOS were integrated into student activities conducted through lessons 1–4 of the Exploring Databases curriculum used in this study, although they were not explicitly presented in the context of the NOS. As discussed by others, students might have better recognized the NOS elements they were conducting had they been versed explicitly in what constitutes the NOS (Lederman and Lederman, 2004; Sadler ).

Do Students Perceive Wet Lab (Genotyping) and Dry Lab (Database Experiences) to be “Authentic Science” to the Same Extent?

While the students rated both research experiences to be like real science, they gave a higher rating to the genotyping experiment than to the database research experience. Furthermore, students were more likely to indicate that genotyping “used the same kinds of ‘tools’ and ‘equipment’ that scientists use.” The perception that the wet lab experience is more like real science may reflect how science is most commonly portrayed in the public media, which typically highlights the more visually appealing data-collection aspects of science. In several focus groups, students distinguished between doing experiments and analyzing data, which they equated with the kind of literature research they would do for an English or history project. However, students associated more scientific inquiry tasks with the database experience than the genotyping activity (Table 4). They also reported a better understanding of what they were doing in the database lab, which may reflect the more structured directions for completing the database research than the genotyping protocol.

In the parlance of Chinn and Malhotra (2002), the genotyping experiment most closely resembles a “simple experiment” in which students are provided the research question, follow a given protocol with variables and control group specified for them, and are told what to measure. The authentic aspect of this experiment is that they do not know what the outcome is for the sample they are analyzing, so they must draw their conclusions based on their own results and then verify their findings by comparing their data with those of other student groups in their class who are analyzing the same samples. In addition, the genotyping data they produced are included in a bona fide research database as part of an ongoing research study. In contrast to genotyping, the database research is most closely aligned to “authentic inquiry” as described by Chinn and ­Malhotra (2002), because students generate their own research questions within the context of what can be asked using the smoking behavior database and select their variables by defining parameters for “exposed” and “not exposed” when calculating their odds ratios. They also present their research to their fellow students and defend or alter their claims in response to critiques from their colleagues.

Focus group results suggest that the participation in both types of scientific inquiry broadens student understanding of the tools, processes, and approaches used in doing science. While students tend to hold fast to their stereotypical visions of scientists and what they do, many students were able to articulate an expanded perspective that science is accomplished in many ways and that a case control study, for example, is another legitimate method of generating scientific knowledge. Moreover, the content of the database—real data about smokers and nonsmokers along with their environmental and genetic exposures—made the database inquiry activity interesting, with the feel of an authentic science activity. The same perspective was shared about the genotyping activity—students were excited to be working with genetic data from real people. Thus, after both research activities, students identified aspects of the NOS embedded within their experiences, including the multifaceted approaches to doing research in a particular field and the significance of doing research in an area that has personal and/or social significance, in this case smoking behavior.

Limitations of the Study

While results of the curriculum implementation are encouraging, there are cautions that should be acknowledged. In any human research project, the assigning of teachers and students to condition groups is challenging. Some teachers were unable to implement the curriculum as initially planned or to provide times for complete data collection. The length of time that teachers spent on the curriculum was determined by the demands and opportunities that faced each teacher. The final distribution and numbers of students in each condition group were not as balanced as we had intended because of incomplete participation from some classrooms. In addition, while we tried to match at the classroom level, it is possible that some results were influenced by unaccounted variables. We addressed some of these issues by collecting a variety of data types in order to triangulate our findings.

CONCLUSIONS

In spite of their own recognition that the database lab incorporates more scientific tasks than the wet lab, some students persisted in maintaining a stereotypical view of science based on the use of tools and equipment to collect data as in the wet lab. This has been reported by others (Driver ). However, after completing both lab experiences, students expressed more sophisticated views of the nature of scientific research, including the theory-laden aspects of science and the multiplicity of approaches used by scientists to conduct their research. This outcome highlights the importance of immersing students in many kinds of research experiences through their precollege education so they can develop a richer understanding of how scientists conduct their research compared with the very procedural scientific method traditionally taught in science textbooks (Windschitl ).

Through the two science education programs described in this paper, we developed research experiences that could be integrated into a wide variety of high school courses, including introductory biology, so that students at all levels could be involved. The two research experiences included in this study each present advantages and disadvantages for integration into a science class. The genotyping experience enabled students to use some of the classical tools and techniques of molecular biology to generate authentic data for the ongoing research study. For this application, it was critical that students follow a standardized protocol to ensure data quality, and the experience did not have the flexibility for students to pose research questions or design their experiments. However, the opportunity to use those tools and techniques and contribute data to the study was clearly an experience students valued. In contrast, the database experience allowed students to pose and test their own research questions, albeit in the context of a larger research study that they had not designed using data that they had not collected, and again, they placed a high value on the experience. In both cases, students could take ownership of the research they conducted because of their unique contributions to the study, an important aspect of developing interest in science, as discussed by others (Polman and Pea, 2001; Hanauer and Dolan, 2014).

The proliferation of publicly accessible scientific databases and data-analysis tools presents exciting opportunities to engage students in authentic scientific research using pre-existing data, similar to what many research scientists do (Hug and McNeill, 2008). As teachers strive to engage their students in meaningful activities that reflect scientific practice as recommended in the NGSS (NGSS Lead States 2013), the use of databases is a viable option. To increase students’ recognition of database research as a scientific practice, we argue that the science education community will need to address the following questions:

How can we foster students’ understanding that manipulation and analysis of data collected by others is a valid research activity conducted by practicing scientists?

How can we help teachers develop the necessary conceptual understanding to guide students in recognizing the validity of different approaches in science?

We have attempted to address these issues through a curriculum that exposes students to the many practices of scientists and immerses them in activities that foster engagement in science practices, including developing their own theoretical and conceptual framework for conducting research on smoking behavior and designing research questions that they can test using the smoking behavior database. To broaden their teachers’ understanding of science practices, teacher professional development workshops provided opportunities for teachers to engage in discussions of scientific research with scientists and science educators. The understanding of science practices in the context of discipline-specific content and cross-cutting concepts by all students is an important goal of the NGSS that should guide future curriculum development and classroom practice. On the basis of our results, we recommend that teachers and students have opportunities to engage in science practices that reflect the unique approaches taken in different fields and to participate in deliberate discussions of those approaches so they develop a more nuanced view of the practices and nature of science.

ACCESSING MATERIALS

The current Exploring Databases teacher professional development website, curriculum, PowerPoint presentation, and the smoking behavior database can be downloaded from https://gsoutreach.gs.washington.edu/instructional-materials/exploring-databases.

The curriculum from the foundational project, Investigating the Effects of Genes and Environment on Smoking Behavior, from which the genotyping protocol is taken, as well as a related ethics unit and all PowerPoint presentations, can be downloaded from https://gsoutreach.gs.washington.edu/instructional-materials/investigating-smoking-behavior.