Developing the inner scientist: book club participation and the nature of science.

The leap from science student to scientist involves recognizing that science is a tentative, evolving body of knowledge that is socially constructed and culturally influenced; this is known as The Nature of Science (NOS). The aim of this study was to document NOS growth in first-year premedical students who participated in a science book club as a curricular option. The club read three acclaimed nonfiction works that connect biology to medicine via the history of scientific ideas. Students' NOS status was assessed as informed, transitional, or naïve at the beginning and end of the academic year using the Views of Nature of Science Questionnaire-Form C (VNOS-C). Focus group interviews and document analysis of assignments and exams provided qualitative evidence. VNOS-C scores improved over the academic year regardless of book club participation. Students who participated in book club had marginally better NOS status at the end of the year but also at the beginning, suggesting that book club may have attracted rather than produced students with higher NOS status. It is notable that an improvement in NOS understanding could be detected at all, as there have been few reports of NOS growth in the literature in which NOS was not an explicit topic of instruction.

INTRODUCTION

Physicist Richard Feynman is credited with having said that philosophy of science is about as useful to scientists as ornithology is to birds. However, the study of the scientific enterprise by outsider philosophers and sociologists of science occasionally yields works of great influence, even to scientists themselves (Kuhn, 1970). Indeed, understanding how the enterprise of science operates is considered so important by practicing scientists that The Nature of Science (NOS) has become a significant feature of science education reform (National Academy of Sciences, 2005). An understanding of how knowledge is justified and how that knowledge grows is the first of 11 overarching principles guiding new recommendations for science courses for future physicians (American Association of Medical Colleges and Howard Hughes Medical Institute, 2009).

Epistemology is a branch of philosophy that concerns itself with what qualifies as knowledge in a discipline and how that knowledge comes into being. For science, epistemology considers the process by which scientists generate, evaluate, and negotiate the meaning of observations of natural phenomena from biochemical structures to astronomical events. These are creative processes that occur through the lens of existing paradigms in the field and the scientists’ experiences, values, and beliefs. These aspects of science have come to be known in science education as the nature of science (Lederman, 2007). Although the definition of NOS varies, there is wide agreement that scientific knowledge is durable but subject to change. Furthermore, scientific knowledge is empirically based but subjective, due to interpretation being influenced by current models, or theory-laden. As a human enterprise, it necessarily involves inference and creativity and is socially and culturally embedded. These are aspects of NOS that science students should come to understand implicitly or explicitly as a result of their education. The aim of this study was to document NOS growth in first-year premedical students who participated in a science book club (BC) as a course option in biology.

During a scientist's education from undergraduate to researcher, he or she comes to understand NOS as a result of immersion in the discipline, first in classrooms and teaching labs and later in authentic research. It is not clear which among a student's vast set of experiences contribute most to NOS growth or how they interact. Without explicit attention to NOS during instruction, inquiry activities on their own do not lead to more informed NOS views (Khishfe and Abd-El-Khalick, 2002). Even authentic research experiences do not reliably improve NOS understanding (Bell ; Morrison ), and practicing scientists themselves have diverse views that are not always consistent with target NOS understanding (Schwartz and Lederman, 2008; Wong and Hodson, 2009). Surprisingly, even courses in the history and philosophy of science do not reliably improve NOS status (Abd-El-Khalick, 2005), and upper-level courses for science majors do not improve NOS status any more than do general education science courses (Miller ). It seems that explicit instruction is required (Sandoval and Morrison, 2003).

Science educators have advocated using stories from the history of science to convey the numerous cultural, personal, and technical variables that come to bear on the development of an idea (Irwin, 2000; Tao, 2003; Clough, 2011). A source of such stories appropriate for undergraduate students is science trade books commercially produced for the general public. Three years ago, P.B.G. established an optional BC for students enrolled in our medical college's premedical biology courses. Each Spring, one-third to one-half of the class has chosen BC as their weekly homework option. The assigned books, Genome (Ridley, 1999), Why We Get Sick (Nesse and Williams, 1995), and Good Germs, Bad Germs (Sachs, 2008), were selected because they relate the major topics of the course—genetics, evolution, and ecology—to human biology and medicine. These narrative expository works can be considered lessons in the history of science, because they document the development of thought and paradigms with respect to genes, illness, and microbes, respectively. This genre is distinct from other science nonfiction genres, such as reference books, biographies, field guides, and how-tos. Although there has been recent attention to the value of adapted primary literature (Ford, 2009), we were unable to identify studies that evaluate the use of popular science nonfiction in undergraduate science.

The BC originally was established to offer an additional enrichment activity to broaden premedical students’ interest in biology and connect biology to medicine. At the outset, there was no goal of stimulating or measuring NOS growth. However, anecdotal evidence in the first year of BC suggested a more sophisticated understanding of NOS than the author expected of freshmen. This led us to ask the following research questions:

Does NOS understanding among our premedical students indeed improve over one academic year?

If so, does participation in the BC contribute to this improvement?

To address these questions, we conducted surveys and interviews and collected documents the following academic year. The Views of Nature of Science–Form C (VNOS-C) questionnaire is a validated conceptual diagnostic test (Lederman ) that was administered to all students at the start and end of the academic year. Postcourse focus group interviews were conducted with students who had and had not been in BC, using the VNOS-C questions as probes. Learning artifacts, such as BC homework and optional exam questions, also were evaluated. The improvement in NOS status for both groups was significant, indicating that the sum of their experiences in their first year of our program contribute significantly to their NOS growth. This claim was triangulated by qualitative evidence from interviews and course documents. BC participants had marginally higher VNOS-C scores than non-BC students at the end of the year; however, these scores were also slightly higher before BC began, suggesting BC may have attracted rather than produced students with slightly higher NOS status. Thus, the effect of BC could not be isolated as a cause of NOS growth against a backdrop of the entire set of first-year experiences in our program.

METHODS

Instructional Context

Students participating in this study were in their first year of an intensive 2-yr premedical program at Weill Cornell Medical College in Qatar. Admission is selective (average Math SAT: 682) and heavily dependent on the student's motivation toward medicine. Most students speak English as a second language but received secondary education in English medium schools that use diverse curricula, including the Advanced Placement (AP; United States), International Baccalaureate Diploma Programme (IB), A-levels (United Kingdom), and national curricula from Qatar and other countries in the region. Few students came from educational systems that emphasize inquiry or hands-on investigations. P.B.G. was the sole instructor of both of the required first-year biology lecture courses. There was attention to current biology research and occasionally to aspects of NOS throughout the course, but no lessons were designed to address NOS explicitly.

In addition to three lectures per week, students are required to attend a review session. Two different review sessions were offered, and students were free to choose the one they preferred. One was a traditional review with problem solving and extensions; the other option was a book/journal club. Both options were introduced in the first class of the semester. Students wishing to join BC could then reserve loaner copies of the books. Some said they joined to try something new or to be sure they were reading more than just textbooks in their science-heavy curriculum. Peer influence seems to have played some role in participation, as groups of friends often participated together. Some who did not participate said that they felt the review session would help them in the course more or perceived the readings to be the more onerous assignment. Other than the associated homework assignments (see Document Analysis), BC and non-BC students attended a common lecture and were assessed by common exams. BC was neither established nor modified for the purpose of this study.

The Book Club

For each book, the BC met for 1 h/wk over 3–4 wk. The books were chosen because they were favorably reviewed works that directly relate biology to medicine. P.B.G. was a participant observer, and T.M. and S.K. were observers. The BC sessions were videotaped but not further considered for this study. Students provided informed consent prior to participating and being videotaped.

The BC first read Why We Get Sick: The New Science of Darwinian Medicine (Nesse and Williams, 1995), in which the authors argue that it is essential that physicians consider illness and disease from an evolutionary perspective, and that evolutionary medicine should become an important part of their medical education. Beginning with fever and inflammation, the authors take a fresh look at the adaptive significance of pathogens, injury, blood type, toxins, allergy, and even mental illness. At the time we read this book, the class had not yet studied the mechanisms of natural selection and speciation; however, evolution had been introduced as an overarching theme in the first semester and revisited in nearly every topic studied to that point.

At the same time we were studying genetics, the BC read Genome (Ridley, 1999), subtitled Autobiography of a Species in 23 Chapters. This national best-seller was written when the human genome was first mapped. As the cover summarizes, “by picking one newly discovered gene from each pair of chromosomes and telling its story, Matt Ridley recounts the history of our species and its ancestors from the dawn of life to the brink of future medicine.” Beginning with the 5S rRNA gene on chromosome 1, Ridley chose a gene purposefully from each the remaining 22 chromosomes to ponder history, fate, self-interest, and disease via stories as old as the prebiotic soup and as new as epigenetics.

By the time we studied ecology, we were reading Good Germs, Bad Germs: Health and Survival in a Bacterial World (Sachs, 2008), which the cover explains “explores our emerging understanding of the symbiotic relationship between the human body and its resident microbes” in its recounting of the history of sanitation, antibiotics and resistance, the human microbiome, and the emerging hygiene hypothesis.

VNOS-C Instrument

Researchers use a variety of methods to assess the NOS status of a learner, including closed- and open-ended survey instruments and interviews. The VNOS-C is the open-ended instrument used in this study (Supplemental Material; Lederman ). The VNOS-C was administered during scheduled class time as a pretest to the cohort of entering students (n = 62) in the first week of the academic year (precourse) and again near the end of the academic year (postcourse: n = 58). No incentives were offered, and providing names was optional. Approximately 40 min were allowed for completing the survey. All students selected for the focus group interviews completed the postcourse VNOS-C at the start of the interview. The remaining students completed the VNOS-C at the end of the course within 2 wk of those completed by the focus groups. Information about BC membership and date of administration was masked in a code and the pre- and post-BC and non-BC questionnaires were thoroughly mixed to avoid bias during scoring. To maximize scoring reliability and reduce bias, a single evaluator scored the randomized, blinded questionnaires as a single set.

The VNOS-C was designed to be used with a follow-up interview to assign NOS status holistically. No scoring rubric has been published for the VNOS-C; therefore, one was devised for this study by adapting a published rubric used for the open-ended questions of a newer, stand-alone questionnaire, the Student Understanding of Science and Scientific Inquiry (SUSSI; Liang ), which was not yet available at the start of this study. The SUSSI survey questions correspond one-to-one with stated NOS aspects, but the VNOS-C does not. However, neither instrument explicitly assesses all aspects of NOS understanding. For example, there is not a VNOS-C item that explicitly prompts the respondent to distinguish observation from inference; thus, it would be incorrect to assume that a student does not understand the inferential aspect of NOS if his or her response does not address it. Therefore, only the NOS aspects that were directly prompted by the VNOS-C questions were scored, namely empirical NOS, tentative NOS, theories and laws, social/cultural NOS, and creative NOS. The scoring criteria for each level and examples from student responses are provided in Table 1 and explained further in the Results and Discussion.

Table 1.

Rubric for scoring VNOS-C, including illustrative examples from written student responsesa

NOS aspect	NOS status	Target understanding	Illustrative examples from student VNOS-C responses
Empirical NOS		Naïve	Science is based only on objective observations that lead to discovery of facts. This can only be arrived at by experimentation.	Science is a way of displaying facts and analyzing them, in contrast to religion and philosophy
Transitional	Science knowledge is based on observations of natural phenomena directly or indirectly. Ideas are tested with experiments or investigations.	Science … relies heavily on evidence. … experiment is a means to prove, justify or simply observe a physical, biological or chemical phenomenon.
Informed	Science deals with theories, which are explanations of observations in nature. Observations can occur using experimental or naturalistic approaches.	Two scientists may have opposing ideas on how something works. If science did not require the validation of these ideas then any … journals written on the subject matter are not so much advances … as they are personal perspectives.
Tentative NOS		Naïve	Theories do not change unless the original experiment was wrong	They are extremely certain [about textbooks]. They use things like spectrophotometers and the properties of elements.
Transitional	Theories can change, for example when new technology becomes available	Science is never complete. Every year thousands of discoveries are made in science and a lot of these updates are crucial to understand the world we live in.
Informed	Theories get updated or modified as new information accumulates; they also can change when existing evidence is reinterpreted	Scientists don't intend to falsify a theory, but if their findings oppose a certain theory it would have to change. If such a finding was discovered it usually drives the scientist to look at the theory in a different aspect.
Nature of theories and laws		Naïve	Laws are more certain than theories or theories become laws when proven	A law was supported a longer time than a theory.
Transitional	Theories are explanations; laws are relationships	Scientific theory is mainly a generalization with some defects that might occur, e.g., theory of evolution. Scientific law is something concrete that is evident and not possible to argue on, e.g., law of gravity.
Informed	Theories are explanations (hows and whys) of observed phenomena, whereas laws are statements of relationships between them (usually mathematical) observed directly	Laws are accepted as facts and can often be expressed in terms of mathematical equations … A theory is more complex and describes a group of related phenomena
Social/cultural NOS		Naïve	Science is a search for universal truth and fact that is not affected by culture or society	Science is the same everywhere in the world, while religion or philosophies of individuals differs from one individual to the next.
Transitional	Society or culture determines what or how science is conducted or accepted.	Science is universal. No matter where you come from, a theory will still be a theory. It's just the usage and interpretation of these theories varies depending on your background and social norms of your own culture.
Informed	Society or culture affect how a scientist interprets his or her data, including existing paradigms.	Science is universal—its explanations can be used to explain phenomena across cultures and countries. Science, however, may be slightly affected by these values because they may affect how a scientist thinks, because people are affected by their environment.… many important discoveries were reached through detaching the mind from the boundaries of following a fixed method.
Creative NOS		Naïve	Scientists do not use creativity because it affects their objectivity	No, they can't use imagination.
Transitional	Scientists use creativity in designing experiments but not in interpreting data, or vice versa	Scientists start their researches using their imagination … but this creativity stops whenever the idea is proved wrong, illogical or inapplicable.
Informed	Scientists use creativity throughout their investigations, including interpretation.	Yes, scientists do use creativity and imagination in experiments, I think they have to be creative in all the stages mentioned above. … At every step he/she must think outside the box to get somewhere!

aModified from Liang .

Each questionnaire was scored for the five aspects of NOS noted above by assigning a value of 1, 2, or 3 for a naïve, transitional, or informed view, respectively, of each NOS aspect using the criteria in Table 1. Overall NOS status was the average of these five subscores. Unclassified or blank responses were not scored, and NOS status for these questionnaires was based only on the number of questions receiving a score. Questionnaires with two or more unscored items out of nine were excluded. Evaluations were low-inference: no additional meaning was ascribed to a student's understanding beyond the words on the page. The criteria used to assign these ordinal variables, although consistent with similar studies, are still subjective. These place limitations on claims based on these quantitative data. As will be evident in the results, the qualitative data suggest that taking additional measures to improve the precision of each score would not likely have changed the overall findings of the study.

For the statistical analysis of the VNOS-C scores, variables were summarized using means and SDs. The averages used to represent NOS status are based on ordinal variables operationalized using subjective but consistent criteria. Doing so assumes the NOS scores fall on a linear continuum in a normal distribution. Because questionnaires could be completed anonymously, pre- and postcourse scores could not be matched for each student completing the VNOS-C. Therefore, the difference between the means for the pre– and post–VNOS-C scores for the total sample was assessed using the independent t-test. Subscores for individual aspects of NOS were not subjected to significance testing. Postcourse score comparisons between all students in the BC and those who did not participate in the BC were made using the independent t-test. A subset of the pre- and postcourse questionnaires that could be matched by names were analyzed separately using the Wilcoxon ranked-sum test (n = 16). The matched scores available for students later invited for focus group interviews were subjected to analysis of covariance (ANCOVA). A p value of 0.05 or less was declared significant. All analyses were done using IBM-SPSS (Armonk, NY).

Written questionnaire responses were also used as qualitative data for comparing BC and non-BC students with respect to aspects of NOS. Illustrative examples transcribed here from written documents are labeled “BC” and “non-BC.”

Focus Group Interviews

Near the end of the second semester, two semistructured focus group interviews were conducted, one with eight BC members and the other with eight students who had not participated in BC (non-BC). The class cohort was sampled purposively to compose the focus groups. First, only students for whom pre- and postcourse questionnaires could be matched were considered. Among these, two groups of eight were selected that were balanced for gender ratio and for a range of biology grades to that point in the course. The latter measure was taken to validate that any improvements in NOS understanding could be attributed to BC, not simply academic ability. In the end, the average course grade of non-BC students chosen for the focus group was marginally but not significantly higher than BC focus group students (3.46 vs. 3.17 on 4.0-point scale, p = 0.13).

The selected students were invited personally by individual email, and all who were invited attended. No incentives were offered for participation. The role of BC participation in the focus group composition was not revealed, and the interviewers made no mention of BC during the interviews. The students were therefore unaware that the research interviews would be probing the role of BC participation. Interview questions were paraphrased from the VNOS-C. A 5-s wait time was observed before moving to a new question. Additional questions were posed as needed to redirect the conversation or to explore ambiguities. Focus group interviews were videotaped (the segment of the BC focus group about creative NOS was not recorded). Video was imported, transcribed, and coded using NVivo 8 software (QSR International, Doncaster, Australia). Transcripts were initially parsed and coded by NOS aspect: empirical, inferential, tentative, creative, social/cultural, theory-laden, myth of scientific method, nature of theories, and laws versus theories. Codes within these categories were not determined a priori, but familiarity with reported student ideas in the literature informed recognition of categories within each aspect. For example, “technology” was the label for one category within “tentative,” as an explanation for why theories change.

Verbal data from interchanges between the Investigator (I) and Students (S1, S2, etc.) are labeled as such. Isolated comments that are not part of a sequential interchange are bulleted.

Document Analysis

To identify any additional evidence of epistemological growth, we included homework assignments and exam essay questions about BC readings in the document analysis. No comparable assignments were completed by non-BC students. The weekly homework option for BC participants was to complete a reflective journal entry about that week's readings (Supplemental Material). In the analysis, particular attention was paid to sections in which students identified “at least three new insights you had while reading this section and at least three passages or ideas that are still confusing to you or new questions that emerged as a result of this reading.” On each exam, students answered one of three essay questions offered, of which one was about the BC, for example: “Using three relevant examples presented in Ridley's Genome, discuss several insights you had about the human genome or insights about science in general, and relate them to your understanding of genetics prior to reading the book.” Approximately 50 submitted homework assignments and 11 exam essay responses were analyzed for evidence of understanding of NOS using criteria in the rubric (Table 1). Neither task was designed to assess NOS status, but comments that were indicative of NOS status were noted.

Measures to Maximize Reliability, Validity, and Trustworthiness

All reasonable post facto measures were taken to ensure validity and reliability and minimize bias from the authors' role in the BC and course. Prior to our establishing the scoring rubric, an external evaluator involved in the VNOS-C development qualitatively assessed a sample of the questionnaires independently. The evaluator made similar assignments of naïve, partially informed, and informed as we did, which helped us refine criteria and thus improve validity of our adapted rubric. Masking, randomization, and scoring by a single evaluator reduced the likelihood of bias. Although these measures improve reliability to allow comparisons within this population, scores generated here cannot be compared with other groups or by other evaluators, because our modified rubric was not further validated with other samples or populations.

Human Subjects

This project was granted exemption from oversight by the institutional review board of our medical college. Students gave informed consent prior to BC, VNOS-C completion, and focus group interviews. Non-BC students were given the opportunity to exclude their homework and exams from consideration for research purposes, but none did so.

RESULTS AND DISCUSSION

Questionnaires, interviews, and documents provided quantitative and qualitative evidence of NOS status for BC and non-BC students at the beginning and end of the academic year. Open-ended VNOS-C questionnaire responses were scored to assess the effect of BC participation on NOS status. These scores were compared several ways. First, comparison of precourse and postcourse NOS scores showed a significant improvement for both BC (p = 0.01, n = 22) and non-BC students (p < 0.001, n = 34) from the beginning to the end of the academic year (Table 2 and Figure 1). Therefore, our first claim is that NOS understanding improved for the class as a whole during their first year of our premedical program.

Table 2.

Pre- and postcourse VNOS-C scores of BC and non-BC students as compared using independent t tests

	Pre	Post	p Value
BC	1.68 (n = 22)	2.00 (n = 24)	0.01*
Non-BC	1.55 (n = 40)	1.89 (n = 34)	< 0.001*
p Value	0.11	0.32

*Denotes differences that were significant at p < 0.05.

Figure 1.

Pre- and postcourse VNOS-C scores of all BC and non-BC students.

Because providing names was optional, precourse and postcourse questionnaires could not be matched for every student, and it therefore cannot be asserted that each and every student's NOS status improved. We were, however, able to match pre- and post-questionnaires completed by students who eventually participated in BC. When this subset was analyzed, NOS status of future BC students was marginally but not significantly higher than non-BC students before BC began (1.68 vs. 1.55, p = 0.11), and their improvement from pre- to postcourse was not significantly different from the improvement shown by non-BC students (0.29 vs. 0.40, p = 0.32; Table 2 and Figure 1). The same result was observed for participants in the focus group interviews for whom pre– and post–VNOS-C scores could be matched (n = 16 students, 32 questionnaires; 2.28 BC vs. 2.01 non-BC, p = 0.14; Table 3 and Figure 2). A statistically significant difference may have been detectable in the whole class sample using ANCOVA; however, because this requires matched pre- and postcourse scores, we could only subject the smaller focus group samples to ANCOVA. After adjusting for their pretest scores, the BC and non-BC groups were not significantly different (p = 0.191).

In every comparison, the VNOS-C scores of BC students were marginally higher than scores of non-BC students. To rule out the possibility that this could be attributed to general academic ability or biology knowledge, we compared final grades in the biology course. BC students’ final grades were marginally but not significantly higher than the class as a whole (3.36 for BC vs. 2.97 for the class on a 4.0-point scale, p = 0.065), but among the 16 focus group participants, the BC students had a slightly lower final course grade than the non-BC focus group participants (3.17 for BC vs. 3.46 for non-BC on a 4.0-point scale, p = 0.13). Therefore any differences between BC and non-BC groups are not attributable to differences in academic ability or biology content knowledge. Because BC students’ NOS status was slightly higher even before BC began, our second claim is that BC attracts students with higher NOS status.

Postcourse questionnaires completed as part of the focus group interviews had higher NOS scores than those collected outside the focus groups. This was true of both BC and non-BC students, and may reflect a more authentic engagement with the task by interviewees than by students who would not be asked to discuss their responses afterward.

In summary, we claim that NOS understanding improved for the class as a whole during their first year of our premedical program, and that BC may attract rather than produce students with higher NOS status.

NOS Aspects

The NOS scores analyzed in the previous section are averages of scores assigned for each of five different NOS aspects: empirical NOS, tentative NOS, theories and laws, social/cultural NOS, and creative NOS. In this section, we look more closely at each NOS aspect using data from the questionnaires, focus groups, and learning artifacts from homework and exams. Note that the means on individual aspects of NOS were not subjected to significance testing. Therefore, the assertions in this section are based on triangulations of scores with verbal data.

On the VNOS-C questionnaire, BC students scored higher than non-BC students with respect to understanding of empirical NOS, tentative NOS, social/cultural NOS, and creative NOS (Figure 3). BC students as a group scored higher than non-BC students on every aspect of NOS except one, the distinction between theories and laws in the precourse questionnaire.

Figure 3.

Pre- and postcourse status of BC and non-BC students with respect to five aspects of NOS.

Pre- and postcourse VNOS-C scores of eight BC and eight non-BC students who participated in focus group interviews.

Focus group interviews generated verbal data that made it possible to explore nuances of NOS understanding that were not evident in the questionnaire responses. Although the VNOS-C differences between BC and non-BC were not statistically significant, the conversations in the focus groups were starkly different. The level of questioning and debate and the sophistication of examples were higher in the BC focus group; quotes from the focus group transcripts illustrate the level of understanding of each aspect of NOS (discussed in relevant sections below).

The BC homework and exam essay options were analyzed for evidence of more informed views of NOS among BC students, particularly the tentative aspect of NOS (Table 4). Neither of these data sources directly prompted consideration of the above aspects of NOS; therefore, these data from BC students are taken as evidence of informed views that are implicit rather than explicit. The data provide credible evidence that BC prompts consideration of NOS when completing tasks requiring reflection on the readings. These data further support our claim that BC participation prompts consideration of NOS. Because non-BC students do not complete the BC homework and exam options, we cannot say whether BC students give more consideration to NOS than non-BC students do.

Table 4.

Evidence of informed NOS views from document analysis of BC students

Students’ written responses	Source	NOS aspect: Significance
[T]he notion of the genome as a battlefield between two powers: parental and childhood genes, has greatly affected the philosophical foundations of biology [regarding genetic imprinting].	Insight in homework about Genome	Nature of theories: how knowledge accretes to generate new big ideas Creative NOS: value of metaphor in spreading an idea
Ridley also discussed the history of eugenics. Prior to reading this book, not only did I have no knowledge of many 20th century European [practices], I also had a very limited perspective pertaining to it … Ridley showed me the errors of my ways. Eugenicism can take a more integrated and disguised form in society than one would think. In the early 1900s many Europeans were prevented from the right to reproduce, based on genetic prejudice. According to some, their bad genes were plaguing society … [Same student] Although Ridley did not so much as take a position based on irrefutable evidence, I realized that maybe he couldn’t. That maybe science is limited and people have different ways of interpreting the world so maybe they use different branches of science as well.	Exam response regarding Genome	Social/cultural NOS Theory-laden NOS
There are so many aspects of science in which it is sometimes difficult to differentiate between cause and effect.	Insight in homework about Genome	Inferential NOS
Most importantly, the book taught me that after all it doesn't matter if we believe that evolution is true or not, but we must acknowledge the fact that many aspects of sickness and medicine make much more sense in the light of evolution.	Exam response, regarding Why We Get Sick	Empirical NOS Inferential NOS Social/cultural NOS
I like Thomas Huxley's quote about how error is better than being vague, in the chapter about allergies. It is better to speculate than to completely ignore something and I think it is an interesting and productive way of thinking. [Elsewhere in same homework] Do people with allergies have an immunological advantage? Does this mean that there is a misconception that allergies are a sign of a weak immune system?	Insight in homework about Why We Get Sick	Tentative NOS: hypotheses, even naïve ones, have value Tentative NOS: recognizing a medical idea in transition
The book's views on the appendix ignore the new positive functions of the appendix … maybe its views on the so-called imperfect evolution of human eyes and breathing apparatus is also flawed and will be rewritten in the future.	Insight in homework about Why We Get Sick	Tentative NOS: spotting an outdated view and wondering which current views are subject to change
Maybe the use of hand sanitizers in our daily lives is not such a good idea. After all we are robbing skin of a vat of bacteria that could be performing some unknown function we are unaware of?	Insight in homework about Good Germs, Bad Germs	Tentative NOS: Even health practices can change as theories change
“All truth passes through three stages: First, it is ridiculed; Second, it is violently opposed; Third, it is accepted as self-evident.” Arthur Schopenhauer	Cited by several students as a favorite quote opening a chapter of Genome	Tentative NOS Social/cultural NOS

The remainder of this section compares BC and non-BC students with respect to each aspect of NOS using both questionnaire and interview data.

Empirical NOS.

Most students understood that science differs from other disciplines because it is based on evidence, but few were scored as informed on this aspect of NOS in the questionnaire because many said that experiments are necessary in science. These responses were scored as transitional. Very few went beyond the question before them to state in the questionnaire that qualitative observations are as valid in science as quantitative data generated by experiments, which would have been scored as informed. Comparing BC and non-BC students with respect to this NOS aspect, BC students had more informed (Figure 3, green) and fewer naïve (Figure 3, blue) scores at the beginning and end of the year. In the focus groups, however, both BC and non-BC students stated that nonexperimental data are valid evidence in science. The validity of qualitative and “discovery” science had been addressed in the first week of the academic year both in class and in the textbook.

BC

S1: It doesn't have to be specifically manipulating variables, for example, when the circulatory system was discovered by William Harvey, he experimented, right? [C]ut people's arms open, he was experimenting. He didn't know what will [sic] happen. I guess that's what we call an experiment.

Non-BC

S1: But an experiment can also be an observation, it doesn't necessarily have to be a controlled environment.

S2: An experiment might use observable data so we should go to nature and see that, or to see the cell in the microscope.

S3: Yeah like in the butterfly [probably referring to industrial melanism in moths], was it the monarch butterflies? The one with the trees. That wasn't experiments. That was from observations. [He looked] at different environments, he didn't change anything.

Interestingly, students in the BC focus group mentioned “natural experiment” twice as another valid source of data for testing a hypothesis.

BC

I: Is a study considered an experiment?

S1: No, in an experiment you have to change something.

S2: Yeah, you have to manipulate.

S3: Manipulate or work off of already [collected data], how nature manipulates the variable, then you will just [use] opportunistic sampling or something.

BC

I: Are all studies experiments?

S1: No.

S2: No, if you’re not changing anything.

S3: No because in an experiment you control a variable and change something.

S4: Not necessarily.

S5: Like the natural experiment in lecture, they didn't do anything, they just observed [referring to the Grants’ Galapagos Finch study discussed in lecture that week].

Questions that arose during the BC focus group suggest that students may be failing to distinguish “experiment” and “investigation” in question 2 of the VNOS-C. Therefore, the VNOS-C scores may underestimate the students’ understanding for the empirical aspect of NOS.

BC

I was just wondering, when you make observations, like when you’re seeing stuff, is it an experiment?

Is a study considered an experiment?

So studies and experiments are the same thing.

In summary, with respect to empirical NOS, both groups recognized ambiguities in their concepts of “experiment” but acknowledged the value of qualitative data in science. In addition, the BC focus group cited that natural experiments also provide valid data that can be used to test a hypothesis, even though humans did not manipulate the variables directly. On the basis of the VNOS-C responses (Figure 3) and focus groups, we assert that BC students had a slightly better understanding of empirical NOS than did non-BC students.

Tentative NOS.

Nearly all students responded on the VNOS-C that theories do change over time. Responses were scored as naïve if they stated that theories only change when proven wrong. A response was also scored as naïve if the student said in question 2 that theories change but contradicted that in questions 5 and 6 by saying that scientists are very certain of information in textbooks. Most who said that theories do not change also said that a record or memory of the theory would not cease to exist even after a better theory became favored. This interpretation was scored as transitional rather than naïve. Responses that cited technology as the main reason for changes to theories were scored as transitional. The few responses that specifically mentioned reviewing old data in light of new theories were scored as informed. For example:

Scientists don't intend to falsify a theory, but if their findings oppose a certain theory it would have to change. If such a finding was discovered it usually drives the scientist to look at the theory in a different aspect …

Comparing BC and non-BC students with respect to the tentative aspect of NOS, BC students had more informed (Figure 3, green) and fewer naïve (Figure 3, blue) scores at the beginning and end of the year. In both the BC and non-BC focus groups, the verbal response to the question “Do theories change over time?” was quick and unanimous. However, in the BC focus group, students had a more sophisticated understanding of what it means to revise a theory because it is wrong.

BC

S: I think when something is revised it's not wrong. Even when publishing, the scientific community accepts it and even if it turns out to be wrong I don't think they label it wrong. They label it, like he said, the best they had at the time. They wouldn't say it's like completely wrong.

Non-BC

S: But you have some theory that some accept it. You can have it for some time then someone does a new experiment and realize[s] it's wrong.

When asked about whether scientists are certain about information in textbooks, the reply was that they were as certain as can be given the available information. One BC focus group participant cited how the model of the cell membrane has changed from the sandwich model to the fluid mosaic model. Another BC student commented about additions of information to each new edition of their textbook, such as about aquaporin protein channels we had studied in class. None mentioned examples from the BC readings. As in the questionnaire, some students in the focus group held the view that improvements in technology are the reason some theories change. In response to such a view expressed in the BC focus group, a student with a more informed view said:

A lot of times it's not technology. It could be things like culture because a lot of things, like you said in lecture, a lot of the greatest achievements ever were as plain as the noses on our faces, it just took someone to point it out.

Theories and Laws.

The lowest score in the questionnaire for both groups at the beginning and end of the year was the distinction between theories and laws. However, there was not a separate VNOS-C item that asked explicitly for the definition of a theory, nor for students to distinguish theories from laws. This question was left blank more than any others in the VNOS-C. Most correctly described theories as tentative explanations, but very few were able to explicitly state what a scientific law is. The target understanding is that scientific laws are statements that describe predictable relationships between phenomena, but without positing a mechanism for them. Many of these can be expressed as equations, but none is an explanation, as is a theory. Transitional status was assigned to responses mentioning mathematics or relationships or observations. The few responses that were scored as informed were in the BC postcourse group. At the end of the year, the BC VNOS-C scores had improved for this aspect of NOS, but the non-BC scores had not.

An erroneous idea that emerged both in questionnaires and focus groups was that theories that have withstood the test of time become laws. In focus groups, the immediate response was similar for both BC and non-BC students, but this was followed by doubt and debate. Long pauses and a questioning tone suggested they had never been asked to consider the distinction between laws and theories before. In the BC focus group, one student gave evidence of an informed view: “Laws are just observations, and theories are explanations of those observations.” This student was a lone voice among the BC focus group, until another student agreed, saying, “I think laws are observations,” and another followed with, “Laws are actually an attempt to understand nature.” Each comment got closer to the target understanding.

BC

S1: Someone makes a law to make it easier to understand the natural phenomenon going on.

S2: I thought laws are looking at quantitative, I think if we could just plug in numbers and test whether we are right or wrong. And theories are more qualitative, speculations, you make models.

There was less progress in the non-BC focus group. The “theory maturation” orientation was evident in the following exchange.

Non-BC

I: Once a theory becomes very accepted is it still a theory?

S1: No it's a law [responds quickly, but then smiles doubtfully and says] It goes hypothesis to theory to law to … right?

S2: A law is more proven, has more basis behind it, but a theory is just accepted but it may not have solid evidence behind it.

On the basis of the VNOS-C scores and focus group data, we assert that BC students have a better understanding of the distinction between theories and laws than non-BC students.

Some responses mentioned “facts” as a rationale for accepting a law. While this is a correct use of the definition of facts as empirical observations, it was not assumed that the respondent used the word in this way. It was explained in class that theories are current best explanations, but the nature of scientific laws was not discussed in either the course or the BC. Therefore, it was not surprising that the low score on this aspect did not improve much over the year. This is also evidence that studying numerous laws in their first-year chemistry and physics courses did not clarify this distinction. The naïve view has also been found to be widespread among other undergraduate students (Miller ) and scientists (Wong and Hodson, 2009), which begs the question of whether this aspect of NOS is important for being successful in science.

Social/Cultural NOS.

The VNOS-C scores for this NOS aspect were very similar for the BC and non-BC students and indicated that students in both groups were fairly well informed, even at the start of the year. The improvement seen was similar as well, which may be due to the fact that there was attention paid in class to the historical context of ideas. For example, the historical context of Mendel’s, Darwin’s, and Morgan's contributions and the social context of Franklin's role in discovering the structure of DNA were discussed in relevant parts of the syllabus. This may explain why BC could not be isolated as a reason for improvement for this aspect of NOS. Biology lecture also contributed to an understanding of the role of consensus in building models. For example, in response to VNOS-C question 6 about the definition of a species, several students wrote that scientists are very certain about the definition of a species because they invented the definition themselves, suggesting an informed understanding that species is a human construct. Species concepts had been discussed in the previous week's lectures about how biologists’ classification schemes have changed over time. Understanding of social/cultural NOS was also enhanced in other courses, such as the First-Year Writing Seminar, in which epistemology, the nature of objectivity, and theoretical lenses are discussed. In response to VNOS-C question 7 about dinosaurs, one student in the BC focus group recognized a parallel between analysis of fiction literature and the role of personal experience and agenda in scientific data interpretation.

BC

S: Whatever the scientist uses, they have their own interpretations and sometimes they put their own kind of personal agenda. And they’ll look at the data in a way to support what they think happened, and another scientist will interpret in a different way … Science becomes a lot like literature, like it's how you interpret the data. Like we could all read the same book or the same poem and we would all see it in different ways according to what we want to see.

Others recognized the impossibility of complete objectivity.

BC

S: It's not intentional to factor in social or … like intentionally factor those into your model. It just happens. People can't be completely objective ever because you always have the element of their upbringing.

Non-BC

S: How our mind processes information and gives us an outlook, it depends on past experience. It can't just be based on the data. Even if you think it's just based on the data, unconsciously we are influenced by our past experience, which includes social, cultural …

One BC student recognized the theory-laden aspect of science, another feature of social/cultural NOS. This student cited social construction, a concept presumably discussed in the First-Year Writing Seminar, since it had not been discussed in biology lecture. There was only one comment made in either focus group interview that cited an example from BC.

BC

S: I think to some extent we can have social construction in science. In book club (Genome) we did the part where for a long time they thought we had 24 chromosomes, and people kept seeing 24 just because one person said 24, even though more people conducted experiments they kept believing 24. So there was social construction there and they were just following previous scientists …

In interviews and questionnaires, many responses about social and cultural influences were about government policy and business decisions regarding research grants or drug development: “Culture can also affect science, for example you are doing research about drugs, you would study drugs … more related to your country.” The majority of our students have had a strong Muslim upbringing, and many VNOS-C responses cited undue religious influence on cloning, stem cell research, and/or or beliefs in evolution and stated that the objectivity of science is a welcome improvement over this past influence on the practice of science. Such responses were scored as transitional with respect to the social/cultural NOS. Responses that did not connect how these factors influence theories were scored as transitional. Recalling an example from that day's research report in class, one BC focus group student recognized that public naïveté about the meaning of scientific findings exacerbates this influence.

BC

S: I believe that experiments are not [socially or culturally influenced], like once the results get released I think there is pressure, like public pressure. Like what you said this morning, when scientists for example say “this is what the data supports,” and politicians say, “but it's not proven.” I guess that it's kind of like there is this little gap between the way scientists think and the way the public thinks …

In the case of the social/cultural aspect of NOS, the VNOS-C data suggest no difference between BC and non-BC students. This may be due to regular attention paid in class to the sociology of science. However, examples from the focus group transcripts suggest that BC students have a more sophisticated view, perhaps because they have been primed by the BC readings to notice relevant examples they encounter in class.

Creative NOS.

Of the five aspects of NOS assessed on the questionnaires, students seemed most informed about the creative aspect of NOS. Nearly all students in both groups were scored as transitional or informed on this aspect. This was triangulated by verbal data in the non-BC focus group interview.

Non-BC

I: You’re allowed to be creative when you’re interpreting the results?

S: You have to be, that's how your mind works, creativity, you can't just like [turn it off]. It's not an exact process, that's why people have different views of the same things.

We do not know whether this finding differs from other generations or nonmedical disciplines. Our interpretation is that there has been more widespread acceptance of the role of creativity in science in this postpositivist generation of learners than in the past.

Other Aspects of NOS.

Some misconceptions about NOS reported in the literature were not observed in either the VNOS-C or the focus groups. For example, other researchers have observed that the myth of the scientific method is common (Abd-El-Khalick, 2004). However, there was no mention in the focus groups and only one reference in all the questionnaires of a stepwise approach to science from hypothesis to conclusion. Therefore, this myth either was not elicited by the survey or it is not a significant barrier to understanding NOS in this population. Likewise, there were no explicit observations to suggest that students could not distinguish observation from inference (inferential NOS). The scientific method is not in the current Qatar National Curriculum Standards for secondary science, and other curricula are possibly also moving away from this misleading orientation.

A comparison of the questionnaire responses with those cited from upper-level students enrolled in a history of science course (Abd-El-Khalick, 2004) showed similar strengths and weaknesses. There was widespread recognition that science knowledge is based on evidence, unlike other disciplines, but also that science requires experiments. As in the other study, some undergraduates continue to use “prove,” “right,” and “wrong” naïvely when referring to validity of ideas. They also held naïve views that laws are stronger theories. Students in this study likewise cite technology as the reason theories change. However, one area in which our students were more informed was that they were aware that theories are best current explanations, not opinions. On the basis solely of these comparisons, we assert that these premedical students’ NOS understanding is as informed as that of upperclassmen and graduate students interviewed in the Abd-El-Khalick study.

CONCLUSIONS

The research questions that drove this study were:

Does NOS understanding among our premedical students improve over one academic year?

If so, does participation in the BC contribute to this improvement?

Scores on VNOS-C questionnaires, focus group interviews, and responses in BC homework and exams lead us to conclude that NOS understanding does improve over one academic year and that BC contributes to this improvement. It is likely that the instructor's familiarity with NOS affected instruction and thus played a role in the observed improvements seen in the whole class. Limitations of sample size and a high background improvement made it difficult to detect a statistically significant effect of BC on NOS growth. However, the qualitative data from questionnaire responses, focus groups, homework, and exams suggest that BC has had a positive effect on NOS understanding. Because the NOS status was marginally higher even before students began their first university courses, we cannot attribute the difference solely to BC participation. Instead it appears that BC participation is more attractive to students with a higher NOS status, those genuinely curious about how science knowledge grows and changes over time.

Even in a program with a very focused curricular goal, narrow admissions criteria, and a regional student body, there is considerable heterogeneity among our students with respect to academic propensities for extracurricular academic activities, such as creative writing, debate, lab research, and even attraction to BC. Book club was initially offered to extend the learning opportunities for high-achieving students without imposing an additional burden in a compressed curriculum. However, because of the insightful and engaging presentation of the ideas in these three books, even academically at-risk students continued to participate. Several said that BC showed them that the genre is accessible even to novice science students, more accessible than primary literature but a satisfying alternative to popular fiction. Considering the VNOS-C scores alone, these premedical students showed NOS growth during their first year in the premedical program regardless of BC participation. However, the qualitative data suggest the difference in scores may underestimate these differences, and exposure to science stories makes NOS understanding more informed and sophisticated. Collectively, these data indicate that reading and discussing science nonfiction trade books may hold value as a curricular adjunct, even if the effect of BC participation could not be measured against the background effects of a biology course and writing courses that attend to epistemology across the curriculum. Students take traditional laboratory courses, but there were no observations in regular weekly visits that any of the features of NOS were addressed explicitly or implicitly during instruction. It is fortunate that an improvement was measured at all, given that there has been little change observed in college students in other studies (Abd-El-Khalick and Lederman, 2000; Miller ).

Table 3.

Matched pre- and postcourse NOS scores of BC and non-BC students

	Pre	Post	p Value
BC (n = 8)	1.72	2.28	0.008*
Non-BC (n = 8)	1.49	2.01	0.005*
p Value	0.20	0.14
ANCOVA			0.191

*Denotes differences that were significant at p < 0.05.