Bioinformatics education in high school: implications for promoting science, technology, engineering, and mathematics careers.

We investigated the effects of our Bio-ITEST teacher professional development model and bioinformatics curricula on cognitive traits (awareness, engagement, self-efficacy, and relevance) in high school teachers and students that are known to accompany a developing interest in science, technology, engineering, and mathematics (STEM) careers. The program included best practices in adult education and diverse resources to empower teachers to integrate STEM career information into their classrooms. The introductory unit, Using Bioinformatics: Genetic Testing, uses bioinformatics to teach basic concepts in genetics and molecular biology, and the advanced unit, Using Bioinformatics: Genetic Research, utilizes bioinformatics to study evolution and support student research with DNA barcoding. Pre-post surveys demonstrated significant growth (n = 24) among teachers in their preparation to teach the curricula and infuse career awareness into their classes, and these gains were sustained through the end of the academic year. Introductory unit students (n = 289) showed significant gains in awareness, relevance, and self-efficacy. While these students did not show significant gains in engagement, advanced unit students (n = 41) showed gains in all four cognitive areas. Lessons learned during Bio-ITEST are explored in the context of recommendations for other programs that wish to increase student interest in STEM careers.

INTRODUCTION

Careers in science, technology, engineering, and math (STEM) command higher salaries, make significant contributions to the economy, and provide paths out of poverty, yet student interest in STEM careers is on the decline (Lowell ; University of the Sciences [USciences], 2012). When high school students are asked to explain their lack of interest, 18% cite a lack of knowledge about health science careers, while others either blame poor preparation in high school (16%) or an inability to do well in these subjects (21%; USciences, 2012). A recent review of the literature concluded that “high school appears to be a key point at which young people's impressions of science influence their future career decisions” (Subotnik ). Understanding how high school students become aware of STEM career options, how educators can help students translate awareness into pursuit of STEM careers, and how to provide students with the support and skills they need to succeed are crucial elements for ensuring the future of our STEM workforce. According to the U.S. Department of Labor's Bureau of Labor Statistics, life sciences and other STEM professions are expected to grow at a faster rate than non-STEM professions (U.S. Department of Labor, 2007), spurred on by both the fast pace of technological innovations and the changing demographics of the U.S. population (Stine and Matthews, 2009). Consequently, national attention is focusing on the need to better promote STEM careers among young people (National Research Council [NRC], 2002; President's Council of Advisors on Science and Technology [PCAST], 2010). We present here an investigation into how the Bio-ITEST program offers a teacher professional development model and student classroom experiences that seek to increase teacher and student awareness of and interest in bioinformatics-related STEM careers.

Cognitive-Behavioral Building Blocks of Career Development

A large body of work addresses various cognitive-behavioral processes that lead to career choice (or discouragement), including the concepts of awareness, self-efficacy (and closely related notions such as autonomy, confidence, proficiency, and competence), engagement (or involvement), and relevance (Bandura, 1994; Blustein and Flum, 1999; Dorsen ). Though different conceptual frameworks use varying terminology, the approaches are complementary. These cognitive processes, occurring internally and mediated by both intrinsic and extrinsic motivators, result in external behaviors whose consequences in turn further reinforce or detract from emerging career choices.

Awareness of STEM careers is an essential precondition for engagement, self-efficacy, and a sense of relevance to develop; however, students often have limited understanding of available careers and requirements for success (Dorsen ). Student awareness includes knowledge and appreciation of the required skills, education, and work/life issues associated with a variety of STEM careers. Tai demonstrated that early expectations of a career in science are a potent predictor, independent of academic preparation, of later STEM career choice, emphasizing the need for early exposure to and encouragement in the pursuit of STEM careers.

Self-efficacy is defined by Bandura (1994) as “people's beliefs about their capabilities to produce effects,” that is, to achieve particular results. Bandura further posits that, as people prepare for careers, perceived self-efficacy is the foundation of cognitive, self-management, and interpersonal skills informing career choice and success. A large body of work supports Bandura's theory as it relates to achievement in STEM fields (Hackett and Betz, 1989; Lent ; Mau, 2003; Zeldin ). Career exploration activities can raise awareness and at the same time foster a sense of self-efficacy and ownership that becomes intrinsically motivating (Blustein and Flum, 1999).

Engagement with subject matter and STEM careers can be demonstrated by students showing interest in learning and experiencing more in the science classroom, including active participation in discussions and asking questions that go beyond the content presented. Engaging students in real-world research projects is a proven strategy to encourage interest in science careers (O’Neill and Calabrese Barton, 2005). Schneider have also found that students who report high motivation and challenge in their schoolwork are more likely to engage in future educational opportunities.

Relevance is a concept that describes when students find a meaningful connection to STEM content or related careers and is a critical component in fostering the positive feelings associated with intrinsic motivation (Shernoff ). Situations that require students to solve real problems serve to increase perceptions of relevance. Science relevance is typically measured in terms of student beliefs that science may be useful in everyday life and in the future (Siegel and Ranney, 2003).

While these conceptual elements of the cognitive-behavioral processes that lead to career choice are causally related to one another, each component may operate in multidirectional, simultaneous, or mutually reinforcing ways. For example, engagement can lead to increases in self-efficacy, heightening both awareness and a sense of relevance, while a sense of self-efficacy fostered by success can encourage further exploration of the subject matter, underscoring the relevance of the material to the student's life and promoting greater engagement.

Bioinformatics and the Data-Driven Nature of Today's Biology

Advances in data-intensive sampling methods, such as high-throughput DNA sequencing, proteomics, metabolomic characterization of complex biological samples, and high-resolution imaging of various living systems, have led to exponential growth in the amount of biological data available and rapid changes in how biological information is used. Bioinformatics can be defined as the application of computer science to biology and biomedicine. It is an interdisciplinary field that combines information technology, software engineering, and biology to analyze the massive data sets generated in biology today. Bioinformatics databases and analysis tools have become ubiquitous in modern biology: from DNA and protein comparisons to working with molecular structures and metabolic pathways, bioinformatics is integral to our understanding of biology. The need for individuals who can understand and analyze this wealth of information and utilize bioinformatics tools for data analysis has grown rapidly, with serious implications for our future STEM workforce. The NRC, in Building a Workforce for the Information Economy, notes that “the wealth of biotechnology-related data continues to expand, along with the need to analyze and understand it, and specialists in bioinformatics … are in great demand relative to supply … There is no sign that the demand for bioinformatics specialists is abating. Indeed, the demand will continue to grow rapidly, given estimates that as many as 40% of the biotechnology companies that survive will be selling information rather than products” (NRC, 2001, p. 328).

Despite the ubiquity of bioinformatics in biology today, these tools and concepts receive little attention in most high school science classes. This paper describes our efforts to address this need through Bio-ITEST: New Frontiers in Bioinformatics and Computational Biology. The Bio-ITEST program and curricula are designed to provide secondary science teachers with the knowledge, skills, and resource materials to engage their students in the newly developing fields of bioinformatics and related careers at the intersection of biology and information technology, encouraging student participation in these important new workforce areas.

Many bioinformatics tools used by scientists are freely available and can be readily implemented in high school settings with little to no up-front costs if student computers are available. This provides students with the opportunity to use authentic bioinformatics and tools and databases utilized by practicing scientists. We predicted that the experience of using the same tools that scientists use in conjunction with sequential skill building from lesson to lesson would increase students’ sense of self-efficacy. Each Bio-ITEST lesson encourages career awareness by featuring a different STEM professional who uses bioinformatics in his or her work or whose work is made possible by bioinformatics. Bio-ITEST curriculum-development teachers selected genetic testing and evolution as unit topics, which they have found to be particularly engaging for high school students. These topics also provide narrative frameworks that can be investigated with bioinformatics tools and databases. We predicted that these topics would increase student perceptions of the relevance of unit content. In addition, both topics involve socio-scientific issues that can be explored through ethical analysis and discussion strategies. In our prior work, we have found that socio-scientific issues promote student interest in science content and foster critical-thinking skills (Chowning, 2005, 2009a, 2009b; Chowning ). A feedback loop in which students document their increasing proficiency in using bioinformatics tools through résumé writing can serve to further increase self-efficacy and relevance. The concepts of awareness, self-efficacy, engagement, and relevance were also used to evaluate the effectiveness of the Bio-ITEST program by measuring changes in these attitudes relative to bioinformatics, bioethics, and related STEM careers.

Through bioinformatics curriculum development and teacher professional development, the long-term goals of the Bio-ITEST program are to increase teacher and student understanding of the application of information technologies to the biological sciences; the ethical implications of the acquisition and use of biological information; and the career possibilities in the fields of bioinformatics, computational biology, and related STEM careers.

MATERIALS AND METHODS

Bioinformatics Curriculum Development Approach

The Bio-ITEST curriculum development was led by the Northwest Association for Biomedical Research (NWABR), a 501(c)(3) nonprofit organization that has been fostering students and teachers in bringing the discussion of ethical issues in science into the classroom since 2000 (Miller, 2008; Chowning, 2005, 2009a, 2009b; Chowning ). NWABR's mission is to promote an understanding of biomedical research and its ethical conduct through dialogue and education. As part of this mission, NWABR has a demonstrated history of success in developing curricular materials and providing teacher professional development focused on the science and ethics of diverse topics, including HIV vaccines, stem cells, animals in research, and the nature of science (freely available at www.nwabr.org). NWABR connects the scientific and education communities across the Northwest and helps the public understand the vital role of research in promoting better health outcomes. NWABR's curriculum-development process is informed by the principles of Understanding by Design by Wiggins and McTighe (1998), and “constructivist” perspectives that recognize that learners build their understanding based on prior experience and construct conceptual scaffolds upon which to integrate new learning. Teacher partners work with NWABR in all aspects of the curriculum development process, selecting central ideas, conceptualizing lessons, field testing, and sharing their experience in teaching students and their knowledge of effective implementation strategies in the areas of state and national education standards.

Six experienced teachers from Washington and Oregon were recruited to provide the broad outlines of two bioinformatics-related curriculum units during the 2-wk 2009 Bio-ITEST curriculum development workshop. During the first week, teachers were immersed in bioinformatics and molecular biology through the use of wet-lab and computer activities, meetings with scientists, and tours of research facilities in the Seattle area. Computer activities included exploration of the molecular visualization program Cn3D (Wang et al., 2000) utilizing previously developed molecular structure activities that had been shown to significantly increase student learning among high school and college students (Porter ). On the basis of these experiences, teachers chose overarching themes for the two bioinformatics units. The introductory curriculum focuses on genetic testing, which teachers believed would be relevant and engaging for students, particularly with new companies like 23andMe (Mountain View, CA) offering personalized, direct-to-consumer (DTC) genetic testing. This topic also offers the opportunity to address topics in molecular biology, bioinformatics, and ethics-related concepts. The advanced curriculum focuses on genetic research and evolution, utilizing DNA sequence data and bioinformatics tools to explore species relatedness.

Lesson materials were developed and further refined by Bio-ITEST staff following review by our advisory board of scientists and educators, and were then shared with teachers at semiannual professional development workshops (described below in Bioinformatics Teacher Professional Development). Workshop teachers then piloted and field-tested the lessons, providing written and oral feedback that informed additional lesson refinements in an ongoing and iterative process over a 2-yr period. Detailed site observations and teacher interviews conducted by the external evaluation team further informed revisions, particularly to the career materials contained in each unit. To ensure the accuracy and authenticity of Bio-ITEST curricular materials, as well as to obtain feedback about lesson composition and flow, NWABR recruited additional bioinformatics experts to review both curriculum units. High school and college students were recruited to provide feedback on the content, flow, and usability of each lesson. All lessons were mapped to the Washington State K–12 Science Learning Standards (Office of the Superintendent of Public Instruction, 2010), the National Science Education Standards (NRC, 1996), and the A Framework for K–12 Science Education (NRC, 2011). All lessons are freely available as PDF documents with accompanying PowerPoint slides, sequence and structure files, and supporting animations. These materials can be downloaded from NWABR's introductory bioinformatics curriculum Web page (www.nwabr.org/curriculum/introductory-bioinformatics-genetic-testing) and advanced curriculum Web page (www.nwabr.org/curriculum/advanced-bioinformatics-genetic-research).

Introductory Curriculum:

The introductory bioinformatics curriculum, Using Bioinformatics: Genetic Testing, introduces students to a collection of bioinformatics tools and explores the ethical issues surrounding genetic testing. Students investigate the genetic and molecular consequences of a mutation in the Breast Cancer susceptibility 1 (BRCA1) gene, using the Basic Local Alignment Search Tool (BLAST) bioinformatics tool to compare DNA and protein sequences from patients with those of the BRCA1 reference sequence. Students then use Cn3D to visualize molecular structures and the impact of mutations on protein structures. The curriculum begins by having students perform Meet the Gene Machine, a play developed by the Science Communication Unit (www.science.uwe.ac.uk/sciencecommunication) at the University of the West of England, funded by the Wellcome Trust, and used with permission (Table 1). The play sets the stage for the rest of the curriculum, helping students explore some of the myths and realities of genetic testing today as they follow the story of a family considering using genetic testing to learn if they possess mutations in BRCA1. Students are also introduced to principles-based bioethics in order to support their thoughtful consideration of the many social and ethical implications of genetic testing. With the Bio-ITEST program's emphasis on promoting student interest in STEM careers, each lesson features an individual who uses bioinformatics in his or her work or whose work is made possible by bioinformatics (Table 1). In the culminating career lesson (lesson 7), students explore each featured career in greater depth, reading transcripts of interviews with the career professionals and writing their own résumés to document their experience in molecular biology and bioinformatics.

Table 1.

Introductory curriculum, Using Bioinformatics: Genetic Testing: lesson activities and learning objectives

Title	Learning objectives	Activities	Featured career and rationale
Lesson 1: Bioinformatics and Genetic Testing	Genetic tests are available for many conditions, but vary in their clinical validity and usefulness. Genetic tests can have social and ethical implications.	Student-led play, Meet the Gene Machine Teacher-led exploration of DTC company website,23andMeNOVA video, “A Family Disease”	Bioengineer: develops devices like the “gene machine” featured in the play
Lesson 2: Navigating the NCBI	Biological data are stored in public databases such as the one at the NCBI. Genetic tests are developed using the biological information available in databases such as the one at the NCBI. All organisms need DNA repair proteins like BRCA1, including cats and dogs.	Student-led exploration of the NCBI Understanding databases through a comparison of the NCBI and iTunes	Veterinarian: genetic testing is now available for animals, too
Lesson 3: Exploring Genetic Testing: A Case Study	Genetic testing can have implications for family members of the patient, as they share the same genetic material. Ethical principles can be applied to many situations, assist in considering alternative perspectives, and facilitate engagement in discussion and decision making.	Structured academic controversy using a short case study about a woman considering BRCA1 genetic testing	Genetic counselor: helps people consider the risks and benefits of genetic testing
Lesson 4: Understanding Genetic Tests to Detect BRCA1 Mutations	Reference sequences are used to determine whether patient DNA sequences contain mutations. The bioinformatics tool BLAST can be used to compare DNA and protein sequences.	Use a pedigree and Punnett squares to identify family members who should consider testing for BRCA1 mutations Align patient DNA and protein sequences against a reference sequence to identify a mutation using BLAST	Laboratory technician: processes patient samples for genetic testing
Lesson 5: Learning to Use Cn3D: A Bioinformatics Tool	Bioinformatics tools like Cn3D help scientists visualize molecular structures. A protein is a physical “thing” with a three-dimensional structure that determines its function. A mutation can impact the three-dimensional structure (and therefore the function) of a protein.	Student-led exploration of macromolecular structure using Cn3D Teacher-led exploration of the impact of mutations on the BRCA1 protein using Cn3D	Three-dimensional animator: utilizes biological information to make difficult concepts understandable (such as the animation featured in this lesson)
Lesson 6: Evaluating Genetic Tests: A Socratic Seminar Discussion	Genetic tests differ in their clinical validity and usefulness. There are some conditions for which there are genetic tests but no effective treatment. Medical conditions differ in their penetrance and the number of genes involved.	Socratic seminar discussion utilizing one of two readings	Bioethicist: helps scientists and society consider the ethical implications of scientific endeavors, including genetic testing
Lesson 7: An Introduction to Bioinformatics Careers	Bioinformatics tools are used by people in many different careers. Different careers require different skills and education. Jobs in many fields require submission of a résumé specific to that job.	Select a career and read an interview transcript with a career professional from lessons 1–6 Perform Internet research about a selected career Prepare a résumé	Students select one career from previous lessons to explore further
Lesson 8: Genetic Testing Unit Assessment: ALAD and SOD1	Demonstrate proficiency using BLAST, Cn3D, and ethical reasoning	Application of BLAST, Cn3D, and ethical reasoning skills to a new genetic disease and associated genetic test	None

Advanced Curriculum:

DNA barcoding is a taxonomic method that uses a short genetic marker in an organism's DNA to identify it as belonging to a particular species (Folmer ; Hebert ). For animals, the mitochondrial-encoded cytochrome c oxidase subunit 1 (COI) gene is used (Folmer ). The advanced curriculum, Using Bioinformatics: Genetic Research, explores DNA barcoding of animals, building on lessons from the introductory curriculum and incorporating additional informatics resources to teach concepts related to species diversity and evolution. DNA barcoding is a technique with several advantages for use in an educational setting. Because the COI gene is mitochondrial, the DNA is more abundant and less prone to degradation (for classes performing their own wet-lab experiments). The region that is sequenced is short, eliminating the need to generate several overlapping sequences and assemble them. Mitochondrial DNA lacks introns in many organisms, which also simplifies the analyses. In addition, DNA barcoding provides concrete connections for students between DNA sequences and the surrounding world. Barcoding can be used to catalogue and confirm the discovery of a new species or to identify the species of an unknown sample. In the Using Bioinformatics: Genetic Research curriculum, students use BLAST to identify an unknown DNA sequence, perform multiple sequence alignments, and build phylogenetic trees (Table 2). They also learn to use the bioinformatics tool ORFinder to identify open reading frames in a DNA sequence.

Table 2.

Advanced curriculum, Using Bioinformatics: Genetic Research: lesson activities and learning objectives

Title	Learning objectives	Activities	Featured career and rationale
Lesson 1: The Process of Genetic Research	Science is a process involving observations about the natural world, and the generation and testing of hypotheses. Genetic research and bioinformatics can be used to answer research questions in many different STEM fields. DNA sequence data can be used to evaluate species relatedness.	Think–pair–share exploration of genetic research questions in various STEM fields Review of the scientific process Student-generated hypothesis of canid relatedness Pairwise comparisons of short paper DNA sequences	DNA sequencing core lab manager: helps scientists obtain DNA sequence data for their research studies
Lesson 2: DNA Barcoding and the Barcode of Life Database (BOLD)	Scientists often collaborate with one another to conduct research. Biological data are shared by scientists and stored in public databases such as the one at the NCBI and BOLD. The bioinformatics tool BLAST can be used to identify unknown DNA sequences.	Use BLAST to identify “unknown” DNA sequences provided by NWABR Obtain taxonomic information about species using BOLD Form collaborative groups with other students whose identified species are in the same taxonomic class	Postdoctoral scientist in DNA and history: uses genetic data to study the history of human populations and migrations
Lesson 3: Using Bioinformatics to Study Evolutionary Relationships	Scientific collaboration and data sharing are vital to the scientific process. Bioinformatics tools like JalView/ClustalW2 can be used to analyze long DNA sequences. Phylogenetic trees can be used to draw conclusions about evolutionary relationships.	Use JalView/ClustalW2 and DNA sequence data from lesson 2 to compare multiple sequences Use BLAST and an outgroup (provided by NWABR) to create a phylogenetic tree and draw conclusions about evolutionary relationships	Microbiologist: uses genetic data to study microbes that cause diseases such as tuberculosis or influenza
Lesson 4: Using Bioinformatics to Analyze Protein Sequences	DNA is composed of two strands that are complementary and antiparallel. There are six potential reading frames for protein translation in each strand of DNA. Bioinformatics tools can be used to identify open reading frames and compare protein sequences.	Paper exercise to understand the complementary nature of DNA and six reading frames of protein translation Use ORFinder to identify the likely reading frame for a DNA sequence Perform multiple sequence alignment using a group's protein sequences	Biological anthropologist: uses genetic data to study the evolution of humans and other hominids
Lesson 5: Protein Structure and Function: A Molecular Murder Mystery	Mitochondria are the site of ATP production in the cell. Cytochrome c oxidase is involved in ATP production. The active site of a protein is vital to the function of the protein. Substances that bind to the active site can interfere with protein function.	Identify the active site of cytochrome c oxidase using Cn3D Identify a foreign substance (a poison) bound to the active site of cytochrome oxidase	Molecular diagnostics researcher: uses genetic information about infectious organisms to develop diagnostic tests
Lesson 6: Assessment: Writing Research Reports	Scientists share their work with other scientists in the spirit of collaboration and to advance scientific knowledge. The components of a research report correspond to the steps of the scientific method.	Write a research report with instruction, methods, results, and discussion sections and figures Assessment alternatives: scientific poster, scientific abstract, or a science-related magazine article	Science and technical writer: helps scientists communicate effectively to the public and to other scientists
Lesson 7: Who Should Pay? Funding Research on Rare Genetic Diseases	Rare genetic conditions affect a limited number of people but can cause great suffering. Much scientific research in the United States is funded by taxpayer money. There is a limited amount of money that must be allocated based on our values and the needs of stakeholders. Bioethical principles can provide a structure for making complex decisions.	Jigsaw exercise: meet in “like” and then “mixed” groups of stakeholders (parent, researcher, doctor, or advocate) Use bioethical principles to draft recommendations on allocation of public resources for research on rare genetic diseases	Pediatric neurologist: uses genetic testing results to help diagnose and treat children with diseases of the brain or spinal cord
Lesson 8: Exploring Bioinformatics Careers	Bioinformatics tools are used by people in many different careers. Different careers require different skills and education. Jobs in many fields require submission of a résumé and cover letter specific to that job. Job interviews include questions about your skills and experience (optional).	Select a career and read an interview transcript with a career professional from lessons 1–6 Perform Internet research about a selected career Create or update a résumé Critique and write a cover letter Mock job interview (optional)	Students select one career from previous lessons to explore further
Lesson 9: Analyzing DNA Sequences and DNA Barcoding	DNA sequences can be used to identify the origin of samples. DNA data (called a chromatogram) are generated by DNA sequencing. For increased accuracy, both strands of DNA are often sequenced. Data can be used to guide decision-making when reconstructing a DNA sequence.	Use BLAST and FinchTV to analyze DNA chromatograms (provided by NWABR or generated in class using the wet lab) Identify and edit discrepancies between sequence data from both strands of DNA Use a phylogenetic tree from BOLD for sample identification	None
Wet lab: DNA Barcoding: From Samples to Sequences	DNA barcoding involves multiple laboratory experiments prior to bioinformatics analysis. DNA must be purified through a process involving cell lysis and separation of the DNA from the rest of the cell debris. PCR is used to make many copies of a gene or region for use in subsequent analyses. Agarose gel electrophoresis is performed to confirm whether a PCR was successful. A purified DNA product is used for DNA sequencing.	Lab 1: DNA purification for DNA barcoding Lab 2: Copying the DNA barcoding gene using PCR Lab 3: Analyzing PCR results with agarose gel electrophoresis Lab 4: Preparation of PCR samples for DNA sequencing	None

As in the introductory curriculum, each advanced lesson features a professional who uses bioinformatics in his or her work or whose work is made possible by bioinformatics (Table 2). In the culminating career lesson (lesson 8), students read interview transcripts and perform career-related Internet research. They also build on the career skills developed in Using Bioinformatics: Genetic Testing lesson 7 by updating their résumés, learning to write a cover letter, and participating in mock job interviews.

A key element of the Bio-ITEST program is the incorporation of an authentic bioinformatics research project into the curricular materials, as this has been shown to increase student engagement and interest in STEM careers (O’Neill and Calabrese Barton, 2005). DNA barcoding offers exciting opportunities for students to participate in authentic research, generate testable hypotheses, and learn how to use the tools of science. Lesson 9 is an optional extension lesson in which students learn how to analyze DNA sequence data provided by the Bio-ITEST program or generated in their classrooms (wet lab).

Bioinformatics Teacher Professional Development

NWABR's professional development workshops are based on five principles of professional development consistent with research on adult learning (Sparks and Hirsh, 1997). These principles include building upon the teacher's current knowledge and skills and providing engaging and varied opportunities to practice new skills and receive feedback about progress. Successful professional development should result in measurable increases in teacher knowledge and skills that are linked to outcomes in student achievement.

The Bio-ITEST curriculum is shared with teachers in two different professional development formats: a 1.5-d workshop, An Introduction to Bioinformatics, held in late Winter, and a 2-wk workshop, Using Bioinformatics: Genetic Research, held each summer. Both formats provide teachers with the opportunity to see where bioinformatics is used in real-world situations through visits to Seattle-area research institutions and interactions with scientists, and to experience the Bio-ITEST curriculum firsthand. To measure the effect of the Bio-ITEST curriculum and professional development on STEM career awareness and related outcomes, we recruited the 24 teachers from around the country who participated in the 2010 Summer workshop to participate in the Bio-ITEST research study. After teachers completed the workshop, their students were also recruited to participate in the study during the 2010–2011 academic year.

The 2-wk Bio-ITEST Summer workshop, held at Shoreline Community College (SCC) in Shoreline, Washington, exposed teachers to both the introductory and advanced bioinformatics curricula. Given the complex nature of the subject matter and the amount of information covered during only 10 d of instruction, the participating teachers received background readings and homework assignments introducing them to bioinformatics, genetic testing, and DNA barcoding in advance of the workshop. NWABR's previous experience in professional development has been in the context of instructing teachers on using ethics in their classrooms (Miller, 2008; Chowning ). This experience has informed a model for professional development wherein workshop teachers begin the workshop by setting “group norms” to create a supportive learning environment (such as respecting others’ opinions, sharing speaking time, and refraining from being judgmental) and devising strategies to let program staff know when they felt overwhelmed by complex material. Questions could be asked verbally during workshop activities, written on large Post-It notes around the perimeter of the room, or asked anonymously at the end of each day on Stars and Wishes forms. These Stars and Wishes sheets, developed by science and ethics teacher Paula Fraser, were used by Bio-ITEST staff to quickly assess what worked (Stars) and what did not (Wishes), to identify topics that needed additional clarification, and to modify instruction for the next day. Teachers received training in using the introductory lesson materials, including review of pedagogical strategies included in the procedures section of each lesson, and experienced each of the full-length lessons themselves in order to practice their new skills.

As the focus of the advanced curriculum is DNA barcoding, a primary goal of the workshop was for teachers to experience the entire barcoding process: obtaining a sample from an organism, purifying the DNA, using polymerase chain reaction (PCR) to amplify the barcode sequence, sending it to be sequenced, analyzing the DNA data, and comparing those data with sequences from database repositories at the NCBI (www.ncbi.nlm.nih.gov) and the Barcode of Life Data Systems (http://barcodinglife.com). Teachers isolated DNA from an “unknown” tissue sample (Oncorhynchus kisutch [Coho salmon] purchased at a local grocery store), used PCR, checked their PCR products by agarose gel electrophoresis, purified their PCR products, and submitted them for sequencing (see “wet lab” in Table 2). Experimental protocols from commercially available kits and protocols obtained from members of the DNA barcoding community (Yancy ) were adapted for classroom use. These protocols were supplemented with educational support resources available online from the Howard Hughes Medical Institute and the DNA Learning Center (www.dnalc.org and www.dnai.org). Teachers used the DNA sequence data they generated in week 1 for their exploration of the advanced strand lessons during week 2 to help them become more familiar with using bioinformatics tools and databases. Teachers also received training in using software (FinchTV; Geospiza, Seattle, WA) and online programs for analyzing DNA sequences, performing multiple sequence alignments, and using bioinformatics to study evolutionary relationships. The goal of the activities was to help teachers understand the flow of biological data from the lab bench to the computer.

Echoing the 2009 curriculum development workshop format, Summer workshop teachers deepened their exploration of bioinformatics and related careers by touring local research facilities and learning about next-generation DNA-sequencing technology and other high-throughput data-generation and analysis techniques. Guest speakers and panel discussions with scientists who perform genetic research were included in the 2-wk program to illustrate diverse careers and areas of research influenced by bioinformatics.

Bio-ITEST Program Evaluation and Research Study

An external evaluation team conducted a formative and summative program evaluation that addressed two questions: 1) In what ways does the Bio-ITEST model of curriculum development and teacher professional development add to our understanding of how to best prepare teachers to develop the knowledge and skills necessary among their students for participation in the STEM workforce? 2) In what ways does Bio-ITEST contribute to our understanding of how to engender student awareness of and interest in STEM careers? The program evaluation used teacher and student structured interviews, focus groups, site observations of professional development and classroom activities, and numerical and open-ended survey questions to determine program effectiveness.

As part of this larger program evaluation, the Bio-ITEST research study focused on the following questions:

What were the effects of Bio-ITEST program participation on teachers’ knowledge and perceptions of bioinformatics and related STEM careers?

What were the effects of Bio-ITEST participation on students’ knowledge and perceptions of bioinformatics and related STEM careers?

Did change in participating teachers’ knowledge and perceptions correlate with change in students’ knowledge and perceptions of bioinformatics and related STEM careers?

All 24 teacher participants from the 2010 Summer Bio-ITEST professional development workshop Using Bioinformatics: Genetic Research were recruited to take part in the evaluation and the research study. The Bio-ITEST research study utilized pre- and postsurveys of teacher and student participants to measure changes in awareness of STEM career opportunities, particularly in bioinformatics (“awareness”); sense of self-efficacy in using bioinformatics tools (“self-efficacy”); perceptions of the relevance of biology content to their lives (“relevance”); and engagement with science (“engagement”). Teacher results are based on surveys tracking growth over three points in time: at the beginning (baseline, “preworkshop”) and end (“postworkshop”) of the August 2010 Using Bioinformatics: Genetic Research workshop, and at the end of the school year (May or June 2011, “end of year”). An additional goal of the study was to determine whether changes in teachers’ perceptions correlated with changes in students’ perceptions in each of the four study areas (awareness, self-efficacy, relevance, and engagement). Two hundred eighty-nine students of participating teachers who taught the introductory unit, Using Bioinformatics: Genetic Testing, completed pre- and postunit surveys. Surveys were administered online where possible or via paper and pencil. All teacher and student surveys are available in the Supplemental Material. A separate postunit survey was administered to 41 students participating in the advanced unit, Using Bioinformatics: Genetic Research, at the conclusion of instruction.

In addition to the correlational research study, the program evaluation addressed the question of student impacts by soliciting feedback from teachers and students through open-ended queries on three questionnaires: a prequestionnaire for the May 2011 reunion of Bio-ITEST teachers, the Bio-ITEST Educators End of Year Survey, and the Bio-ITEST (advanced unit) Genetic Research Curriculum Student Survey. Teacher comments are based on their experiences teaching both the introductory and advanced curriculum units and focused on the following themes: impacts of the professional development workshop, impacts of the curriculum units and program overall, and effectiveness of conveying bioinformatics and related career materials to students.

Institutional Review Board (IRB) Approval

This study was reviewed and approved by Quorum Review IRB (Quorum Review File # 24134/1). All study participants and/or their legal guardians provided written informed consent for the collection and subsequent analysis of verbal and written responses.

Survey Development

Face validity for the survey pre/postunit constructs (awareness, self-efficacy, engagement, as well as relevance for students) was established through an iterative item-construction process by the research team, and the content validity of the four constructs was empirically tested using exploratory factor analysis (EFA) of the preunit survey items (Stevens, 2002; Tabachnick and Fidell, 2007). The EFA used maximum-likelihood estimation (Tabachnick and Fidell, 2007) and a Varimax orthogonal rotation. For the teacher survey, an EFA showed that the three-factor model fitted the data well (χ2(3) = 0.84, p > 0.05), and the set of factors accounted for 68% of the variance in the items (internal consistency across all items was 0.84). For student results, an EFA showed that a four-factor solution fitted the data fairly well, in that all communalities were > 0.40, and further, the set of four factors together accounted for 61% of the total variance in the set of items. The results of the four-factor EFA solution show that the item-factor loadings corresponded well with the four constructs overall. Internal consistencies were as follows: awareness: 0.85; relevance: 0.65; self-efficacy: 0.76; and engagement: 0.83.

Statistical Analyses

Basic one-sample t tests were used to evaluate whether teachers’ pre- to postworkshop and postworkshop to end-year changes were statistically significant, as well as whether students’ pre/postunit changes were statistically significant. We corrected for type I error inflation familywise by grouping the statistical tests by survey item type (i.e., teachers’ pre–post responses, teachers’ retrospective responses, introductory unit students’ responses, introductory unit students’ retrospective responses, and advanced unit students’ retrospective responses) and adjusting our per-comparison type I error rate using the Bonferroni adjustment (which divides the alpha level by the number of tests performed). Familywise alpha level was set at 0.05. SPSS/PASW (IBM Corp., Armonk, NY) was used for these analyses.

In addition to item-level t tests, the correlations between teacher and student gains were tested within a multilevel modeling framework using HLM 7 (Raudenbush ). This method of analysis accounts for the nonindependence of students’ scores within a classroom and uses appropriate degrees of freedom for testing teacher gain correlations with student gains. For these analyses, we created composite construct scores for each teacher and student at each survey wave (pre- and postunit) by computing the mean of the scores of each item related to the construct (see prior discussion of EFA results in Survey Development). Twelve of the 24 workshop teachers participated in this phase of the research. Three retrospective items (administered only on the postunit survey) asking students to estimate their knowledge and skills “before” and “after” the unit, were computed as the students’ difference scores.

RESULTS

Teacher Effects: Impacts of the Professional Development Experience

Background Characteristics.

Characteristics of the 24 teachers who participated in the 2010 Summer professional development workshop, Using Bioinformatics: Genetic Research, are summarized in Table 3. Three-quarters of workshop participants were female and white, and of those indicating ethnicity, one teacher was Hispanic/Latino. Teachers represented 21 different schools. Most had master's degrees (79%), as well as teaching certifications in biology (83%) and/or another science certification (63%). Three teachers (12%) had career or technical education (CTE) certifications. This group of teachers was quite experienced overall, with an average of nearly 13 yr of high school teaching experience and over 10 yr spent teaching biology. One-third of the teachers had prior professional development in bioinformatics (including two who participated in the 2010 1.5-d workshop, An Introduction to Bioinformatics, and one who participated in the 2009 Bio-ITEST curriculum development workshop). Nearly two-thirds (63%) had prior professional development in integrating ethics into science curricula.

Table 3.

Characteristics of teacher participantsa

	All 2010 teacher participants		Curriculum implementers
Number of teachers	24		12
Gender	75%25%	Female (18) Male (6)	67% 33%	Female (8) Male (4)
Ethnicity	75%4%21%	Non-Hispanic white (18)Hispanic (1)Unknown (5)	75%0%25%	Non-Hispanic white (9)Hispanic (0)Unknown (3)
Race	75%4%8%0%0%0%0%8%	White (18)Black/African American (1)Asian/Southeast Asian (2)American Indian (0)Alaska Native (0)Native Hawaiian (0)Pacific Islander (0)Other (2)	67%0%17%0%0%0%0%17%	White (8)Black/African American (0)Asian/Southeast Asian (2)American Indian (0)Alaska Native (0)Native Hawaiian (0)Pacific Islander (0)Other (2)
Highest level of education completed	8%79%13%	Doctorate (2)Master's degree (19)Bachelor's degree (3)	8%83%8%	Doctorate (1)Master's degree (10)Bachelor's degree (1)
Certifications	83%63%13%13%	Biology (20)Other science (15)CTEb (3)Conditional (3)	83%50%17%0%	Biology (10)Other science (6)CTEb (2)Conditional (0)
Prior professional development	63%33%	Ethics (15)Bioinformatics (8)	58%50%	Ethics (7)Bioinformatics (6)
Mean years of teaching experience	131013	High schoolBiologyAll sciencesc	131214	High schoolBiologyAll sciencesc

aPercentages of individual items may not total 100% due to rounding or classification of individuals into multiple categories.

bCareer and Technical Education.

cIncludes biology.

Qualitative Findings.

As part of the postworkshop survey, teachers had the opportunity to comment on the “most significant take-aways” from their workshop experiences. These remarks were supplemented by the teacher interviews and focus groups conducted by the evaluation team. Many participant comments related to the general theme of increased understanding of bioinformatics, biology, and biotechnology, and greater comfort in the skills they had learned:

“I am so excited that I am gaining some comfort and familiarity with bioinformatics software and databases; it will become part of the tool kit I can use when designing lessons.”

“An additional take-away is a much more well-formed understanding of how bioinformatics fits with biological research and biotechnology. I had a very unclear understanding of this prior to the workshop, and now feel I can easily articulate this understanding to my students, particularly those that show interest and aptitude both in biology and technology.”

Teachers also remarked on the wealth of bioinformatics resources available to teachers and the importance of hands-on experience using them, including the curriculum materials, bioinformatics databases, and professionals in the field who are interested in providing support:

“I feel confident that I’ll be able to turn around and teach it to my students this next school year. The curriculum is extremely well thought out, very teacher friendly, and will be interesting to students. I cannot wait to bring this curriculum to my school, and I am very proud that I’m able to offer my students an opportunity to learn about and actually do science that is on the leading edge of what is being done in our area.”

In addition, teachers noted the importance of exposing students to STEM careers, bioinformatics, and bioethics:

“Two things that I’ve been trying to bring to my classes is [sic] already woven into the curriculum: ethical studies, and career information. Before coming to this class, I realized that I’ve done a good job of getting students interested in science, but a poor job of guiding them toward careers based on that interest.”

“The most significant take-away from this workshop is the importance of exposing students to STEM careers, particularly those using bioinformatics. This is an area in which I have much room for growth and really value the resources provided.”

Teacher Survey Findings.

The effects of the 2010 Bio-ITEST Summer professional development workshop Using Bioinformatics: Genetic Research were evident in survey findings measuring teacher self-reported changes in career awareness, engagement and self-efficacy at three points in time (Figure 1): before the Summer workshop (“pre-workshop”), immediately after the Summer workshop (“postworkshop”), and at the end of the school year (“end of year”). When compared with responses to survey questions on the preworkshop survey (n = 24), teachers showed significant increases postworkshop (n = 24) and at the end of the academic year (n = 23) in all three conceptual areas measured. All pre/postworkshop gains were statistically significant (adjusted for multiple comparisons using the Bonferroni procedure), with the largest gains in the areas of career awareness and self-efficacy using the tools of bioinformatics. All preworkshop/end-of-year gains were statistically significant, except “My interest in analyzing biological information,” a measure of engagement. Mean ratings generally declined somewhat from postworkshop to end of year, but declines on three of the items were not statistically significant, and all end-of-year ratings were still significantly higher than the corresponding preworkshop means.

Figure 1.

Teacher survey responses preworkshop, postworkshop, and end of year. Survey items are arranged in the order of preworkshop response means, as less positive change is possible for higher preworkshop responses. Bars represent SE of the mean. The conceptual category for each survey item is contained in parentheses: A, awareness; E, engagement; and SE, self-efficacy. Postworkshop and end-of-year comparisons are made to preworkshop means. Unadjusted p values: *, p < 0.05; **, p < 0.01; ***, p < 0.001; after Bonferroni adjustment, only p < 0.001 should be considered statistically significant.

Retrospective questions on the postworkshop and end-of-year surveys asked teachers to rate themselves “before” the workshop (retrospectively) and “now” (at the end of the workshop) on survey items measuring career awareness and self-efficacy. Similar retrospective questions were asked of teachers on the end-of-year survey. Teachers demonstrated statistically significant gains on all retrospective survey items (after adjusting for multiple comparisons) following the 2010 professional development workshop “Using Bioinformatics: Genetic Research” (n = 24) and at the end of the 2010–2011 academic year (n = 23; Figure 2), with the largest reported gains for self-efficacy items.

Figure 2.

Teacher retrospective survey items postworkshop and end of year. Survey items are arranged in the order of “Before” at postworkshop response means, as less positive change is possible for higher “Before” responses. Bars represent SE of the mean. The conceptual category for each survey item is contained in parentheses: A, awareness; E, engagement; and SE, self-efficacy. All comparisons are made within survey waves (i.e., “Before” at postworkshop compared with “Now” at postworkshop). Unadjusted p values: ***, p < 0.001; after Bonferroni adjustment, all findings remain statistically significant.

Student Effects: Impact of the Introductory Curriculum Unit

Of the 24 teachers who participated in the 2010 Summer workshop, 13 are known to have implemented four or more of the eight introductory lessons, of whom 12 returned consent forms to allow inclusion of their students’ data in analyses. Chi-square and t tests were used to compare the demographic characteristics shown in Table 3 of all 2010 teacher participants and the 12 curriculum implementers who participated in the research study. No significant differences were found.

The introductory lessons were trialed in at least 28 classes, with students representing various levels of science background and interest, and in a diversity of settings. For example, although the lessons are designed primarily for classroom use, one of the teachers introduced both the introductory and advanced lessons in an after school club. At least two of the teachers implemented the lessons in courses that were part of a science career track aimed at highly motivated science students. In other cases, the lessons were introduced in a required biology class, in which students may not have much prior interest in science. Of the 699 students in the 28 classrooms of the 12 teachers included in the research study, 374 (54%) students consented to take part in the Bio-ITEST study and 289 (41%) completed both preunit and postunit surveys measuring conceptual constructs similar to those measured for their teachers. Characteristics of student participants are shown in Table 4.

Table 4.

Characteristics of student participantsa

Curriculum unit	Introductory		Advanced
Number of students	289		41
Gender	63% 37%	Female (181) Male (108)	56% 44%	Female (23) Male (18)
Ethnicity	95%4%1%	Non-Hispanic white (274)Hispanic (13)Unknown (2)	90%5%5%	Non-Hispanic white (37)Hispanic (2)Unknown (2)
Race	70%6%13%1%0%1%1%8%0%	White (203)Black/African American (17)Asian/Southeast Asian (37)American Indian (3)Alaska Native (0)Native Hawaiian (2)Pacific Islander (3)Other (23)Unknown (0)	68%12%10%2%0%2%0%0%5%	White (28)Black/African American (5)Asian/Southeast Asian (4)American Indian (1)Alaska Native (0)Native Hawaiian (1)Pacific Islander (0)Other (0)Unknown (2)
Grade level	18%33%25%24%0%	Freshman (51)Sophomore (95)Junior (73)Senior (69)Unknown (0)	0%2%32%63%2%	Freshman (0)Sophomore (1)Junior (13)Senior (26)Unknown (1)

aPercentages of individual items may not total 100% due to rounding.

Students using the introductory curriculum made significant pre/postunit gains (adjusted for multiple comparisons) on all items measuring awareness and self-efficacy (Figure 3). The pre/postunit changes on the two relevance items were not significant. One relevance item, “I am interested in how science knowledge can guide ethical decision making,” showed little change preunit (mean = 5.19) to post-unit (mean = 5.06). The second relevance item on the pre/postunit surveys, “I think it is important for people in our society to learn about science,” showed high preunit scores (mean = 5.76) that changed little postunit (mean = 5.86). However, students reported significant gains on the retrospective postunit item measuring relevance, evaluating their perceptions of the connection between biology content and their personal lives (“before this unit” mean = 3.88; “now” mean = 5.64; Figure 4). The retrospective survey items measuring self-efficacy and awareness also showed significant gains (greater than two points).

Figure 3.

Student pre–post survey responses: introductory unit. Survey items are arranged in the order of preunit response means, as less positive change is possible for higher preunit responses. Bars represent SE of the mean. Responses are categorized into four conceptual areas: A, awareness; E, engagement; R, relevance; and SE, self-efficacy. Unadjusted p values: *, p < 0.05; **, p < 0.01; ***, p < 0.001; after Bonferroni adjustment, only p < 0.001 should be considered statistically significant.

Figure 4.

Student retrospective survey responses: introductory unit. Items arranged in the order of “Before this unit” response means, as less positive change is possible for higher “Before” responses. Bars represent SE of the mean. Responses are categorized into four conceptual areas: A, awareness; R, relevance; and SE, self-efficacy. Unadjusted p values: ***, p < 0.001; after Bonferroni adjustment, all findings remain statistically significant.

Two of the four pre/postunit survey items measuring engagement showed no significant change. Students began the unit with a fairly high level of engagement, indicated by a mean of 5.34 on the question “In general, I enjoy learning about science topics.” Postunit, this score remained essentially unchanged (mean = 5.37). Student preunit scores were similar, though slightly lower (mean = 4.88), on a survey item measuring engagement with STEM careers (“I see myself working in a career that involves scientific information.”) These scores also changed little postunit (mean = 4.90). Two of the four pre/postunit changes on the engagement items were significant and negative. One question measured engagement with computer programs to visualize three-dimensional images of molecules (i.e., Cn3D), while the other queried about interest in analyzing biological information. These declines were modest, < 0.5 on the 7-point scale (Figure 3). The retrospective postunit survey did not measure student engagement.

To further explore student engagement and the relationship between engagement and self-efficacy, we performed correlational analyses on the pre/postunit responses of the 289 students who participated in the introductory lessons. The preunit correlation between engagement and self-efficacy was r = 0.50 (25% shared variability, p < 0.001), and the post-unit correlation increased to r = 0.63 (40% shared variability). The correlation between pre–post gains on engagement and pre–post gains on self-efficacy was r = 0.34 (12% shared variance).

Student Effects: Impact of the Advanced Curriculum Unit

A postunit survey measured the students’ perceived effects from the advanced genetic research curriculum. Three teachers who implemented the unit in three classrooms were included in this study. These three teachers were more experienced teaching at the high school level (mean = 23 yr) than the nine teachers who only implemented the introductory unit (mean = 9 yr). These three teachers also scored higher on preworkshop survey measures of self-efficacy (mean = 3.67 vs. mean = 5.00). However, these differences did not remain statistically significant after Bonferroni adjustment for multiple comparisons. Usable surveys were obtained from a total of 41 advanced unit students. Characteristics of student participants are shown in Table 3.

Student postunit retrospective mean ratings comparing “before this unit” with “now” showed large gains on all items, ranging from 1.1 to 3.1 on the 7-point scale (Figure 5). The largest increases were among items measuring self-efficacy (confidence in accessing biological databases and understanding how databases store biological information) and engagement (interest in analyzing biological information). Additional survey items measuring engagement (see myself working in a STEM career and interest in using computer programs to visualize three-dimensional images of molecules) also showed gains of 1.3–1.9 points. All changes were significant, even after adjusting for multiple comparisons.

Figure 5.

Student survey responses post–advanced unit instruction. Items arranged in the order of “Before this unit” response means, as less positive change is possible for higher “Before” responses. Bars represent SE of the mean. Responses are categorized into four conceptual areas: A, awareness; E, engagement; R, relevance; and SE, self-efficacy. Unadjusted p values: ***, p < 0.001; after Bonferroni adjustment, all findings remain statistically significant.

Teacher and Student Comments on Student Effects

Teachers and students were asked to reflect on their most significant take-aways or lessons learned from the bioinformatics curriculum units. Teacher comments are based on their experiences teaching both the introductory and advanced curriculum units. Student comments generally represent more capable students, many of whom were already interested in science, because only the advanced curriculum survey collected open-ended student responses. Comments were categorized according to each of the four career constructs. Some quotes could be classified into more than one category but are assigned to one construct for simplicity. A representative sample of comments is presented in Table 5.

Table 5.

Teacher and student comments on student effects

Construct	Teacher comments	Student comments
Awareness	“It opened up a whole world to them—they knew nothing about the topic before. Now they understand and can use some bioinformatics tools, and they have a clear understanding that there are jobs available in this area, as well as some knowledge about the types of jobs, and the education required to get them.” “They also realized that there are SO many career opportunities they had never heard of.” “Introduced the students to careers they had not thought about before. Infusing career-awareness into my curriculum has not been something I have really done before, so the bioinformatics unit was really the only exposure the students had all year.”	“Some of the most important things were learning about different careers in this unit.” “I learned that bioinformatics is extremely useful in a wide variety of careers and applications.” “It really introduced me to new possible career choices.”
Relevance	“The lessons on ethical issues and awareness of the different careers that use bioinformatics had the most impact. Understanding how technology has changed science and how many different career options there are in biology. I have had many students tell me ‘I didn't know I could do this in science’—it really is an open ended, making connections ‘real’ curriculum.”	“It opened a door that I could go through, it introduced me into something I might be interested in.” “I still want to be a mechanical or electrical engineer, but I might be interested in designing systems to work with biological and bioinformatics technologies.” “I already wanted to pursue a career in the biomedical field, so this unit just added interest to the field.” “There are many ways to help people besides being a doctor.”
Self-efficacy	“They were excited about NCBI. They are so good on computers anyway, one kid became the class teacher. He helped everyone else. Some of them were surprised they could use a tool like that even though they are not scientists.” “They gained significant confidence in their ability to read, understand, and analyze data. It was fabulous!” “The ability to use various tools and databases increased the students’ skills and confidence in applying biology topics.”	“There are different sites available for the public to use for themselves than relying on others to do it for them, and it allows others to learn how the whole process works.” “I didn't know it was so easy to access that information.” “That there are massive databases online that you can plug DNA into and get results of which species it is.”
Engagement	“Students told me that they enjoyed this unit … It really opened their eyes to new ideas and scientific ways of thinking.” “They were really into the Gene Machine and role playing. … They were also interested in the recurring theme of the breast cancer gene and how it could be looked at from multiple levels of understanding, for example the consequences of the disease and the molecular structure of a mutated protein.” “They were fascinated by 23andMe. They really used it. I took it one step further, and did a genetics project. I had them look for diseases in their families on the website (e.g., sickle cell anemia), and they did a research project which they presented with PowerPoint. They learned more by them doing the research themselves.”	“I am very interested in a way I can work in bioinformatics and combine that with engineering and physics.” “[The] DNA barcoding unit made me consider more database related careers in science. Before I considered science careers using primarily databases to be boring jobs but now I think it would be very interesting and more than just sitting at a computer all day.” “I am now looking at pursuing a career in biological research for the benefit of global health.” “It created a new possible job career. I love science and I never knew much about this type of science and it is very fascinating.”

Correlation of Teacher and Student Gains

We tested whether or not participating teachers’ pre/postworkshop gains in knowledge and perceptions of bioinformatics and related STEM careers correlated with growth in students’ knowledge and perceptions of bioinformatics and related STEM careers using multilevel modeling (students [level 1] nested within teachers [level 2]). Results showed no evidence of a relationship between teacher and student growth on any construct, including teacher's total pre/postworkshop change, total postworkshop (point in time) composite scores, and total end-of-year (point in time) composite scores, as well as postworkshop and end-of-year retrospective ratings. In other words, teacher subscale composites did not correlate with student composites (i.e., teacher career awareness did not correlate with student career awareness and vice versa). It is important, however, to note that these analyses were severely limited by a small teacher sample size (n = 12 available) and a lack of data to control for lesson fidelity implementation effects on student gains.

DISCUSSION

The Bio-ITEST program responds directly to the learning goal of the NSF's strategic plan: to cultivate a world-class and broadly inclusive science and engineering workforce and to expand the scientific literacy of all citizens (NSF, 2006). Our program emphasizes cutting-edge bioinformatics resources and provides a broad range of instructional materials that serve both highly motivated science students and the broad base of general biology students who will comprise the majority of our future citizenry. NWABR's teacher workshops are based on five principles of professional development consistent with research on adult learning (Sparks and Hirsch, 1997). Successful professional development experiences build upon the learner's current knowledge, provide multiple opportunities to practice new skills, provide ongoing feedback on the learner's performance, and are linked to measurable outcomes in student performance. Utilizing this model, NWABR's professional development workshops included teachers as program partners, valuing their prior experiences and building on their existing knowledge during training and curriculum development. All teacher workshops offered extensive opportunities to practice new skills and to give feedback on both teacher and staff performance. Teachers who participated in the Bio-ITEST program had significant increases in awareness of bioinformatics and related STEM careers, as well as improved reports of self-efficacy using bioinformatics tools and databases and integrating these resources into their classrooms. On the basis of numerical and open-ended responses, following instruction with the Bio-ITEST curriculum units, students demonstrated increased understanding of the application of information technologies to the biological sciences, the ethical implications of the acquisition and use of biological information, and the career possibilities in the fields of bioinformatics and related careers. While the Bio-ITEST research study did not find a correlation between changes in teacher and student knowledge and perceptions of STEM careers, the magnitude of change among both groups was significant given the short duration of the educational intervention. In the following sections we highlight some of our key program findings in the context of suggestions for teacher professional development and student instruction in other STEM fields.

Lessons for Effective Teacher Professional Development

Collaborate with Teachers as Program Partners.

Working directly with teachers as respected and valued partners throughout the iterative curriculum-development process, including during lesson design, field-testing, and revision, is a crucial component for program success. In addition to being familiar with state and national education standards, teachers are uniquely qualified to identify instructional approaches and lesson components that will appeal to today's students. Involving teachers with every phase of curriculum development provides ample opportunity for both oral and written lesson feedback and revision, including identification of areas in which teachers or students may struggle with material in the classroom. Feedback can be solicited from teachers as they experience the lessons in the professional development workshops and following implementation of the lessons with their students in the classroom. For our bioinformatics curricula, teacher feedback led to the expansion of teacher background and procedure instructions in each lesson, as well as incorporation of multiple screen capture images in student instruction handouts to help students navigate and use bioinformatics tools and databases. Improved visual aids permitted most lessons to be completed in the time allotted to the average high school class and reduced reports of student frustration. The modifications teachers made when implementing lessons were incorporated into later versions—most notably in the case of career activities, such as attending a “social mixer,” participating in mock job interviews, and writing scientific abstracts and bioinformatics magazine articles. During professional development workshops, Stars and Wishes forms, group discussions, and opportunities for teachers to critique curriculum lessons provided ongoing program feedback and assessment and helped to foster a sense of community among teachers and program staff. Respecting teachers as peers and active participants in the iterative curriculum-development process is likely to increase their sense of ownership of the curricular materials, as well as improve the likelihood of successfully meeting program goals.

Support Teachers as Students.

Following the 2-wk professional development for teachers in Summer 2010, teachers demonstrated significant gains in the areas of career awareness, self-efficacy, and engagement, and these gains were largely sustained throughout the 2010–2011 academic year. Comments from teachers revealed numerous benefits of the professional development model utilized by NWABR. Teachers “became the students,” experiencing each lesson “firsthand,” as their students would. This provided opportunities to ask questions and gain insights about how each lesson could be implemented in their classrooms, to learn how advances in technology had contributed to the materials presented, and to make more explicit connections between curriculum content and STEM careers. Teachers also noted that hands-on wet labs and computer activities provided many opportunities for them to become more familiar with computational and bioinformatics tools. This is supported by the significant retrospective postworkshop gains in teachers’ perceptions of their self-efficacy (Figure 2).

Having teachers experience the curriculum lessons helped uncover a number of teacher misconceptions related to bioinformatics and molecular biology. These included a lack of understanding or knowledge about: the presence of genes on both strands of DNA in a given chromosome; the strand of DNA (anti-sense vs. sense) used for protein transcription and translation; ATG/AUG codons not necessarily acting as “start” codons; proteins’ ability to form bonds to metal ions; and the shape of phylogenetic trees varies with the genes and organisms that are chosen for analysis. These misconceptions were explicitly addressed in later versions of the lessons and teacher professional development workshops. Ample time for teacher feedback, opportunities and diverse venues for questions, and support by program staff and guest scientists are especially valuable when teaching complex subjects such as bioinformatics.

It became clear during the workshops that teachers varied widely in their technical skills. During the Summer 2010 workshop, it became clear that there was a need for instruction in the basic computer skills necessary for bioinformatics analyses, such as finding downloaded files, copying and pasting text (DNA and protein sequences), understanding file formats, finding specific text or sequences within a document or on a Web page, capturing screen images, and bookmarking Web pages. This led to the implementation of Computer Skills 101 in subsequent Summer professional development workshops (i.e., 2011 and 2012) and additional computer instructions in lesson revisions. Many of these skills also proved to be useful for teachers in other settings. Preworkshop surveys of teacher computer skills, practice exercises on preworkshop homework, and having additional program staff and guest scientists available to assist early in the workshops also helped improve instruction. Peer mentoring (pairing more experienced teachers with less experienced teachers during workshop activities) was also an effective approach, resulting in both enhanced learning and a sense of teamwork and camaraderie.

Hands-on experiences using the tools of science, time for questions and uncovering common misconceptions that can present barriers to learning, preassessment of existing skills, and peer mentoring are all effective approaches to teacher professional development that can be implemented in a variety of STEM fields.

Promote Career Awareness Among Teachers by Including Diverse STEM Professionals.

Teachers indicated that the chance to network with professional scientists, as well as with other science teachers, was of great benefit to them. Including scientists throughout the sessions—as tour guides, members of panel discussions, and during one-on-one interactions—allowed teachers to interact with and question the scientists about their work and experiences using bioinformatics on a daily basis. Personal stories from scientists, such as how they chose their current career and what they love most (and least) about their jobs as scientists, resonated with teachers and students alike. These anecdotes help humanize scientists for both teachers and students. Exposure to diverse career professionals illustrates the many different approaches that are utilized in a particular field, and the diversity of career professionals themselves may help to dispel many of the stereotypes of scientists that persist in the American psyche.

Lessons for Promoting STEM Careers among Students

Integrate Information about STEM Professionals into Curriculum Units.

Introductory unit students made significant pre/postunit gains on survey items measuring career awareness and self-efficacy, as well as on postunit retrospective survey items measuring relevance and self-efficacy. Similar gains were found among advanced unit students. Intentional integration of careers into each curriculum lesson, as well as the culminating career lessons, helped students understand the many different careers related to genetic testing and genetic research. Interviews and photos were provided with each career to help students connect that career with a real-world STEM professional.

Diversity of STEM individuals featured was a key consideration in lesson development, with an emphasis on women and individuals from backgrounds underrepresented in STEM, including people who were the first in their families to attend college. Many students do not have access to STEM professionals as role models. On seeing the photo of the veterinarian featured in lesson 2 of the introductory curriculum holding her infant son, one female student remarked to her teacher, “I didn't know that you could be a veterinarian and have a family.” Providing diverse role models to all students can promote equity in the STEM fields. According to one Bio-ITEST teacher:

“I think being able to learn about various careers and how the learning each day is relevant to a particular career has been very valuable for my students. I struggle to incorporate this piece into my teaching on a regular basis. To have the career tie-in treated with intentionality and structured in a way that encourages students to really pay attention, was fabulous. They also were able to see an application of technology in science, and for my students, I think that was greatly valuable as well.”

Including stories or examples of STEM professionals throughout curriculum units may be an effective approach in a number of different STEM fields to promote career awareness among students.

Encourage Students to Use Authentic Scientific Tools and Approaches.

The sense of self-efficacy that can arise from the ability to use the same tools used by practicing scientists may be a key factor in encouraging young people to consider a science career. In fact, the largest pre/postunit gain on a survey item was an indicator of self-efficacy (measuring understanding of how databases that store biological information are used in research settings). A Bio-ITEST teacher noted that, while the incorporation of career profiles and work responsibilities may have increased student career awareness, actually performing the types of computer analyses used by scientists is also compelling for students:

“I think the Bio-ITEST curriculum does a good job of generating career interest, but I suspect that simply doing the work of scientists is a compelling incentive for students to pursue this pathway.”

Emphasis on the practices of science via bioinformatics databases and tools is particularly timely, given the release of A Framework for K–12 Science Education (NRC, 2011). The fact that the tools used (including BLAST, Cn3D, and FinchTV) are authentic bioinformatics tools used by scientists, not simplified or scaled-down versions made specifically for students, was also compelling:

“What they liked was that it (BLAST) wasn't a ‘made for students program.’ They got the idea that they were able to use some of these tools that real researchers are using, that they can just look up stuff and find things. They really liked it, they asked, ‘Is this a thing that scientists use?’ They liked that idea that they were learning something real, not a made-up situation.”

While postunit measures of self-efficacy were significantly higher among students who experienced both the introductory and advanced units, only the advanced postunit survey showed significant gains in student engagement. This could be a result of a number of factors. Only the advanced unit contained hands-on wet-lab activities that, coupled with the computer-based activities, may have promoted greater engagement among these students. Students participating in the introductory unit were much more likely than advanced unit students to experience significant technology problems, such as limited access to computers and required programs (e.g., Cn3D was needed to view three-dimensional molecular images; described below in Anticipate Technology Challenges in the Classroom and Develop Potential Alternatives), which may have reduced measures of engagement. In addition, students who participated in the advanced unit spent more time on the Bio-ITEST activities (a total of 10–15 lessons for the introductory plus advanced unit students vs. four to eight lessons for the introductory unit only). It is important to note, however, that self-efficacy and engagement are intrinsically related to one another. When students are engaged with lessons or activities, they may be more motivated to improve their skills, leading to increases in self-efficacy. Conversely, a sense of self-efficacy can encourage students to explore subject matter more deeply, promoting greater engagement. Among students who participated in the introductory unit, the correlation between engagement and self-efficacy increased from r = 0.50 to r = 0.63 from preunit to postunit. Pre/postunit gains on survey items measuring engagement and self-efficacy were similarly correlated. In other words, students started the introductory unit having a positive relationship between self-efficacy and engagement (i.e., if they were already self-efficacious, then they were likely to also already be more engaged). Further, this relationship grew stronger over the course of the introductory unit intervention. If a student made gains in self-efficacy, he or she also tended to make gains in engagement. Importantly, this correlation works in either direction: if a student made gains in engagement, he or she also tended to make gains in self-efficacy. Because this is a correlational study, we cannot untangle the causal direction of the two variables and suspect that for some students self-efficacy may lead to greater engagement, while for other students engagement may lead to greater self-efficacy. Additional research is needed to disentangle these effects.

Whether by means of self-efficacy or engagement, or both, providing ample opportunities for students to use the authentic tools from a particular STEM field helps them understand what a career in that STEM field might entail. Using the “real” tools of the STEM field in lieu of “student versions” may increase both student interest in science content and a sense of self-efficacy.

Use Socio-Scientific Issues and Ethical Analysis to Increase Student Interest in Science Content.

The National Education Science Standards (NRC, 1996) and A Framework for K–12 Science Education (NRC, 2011) emphasize not only science content and processes, but also the social contexts of science and the real-world decision making that students will face in their everyday lives. NWABR has successfully utilized socio-scientific issues and ethical discussion strategies to promote student interest in science content (Chowning, 2005, 2009a, 2009b). We have also shown that discussion of socio-scientific issues in science classes promotes higher-order justification skills among students and increases self-reported interest in and engagement with science content (Chowning ). In our prior program evaluations, NWABR teachers have also reported increases in student interest in science-related material when it is framed within a socio-scientific narrative. Our prior work and the input of our curriculum-development teachers provided the foundation for our selection of genetic testing as the focus of the introductory unit and informed the inclusion of bioethics lessons in both curriculum units.

Among the two pre/postunit survey items measuring changes in student perceptions of relevance, neither of the changes was significant. However, students began the unit with fairly high scores measuring their interest in how science knowledge guides ethical decision making (a measure of relevance), and these scores changed little postunit. In addition, students reported significant gains on a related retrospective question evaluating their understanding of ethical issues in genetic testing and similar gains on a retrospective question about their understanding of the connection between biology content and issues they might face in their personal lives. The second relevance question, “I think it is important for people in our society to learn about science,” also had fairly high preunit scores. These scores changed little postunit, suggesting that students did in fact believe that having society learn about science is important, and exposure to the curriculum unit did not change those beliefs. In general, when students can see the connection between the science they learn in the classroom and real-world problems, engagement increases, and the material they are learning seems relevant (Shernoff ; Siegel and Ranney, 2003). This may be particularly important for girls (Tucker et al., 2008; Modi ), who have traditionally been underrepresented in STEM. During our research, we noted the key role of teachers in facilitating STEM career awareness; this role and its implications are discussed in detail elsewhere (Cohen et al., 2013).

Challenges for Program Implementation

Anticipate Technology Challenges in the Classroom and Develop Potential Alternatives.

Some teachers were unable to implement particular lessons due (at least in part) to technology challenges, primarily lack of computer access for students and difficulty receiving school or district approval to install Cn3D and/or FinchTV on school computers. Many schools or districts require notification of the IT department weeks or months in advance of unit instruction to ensure computer availability and program installation. For some teachers, even this advanced notice was insufficient to overcome logistical issues, such as the extensive use of school computers for high-stakes testing. As a result, in some classes, computer-based lessons were covered out of their intended lesson-order sequence or were skipped entirely. Either of these approaches could compromise the curriculum narrative of a patient considering genetic testing, which is followed by “performing” the genetic test, using DNA and protein alignments to identify mutations, and then mapping the mutations to the protein structure. More than half of the teachers reported difficulty obtaining access to computers for student use. One teacher adjusted to the dearth of student computers by installing Cn3D on her personal or “teacher” computer and adapting the introductory unit molecular visualization lesson from a student-led activity to a teacher-led demonstration supplemented with PowerPoint slides. This approach still covered lesson material, but the hands-on experience for her 28 students was compromised. In classes with limited access to computers, computer activities were performed in medium (3–4) or large (8–10) groups of students working together on a single computer. The frustration of inadequate computer support may have contributed to the declines in student engagement observed in the pre/postintroductory unit survey items about analyzing biological information or using Cn3D. Students in the three classes that participated in the advanced unit showed significant retrospective postunit gains in all areas measured, including engagement. Two of these classes had no impediments to students’ access to computers. The third advanced unit class had a ratio of three to four students per computer.

The evaluation was not designed to stratify student responses based on access to or challenges with computer technology. When implementing technology-rich curricula, it is important to be aware of school and district policies regarding installation of computer programs, request access to student computers well in advance of unit instruction, and develop backup plans in the event of technology problems. Questions of student access to technology (i.e., computers) are larger policy and education funding issues with serious implications for equity among different student populations.

Balance the Challenges of Curriculum Fidelity versus Flexibility.

While the evaluation did not document permutations in implementation, it is important to note that even for those who trialed most or all of the lessons, fidelity of implementation varied. Some teachers were constrained by school or district mandates on curriculum content, and thus did not teach courses in which the material would be relevant during that academic year, or believed that the material was too difficult for their introductory biology students. Some lessons were skipped, taught out of sequence, or modified due to the technical issues described above. For those who did implement the lessons, some noted that in some cases they skipped lessons entirely, chose to introduce only parts of a lesson, and/or replaced activities with their own innovations. In some instances, these derivations led to new ideas and approaches to lesson implementation, extension exercises, and other lesson revisions. For example, the mock job interview and social networking activities in the culminating career lessons were initially developed by field-test teachers as lesson permutations prior to their incorporation into the final version of the curriculum units.

Nonetheless, the gains in student career awareness, relevance, self-efficacy, and (post–advanced unit) engagement were encouraging, given the length of the intervention (usually six to nine lessons taught in a 1- to 2-wk period) and the challenges to implementation noted above.

One can view fidelity and flexibility as two sides of the same implementation coin. It is impossible to design a curriculum for implementation in varied educational settings that is “one size fits all,” and indeed, teacher innovation was important in refining the curriculum over time. However, flexibility and teacher adaptation of lesson components has the potential to dilute what may already be a short-term intervention, which makes measuring the outcomes of the intervention more challenging.

Consider Challenges in Program Evaluation and Areas for Additional Study.

The survey instruments and career constructs utilized in this study provided a valuable means to assess program impacts on teachers and students; however, they have their limitations. The development of STEM identities among students is not a linear process, nor do all students experience to the same degree every component of the career development model that guided this study. While awareness of career possibilities is a necessary prerequisite for future pursuit of a given career, some students may be motivated to pursue additional STEM studies based on a sense of self-efficacy or mastery of the bioinformatics tools featured in the lessons (“I’m good at this, therefore I will try more of it”). Others, particularly girls, may be compelled by feelings of relevance (“this is important, this can help others”; Modi ). The limited number of teachers who participated in the research study may have impacted the ability to detect a correlation between teacher change and student change. In addition, variations in fidelity of implementation could also influence these results. Finally, there may be some areas in which a correlation might not be expected. For example, students may be more comfortable with some aspects of technology than teachers are, so some areas in which teachers needed professional development and showed gains on survey items may not be the same areas in which change among students would be expected. Future studies would benefit from a greater number of teacher participants, as well as additional refinement of survey instruments, such as inclusion of a measure of engagement on the retrospective postintroductory unit survey. It would also be valuable to determine the generalizability of these findings to other STEM fields that utilize curriculum development and teacher professional development to promote student interest in STEM careers. For example, we believe that the effects of socio-scientific discussions on student interest in course content and careers in other STEM fields warrant further investigation.

CONCLUSION

According to the National Science Board, the number of STEM workers from 1980 to 2000 increased at an average annual rate of 4.2%, while the rate of STEM degrees acquired grew at only 1.5% annually (National Science Board, 2008). PCAST found “that the problem is not just a lack of proficiency among American students; there is also a lack of interest in STEM fields among many students” (PCAST, 2010). High school science curricula that explore real-world problems and utilize authentic science tools appear to be an effective way to interest students in STEM content and promote self-efficacy. Additionally, incorporating ethical theory and discussing the socio-scientific issues arising from emerging genetic technologies appears to help students understand the relevance of the science material. Intentionally integrating career components into the curriculum helps acquaint students with the types of STEM careers available, the type of work each STEM professional performs, and the training and education requirements. Like many emerging areas of STEM, bioinformatics tools are complex and their use is challenging to teach. In addition, their user interfaces change often. However, utilizing the same bioinformatics tools that scientists use, such as the NCBI databases, BLAST, and Cn3D, promotes student interest and provides access to the wealth of biological data accumulated by scientists around the world. As one student aptly put it, “I had no idea that the general public has access to all of these databases and information.” Another Bio-ITEST student noted:

“Careers in science look more desirable than they did before, as now they are better explained … They don't seem as tedious or difficult with the added features from bioinformatics.”

Familiarity with these tools will serve these students well if they pursue careers in STEM fields. For students who do not pursue careers in STEM, understanding the applications and limitations of bioinformatics tools and emerging genetic technologies will assist them in making informed decisions about medical advances they read about in the popular press or in the ballot booth.