Scientific teaching: defining a taxonomy of observable practices.

Over the past several decades, numerous reports have been published advocating for changes to undergraduate science education. These national calls inspired the formation of the National Academies Summer Institutes on Undergraduate Education in Biology (SI), a group of regional workshops to help faculty members learn and implement interactive teaching methods. The SI curriculum promotes a pedagogical framework called Scientific Teaching (ST), which aims to bring the vitality of modern research into the classroom by engaging students in the scientific discovery process and using student data to inform the ongoing development of teaching methods. With the spread of ST, the need emerges to systematically define its components in order to establish a common description for education researchers and practitioners. We describe the development of a taxonomy detailing ST's core elements and provide data from classroom observations and faculty surveys in support of its applicability within undergraduate science courses. The final taxonomy consists of 15 pedagogical goals and 37 supporting practices, specifying observable behaviors, artifacts, and features associated with ST. This taxonomy will support future educational efforts by providing a framework for researchers studying the processes and outcomes of ST-based course transformations as well as a concise guide for faculty members developing classes.

INTRODUCTION

Recognizing the importance of undergraduate science ­education, national organizations have issued dozens of reports over the past several decades calling for dramatic alterations to undergraduate curricula and teaching methods. Written by scientists, educators, and policy leaders, these reports have three recurrent themes. First, they propose that students should learn about the nature of science and engage in scientific practices (American Association for the Advancement of Science [AAAS], 1989, 2011; National Research Council [NRC], 1999). Second, they stress the need to incorporate learning principles from the cognitive sciences and student performance data in the ongoing development of teaching methods (NRC, 2000, 2003b, 2012). Finally, they call attention to the persistent achievement gap for members of historically underrepresented groups and recommend teaching practices that promote the success and persistence of all students (NRC, 2011; President’s Council of Advisors on Science and Technology [PCAST], 2012).

These calls have had broad impacts within the life sciences community, serving as the impetus for local and national transformation efforts. In 2003, the National Research Council’s BIO2010 report initiated an important movement within biology education by specifically calling for a professional development workshop to help faculty members cultivate their teaching skills (NRC, 2003a). In 2004, this call was answered through the founding of the Summer Institutes on Undergraduate Education in Biology (SI) with support from the Howard Hughes Medical Institute (HHMI) and the National Academy of Sciences (NAS; Pfund ). Initially focused on biology faculty members at research institutions and held only once per year, the SI has since expanded to seven regional sites, and more than 1000 faculty members have attended the 5-d workshop as of 2014. These faculty members are primarily—but not exclusively—biologists, and they represent more than 200 institutions from across the country, including 2- and 4-yr colleges and almost all of the nation’s research-intensive universities. SI participants are trained to develop, implement, and disseminate innovative teaching practices at their home institutions, leading to an extensive network of faculty members who have been influenced by the SI program.

The SI curriculum promotes a pedagogical framework called Scientific Teaching (ST) (Handelsman ), an approach described in a book by the same name (Handelsman ). Reflecting the national calls from which the SI emerged, ST builds on the foundational idea that the way science is taught should reflect the way science is practiced (AAAS, 1990). ST aims to capture the spirit of scientific research by immersing students in the scientific discovery process and using evidence, either local or published, to justify the selection of teaching methods (Cross and Steadman, 1996; Angelo, 1998; Hutchings and Shulman, 1999; Handelsman ).

ST encompasses three central tenets: active learning, assessment, and inclusivity. Active learning refers to exercises in which students do something (e.g., writing, discussing, solving, or reflecting), rather than passively listening to a lecture (Crouch and Mazur, 2001; Prince, 2004; Michael, 2006; Wood, 2009; Osborne, 2010). Assessment can be used during a learning event (formative assessment) or at the completion of a unit (summative assessment), in each case providing information to students and instructors regarding student progress (Black and Wiliam, 1998; Tanner and Allen, 2004). Inclusivity embodies the idea that undergraduate science courses contain students of diverse backgrounds and that conscious efforts are required to achieve course environments that minimize potential biases and promote the success of all students (Milem, 2001; Tanner and Allen, 2007). In the past decade, ST has spread throughout the biology education community, providing an overarching framework for biology education research projects and serving as the basis for a number of professional development workshops (Ebert-May and Hodder, 2008; Miller ).

The increasing prominence of the ST approach has created a specific need to identify and define its core elements and supporting practices. ST represents a specific articulation of key educational principles pertaining to undergraduate science instruction. It is consistent with broader consensus reports, but it is distilled and packaged in a manner suitable for biology faculty with little pedagogical training. ST implementation involves complex human behaviors modified by social interactions, classroom environments, and task characteristics, such as the content and cognitive demand of a given activity (Hora and Ferrare, 2013). ST can be embodied to different degrees, with one practitioner incorporating a short classroom activity, another revamping an entire course according to the ST paradigm, and both self-reporting as engaging in ST. Some ST practices are readily apparent in the classroom environment, while other important elements are less visible, occurring behind the scenes as an instructor makes plans and adjustments throughout the semester. Several observation protocols have been developed to document classroom practices, but the degree to which they align with ST has not been defined (e.g., Piburn and Sawada, 2000; Hora and Ferrare, 2013; Smith ). Furthermore, students can engage in course-related activities either during class or outside class through homework, projects, online forums, or other exercises. For these reasons, future efforts to study ST implementation and associated student outcomes will require systematic definition of ST in a way that accounts for its diverse applications.

Taxonomy development has been used as a research methodology in many disciplines to clarify and elaborate overarching processes, structures, and goals. For example, within the medical education community, taxonomies have been used to better describe medical errors as well as to specify desired competencies for medical residents (Zhang ; Graham ). Often adopting a hierarchical organization, taxonomies use explicit criteria to systematically identify, classify, and define elements that fit within a broader structure. Specifying a given domain through taxonomy development is recommended as preparation for curriculum building, program evaluation, and instrument construction (Chatterji, 2003). In addition to informing these activities, taxonomies can also guide future research efforts by summarizing current understandings and providing a defined reference point for future studies (Bordage, 2009).

In this article, we describe the development of a taxonomy that operationalizes ST principles through the specification of observable teaching practices associated with ST. We detail the process underlying its initial construction and iterative revision, and we report on classroom observations demonstrating the applicability of the taxonomy within the course context. We also provide data from faculty surveys supporting the comprehensiveness of the taxonomy. The resulting taxonomy identifies core pedagogical goals of ST and articulates specific practices aligned with each goal. By defining observable indicators of ST practice, this taxonomy provides a common framework for researchers studying the processes and outcomes of ST-based educational transformations as well as an important resource for faculty members engaged in course transformations. This taxonomy can also serve as the basis for formal instruments designed to document the implementation of ST within a course.

METHODS

Project Scope

The overall goal of this project was to make explicit the pedagogical goals of ST and to develop a list of observable practices supporting these goals (Figure 1). Using the book Scientific Teaching as the primary guide (Handelsman ), we sought to define ST elements that apply within the undergraduate course context, acknowledging from the outset that the resulting product would not include all valuable teaching practices or address other important considerations surrounding higher education. For example, though firmly rooted in the principles of teaching inclusively and embracing diversity, ST does not extensively address issues of affordable access or student disabilities. These issues fall outside the purview of a typical instructor, being governed largely by institutional policies and student support offices. There are also general behaviors associated with quality teaching (e.g., speaking clearly, being organized, or maintaining professionalism) that are not explicitly emphasized in ST.

Figure 1.

Flowchart providing a general overview of the taxonomy development process.

Identification of Pedagogical Goals

The first part of the taxonomy development process involved identifying ST’s core pedagogical goals. Here, pedagogical goals are defined as learning processes, course structures, and classroom environments that are desired by the instructor. We began by deconstructing the book Scientific Teaching into a comprehensive list of its recommended teaching practices, using other education literature to elaborate certain topics (e.g., see references included in Table 1). We next identified the central intentions and features of each teaching recommendation. For example, the specific suggestion to employ group problem solving in the classroom could be generalized into two components: having students work together and having students solve problems. These broad components were further consolidated based on related features into a list of core pedagogical goals, which were subsequently refined and translated into student-centered terms.

Table 1.

References related to each ST goal

Course alignment
Students understand learning an row d performance expectations based on information from the instructor that defines what students should know and be able to do at course completion.	Wiggins and McTighe, 2005
	Allen and Tanner, 2007
	Mestre, 2008
	Wood, 2009
Students work to accomplish course objectives by participating in exercises and formative assessments that align with the desired outcomes.	Biggs, 2003
	Phillips et al., 2008
	Blumberg, 2009
Student achievement of course objectives is accurately measured using summative assessments that are aligned with the desired outcomes.	Danili and Reid, 2005
	Allen and Tanner, 2006
	Brilleslyper et al., 2012
	Kishbaugh et al., 2012
Students inform course curriculum decisions by providing feedback and performance data to the instructor.	Novak et al., 1999
	Richardson, 2005
Science practices
Students explore the relationship between science and society by reflecting upon science in the context of society throughout history and in the present day.	Sadler et al., 2004
	Zeidler et al., 2005
	Chamany et al., 2008
	Labov and Huddleston, 2008
	Pierret and Friedrichsen, 2009
Students use science process skills by engaging in practices integral to the performance of science.	Hanauer, et al., 2006
	Bao et al., 2009
	Coil et al., 2010
	Wei and Woodin, 2011
	Goldey et al., 2012
Students synthesize experimental results by critically evaluating multiple pieces of data and drawing conclusions based on evidence and reasoning.	Svoboda and Passmore, 2013
	Wiley and Stover, 2014
	Osborne, 2010
Students engage in formal scientific discourse by interpreting and communicating scientific ideas.	Hoskins et al., 2007
	Brownell et al., 2013
	Stanton, 2013
Student participation
Students engage in class by participating in active-learning exercises that serve as formative assessments.	Black and Wiliam, 1998
	Hake, 1998
	Prince, 2004
	Nicol and Macfarlane-Dick, 2006
	Freeman et al., 2007
	Armbruster et al., 2009
Students refine their knowledge through peer interactions by participating in small-group ­activities that require discussion.	Springer et al., 1999
	Wright and Boggs, 2002
	Tanner et al., 2003
	Smith et al., 2009
	Tanner, 2009
Students participate at the whole-class level, because the instructor provides mechanisms and formats that facilitate class-wide participation.	Nicol and Boyle, 2003
	Wood, 2004
	Crossgrove and Curran, 2008
	Kay and LeSage, 2009
Students of diverse backgrounds are affirmed as members of the class and scientific community by considering the perspectives and contributions of people with different origins, genders, and affiliations.	Steele, 1997
	Seymour, 2000
	Dasgupta and Greenwald, 2001
	Uhlmann and Cohen, 2005
	Tanner and Allen, 2007
Cognitive processes
Students practice higher-order cognitive skills by applying, analyzing, synthesizing, or evaluating evidence, concepts, or arguments.	Dori et al., 2003
	Miri et al., 2007
	Crowe et al., 2008
	DeHaan, 2009
Students transfer knowledge and skills across disciplines by utilizing skills or concepts from multiple disciplines to solve scientific problems.	Bialek and Botstein, 2004
	Labov et al., 2010
	Tra and Evans, 2010
Students learn to think metacognitively by reflecting on the effectiveness of their learning and problem-solving strategies.	Ertmer and Newby, 1996
	Pintrich, 2002
	Schraw et al., 2006
	Tanner, 2012

Elaboration of Supporting Practices

To operationalize ST in an explicit manner, we further described each pedagogical goal in terms of specific practices that support its achievement. For example, one goal is for students to “learn to think metacognitively,” a process that can manifest itself in many ways and be elicited by different kinds of activities. To elaborate each goal, we articulated a general approach to encapsulate the different ways that the goal could be achieved and then compiled a series of supporting practices that exemplify each general approach. Many supporting practices were found within the Scientific Teaching book, and we drew from our collective experiences as students and instructors to supplement these practices. Once a draft list had been developed, we conducted informal classroom observations of ST-trained instructors to identify additional practices that had been potentially overlooked. Several rounds of iterative revisions ultimately led to the production of a complete draft taxonomy.

Testing the Applicability of the Draft Taxonomy

Like many educational approaches, the nature of ST precludes definition in absolute or authoritative terms. The growing community of practitioners who implement and disseminate ST shapes its features in an ongoing manner. In light of these qualities, we conducted classroom observations to determine whether the supporting practices listed were in fact detectable within classroom environments. The intent of this exercise was not to use the taxonomy as a measurement tool but rather to ensure that the supporting practices we had compiled were being implemented in practice. Ten faculty members from biology and other science disciplines at the University of Colorado were recruited by email and observed for one class meeting each (Table 2). These faculty members were primarily, but not exclusively, former SI participants and/or had a reputation for utilizing transformed teaching practices. For each observation, two investigators recorded field notes regarding student and instructor activities. After class, each investigator separately determined which of the initial taxonomy’s 38 supporting practices occurred at least once during the class. Initial interrater agreement was 84%, and consensus was reached on the remaining items through discussion. Importantly, 89% of the supporting practices (34 of 38) were scored at least once during this series of classroom observations. These observations suggest that the supporting practices in the draft taxonomy represent a reasonable list of behaviors that are both used and observable.

Table 2.

Sample demographics for class observations and faculty surveys

Class observations (n = 10 total)
Instructors trained at SI	5
Instructors trained elsewhere	2
Instructors informally trained	3
Lower-division courses	5
Upper-division courses	5
Small enrollment (10–25 students)	4
Medium enrollment (26–100 students)	3
Large enrollment (>100 students)	3
Biology courses	7
Other STEM courses	3
Faculty surveys (n = 14 total)
Instructors trained at SI	9
Instructors trained elsewhere	5
Biology instructors	10
Other STEM instructors	4

As a supplement to classroom observations, we also conducted an online survey to determine the extent to which a sample of faculty members would recapitulate our list of supporting practices. Fourteen additional faculty members, including 11 from other institutions, who had completed SI or other similar training were recruited by email to complete a 30-min online survey administered via Qualtrics (Table 2). In the survey, participants were presented with a series of pedagogical goals along with corresponding general approaches and were asked to describe practices they use to achieve these goals in their classes. Participants were given five text-entry boxes for each goal as well as an additional box for general comments. For prevention of survey fatigue, each participant was presented with roughly half of the 17 different goals from the draft taxonomy. From this survey, we collected a total of 288 reported practices, which were subsequently reviewed to determine alignment with the existing supporting practices.

The faculty surveys allowed us to address two principle objectives: to determine whether the supporting practices on the draft taxonomy were similar to those generated by an independent group of faculty members and to identify additional supporting practices reported by faculty members. Two investigators worked together to determine the degree of alignment between the so-called faculty-generated (FG) practices and the existing practices by determining whether each FG response qualified as a general restatement or a specific example of an existing practice based on related keywords, themes, and characteristics. For example, when presented with the goal of students “using science process skills,” the FG response of having “students generate hypotheses and make predictions” was judged to be analogous to the existing practice of having “students identify, construct, or evaluate hypotheses and make predictions based on their hypotheses.” One investigator initially aligned the FG practices to the existing supporting practices, a second investigator reviewed all of the assignments, and then the two investigators discussed any disagreements. Again, 89% of the existing practices (34 of 38) were aligned with one or more FG responses, providing confirmation that the existing practices could largely be corroborated by faculty practitioners. Faculty members employ similar practices to achieve certain pedagogical goals, and many supporting practices were therefore corroborated by several FG responses. While the majority of the FG practices could be paired with an existing practice, roughly 10% (28 of 288) did not align with an existing practice for the pedagogical goal under which they were originally submitted. Among these, some aligned with other pedagogical goals, while others were deemed to be outside the scope of ST. The few remaining FG practices were added during a final round of taxonomy revisions.

Final Taxonomy Revisions

While the draft taxonomy (with 38 practices) showed considerable alignment with observed and reported teaching practices, our efforts revealed a few areas for further revision. Language throughout the taxonomy was clarified to be more parsimonious, including the merging of two pairs of pedagogical goals based on overlap within faculty responses (e.g., faculty members did not make distinctions between collaborative and cooperative learning approaches). Four supporting practices were removed because they were redundant with other items, occurred outside the observable course context, or were not reported by faculty members. Three new supporting practices were added to reflect previously unlisted FG practices. As an example, for the goal of “affirming students of diverse backgrounds,” multiple respondents mentioned “employing mechanisms to enhance diversity within student groups.” This practice was added to the taxonomy. After final taxonomy revisions, only one supporting practice remained that had not been observed in the classroom or mentioned on faculty surveys. This was having “students analyze data using appropriate methods.” Because this practice had been identified in national reports as an important component of developing science process skills (NRC, 2003a; AAAS, 2011), it was retained in the final taxonomy (37 practices altogether).

RESULTS

Taxonomy Structure

The final ST taxonomy consists of a series of 15 pedagogical goals, 15 general approaches, and 37 supporting practices arranged in a hierarchical manner (Table 3). The taxonomy operationalizes ST by identifying its core elements and elaborating explicit behaviors, artifacts, and features associated with each element. Several aspects of the taxonomy reflect the ongoing evolution of ST since its original publication. For example, while ST was developed within the context of biology education, the taxonomy maintains its applicability throughout the sciences by utilizing interdisciplinary language. Furthermore, the SI curriculum and Scientific Teaching book are geared toward instructors, and they fittingly describe actions that instructors can take to build productive learning environments for their students. In contrast, the taxonomy is phrased with an explicit focus on student actions and perceptions. This student-centered language is not intended to diminish the importance of the instructor role but rather to emphasize that ST is ultimately about what students do and perceive.

Table 3.

The complete taxonomy of observable ST practices

Pedagogical goal: a particular learning process, structure, or environment desired by the instructor	General approach: a general statement of how the given pedagogical goal will be achieved	Supporting practices: specific actions, materials, or capabilities that exemplify the general approach
Course alignment
Students understand learning and performance expectations	based on information from the instructor that defines what students should know and be able to do at course completion.	1. Students are provided learning goals detailing conceptual understandings, content knowledge, and process skills they are expected to master.
Students work to accomplish course objectives	by participating in exercises and formative assessments that align with the desired outcomes.	2. Students are able to connect activities and formative assessments with specific learning objectives.
Student achievement of course objectives is accurately measured	by using summative assessments that are aligned with the desired outcomes.	3. Students are able to connect material on summative assessments to specific learning objectives.
		4. Student summative assessments use different formats or multiple types of answer input.
Students inform course curriculum decisions	by providing feedback and performance data to the instructor.	5. Students are given the opportunity to provide feedback on course structure and content.
		6. Students ask questions or state interests that are pursued during class.
		7. Students are given supporting activities when assessment reveals a problem area.
Science practices
Students explore the relationship between science and society	by reflecting upon science in the context of society throughout history and in the present day.	8. Students use historical information to recognize why certain discoveries represent paradigm shifts or major technological advancements.
		9. Students relate scientific concepts to everyday phenomena or human experiences.
		10. Students utilize scientific judgment to address challenges facing nature or society.
Students use science process skills	by engaging in practices integral to the performance of science.	11. Students identify, construct, or evaluate hypotheses and make predictions based on their hypotheses.
		12. Students design and evaluate experimental strategies.
		13. Students analyze data using appropriate methods, such as descriptive or inductive statistics.
		14. Students construct graphs or tables and analyze results presented in these formats.
Students synthesize experimental results	by critically evaluating multiple pieces of data and drawing conclusions based on evidence and reasoning.	15. Students formulate or evaluate conceptual models based on data and inference.
		16. Students attempt to reconcile conflicting pieces of data.
		17. Students develop arguments or make decisions based on experimental data.
Students engage in formal scientific discourse	by interpreting and communicating scientific ideas.	18. Students read and evaluate scientific literature, including peer-reviewed and popular media articles.
		19. Students present scientific ideas in written or oral formats.
Student participation
Students engage in class	by participating in active-learning exercises that serve as formative assessments.	20. Students answer questions, solve problems, or construct representations.
		21. Students complete formative assessment activities and receive feedback on their answers.
Students refine their knowledge through peer interactions	by participating in small-group activities that require discussion.	22. Students complete worksheets, discuss problems, and perform other activities in groups of two or more.
		23. Students provide peer feedback on projects, assessments, or other activities.
		24. Students complete tasks wherein the success of the group involves the participation of each group member.
Students participate at the whole-class level	because the instructor provides mechanisms and formats that facilitate class-wide participation.	25. Students use an audience response system or other polling method to answer content questions.
		26. Students report the results of group work to the whole class.
		27. Students are encouraged to respond to other student ideas.
Students of diverse backgrounds are affirmed as members of the class and scientific community	by considering the perspectives and contributions of people with different origins, genders, and affiliations.	28. Students consider contributions of diverse people and perspectives in the realm of scientific discovery.
		29. Students utilize examples and analogies that reflect diverse people and cultures.
		30. Students are grouped using mechanisms that enhance the diversity of each group.
		31. Students are aware of instructor sensitivity to socially controversial issues.
Cognitive processes
Students practice higher-order cognitive skills	by applying, analyzing, synthesizing, or evaluating evidence, concepts, or arguments.	32. Students incorporate lower-order knowledge into higher-order cognitive skills development.
		33. Students interpret or construct conceptual representations in a variety of formats, including video, pictorial, graphic, or mathematical.
		34. Students engage in structured, open-ended inquiry exercises, such as case-based or problem-based activities.
Students transfer knowledge and skills across disciplines	by utilizing skills or concepts from multiple disciplines to solve scientific problems.	35. Students apply knowledge from mathematics, computer science, biology, chemistry, physics, or other disciplines within the context of a different discipline.
Students learn to think metacognitively	by reflecting on the effectiveness of their learning and problem-solving strategies.	36. Students consider assumptions, appropriateness of skills utilized, or thought processes when solving problems or answering questions.
		37. Students reflect on the effectiveness of their study habits.

While ST is traditionally defined according to the general tenets of active learning, assessment, and inclusivity, the taxonomy is divided more specifically into four sections pertaining to course alignment, science practices, student participation, and cognitive processes. The course alignment section focuses on the interrelation of three different curricular components: learning goals, instructional activities, and summative assessments. Learning goals define what students should know and be able to do upon course completion, instructional activities provide students with opportunities to accomplish this learning, and summative assessments gauge the degree to which students achieved the original goals. ST advocates that instructors explicitly communicate their learning goals to ensure that students are aware of the conceptual understandings, content knowledge, and process skills they are expected to master. ST also stresses the requirement that activities and assessments must align with course learning goals to provide suitable avenues for intended learning and accurate measures of student achievement. This alignment creates an important feedback loop that enables instructors to use student data to help make decisions on how to spend valuable class time and improve their teaching methods.

The science practices section elaborates the idea that undergraduate science courses should capture the spirit of scientific discovery by engaging students in scientific processes. This differentiates ST from more general pedagogical approaches (e.g., student-centered learning or team-based learning) that do not explicitly focus on incorporating scientific reasoning. By articulating that students should explore the relationship between science and society, engage in experimental design and interpretation, and participate in formal scientific discourse, this section addresses the dual intentions of preparing scientifically literate citizens as well as training future scientists.

The student participation section of the taxonomy contains pedagogical goals related to how students participate within a course. These goals embody a constructivist approach by acknowledging the importance of student knowledge and the value of enabling students to build their own mental models through active engagement and peer feedback. One pedagogical goal describes involving students at the whole-class level through the use of classroom response systems such as clickers or the reporting out of group work. Another goal in this section serves to help students feel affirmed as members of the class and the larger scientific community, irrespective of their backgrounds or future career aspirations. Instructors are challenged to demonstrate “cultural competence” by adopting teaching materials and practices that help students from diverse backgrounds self-identify as science practitioners (Tanner and Allen, 2007).

The final section of the taxonomy focuses on different types of cognitive processes that should be cultivated. For students to be innovative and productive members of the future workforce, they must be able to apply, analyze, synthesize, and evaluate disparate pieces of information and to remember and comprehend essential definitions and processes (Bloom ; Anderson and Krathwohl, 2001). The increasingly interdisciplinary nature of modern research fields has created demand for the ability to integrate concepts across disciplinary boundaries. Furthermore, students need to be encouraged to develop metacognitive habits that allow them to self-reflect in order to optimize their problem-solving and study skills (Tanner, 2012).

To visualize the relationship between the taxonomy and ST’s traditional tenets, we categorized each supporting practice as supporting active learning, assessment, or inclusivity (Figure 2). Not surprisingly, we found that there was extensive overlap between these tenets, with most supporting practices addressing multiple tenets. Scientific Teaching itself suggests that active learning and assessment are inextricably linked, in that active learning incorporates assessment of student understanding, and assessment necessarily elicits active student engagement. This is borne out in the taxonomy, as there are 27 practices that address both active learning and assessment. ST is intended to be inclusive, and there is growing evidence that implementation of interactive teaching practices and the development of science process skills can have beneficial outcomes for members of traditionally underrepresented groups (Dirks and Cunningham, 2006; Haak ; NRC, 2012). Within the taxonomy, we found that 21 supporting practices were related to inclusive teaching. While these designations are not intended to be absolute, they do serve to illustrate the relationships between the traditional ST tenets.

Figure 2.

Venn diagram showing the classification of supporting practices under the ST pillars of active learning, assessment, and inclusivity. Numbers from the ST taxonomy are used to indicate the categorization of each supporting practice.

ST consolidates research-based practices from across the science education literature, but it contains particular terminology and emphases that make it unique. After constructing the final taxonomy, we sought to align the ST taxonomy to the broader education literature by collecting reviews and research articles describing and justifying ST approaches. Table 1 contains a listing of several references related to each pedagogical goal. These citations support the consistency of the ST pedagogical goals with broader educational dialogue. Most articles apply to more than one pedagogical goal, and so the designation of an article under a particular goal does not negate its applicability to other parts of the taxonomy. For example, in addition to an overall focus on helping students develop higher-order cognitive skills, the Blooming Biology Tool described by Crowe has applications for course alignment, activity development, assessment, and student metacognition. We hope that this table will provide a helpful starting point for practitioners wanting to learn more about ST’s research foundations.

DISCUSSION

In response to ongoing national calls, ST was formulated as a way to help scientists bring their expertise into the classroom in more authentic and productive ways to improve student learning. With numerous copies of Scientific Teaching in circulation and hundreds of educators being trained in ST-based programs each year, ST has achieved significant influence within the education community. Future efforts to monitor ST’s use and impact will depend on having ways to identify its application. In this study, we have operationalized ST practices through the development of a taxonomy that identifies the core goals of ST and defines observable practices associated with each goal, thus providing faculty members with a concise, inclusive reference guide representing key ST elements.

To achieve the goal of defining ST in explicit terms, we took the viewpoint of an objective observer with course access similar to that of a student (i.e., someone who can attend class, download course materials, take exams, etc.). While many aspects of ST remain hidden from such an observer, we attempted to identify how these hidden elements are manifested. For example, ST advocates the use of a curriculum design process called backward design (Wiggins and McTighe, 2005). During the backward design process, an instructor first drafts course learning goals, then considers evidence that would indicate student mastery of the learning goals, and finally designs instructional activities that allow students to reach the desired performance level. While this design process provides a valuable guide, it is not possible to objectively determine whether an instructor’s method was in fact “backward” without being present throughout course construction. Nonetheless, the desired outcome of a rational course design can be judged through the provision of written learning goals to students and alignment of subsequent activities and assessments to these goals.

The ST taxonomy is designed to have broad relevance within the context of undergraduate science courses. ST can be used in many different parts of a course, including lecture, lab, recitation, homework, and online forums. The taxonomy describes student actions and experiences that can occur within any of these different settings and includes practices that are found across a large range of course sizes. While some ST practices are easier to implement in smaller classes, all of the practices listed in the taxonomy are feasible in a large-enrollment class, particularly with the aid of instructional technologies or additional personnel (Caldwell, 2007; Otero ). Finally, the taxonomy is intended to be applicable across the sciences, and efforts were made during the validation process to include perspectives from non–biology disciplines. However, given ST’s historical roots and current prominence within biology, it remains possible that the taxonomy does not fully account for teaching practices unique to other disciplines. Understanding these pedagogical differences remains an important research question, particularly in light of ongoing efforts to develop interdisciplinary courses and learning environments (Meredith and Redish, 2013; O’Shea ; Thompson ).

The taxonomy addresses two important issues related to conducting research on the use of transformed teaching practices. First, previous efforts to gauge teaching practices have been criticized for an overreliance on self-reported data, which may not accurately reflect actual classroom practices (Kane ; Ebert-May ). Second, many previous studies on the implementation of research-based instructional practices have focused on the use of specific strategies (e.g., clicker questions) that each contain a number of different components (e.g., question content, peer discussion, group sharing, etc.). Instructors tend to adapt these strategies according to their own classroom needs, resulting in a wide range of different practices that recapitulate the original design with varying degrees of fidelity (Turpen and Finkelstein, 2009; Dancy and Henderson, 2010). By specifying a comprehensive list of practices indicative of ST, the taxonomy begins to address each of these issues by laying the groundwork for the development of observation-based rubrics that separately identify the different layers present within any given course exercise.

While the items on the taxonomy are written in explicit terms, the taxonomy is not intended as a formal observation protocol for classroom evaluation. We anticipate that the taxonomy will serve as the basis for the development of such instruments, which will require additional delineation of scoring mechanisms, response scales, and measurement criteria. The taxonomy is consistent with the frameworks underlying several existing observation protocols, but also identifies elements unique to ST. Developed for K–12 classrooms, the Reformed Teaching Observation Protocol, or RTOP, captures the student-centeredness of a classroom, and most of its items are consistent with the goals and practices listed in the ST taxonomy (Piburn and Sawada, 2000). The Teaching Dimensions Observation Protocol, or TDOP, is based on systems-of-practice theory and accounts for different dimensions of the classroom environment, including teaching methods, pedagogical strategies, student–teacher interactions, cognitive engagement, and instructional technology (Hora and Ferrare, 2013). The TDOP thus includes codes that capture elements of ST as well as practices beyond the scope of ST. Developed under the TDOP framework, the Classroom Observation Protocol for Undergraduate STEM, or COPUS, focuses on faculty and student behaviors, but does not include codes to capture course alignment, science practices, or cognitive processes (Smith ).

In its current form, the taxonomy has several applications for instructors and departments working to improve their educational programs. First, individual faculty members can use the taxonomy to 1) self-assess their own teaching and identify ways to diversify their courses; 2) report and justify the use of transformed teaching practices to promotion and tenure committees; and 3) communicate the rationale behind instructional decisions to students (e.g., on the first day of class in discussing course goals and format). Second, as a professional development tool, the taxonomy can be used to informally document classroom practices and to facilitate dialogue with course instructors (see Supplemental Material 1 and 2). Third, the taxonomy can provide a basis for pedagogy-related conversations at the departmental level: 1) departments engaged in curricular reviews can identify when, where, and how their students are addressing each pedagogical goal within their program; 2) faculty members can use the taxonomy as a common frame of reference drawn from the education literature to guide formative peer feedback (Gormally ), rather than invoking outdated or subjective ideas of what constitutes “good” teaching; and 3) since the content of the taxonomy complements the NSF Pulse Fellows Vision and Change rubrics, it can be used as part of departmental efforts to self-evaluate awareness, acceptance, and use of transformed teaching practices (Aguirre ).

While the ST taxonomy has numerous applications for researchers and instructors with an ST background, we propose that the taxonomy is also useful for individuals engaged in other reform initiatives. The language of the taxonomy is general in nature and captures many key ideas related to educational reform. Thus, we foresee the taxonomy serving as a theoretical underpinning for researchers studying the outcomes of general professional development workshops or other efforts of a more comprehensive nature. We also expect that the taxonomy will be useful for introducing instructors to different aspects of course transformation and providing a concise representation of research-based educational practices. The instructional uses described above would apply equally well for faculty members who have received ST training, non-ST training, or no educational training.

The taxonomy presented in this article identifies and defines observable practices associated with ST. It is not expected that all the goals listed on the taxonomy would be addressed in a single class period or assignment, as the suitability of these practices depends on the scope and goals of a course. Finally, the taxonomy represents an articulation of ST in its current state. Staying true to ST principles, the taxonomy should evolve as future research efforts lead to a deeper understanding of how different teaching practices affect student outcomes. Please contact the corresponding 
author (B.A.C.) if you would like a copy of the taxonomy that has been formatted as a one-page handout.