Identifying and using 'core competencies' to help design and assess undergraduate neuroscience curricula.

There has been a growing emphasis on the use of core competencies to design and inform curricula. Based on our Faculty for Undergraduate Neuroscience workshop at Pomona we developed a set of neuroscience core competencies. Following the workshop, faculty members were asked to complete an online survey to determine which core competencies are considered most essential and the results are presented. Backward Design principles are then described and we discuss how core competencies, through a backward design process, can be used to design and assess an undergraduate neuroscience curriculum. Oberlin College is used as a case study to describe the use of core competencies to help develop learning objectives, activities, and assessment measures for an undergraduate neuroscience major.

As the popularity of Neuroscience at the undergraduate level has grown, many institutions are looking to expand and revise existing curricula. This issue has been addressed at Faculty for Undergraduate Neuroscience (FUN) workshops in the past (Ramirez, et al., 1998; Wiertelak and Ramirez, 2008), but the increased emphasis on defining and assessing learning outcomes suggested to us that it would be useful to reexamine how to go about designing an effective curriculum with clear learning goals. The last decade has also seen a growing emphasis on using core competencies to define and inform curricula. However, there is no defined set of core competencies for Neuroscience programs. For our session at the Pomona workshop, we chose to examine what a set of core competencies for Neuroscience might include and how to use those competencies, through backward design, to design or modify a Neuroscience curriculum.

There are many aspects to designing and implementing a Neuroscience curriculum at the undergraduate level. These include such considerations as (1) the goals of the program in terms of core competencies or learning outcomes, (2) the type of program (major, minor, concentration, etc.), (3) the resources available in terms of faculty and infrastructure, and (4) the location of the program as a stand-alone department or an interdisciplinary program administered through multiple departments. Each of these factors warrants discussion, but we will limit our focus to the first aspect, the development of core competencies.

Thoughtful design of a curriculum must consider what skills students should master by the time they complete the program. For undergraduates, it may be useful to consider where students go after completing their bachelor’s degree. In 2009, FUN faculty members were asked to participate in a survey about many aspects of undergraduate neuroscience programs (Hardwick and Smith, 2010). This survey included questions about the percentage of students that went on to medical schools, PhD programs, other health professions (i.e. nursing, dental, etc.), masters programs, laboratory technician positions or other. There was considerable variability in the responses, but the overall trend was that a significant majority of students were going on to medical school (26.8%), PhD programs (20.2%), or something completely different (24.8%). Thus it is useful to consider what common skills students need for success in these divergent areas.

The American Association of Medical Colleges, in conjunction with the Howard Hughes Medical Institute, recently re-examined the core competencies that they perceive as crucial for an undergraduate premedical curriculum (www.aamc.org/scientificfoundations). For undergraduate programs, these were broken down into eight specific competencies, with six being content specific and two that were more general; quantitative reasoning and scientific inquiry.

Graduate programs in neuroscience do not have such a comprehensive document, but recent workshops at the Society for Neuroscience meeting and informal discussions with directors of graduate programs have suggested a tentative list of desirable attributes that include (a) research experience, (b) critical thinking skills, (c) quantitative and analytical skills, (d) foundational coursework, and (e) the ability to work and learn independently. A recent commentary by Steven Mennerick in the outlines one graduate program (University of Washington St Louis) and emphasized the importance of critical thinking and independent research experience as crucial for success in a graduate program, rather than specific undergraduate course work.

In the broader context of liberal education, the American Association of Colleges and Universities, has developed guidelines for interdisciplinary learning programs with a focus on liberal education. LEAP (Liberal Education and America’s Promise) outlines basic competencies and skills that include integrated learning, application of learning, and social responsibility. The LEAP initiative, which includes participants from over 300 colleges and universities, emphasizes the general competencies that should be included in any liberal arts undergraduate program.

Combining this information with our own experiences in neuroscience education, we developed a list of the following core competencies for an undergraduate neuroscience program:

Independent thinkers, self-motivated learners

Basic knowledge in Neuroscience/Biology/Chemistry/Psychology

Ability to think critically and integratively

Quantitative skills

Scientific inquiry/analytical skills/research skills

Communication skills

In order to determine if this list represents the consensus of a larger population of undergraduate neuroscience faculty, we developed a survey on these core competencies and asked members of the Faculty for Undergraduate Neuroscience to respond. The survey (see supplemental material) was completed by 203 faculty from 128 institutions (not all respondents indicated their institutional affiliation on the survey). There was just one community college represented among the participants. Approximately half of the survey respondents were from institutions offering post-baccalaureate degrees in some discipline. Of those respondents who reported their institution had a graduate program in neuroscience (N = 167), 13.3% offered a masters degree and 24.1% offered a doctoral degree.

Summary of the Survey on Core Competencies

The first question in the survey required respondents to rank the core competencies from most to least essential; from 1 to 6, respectively. The core competency that was judged by the greatest proportion of respondents (37.9%) to be most essential (rank M = 2.22) was the ability to engage in critical and integrative thinking (see Figure 1). Basic neuroscience knowledge was ranked a close second (rank M = 2.84). Combined, 70% of respondents regarded these two components of curriculum to be the most essential competencies for students in undergraduate neuroscience programs to achieve.

Figure 1.

Respondents were asked to rank each of the core competencies with regard to how essential they were for students to master as part of an undergraduate neuroscience program. The graph summarizes the percentage of respondents that regarded the respective core competency as the most essential.

The smallest proportion of respondents regarded communication skills (3.4% of respondents; rank M = 4.60) and quantitative skills (0.5%; rank M = 4.71) as most essential. In fact, quantitative abilities were judged overall to be the least essential of the core competencies by the greatest proportion of respondents (37.5%; rank M = 4.71) (see Figure 1).

To further examine each core competency we asked the survey respondents to rate the importance of various skills or knowledge that contribute to each core competency. In these instances, respondents were not forced to rank these skills; rather they were asked to rate the importance of the component on a scale from essential (1) to non-essential (6). For example, understanding of the cellular and molecular function of neurons, including neuronal communication (77%; M = 1.27) and a basic understanding of neuroanatomy (44.4%; M = 1.90) were rated as most critical components of basic knowledge in neuroscience (see Figure 2). The most valuable competency for students to acquire in support of critical and integrative aspects of thinking was the ability to read and analyze peer-review primary research papers (62.7%; M = 1.42, see Figure 3). Critical thinking was regarded as somewhat more essential than integrative thinking (43.0% vs. 35.8%; M = 1.74 vs. 1.88).

Figure 2.

The proportion of survey respondents indicating they regarded each aspects of basic knowledge in neuroscience to be an essential component of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%.

Figure 3.

The proportion of survey respondents indicating which aspects of critical thinking they regarded to be essential components of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%.

Obtaining proficiency in a wide range of neuroscience research techniques was considered to be relatively non-essential (only 7.8% of respondents regarded this as most essential; M = 2.72). Rather, the abilities to develop hypotheses as well as design experiments to test such hypotheses (72.5%; M = 1.34), and to collect, analyze and interpret the resulting data (50.8%; M = 1.60) were regarded as essential components of core research competencies (see Figure 4).

Figure 4.

The proportion of survey respondents indicating which aspects of research skill they regarded to be essential components of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%.

Independent and self-motivated learning were abilities that respondents generally regarded overall as moderately essential for students to develop. Nearly fifty percent (49.7%; M = 1.63) of respondents regarded the most essential aspect of this ability to be capable of answering questions that may not be specifically addressed in their coursework (see Figure 5).

Figure 5.

The proportion of survey respondents indicating which aspects of independent and self-motivated learning they regarded to be essential components of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%.

Students’ ability to communicate scientific knowledge to the lay public was not typically regarded as an essential skill (25.4%; M = 2.21). However, a majority of all respondents indicated that both the ability to clearly convey information orally (49.7%; M = 1.62) and in writing (40.4%; M = 1.80) are essential communication skills for undergraduate neuroscientists to master (see Figure 6).

Figure 6.

The proportion of survey respondents indicating which aspects of communication ability they regarded to be essential components of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%.

While quantitative competencies had been ranked overall as relatively less essential than the other five core competencies, the ability to interpret quantitative information was regarded by 76.0% (M = 1.29) of respondents as an essential component of quantitative competency (see Figure 7).

Figure 7.

The proportion of survey respondents indicating which aspects of quantitative ability they regarded to be essential components of undergraduate neuroscience instruction. Since each respondent was allowed to rate more than one component as essential, the percentages do not sum to 100%

Respondents were also asked to indicate which of the six core competencies they used to guide their curriculum development, or assess the accomplishments of students and the program (see Table 1). With regard to curricular development, respondents indicated that they primarily utilize assessments of basic knowledge in neuroscience (74.3%). Far fewer (34.5%) assess student independent thinking/self-motivated learning for this purpose. All six of the core competencies were commonly employed to assess student performance. Between 45% and 60% of the respondents indicated that assessments of their students’ proficiencies included measures of research skills, quantitative skills, critical/integrative thinking, and communication skills. In comparison, of the six core competencies, independent thinking and self-motivated learning were least likely to be employed to assess student learning or as a component of curricular planning initiatives and assessment of the overall program. Use of the core competencies for purposes of program assessment was relatively uncommon. Notably, between 10% and 20% of those responding to the survey did not employ any of the core competencies for assessment purposes.

Table 1.

The percentage of respondents reporting the use of core competencies for assessment purposes.

	Curricular Development	Student Assessment	Program Assessment	NA
Independent Thinking - Self-Motivated Learning	34.5%	45.3%	19.4%	37.4%
Basic Neuroscience Knowledge	74.3%	68.4%	40.1%	11.8%
Critical/Integrative Thinking	54.3%	66.9%	38.4%	17.9%
Quantitative Skills	57.2%	64.1%	28.3%	17.2%
Scientific Inquiry/Research Skills	57.4%	63.5%	39.2%	18.9%
Communication Skills	49.3%	72.7%	38.0%	14.0%

Because each core competency could be used in multiple assessment practices, percentages listed either in rows or columns do not sum to 100 (N = 153).

Backward Design Principles, Curriculum Planning and Assessment

It seems likely that development and implementation of the undergraduate neuroscience programs at most of the institutions represented in the present survey did not originate from a consensus among faculty regarding the core competencies an undergraduate neuroscience major should possess. It is more likely that the curriculum in the majority of undergraduate programs began with an emphasis upon specific courses that could be offered given existing faculty expertise. As the programs flourished and grew, more courses were added. In each of the previous FUN workshops discussion regarding neuroscience curricular blueprints remained largely focused on the course offerings themselves rather than upon any emergent consensus regarding core neuroscience competencies (Kerchner, 2005; Ramirez et al., 1998; Wiertelak and Ramirez, 2008). An alternative means of programmatic design and development would work backwards from the neuroscience core competencies that faculty deem essential for all students to determine the scope of the courses in the program and the student learning goals that are the focus within each course. An added advantage of such backward design is that it facilitates the formulation of meaningful formative and substantive assessments of the curriculum (Wiggins & McTighe, 1998).

Wiggins and McTighe (1998, 2005) outline three essential stages of any backward design process; 1) identifying the desired results, 2) determining the acceptable evidence of the desired outcomes, and 3) planning and designing the learning objectives, experiences and pedagogies of instruction that facilitate the desired learning objectives. In the initial stages of the process they suggest that it is helpful to distinguish between what is acceptable for students to gain a general familiarity with but which may not be important, what it is that it is important for them to know, and what it is that should be acquired as an enduring understanding. The latter constitute what may be considered the core competencies. Generally, backward design process is applied to the curriculum at the level of each course, but it can also be used to develop curricula at all levels. Indeed, it is wise to insure that all courses, majors and programs are aligned with the core mission of the institution.

Given the survey results summarized above, how might they be used to shape curricula? For the sake of argument, consider that the survey results represent a consensus view among undergraduate neuroscience faculty regarding the relative importance of the six core competencies in preparing students for careers in neuroscience. One approach toward curricular design/redesign would be to insure that the emphasis within the curriculum reflected these survey results. In other words, the curriculum would emphasize critical/integrative thinking as well as basic neuroscience knowledge. Likewise, cellular and molecular knowledge regarding the function of neurons as well as neuroanatomy would be the focus of coursework intended to strengthen student competencies in basic neuroscience knowledge. Courses intended to hone scientific inquiry and research skills would also focus on developing skills in experimental design, hypothesis testing, analysis and interpretation of results. Perhaps it should be that all courses revisit and emphasize these core competencies in some manner. Conversely, it might not be necessary for the curriculum to insure that all students acquire skill utilizing research technologies across multiple levels of inquiry, from the molecular to the behavioral (e.g., qPCR, stereotaxic surgery, electrophysiology, operant conditioning). This does not mean that the curriculum should not include opportunities for familiarity or mastery of these research tools, only that they should not drive the design of the curriculum. They may be best employed in instances in which they serve learning goals that should be the most enduring.

To complete the backward design process, assessment of students learning must be appropriate and timely. Assessments should at least reflect the emphasis placed upon the core competencies. When this is not the case it may mean that the core competencies and the curriculum are misaligned. According to our survey results, critical/integrative thinking is highly valued among the core competencies, even more highly than basic neuroscience knowledge, while quantitative skills are considered less essential. However, compared to assessments of basic knowledge and quantitative skills, the assessment of critical/integrative thinking is less likely to be a component of ongoing assessments within those neuroscience programs represented in our survey. The Graduate Record Examination (GRE) was revised in 2011 to include three new assessments that encompass dimensions skills related to critical/integrative and analytical thinking (Verbal Reasoning, Quantitative Reasoning, and Analytical Writing). Given that admission criteria for many graduate programs employ minimum GRE scores, incorporating greater emphasis on critical/integrative thinking learning and its assessment in neuroscience curricula would provide a tangible service to our students.

The process of backward design is frequently used to determine how courses may be organized, e.g., determining what texts or teaching formats (pedagogies) to employ, what topics to be covered or those that may be omitted, and in the construction of assessment rubrics. But in addition, we would like to encourage use of backward design as a strategy for developing and assessing programs. In the next section, we describe an example of this use at Oberlin College, one of the first undergraduate neuroscience programs.

Oberlin College: Core competencies, learning objectives, activities, and assessment measures

Oberlin College is a 4-year undergraduate liberal arts college with approximately 2300 students. The field of neuroscience first came on the scene at Oberlin College in the late 1960s when two psychology professors noted that numerous students wanted to do an independent major that combined psychology and biology. A psychobiology major was established, with the first majors graduating in 1972. Subsequently both neuroscience and biopsychology majors were offered as part of the Neuroscience/ Biopsychology Program. By the 1990s these two majors were coalesced into a single major called neuroscience and the Neuroscience Department emerged. Currently, Neuroscience is the second largest major in the natural sciences and graduates 30–40 majors per year. There are currently 5.5 teaching positions in Neuroscience, shared by seven faculty members.

Oberlin neuroscience faculty members have had numerous curriculum-centered discussions. The faculty members engaged in Backward Design and developed written documents that helped make implicit goals more explicit. First, decisions were made about what neuroscience majors should learn, i.e. what goals or core competencies they should achieve. Secondly, those goals were defined in behavioral terms (i.e. learning objectives were developed). Third, faculty members determined what activities the faculty members and department would offer to help students achieve those goals. Lastly some assessment measures were developed to better determine if students have indeed achieved the goals or core competencies. These four inter-related steps are discussed more fully below.

Core Competencies.

Faculty members developed a number of goals or ‘core competencies’ for Oberlin College neuroscience majors (See Table 2). These include: Understand the basics of neuroscience; acquire depth in cellular/systems and behavioral/cognitive neuroscience; learn some basic and advanced laboratory techniques; acquire critical thinking and analytical reasoning skills; acquire oral and written communication skills. Not all of these goals are mutually exclusive: a competency in advanced lab techniques (e.g., data analysis and interpretation skills) also advances analytical reasoning skills. Note that these are the goals that the department has for the majors; goals for an individual class may vary and may include fewer or additional goals.

Table 2.

Neuroscience curriculum at Oberlin College.

Core Competencies*	Learning Objectives for majors	Activities by Neuroscience Department Faculty	Assessment Measures
Basic knowledge in neuroscience.	Able to correctly answer questions about basic concepts in neuroscience.	Lectures and readings in intro and advanced neuroscience courses.	Pre/post quiz in Intro.Exams in courses.Senior survey.
Acquire depth of knowledge in neuroscience.	Able to correctly answer questions about cellular/systems and behavioral/cognitive areas of neuroscience.	Advanced courses: lectures, readings, discussions, papers.Department speakers.	Exams in courses in these areas.Senior survey.
Learn some basic and advanced laboratory techniques.	Can demonstrate conceptual understanding and procedural knowledge of techniques.Can do basic data analysis, graphing, and interpretation.Can work cooperatively.	Introductory lab class.Advanced lab courses.Research experiences.	Homework and exercises in intro and advanced lab courses.Senior survey.
Acquire critical thinking and analytical reasoning skills.	Has an understanding of scientific methodology and experimental design.Able to read a primary scientific article with understanding, summarize it and analyze its strengths and weaknesses.Able to review and integrate a body of scientific literature.Knows how to find relevant scientific literature.Understands what constitutes good evidence in science.	Lectures in courses.Require analyses of scientific articles in advanced courses.Activities, homework in lab classes.Require papers, presentations and discussions in Senior Seminar.Research experiences.	Assessment of senior seminar papers.
Acquire communication skills; oral and written.	Able to clearly write and speak about scientific data from review and original research articles.	Require written analyses in intro course.Require written analyses, discussions and presentations in advanced courses and Senior Seminar.Research experiences.	Assessment of senior seminar papers.

Also knowledge of chemistry, biology, and statistics ascertained by successful completion of courses in those departments.

In addition to the more curricular-oriented skills and knowledge goals noted above, the department also has a number of attitudinal goals for student majors. These include that students are satisfied with the advising they received, they are satisfied with the department and major, they feel they know about postgraduate careers and career choices (including graduate school) and they feel supported in their postgraduate plans.

Although these goals do not explicitly include such worthy items as social responsibility, integrated cross-disciplinary learning, intercultural knowledge, and civic engagement, many of these are part of the college’s goals as a liberal arts college.

The Learning Objectives.

Learning objectives are basically core competencies that have been defined more specifically and in terms of observable behaviors that can be measured. They are sometimes called objectives or measurable outcomes. Defining core competencies as learning objectives is often a difficult and time-consuming step and may require careful thought and numerous iterations. Each core competency may have several learning objectives associated with it. Oberlin College’s learning objectives for neuroscience majors are briefly summarized in Table 2. Note that most of these learning objectives have been further defined. For example, the term “basic concepts in neuroscience” has had to be more precisely defined so that lectures cover the basic concepts and exams can assess knowledge of them.

The Activities.

Activities are the assignments and pedagogical tools that the faculty members use to help students realize the objectives. Each learning objective should have at least one activity designed to help students meet it and most objectives will have multiple associated activities or assignments, usually at both introductory and advanced levels of the curriculum. Activities may range from lectures and readings to discussions, written papers, laboratory exercises, and required student presentations (see Table 2). Again, some activities have been more precisely defined. For example, to help majors acquire analytical reasoning skills the senior seminar requires significant reading, discussion and analysis of primary literature and scientific reviews; at least two formal oral presentations per student which present data from primary sources; and at least one written paper (minimum seven pages) that includes analysis of primary literature.

Assessment Measures.

If the goals, objectives, and activities are well explicated then assessment measures tend to ‘fall out.’ As with all assessment it is best to use multiple measures and numerous types of tools (e.g., self-report, objective test) and include pre- and post-tests when possible. Like other colleges and universities, all Oberlin courses have established assessment measures such as exams and papers which can be used to help assess the department goals for majors. Currently, the Oberlin Neuroscience Department also uses three departmental assessment tools: a pre/post quiz in the introductory neuroscience course to assess changes in basic knowledge of neuroscience; an assessment of papers in the senior seminar class to evaluate critical thinking and analytical reasoning skills (see information in Appendix) as well as written communication skills; and a survey of graduating seniors. The latter is an indirect measure that essentially asks majors if they think they have achieved the core competencies and asks them about their attitudes concerning the department, e.g., if they are satisfied with the academic advising they received. Clearly some of the core competencies such as oral communication skills are not yet adequately assessed by the department.

Although these assessment tools have been very informative, more tools should be developed. One promising resource for the development and dissemination of assessment tools is the recently launched website called Educational Resources in Neuroscience (ERiN: http://erin.sfn.org/). Hosted by the Society for Neuroscience, this site provides links to resources for the teaching of neuroscience and allows users to review and rate resources that others have recommended. If you or your colleagues have useful rubrics for assessment of core competencies, we encourage you to post them online and submit the link to ERiN.

Conclusions/Comments.

It should be emphasized that core competencies, learning objectives, activities, and assessment measures can and should vary across institutions. Many of these can be organized and emphasized differentially in a variety of ways depending on the priorities of a particular department/program. For example, Oberlin’s Neuroscience faculty members have not outlined a departmental core competency in quantitative skills even though it may be a neuroscience goal at other institutions. At Oberlin there is a college quantitative proficiency requirement and basic courses in chemistry, biology, and statistics are required for the major. It is felt that to pass these courses students need to have achieved competency in quantitative skills. Additionally, we have folded in departmental learning objectives for data analysis, graphing, and interpretation of data into the laboratory techniques competency. Moreover, so far Oberlin’s Neuroscience Department has not listed ‘research skills’ as a separate core competency (note that ‘research experience’ is an activity not a competency). However, research experiences are seen as an excellent means to achieve the core competencies and important research skills are included in the learning objectives for the competencies of critical thinking/analysis and knowledge of advanced techniques (see Table 2). Moreover, research experiences are strongly encouraged and faculty members offer courses that include research experiences, offer research internships in their labs and encourage research outside of Oberlin. Generally, the vast majority of students have participated in one or more research experiences.

Together these four steps of Backward Design: generating core competencies, establishing learning objectives, developing activities, and using assessment, form a rubric that can be used to better determine if students are learning what we think they should learn and if there are gaps in the curriculum. Applying this rubric to Oberlin College, it is clear that even though the neuroscience curriculum is fairly well developed there are still many areas that could be improved. Clearly, the curriculum is a work in progress and many of the learning objectives, specific activities and assessment measures are being developed and revised.

Concluding Thoughts

It remains to be determined if there is a consensus regarding the core competencies that should define undergraduate neuroscience curriculum. The results of the survey summarized in the present article constitute a first pass at determining what this consensus may be. Certainly individual programs should be encouraged to consider alternative competencies to those proposed here. We hope that we will have provoked further discussion of what the core competencies may be. What is most important is that the construction, revision and assessment of the curriculum is thoughtful, goal-directed, and incorporates contemporary best practices.

Designing and refining curricula requires a clear set of goals that can shape decisions. In developing our workshop presentation, we tried to define a set of core competencies that would apply broadly to undergraduate neuroscience. The results of our survey suggest that the majority of undergraduate neuroscience faculty agree with our assessment. This then provides a framework for colleges and universities to apply these competencies to their own programs. We would certainly encourage the use of a backwards design approach in such a process, but perhaps more importantly, the feedback from faculty who attended our sessions suggest that establishing a set of neuroscience core competencies will help programs to better define curricula, and ultimately aid in establishing mechanisms to assess programs and courses in the future.

Evaluation of a student’s skill at reviewing and integrating a body of neuroscientific literature

	Score
Builds rationale of research effectively: Presents purpose of research at beginning of paper Structures a progression throughout introduction Shows an understanding of the issues involved Explains significant issues of field Builds a strong conclusion Makes thoughtful comments on work summarized Accurately integrates information from a number of sources
Demonstrates awareness of methodological issues: Presents clear and concise explanation of techniques used and purpose of those methods Considers strengths and weaknesses of approaches used Considers statistical significance of results (if appropriate)
Score each item on a scale of 1 (lowest) to 5 (highest)	Overall Mean: