An approach to determining, delivering, and assessing essential course content in a medical human anatomy course.

Learning objectives typically indicate subject matter judged to be important or that represents essential material to be learned during a course. We report here on our efforts to identify essential course content and determine our effectiveness teaching and assessing this content in our preclinical human anatomy course. Using a consensus driven approach, we identified anatomical structures, relationships, and functional concepts judged to represent essential material in our unit on the thorax that students were expected to be familiar with. We then determined performance on specific examination questions that focused directly on the essential material. Thirty-seven of 48 students (77%) correctly answered all 34 of 51 questions that directly focused on content we defined as essential. The remaining 11 students answered the majority of these questions correctly. The overall mean score was 86% (range 61%-98%). Our review of student performance on the End of Block thorax examination confirmed our belief that we were successful in helping students learn material we defined as essential. We found the process described here to be helpful in defining essential content and for helping focus and improve medical education and learning assessment based on that material. We believe the idea of defining essential content that can be efficiently taught and effectively learned within a proscribed period of time is an important and necessary objective. We believe the approach used here might be successfully utilized in other programs in efforts aimed at quality improvement.

INTRODUCTION

Clearly stated course and session learning objectives are generally understood to represent the most important information students are expected to learn during a course or a particular class session. Ideally, students use these learning objectives to guide their efforts and manage their time in order to ensure success on formative and summative examinations. Learning objectives should address essential concepts and content that all students are expected to learn and be able to utilize in later coursework and work settings. Learning objectives may also address additional topics that might be important or useful in distinguishing among different levels of performance on examinations. Learning objectives also guide faculty in developing specific teaching activities and approaches that will facilitate student success in the course. Importantly, learning objectives serve as a guide for the faculty when writing questions for examination purposes.

Preclinical basic science courses typically include large volumes of content and information that students are expected to become familiar with in order to be successful the course. Efforts to define core course content have been undertaken and several proposed syllabi have been reported in the literature (Berman, 2014; Finn et al., 2018; Leonard et al., 2000; McHanwell et al., 2007; Moxham et al., 2014; Smith et al., 2016; Smith et al., 2018). Recent efforts have been directed toward identifying a core syllabus specifically for the teaching of specific anatomical subjects including the head and neck (Tubbs et al., 2014), musculoskeletal system (Lisk et al., 2014; Webb et al., 2018), embryology (Fakoya et al., 2017), histology (Das et al., 2019), oral anatomy (Moxham et al., 2018), neuroanatomy (Moxham et al., 2015), and thoracic anatomy (Moxham et al., 2020).

Most of these efforts have resulted in extensive lists of topics judged to be important by the authors. However, these descriptions of core material do not typically take into account important considerations such as the number of curricular hours available for teaching a complete anatomy course or a particular subset of anatomy such as limbs, abdomen, or head and neck, factors that vary significantly among different schools. Likewise, these course syllabi do not differentiate between courses that may involve cadaver dissection or how much time may be allocated for dissection and/or other types of learning activities, all factors that can influence the content of a particular course.

In our view these are important considerations since, for example, learning in the dissection laboratory setting is more time consuming and faculty dependent than learning in other settings. In addition, students will likely require more time with the cadaver to prepare laboratory or practical examinations that involve identifying structures on cadaveric material. While we agree that the content identified in the literature represents important if not essential core material, we find it may not be possible to include all of these topics in most anatomy courses. Restrictions based on course length, allocated hours, type of instruction, and variability of assessments will likely necessitate some adjustment in the amount of material included to ensure that students have sufficient time to adequately learn the material.

When developing examinations, either comprised of questions written by the faculty themselves or developed through a process of selecting questions available from external sources such as the National Board of Medical Examination (NBME), faculty strive to match the questions to the material indicated in the stated session or course learning objectives. Faculty may use a process of blueprinting to ensure a fair and representative balance of questions based on the content actually presented in class, considered in the laboratory or identified in the assigned or recommended readings. The extent to which faculty are successful in this endeavor is sometimes uncertain as evidenced by examination scores that are unexpectedly low or comments from students that an examination was unfair or unrepresentative of what students were expecting.

As part of an ongoing effort aimed at quality improvement, we decided to determine how successful we were in teaching and developing examination questions that focused on what we considered to be essential material and concepts in anatomy. We hypothesized that correct answers on examination questions that address the essential material would suggest that we were effective in both teaching this content and in writing examination questions that reliably represented this material. Conversely, high failure rates on these questions would raise concerns regarding our effectiveness in either of these areas and would prompt a careful review of our teaching and examination practices.

We report here on our efforts to determine our success in facilitating learning what we define as essential content related to thoracic anatomy based on student performance on questions included on our End of Block anatomy examination that specifically and directly relate to that material. Our expectation was that if we were successful in addressing those concepts, defined by us as essential, performance on the End of Block anatomy examination questions dealing with that material would be better than on questions dealing with other learning objective‐related topics presented but not defined as essential. Examination related data regarding this latter point would allow us to make informed improvements for future classes regarding teaching approaches for different types of content. Such information would help us ensure that students were taught and learned material that was important and that they were well prepared for subsequent coursework in the curriculum that depended on this content.

MATERIALS AND METHODS

Defining essential content

We began by reviewing the stated learning objectives for the unit of the course in which we considered the anatomy of the thorax, focusing on topics, concepts, and principles we address in either the lecture or the laboratory sessions that we considered to be essential. We conceptualized essential material as foundational knowledge that students must acquire in order to be successful in later class and clinical work in our curriculum. We operationally defined essential content as that which students must understand and be familiar with in order to 1) be successful on our summative examinations, 2) be successful on the STEP 1 examination, and 3) be successful in clerkship rotations that involve patients with diseases involving structures located within or associated with the thorax.

We then reviewed a list of faculty developed terms that include structures, relationships, and concepts that students are expected to identify in the dissecting laboratory and/or be familiar with based faculty lectures and recommended readings. The terms are listed in bold type in our Anatomy Guide & Workbook for the thorax.

The list of terms for the thorax block included 198 terms. This list was distributed to the four course faculty, all of whom are clinicians from different areas of practice (chiropractic, emergency medicine—trauma surgery, physical therapy, and radiology) and who have participated in the course for the previous 5 years. Each individual was asked to select 50 terms from the list that represented essential material as defined above. This first round of review elicited a total of 87 terms from among the 198 terms listed for the thorax unit. These 87 terms were then sent to the faculty for a second review with a request to again identify 50 terms that represented essential content. Fifty‐six terms were identified by all four faculties following this second review and these terms were designated as key terms and served to represent essential material for the purpose of this study (Table 1).

TABLE 1

Fifty‐six key terms representing essential material for anatomy of the thorax

Brachiocephalic trunk	Common carotid arteries
Subclavian arteries	Internal jugular veins
Ligamentum arteriosum	Trachea
Esophagus	Recurrent laryngeal nerves
Pulmonary trunk	Pulmonary arteries
Pulmonary veins	Sternal angle
Carina	Main stem bronchi
Esophageal plexus	Cardiopulmonary plexus
Thoracic duct	Azygos vein
Hemiazygos vein	Sympathetic trunk
Accessory hemiazygos vein	Fossa ovalis
Pectinate muscles	Papillary muscles
Coronary sinus	Cardiac arteries
Coronary veins	Subclavian veins
Mitral valve	Pulmonary valve
Aortic valve	Tricuspid valve
Phrenic nerves	Vagus nerves
Horizontal fissure (lung)	Oblique fissure (lung)
Costodiaphragmatic recess	Costomediastinal recess
Parietal pleura	Visceral pleura
Fibrous pericardium	Visceral pericardium
Left atrium	Right atrium
Left ventricle	Right ventricle
Pericardial sac	Moderator band
Vertebral arteries	Thyrocervical trunk
Internal thoracic artery	Musculophrenic artery
Aortic hiatus	Esophageal hiatus
Brachiocephalic vein	Pulmonary ligament

We next reviewed our current End of Block thorax examination to identify the number of questions that specifically addressed the structures and concepts represented by the 56 key terms. The examination was written by the faculty and consisted of 51 questions of the multiple choice or single/short answer fill‐in‐the‐blank format with or without associated images. We have used this examination with one or two changes based on item analyses for the past 4 years with stable performance and metrics during that time.

Students are informed that the examination is developed around the block and session learning objectives to ensure a fair and balanced sampling of testable material. Students are also informed that the answers to the questions would be from the list of 198 terms included in the course materials for this block. These terms are also used as part of our institutional curriculum mapping process.

Content organization and delivery

Our teaching approach includes faculty lectures, cadaver dissection sessions, dry laboratory sessions, and independent study supported by suggested readings from recommended resources. Anatomy is taught during a 4 h block of time on Tuesday mornings of each week over the course of the 36 week academic year. The basic science curriculum, of which anatomy is a part, is divided into 4, eight week blocks of instruction followed by a ninth week during which examinations are administered.

Typically, a lecture is scheduled during the first hour of each day followed by a 3 h laboratory session. Lectures are delivered live and focus on the subject matter under study for that day. Lectures are recorded and subsequently posted to our educational platform for review at a later time. Attendance at lectures is highly recommended but not designated as mandatory.

The 3 h laboratory dissection sessions are predominantly devoted to work with cadavers. Four students are assigned to a cadaver, each with an assigned responsibility (i.e., dissector[s], reader, and consultant, the person responsible for looking up answers and locating supportive material in texts and atlases). Dissection instructions are provided in the VTCSOM Anatomy Guide & Workbook written by the faculty specifically for our course. The Guide & Workbook includes session objectives and a list of terms representing structures students are expected to identify during dissection. All four core faculty participate in the laboratory sessions with assistance provided by members of our clinical faculty at various times during the year. Dry laboratory sessions utilizing prepared skeletal material, dry skulls, anatomical models, and selected CT and MRI images are incorporated during the course as appropriate.

Attendance at lectures is highly recommended but not designated as mandatory. Attendance at scheduled cadaver laboratory sessions are considered as mandatory based on the different responsibilities of the members of the group, although there is no formal penalty for non‐attendance. Students are expected to notify the faculty in the case of an anticipated or unexpected absence. Our experience has been that absences are infrequent and when they occur, are for appropriate reasons. The Office of Student Affairs must approve all absences.

Determining performance on questions dealing with essential material as defined

The End of Block anatomy examination consisted of 51 questions dealing with the anatomy of the thorax. All questions were linked to one or more of the printed learning objectives for this block of the course. Questions were written in either the multiple choice (n = 21) or single/short answer fill‐in‐the‐blank (n = 30) format. An image (anatomical or radiological) was included in 30 of the multiple choice questions and 14 of the short answer questions. Approximately half of the question were written in the clinical vignette style. Intended answers were checked and determined to be from the list of 198 terms included in the course material. The examination included approximately an equal number of questions from each 50‐min lecture or 2‐h laboratory session.

Thirty‐four of 51 questions focused directly on content related to the 56 key terms that represented material defined as essential. The 17 remaining questions dealt with concepts taught, but not defined as essential. The answers to all questions were from the list of 198 terms included in the course material.

The examination was administered by computer using ExamSoft© and graded using ExamScore© software. All single/short answer fill‐in‐the‐blank responses were reviewed by the faculty for correctness. Appropriate synonymous terms (i.e., brachiocephalic trunk/brachiocephalic artery) were accepted. Minor spelling errors were accepted. Students were allowed 2 h to complete the examination.

RESULTS

Forty‐eight students took the End of Block anatomy examination. Thirty‐four of the 51 questions (67%) dealt directly with content related to essential material as represented by the 56 key terms. Thirty‐seven of 48 students (77%) answered all 34 questions correctly. The number of these questions answered correctly for the remaining 11 students ranged from 32 to 27. The mean examination score of the 37 students answering the all questions dealing with essential material was 88%. The mean examination score of the remaining 11 students was 79%. The mean score on the examination overall was 86% (range 61%–98%).

We reviewed the End of Block anatomy examination metrics provided by ExamScore© and flagged any questions that were answered correctly by fewer than 60% of the students. Our review identified two questions which met this criterion, neither of which dealt with material defined as essential. For these two questions, the number of students who answered them correctly was 57% and 55%, respectively.

We then reviewed the point biserial correlation score for the two flagged questions. With input from our Office of Assessment and Evaluation, we chose a score of 0.26 as a cut score for decisions regarding whether a question would be retained or deleted from the examination. Flagged questions with a point biserial score of less than 0.26 were deleted from the examination and not included in calculations of either student score or examination mean, while flagged questions with a point biserial score of 0.26 or higher were retained and included for final grading purposes. For both questions, the point biserial correlation score was greater than 0.26 and therefore, based on these criteria, both flagged questions were retained and included in determining overall examination mean score.

DISCUSSION

Faculty endeavor to be complete and effective in their teaching efforts and strive to include information considered to be important for the students to know or at least be familiar with for future purposes. This goal is typically achieved by developing content and teaching methods that reliably address the stated learning objectives of the course. The depth of learning and the ways in which the students will be expected to demonstrate their learning must also be carefully considered when developing learning objectives.

For certain topics, knowledge at a lower level of difficulty might be adequate or sufficient, such as in the case of learning objectives that asks the student to simply identify or list the typical branches of the arch of the aorta. For other topics or concepts a higher level of knowledge might be required, such as is reflected in an objective that asks the student to identify the structures of the posterior mediastinum and describe their anatomical relationship to each other. Clarity in the wording of learning objectives is essential. The level and type of knowledge stated in the learning objective must be consistent with the way in which the students will be asked to demonstrate their success in attaining that objective on a formative or summative assessment. Students will rightly expect to be assessed in a way that reliably reflects their learning accomplishments.

Occasionally faculty may lose sight of the amount of information students can learn in a limited amount of time and may include more information than might be manageable during a particular lecture or laboratory session. In some cases, faculty may include information that is beyond the grasp of the student at that particular time. Alternatively, faculty may fail to include foundational facts or concepts that may be of critical importance and mistakenly believe that students will learn this material on their own. In some instances, this may be true but in others, where the subject matter is complex, students might struggle unsuccessfully. In situations such these, students may become confused, having difficulty being able to clearly discern exactly what information is most important at the time, particularly as it pertains to material that will be on the examination.

Similarly, when constructing examinations, faculty may include questions not clearly linked to the stated objectives or may inadvertently embed information in a particular question that might represent a level of understanding beyond that which is implied in the learning objective. Failure to anticipate and address these issues can lead to unreliable evaluations of student knowledge, dissatisfaction on the part of the student and disappointment or frustration of the faculty.

In addition to factors related to faculty efforts and responsibilities that might influence performance on examinations, other influences primarily involving the students themselves must also be considered. Among these are previous experience with the material. Not infrequently, students in health care profession programs prepare prior to admission by taking coursework in some of the subjects that they will encounter in the curriculum. For example, many medical students, during their undergraduate or graduate years, take courses in the biomedical sciences such as biochemistry, anatomy, and physiology. Some have earned graduate degrees in these subjects. These experiences can provide an important foundation for their subsequent course experiences in medical or dental school.

Other factors such as concurrent curricular coursework assignments can significantly influence student learning and subsequent performance on examinations. For many students, time management is a challenge, particularly during the early years of their medical school career when large amounts of material are presented in what students frequently find to be a limited time frame. Students who become overwhelmed, often resort to study strategies that may not be effective. Others, may be hesitant to seek help in identifying particular inefficiencies, or may do so too late to avoid the effects of inefficient of ineffective study habits on test performance. It is important for faculty teaching anatomy courses to be aware of these other legitimate, time consuming, responsibilities that students may have during the academic year.

Increasingly, we have seen students rely on learning materials designed primarily for subsequent high stake examination purposes (e.g., the STEP I examination). Most are designed as examination review aids rather than primary learning resources. While some of these resources may be helpful, they may not adequately cover, or in some cases address at all, certain topics that are included in a particular course at a particular school. These topics may be included on end of course examinations.

We are conscious of these phenomena and as part of our ongoing efforts at quality improvement. We considered it important to review the content of our block on the thorax and come to agreement regarding what topics are essential given the limitations of allocated time and what resources are available to students who may need additional assistance. We examined the topics and material included on our End of Block anatomy examination for the block dealing with the thorax and using a consensus‐driven process, developed a list of those terms/concepts that we judged to represent “essential” content for success in the course and in subsequent parts of the curriculum.

Our anatomy course is located within the first year curriculum, together with other courses in a curricular structure not unlike other medical schools. Consequently, when identifying “essential” content, we took into account time and effort required for students to address other non‐anatomy coursework responsibilities that ran concurrently with the anatomy course. We believe this to be an important responsibility of those responsible for developing courses that are a particular part of a larger program of instruction.

Our next step was to examine our End of Block anatomy examination to determine the extent to which essential content was specifically assessed as compared to content not so identified. We were gratified to find that 34 of 51 questions (67%) focused on material we defined as essential and that 37 of 48 students (77%) answered all 34 of these questions correctly.

Our results confirmed our belief that we were effective in presenting and assessing student success in learning material we defined as essential. In the process, we identified certain topics that were less well understood by the students. This finding has prompted us to search for reasons that might explain these later findings. For example, we may have overlooked or underemphasized this material during lecture or laboratory sessions, a situation that can be remedied by greater attention on the part of the faculty to particular learning objectives and the materials used in lectures and laboratory sessions. We are in the process of revising our materials to address some of these deficiencies.

In addition, we have noted that in several instances, a particular topic we considered to be essential was inadequately or not addressed in some of the resources that students elect to use on their own (e.g., some resources intended for STEP 1 review purposes). We continue to recommend study resources that we consider adequate and appropriate (textbooks and atlases) but recognize that student preference in this regard is based on factors that may be beyond the influence of the faculty.

These observations have prompted us to focus our ongoing quality improvement efforts in several directions. We have recently used the survey method described here to define essential material in the other blocks of our anatomy curriculum (e.g., limbs, abdomen/pelvis, head/neck) and have undertaken a prospective review of our End of Block anatomy examinations to determine the extent to which we specifically assess this material. In addition, we now intend to revise our examinations as necessary to ensure that at least 70% of the questions on each examination directly address topics that we define as representing essential content.

LIMITATIONS

We recognize that the specific results of this work apply to our particular anatomy course and unit examination and may therefore have limited direct applicability to anatomy courses in other programs. We appreciate, for example, that not all courses provide students with a list of structures and concepts for which they are responsible for on examinations. We recognize that programs differ in the time available for anatomy instruction and that not all programs utilize or have access to cadaver material.

We also recognize that many programs may administer examinations involving tagged structures on cadavers which might limit the types of information available for quality improvement analyses. Nonetheless, we believe that formal and specific efforts to identify important or essential content, similar to the efforts described here, are necessary to help ensure student success. We believe also that efforts to continuously revisit and revise teaching approaches and assessment tools are essential for maintaining effective instruction, valid assessments and reliable evaluations of student learning.

CONCLUSIONS

We found the process described here to be helpful in identifying important content in our instructional block on the thorax and as a guide for improving instruction and assessment. We are now using this method to refine our lecture and laboratory sessions and our assessments in other blocks of the course in our continuing efforts at quality improvement. Our review of student performance on the End of Block thorax examination has identified areas where we have been successful and other areas in which we might improve. Our overall goal is to better prepare our students for future learning activities during their undergraduate careers.

We believe the idea of defining content that can be efficiently taught and effectively learned within a proscribed period of time is an important and necessary objective. The need to utilize allocated time and resources efficiently and effectively in the development and delivery of anatomical instruction is widely acknowledged and we believe this process can be helpful in attaining this goal and believe it to be adaptable to other programs with similar goals.