Myth or Reality: Self-Assessment Is Central to Effective Curriculum in Anatomical Pathology Graduate Medical Education.

Self-assessment, a personal evaluation of one's professional attributes and abilities against a perceived norm, has frequently been cited as a necessary component of self-directed learning and the maintenance of competency within regulated health professions, including the medical professions. However, education research literature has consistently shown uninformed personal global assessment of performance to be inaccurate in a variety of contexts, and have limited value in a workplace-based curriculum. Incorporating known standards of performance with internal and external data on the performance improves a learner's ability to accurately self-assess. Selecting content suitable for self-assessment, providing explicit assessment standards, encouraging feedback-seeking behaviors, supporting a growth mindset, and providing quality feedback in a supportive context are all strategies that can support learner self-assessment, learner engagement in reflection, and action on feedback in Anatomical Pathology graduate medical education.

Introduction

Within regulated health professions, self-assessment has frequently been cited as a necessary component of self-directed learning and the maintenance of competency.[1,2] We are focused on developing professional learners who not only achieve the necessary competencies throughout their formal education, but also continue to grow and adapt their knowledge, behavior, and skills as their professions evolve over time.[3] Exploring the values, limitations, and potential roles of self-assessment on learning within clinical settings will assist with awareness and appropriate application of self-assessment within our curriculum.

Critical to the latter goal is a need for clarity surrounding how self-assessment is defined. Not surprisingly, the conceptual framework of self-assessment varies by context and has evolved over time. In an early review of self-assessment in the health professions, Gordon considered valid self-assessment as judging one’s performance against appropriate criteria.[1] An alternative conceptual model of self-assessment is one where the learner’s judgment is informed by standards as well as internal and external data, such as emotions and external feedback from peers and teachers.[4,5] The terminology, informed self-assessment, has since been adopted by a large group of medical educators to capture self-assessment as a reflective process that is informed by data, including standards and multisource feedback.[6-8] While these differences in how self-assessment is defined are significant, they share the characteristic of self-assessment being a personal evaluation of one’s professional attributes and abilities against a perceived norm.[9]

In this article, we review the evidence on the accuracy and effectiveness of self-assessment, examine enabling factors for the persistence of generalized uninformed self-assessment in medical professions, and suggest recommended practices surrounding self-assessment in the clinical setting of Anatomical Pathology graduate medical education.

Is Self-Assessment by Adult Learners Accurate?

The accuracy of self-assessment by adult learners has been extensively studied since the 1970s. In 1991, Gordon wrote the first systematic review on the accuracy of self-assessment by adult learners as compared to external assessment by peers, experts, or standardized tests.[1] Eighteen studies published between 1970 and 1990 were included in this review, of which 14 were on health profession learners. In general, self-assessment correlated poorly with external assessment, with the majority of learners overestimating their actual knowledge or skill performance. A later meta-analysis by Davis et al specifically examined the accuracy of self-assessment among physician practitioners and learners.[2] Of the 20 domains examined in the 17 published studies, 13 (65%) demonstrated little, no, or even an inverse relationship between self- and external-assessment. The most recent systematic review on self-assessment was published in 2008 and included 77 studies involving health profession learners or practitioners published between 1990 and 2005.[9] Similar to the earlier reviews, health professionals demonstrated poor ability to self-assess.[9]

Several learner- and curriculum-related factors influencing the accuracy of self-assessment were identified by these review studies (see Table 1). Learner skill has been consistently identified as a major learner-related factor. Studies by Kruger and Dunning on undergraduate university students, as well as other studies on health profession learners, have consistently found that learners who were the least skilled significantly overestimated their performance, while highly skilled learners underestimated their performance.[10] Several studies have also identified gender differences in the accuracy of self-assessment. A longitudinal study by Edwards et al involving 1152 third-year medical students found that men were 1.7 times more likely to overestimate their grades than women.[11] Other studies have also identified a trend for men to express higher levels of confidence than women.[9] Cultural background of the learners was not found to significantly influence the accuracy of self-assessment, although there is a lack of high-quality literature on this topic.[9] Several curriculum-related factors were also identified. The content of self-assessment was significantly associated with the accuracy of self-assessment. Content unfamiliar to learners (such as asking a surgeon to self-assess their ability to determine patient’s decision-making capacity), complex concepts particularly those involving cognitive skills and knowledge, “soft” skills (such as communication and personal attitude), and self-assessment of global competence were associated with inaccurate self-assessment. On the other hand, learner’s familiarity with the content, performance skills (such as surgical skills or physical exam skills), and self-assessment of task-specific competency, significantly increased self-assessment accuracy.[1,9] External feedback also influenced the accuracy of self-assessment, especially when based on explicit and measurable assessment criteria.[1,9] Performance feedback that pointed out what learners did well and poorly with or without coaching for improvement, and benchmark feedback that informed learners of the expected level of performance, were associated with more accurate self-assessment.[1,9] Reflection also appeared to improve self-assessment accuracy. Several studies examined the use of videotapes in medical training, and found improved self-assessment accuracy when learners were allowed to review and reflect on their videotaped performance.[1,9] Self-assessment accuracy also improved after learners were provided with external feedback beforehand, allowing learners to reconcile this with their unguided self-assessment.[1,9]

Table 1.

Factors Influencing the Accuracy of Learner Self-Assessment.

Facilitators to self-assessment	Barriers to self-assessment
Explicit and measurable assessment criteria (C)	Low- or high-performing learner (L)
Review of performance data by learner (C)	Learner of male gender (L)
Performance feedback (C)	Unfamiliar content (C)
Benchmark feedback (C)	Complex cognitive or “soft” skills (C)
Feedback from external sources (C)	Global assessment (C)

Abbreviations: C, curriculum-related factors; L, learner-related factors.

Several theories were proposed to explain this inability to accurately self-assess. Cognitive psychology studies have shown that individuals naturally and subconsciously use defense mechanisms to diffuse negative information and to enhance positive information, the most common mechanisms being self-serving reasoning, biased hypothesis testing, and biased recall. With self-serving reasoning, individuals attribute success to internal factors such as skills and intelligence, and failure to external factors such as bad luck and distraction. Even when external feedback is provided, biased hypothesis testing ensures that, when confronted with negative feedback, one generally disbelieves this feedback unless solid evidence is provided, while positive feedback requires little evidence. When self-assessing, individuals also commonly rely on memories of their past performance for evidence. However, studies have shown that feedback on success is more memorable than feedback on failure, and negative feedback is preferentially forgotten.[12] To explain why, in their study, learners who were the least skilled were particularly inaccurate in self-assessment, Kruger and Dunning postulated that metacognitive skills essential to learning and performing well are also indispensable for accurate self-assessment, and poorly performing learners cannot accurately self-assess until they are taught these metacognitive skills and their performance improves.[10] In light of these engrained deficiencies in self-assessment, Eva and Regehr argued that perhaps it is simply not possible to accurately assess one’s performance without systematically and intentionally eliciting external assessment.[13]

Is Accurate Self-Assessment Central to An Effective Curriculum?

Contrary to the common assumption in health profession education, the systematic review by Colthart et al found no empirical evidence that supports the role of self-assessment in identifying learning needs, modifying learning activities, and improving clinical outcomes.[9] Instead, research supports the effectiveness of reflection, self-efficacy, and self-directed assessment seeking in learning, metacognitive skills that are related, but not identical, to self-assessment in learning.[13-15] Reflection, as defined by Donald Schön, includes reflection-in-action, which involves concurrent self-evaluation and self-correction while performing a task, and reflection-on-action, which involves summative evaluation of one’s performance after the end of the task.[16] A systematic review by Mann et al found that reflection-in- and on-action effectively stimulated deeper understanding of the knowledge content and complex clinical problem, and were associated with self-reported performance improvement and change in clinical practice.[17] Self-efficacy is defined as the confidence in one’s ability to eventually succeed in performing a task, and is essential for maintaining resilience and psychological well-being. A meta-analysis on academic self-efficacy by Marsh and Martin showed that students with a high level of academic self-efficacy were more likely to achieve subsequent academic success, and past academic success contributed to high academic self-efficacy.[18] Interesting, high self-efficacy has been associated with overestimating one’s current performance. Conversely, individuals with clinical depression and low self-efficacy have been found to be more accurate in self-assessing their abilities.[18] Self-directed assessment seeking is the ability to actively seek out formative external assessment. As discussed previously, external assessment improves the accuracy of self-assessment. In the literature on physician workplace-based assessment, high-quality performance feedback from external sources was generally rated as educational by feedback recipients, and was associated with improved objective performance.[19]

Even when accurate self-assessment is possible, Regehr and Eva hypothesized that this may still be insufficient for health professionals to embark on self-directed learning and practice change, as learning in the areas of one’s weaknesses will invariably necessitate more energy and commitment.[20] Therefore, unless mandated by regulatory bodies or other authorities, health professionals continue learning in the areas of expertise, while resist learning in the areas of weaknesses.[20]

Why Do the Myths of Self-Assessment Persist?

Myths persist in medical education because they fulfill three essential social purposes: (1) shared meaning-making, (2) a vehicle for values and ideologies, and (3) a means of maintaining social power structures.[21] The authors believe that three main factors explain the persistent myth of the central role of self-assessment to effective curriculum in medical education, despite the empirical evidence regarding its limitations.

First, self-assessment is a central tenet of Knowles’ adult learning theory and of other humanistic learning theories, which believe that adult learners are inherently motivated to learn and improve in their self-identified knowledge or skill gaps.[21] These theories are in turn the foundation of curricular design for health professions.

Second, health professions’ signature pedagogy is apprenticeship, a model characterized by “learning in the practice settings of the workplace.”[22] This pedagogic model relies heavily on learners’ ability to self-assess and self-direct their learning. On the other hand, putting learning responsibilities mainly on learners lessens the responsibility of teachers, curriculum designers, and educational institutions for learners’ performance.

Lastly and perhaps most importantly, health professionals’ ability to self-assess is central to effective self-regulation, which is in turn a cornerstone of professional autonomy. According to Regehr and Eva, self-regulation involves the following continuous and repeated steps: ongoing self-assessment of performance, self-identification of knowledge or skills that have fallen below standards of practice, self-directed learning to address these gaps, and practice change incorporating newly learned knowledge or skills.[20] Without the ability to self-assess, external assessors will need to be brought in to evaluate health professionals’ competency and the design of their learning plan, and to monitor their learning progress and practice change.

The 2021 Lens on Self-Assessment and Its Incorporation in the Anatomical Pathology Graduate Medical Curriculum

While it is clear that generalized uninformed self-assessment has limited value in the learners’ ability to identify their strengths and weaknesses, the evidence does support continued incorporation of learner self-assessment within the curriculum.[6,12] The literature identifies three overarching themes: (1) the necessity of learners incorporating standardized outcome measures into self-assessment, (2) the value of learners’ use of feedback (internal and external data) within self-assessment, and (3) the role of self-efficacy in learning and continued professional growth.[6,12] Further discussion with examples is provided below.

Anatomical Pathology Resident Training

As a background, residency training in Canada occurs after three to four years of medical school training, and is the training that leads to specialty (or subspecialty) certification by the Royal College of Physicians and Surgeons of Canada (RCPSC) or certification as a family physician by the College of Family Physicians of Canada (CFPC). Having achieved the RCPSC or CFPC certificate, the physician is prepared for unsupervised medical practice supported by continuing professional development. The University of Toronto’s Anatomical Pathology residency program, one of the largest in Canada, is a five-year training program with a total of 30 to 35 residents. Training occurs at multiple sites including four academic hospitals, affiliated community hospitals, and a provincial Forensic Pathology service. Residents receive ad hoc verbal feedback from faculty and allied health professionals such as pathologists’ assistants and cytotechnologists in the clinical setting, in addition to formal written rotation evaluations 10 to 12 times per year, and slide, written, and oral exams administered semi-annually. Since 2019, Anatomical Pathology residency programs across Canada have moved into a competency-based medical education model called “Competency-by-Design” (CBD), whereby the residency training program is divided into four distinct stages with a list of learning goals and objectives called Entrustable Professional Activities (EPAs) and Milestones.[23] The resident must be observed completing each EPA independently and satisfactorily to progress to the next stage.[23] A key purposed benefit of CBD is increased opportunities for feedback and coaching to guide learning.[23] While each graduate medical training experience is undoubtedly unique, it is anticipated that some of the suggested applications related to self-assessment described below in the four curricular commonplaces are transferrable to other residency programs (see Table 2).

Table 2.

Strategies for Effective Learner Self-Assessment in Anatomical Pathology Graduate Medical Education.

Learner	Teacher
Normalized feedback-seeking behaviors Learning from tasks at the periphery of competency	Assessment based on direct observation of learner’s performance Fostering a culture of trust between learner and instructor
Context	Content
Frequent, dedicated time for learner feedback and coaching	Skill-specific assessment Explicit and measurable assessment criteria Opportunity for learner self-reflection prior to external assessment Provision of performance and benchmark feedback

Learners

For continuous improvement and progression in one’s career, learners at all levels of expertise need to seek feedback, ideally from multiple sources, in order to assess their personal strengths and weaknesses.[24] The model of informed self-assessment, developed by Sargeant and colleagues, supports the concept that learners must receive data, which then must be interpreted through reflection, calibration, filtering, and assimilation.[6] While learners may choose to ignore some data, learners should incorporate valuable feedback into their self-assessment in order to lead to a change in practice. In the Anatomical Pathology clinical setting, residents can explore feedback from numerous sources including faculty, peers, and allied health professionals. Ensuring that all members of the team are valued and that their expertise in different domains within the discipline is clear, will help facilitate learners seeking feedback from multiple sources.

While patient safety should be maintained, learners should be encouraged to engage in tasks that are at the periphery of their competency levels. Normalizing feedback-seeking behavior, and providing necessary supports, are essential in promoting the development of a growth mindset and strong self-efficacy by the learners while learning from these appropriately challenging tasks. For example, a junior resident might request to gross a particularly challenging case, and, while not capable of managing solely, the resident could be coassigned to gross this case with a pathologists’ assistant or a senior resident to permit a sufficient challenge to motivate the resident, while not compromising patient care.

Teachers

Instructors play a major role in informing learners’ self-assessment through feedback. As instructors working within busy field-based settings, it is essential to provide learners with early feedback based on performance that has been directly observed. The credibility of the content of feedback is lost when learners perceive that the feedback is not genuine or is based on content that has not been directly observed.[6,25] To facilitate capturing specific feedback, field notes specific to a case or a teaching session can be used to ensure that specific details regarding the learners’ performance in the clinical setting are captured.

While many factors can contribute to a workplace’s culture, instructors are in large part responsible for creating a culture of trust between learners and themselves. Delivering feedback in a balanced, honest, and non-threatening manner, such that learners are engaged and motivated as well as informed by the feedback, will encourage further feedback-seeking behaviors by the learners.[7,12] In our program, a faculty member will work with a resident sequentially throughout their training. This arrangement provides the opportunity for progressive and repeated feedback on specific skills or behaviors, a practice that has shown to have beneficial effects.[26] Creation and maintenance of field notes on the resident’s performance over time can inform follow-up feedback to encourage the resident to reflect on their past and current performance.

Content

As noted above, incorporating self-assessment on specific skills with explicit and measurable performance markers, particularly when learners can review their performance prior to external assessment, has been shown to improve self-assessment. The latter can also be improved over time, particularly when combined with external feedback and standard outcome measures. For example, in the Anatomical Pathology setting, having residents review the gross description and glass slides of a case that they have grossed, before reviewing the case with a pathologist, would allow them to self-assess their skills in specimen gross examination, gross finding description, and block selection. The provision of criteria by which these specimen grossing skills would be assessed prior to the teaching session would further allow the residents to benchmark their performance against expected standards.

Context

Like in most health care settings, instructors in Anatomical Pathology are tasked with managing both patients and learners within a complex workplace. Learners within the health care settings may be reluctant to seek feedback from their instructors due to an actual or perceived lack of time. In the Anatomic Pathology setting, residents typically get formal evaluation only at the end of a rotation. Instructor overconfidence in residents’ ability to self-assess their clinical skills is a potential contributor to this practice. Scheduling specific times for resident feedback on a daily or weekly basis is warranted to assist with residents’ self-awareness and self-evaluation, in addition to the development of strategies to target weaknesses and the follow-up on previously identified weaknesses.

Conclusions

Within regulated professions, uninformed personal global assessment of performance has consistently been shown to be inaccurate in a variety of contexts including the medical professions, and has limited value in a workplace-based curriculum. Incorporating known standards of performance with internal and external data on the performance improves learners’ ability to accurately self-assess.[11] Selecting content suitable for self-assessment, ensuring standards are explicit to learners, encouraging feedback-seeking behaviors, supporting a growth mindset, and providing quality feedback in a supportive context are all strategies that support learner self-assessment, learner engagement in reflection, and action on feedback in the Anatomical Pathology graduate medical education.[9,12,25]