Applying perceptual and adaptive learning techniques for teaching introductory histopathology.

BACKGROUND: Medical students are expected to master the ability to interpret histopathologic images, a difficult and time-consuming process. A major problem is the issue of transferring information learned from one example of a particular pathology to a new example. Recent advances in cognitive science have identified new approaches to address this problem.
METHODS: We adapted a new approach for enhancing pattern recognition of basic pathologic processes in skin histopathology images that utilizes perceptual learning techniques, allowing learners to see relevant structure in novel cases along with adaptive learning algorithms that space and sequence different categories (e.g. diagnoses) that appear during a learning session based on each learner's accuracy and response time (RT). We developed a perceptual and adaptive learning module (PALM) that utilized 261 unique images of cell injury, inflammation, neoplasia, or normal histology at low and high magnification. Accuracy and RT were tracked and integrated into a "Score" that reflected students rapid recognition of the pathologies and pre- and post-tests were given to assess the effectiveness.
RESULTS: Accuracy, RT and Scores significantly improved from the pre- to post-test with Scores showing much greater improvement than accuracy alone. Delayed post-tests with previously unseen cases, given after 6-7 weeks, showed a decline in accuracy relative to the post-test for 1(st)-year students, but not significantly so for 2(nd)-year students. However, the delayed post-test scores maintained a significant and large improvement relative to those of the pre-test for both 1(st) and 2(nd) year students suggesting good retention of pattern recognition. Student evaluations were very favorable.
CONCLUSION: A web-based learning module based on the principles of cognitive science showed an evidence for improved recognition of histopathology patterns by medical students.

BACKGROUND

Educational practices for teaching histology and histopathology have evolved significantly over the past several decades, driven by technological innovations; primarily digital imaging and web based delivery systems, as well as changes in curricular design and time available for specific disciplines.[123] The expectation for students to master the ability to interpret histologic images remains a goal, yet is still one that many students find problematic. Learning to recognize histopathologic features and processes remains a difficult and time-consuming task for many. With ever-increasing demands to introduce new content in the pre-clinical years, new strategies for improving learning efficiency are needed.

In our experience, the point of difficulty is not the ability to understand the basic concepts that can be learned systematically, or even following and recognizing patterns in a given slide or image when key features are pointed out. The real difficulty arises when a new image is encountered and the students are unable to transfer what they learned in a prior visual setting to the new one, despite knowing what they should be looking for. The feature-based analytical approach followed by novices is slow and often falters in the visual complexity of features, particularly for histopathologic processes. Experienced pathologists, by contrast, largely function through visual pattern recognition and thought processes which are faster and more accurate, utilizing analytical approaches secondarily.[4] Since these pathologists began with the same training that we give our students, relating features on histopathologic slides to the underlying processes characterizing the pathologies, their increased accuracy and fluency have arisen from learning processes that have occurred from viewing and interpreting a large number of examples rather than from a greater understanding of the underlying processes.[4] These are effects of perceptual learning.[56] In this and many other areas of advanced expertise, a key component of learning involves domain-specific changes in the way the brain extracts information from relevant displays. These changes involve both discovery and fluency effects.[7] Discovery effects involve finding the information relevant to important classifications, extracting it selectively and distinguishing relevant structure from irrelevant variation. Fluency effects involve improvements in the extraction and encoding of relevant information. They include most obviously speed, but also more parallel processing and greater automaticity (lower cognitive load), allowing efficient perceptual classification to coexist with other cognitive processes in complex tasks.[6]

These aspects of learning have been notoriously difficult to teach using conventional instructional methods.[6] In most domains, the factual, conceptual and procedural aspects of learning are systematically (usually verbally) instructed, whereas advanced pattern recognition and automaticity are assumed to arise only from long apprenticeship.[6] This leaves crucial aspects of learning to occur in unsystematic fashion over unspecified time periods and the results are often unquantified. Classifying histopathologic processes in images is a task that heavily depends on such processes, especially in the development of abilities to classify novel cases and to perform fluently. Furthermore, research has demonstrated an association between longer response time (RT) and lower accuracy in interpreting visual presentations such as mammograms[8] and skin cancer,[9] as well as in diagnosing a series of written cases.[10] Conventional instructional methods have scarcely recognized and seldom addressed perceptual learning. There has been a lack of methods for developing the expertise that results in shorter RTs and higher levels of accuracy other than to engage the learner in many years of experience in order to see enough cases for such distinctions to be developed.

In recent years, there has been an explosion of research in perceptual learning, much of it focused on low-level perceptual discriminations with the aim of understanding plasticity in the brain (for a review see[6]). There has also emerged a technology of perceptual learning that has proven effective in accelerating the development of expert information extraction skills in domains as varied as flight training, surgery, mathematics, mammography and areas of military field medicine.[1112131415] For example, Sowden, Davies and Rolings demonstrated significantly, the improved discrimination and decision speed by novices in recognizing microcalcification clusters in breast mammograms after perceptual learning training that involved viewing 60 images, in random order, 3 times each day for 4 days with only negative feedback (a computer beep) indicating the wrong choice for cluster location. These developments indicate that perceptual learning can be systematically advanced by particular kinds of interactive learning experiences.

Complementing this emerging technology of perceptual learning is the use of adaptive learning techniques in which learner's accuracy and speed in interactive learning is used to guide the spacing and sequencing of learning events, leading to be more efficient and durable learning, as well as implementation of objective and comprehensive mastery criteria.[16]

Here, we report initial efforts to explore these new techniques to enhance the learning of histopathology image interpretation for beginning medical students. We implemented these learning innovations as web-based perceptual and adaptive learning modules (PALMs).[13151617] PALMs have the potential to develop accurate and fluent pattern recognition by using many short classification trials. Their adaptive features continuously assess each learner's accuracy and RT in interactive pattern classification trials and use this information to space and sequence categories from which exemplars are presented, as well as to track progress. Use of adaptive perceptual learning methods focuses training where it is most needed and leads the learner to increased mastery in seeing intuitively the relevant features and patterns that underlie crucial classifications. These techniques have been shown to significantly increase both the rate and retention of learning for simple factual and complex cognitive, symbolic tasks in various educational and training domain settings.[1116]

If successful, these methods may allow learners at all levels to achieve substantially higher recognition of histopathology patterns in shorter time frames than are possible with current strategies and provide a consistent method of assessing the level of competence. We describe here our experience of developing and implementing a pilot PALM for the histopathology of general pathologic processes (inflammation, neoplasia, cell injury) for beginning medical students.

METHODS

Methodology for Perceptual and Adaptive Learning

Learning to recognize the patterns unique to various structural categories (e.g., different types of disease processes observed on histology specimens) requires discriminating among a large number of exemplars from each category in order to extract the features common, albeit in variable form, to a given category and distinguishable from those of other categories. This process of learning and discrimination is made more efficient by adaptive sequencing of trials that address different learning categories. The sequencing is based on an algorithm[17] that determines a “priority score” for each category, which in turn determines the sequence in which the next exemplar of each category is presented. The algorithm dynamically adjusts priorities after each learning trial based on learner accuracy and speed, as well as the number of trials since a category was last presented. This sequencing algorithm implements several principles that have been shown in learning research to strongly influence learning; these involve spaced practice, dynamics of short and long-term memory and relating recurrence intervals to underlying learning strength. One important principle is that as learning strength increases (as indicated by accurate and quicker responding), the delays in presenting problems that utilize the same concept or pattern (i.e. are in the same “category”) should increase, with interleaved “interfering” problems (i.e., those from different “categories”) occurring in between. This approach was implemented utilizing a core proprietary software program from Insight Learning Technology (http://insightlt.com/#home), adapted to this particular application.

The use of learning criteria that include a “target RT” within which a problem must be answered in order to be considered fluent is used for several reasons. First, fluent processing tends to indicate the operation of pattern recognition processes rather than laborious conscious analyses. Secondly, attaining learning criteria including fluency may predict better retention of learning. Thirdly, fluency, although indexed by response speed, also implies reduced cognitive load, allowing the learner to perform important classifications in more complex and demanding contexts. Once three successive exemplars of a category are answered accurately within the target RT, the category is “retired” so that future questions focus on exemplars from less well recognized categories.

Histopathology Module Development

We developed a short PALM encompassing basic skin histopathology (inflammation, neoplasia and cell injury and normal) in order to enhance and analyze the degree of pattern recognition for distinguishing normal tissue and the three basic pathologic processes, targeted to students in the first curricular block of the year 1 and the year 2 curricula.

The skin histopathology PALM asked subjects to identify images according to eight categories: Cell and tissue injury/repair, inflammation, neoplasia and normal with each pathology presented in images at high (×20-40) or low (×4-10) resolutions (thus, eight categories). Images used were obtained from approximately 120 clinical cases seen on the dermatopathology service at UCLA. Case slides (143) were digitized using Aperio software and multiple still images were obtained from different regions of these, yielding 261 unique images that were used in the PALM. In the PALM, problems were presented as 460 × 363 or 363 × 460 pixel images in flash-card like fashion through the computing device of the student's choosing (e.g., desktop computer, laptop and tablet device) and the student was asked to choose among four choices in identifying the image [Figure 1]. Time was calculated from the beginning of image presentation to the time at which an answer was selected and the PALM had a maximum time allowance of 24 s before a time-out signal was given and had a target RT of 12 s. Accurate answers were always indicated as such in feedback, but for credit toward retirement (mastery), those made within 12 s were given a score of 1 and wrong answers, or accurate answers made after 12 s, were given a score of 0. Although students could, in principle, zoom in on low magnification images to attempt to see more detail, the benefit of this activity was obviated by the lowered resolution that ensued and the 12 s time limit in which a response was required in order to count towards category retirement.

Figure 1

Examples of question and feedback slides for the skin histopathology perceptual and adaptive learning module

In this study, we set the target RT to be twice the average RT observed for experts’ pre-test values (based on the performances of three histopathologists) after excluding 2 s, estimated as the physical RT (i.e. time needed to find the correct answer, move the pointer to it and click). Thus,

RTtarget = 2 × (RTexpert-ave − 2 s) + 2 s

Once an answer was selected or a time-out occurred, the student was given feedback on the correct answer and more detailed information about the image, as illustrated in Figure 1b and d. In addition, for an accurate answer, the RT was shown as part of the feedback. Presentations of images from the same category were spaced by a minimum of three intervening images from other categories, and individual categories were retired once a student made three consecutive correct identifications within the designated RT.

In order to assess the effectiveness of the PALM, three different versions of a test were developed, each consisting of two unique exemplars from each of the eight categories and used to measure the ability of students to identify unseen exemplars of the categories prior to the PALM (pre-test), immediately following the PALM (post-test) and after a delay of 6-7 weeks (delayed post-test). The three different forms of the test were randomly distributed among students for the pre-test in order to average out any possible differences in difficulty among the three test forms and on each subsequent test, a given student received the next form of the test from the one(s) seen in the previous testing situation(s).

In addition to Accuracy and RT for each item, the program captured the sequence of item presentation for each student, the date and time stamp for each item presented and additionally for the PALMs, the total number of items necessary to retire all eight categories and the number of sessions in which the student engaged prior to completing the PALM (a new session being recorded whenever the student logged onto the PALM).

Subjects and Settings

The Histopathology PALM was designed to complement the introductory histopathology curriculum of year 1 students in their first 8 weeks of medical school, during which the basic pathologic processes of cell and tissue injury/repair, inflammation, neoplasia and normal histology are introduced using skin as a representative tissue. Students took the PALM outside of class, according to their own schedule, during the final week of the curriculum after having 5 weekly laboratory sessions that introduced normal skin histology followed by the basic pathology processes as manifested in skin. These laboratories utilized digitized slide examination in conjunction with case studies and had Flash file introductory units that students reviewed prior to the labs. The pathology-related laboratories corresponded to weeks in which the respective topics were the major subject covered in lecture and other settings. This module was completed by all 161 1st-year students during the Fall of 2011 and 155 of 159 1st-year students during the Fall of 2012 (4 students did not finish). The Histopathology PALM was also taken by all 162 2nd-year students in Fall of 2011, during a block in which Dermatology is taught but without a formal histopathology laboratory. Inclusion of the 2nd-year students in 2011 was done to measure retention of the subject from year 1, to serve as review and enhanced learning of the material and to determine whether their performance was any different from that of year 1 students given both the time interval since they had completed the skin histopathology laboratories and the effects of added experience with organ system histopathology throughout the 1st year. For all groups, the module was required as part of the educational curriculum; however, the delayed post-test was voluntary and only offered to students during the Fall of 2011. It was taken by approximately 45% of 1st-year students and 33% of the 2nd-year students.

The PALM and associated tests were delivered online, accessed through the UCLA course management system (“ANGEL”) but delivered from a separate server. A short description and instructions for taking the module were provided on the course web site.

Student Satisfaction Survey

A student satisfaction survey was obtained from both 1st and 2nd-year students in Fall of 2011 after they had completed the PALM. The questions for 1st-year students were:

Did the histopathology exercise make you feel more confident about recognizing basic pathologic processes in histologic images?

Overall, did you find the histopathology exercise to be useful?

Do you anticipate that it would be worthwhile developing comparable modules for other blocks?

2nd-year students (who were not studying skin histopathology in their current course) were only asked question 1. Items were scored on a five-point Likert scale using the choices definitely, somewhat, not sure, only a little and definitely not.

Analyses

Test parameters analyzed were accuracy, retention time and “Score”, which integrates these by counting only those accurate responses answered within the target RT. Differences between pre-, post- and delayed post-tests for each class were assessed by correlated (paired) t-tests. Significance was accepted at α < 0.05 except in cases in which both post-tests and delayed post-tests were analyzed, for which the Bonferroni correction was applied to adjust the accepted significance level to α < 0.025. Effect size was calculated as the differences in means divided by the average standard deviation except in those cases, identified by an asterisk in Table 2, in which the standard deviations were sufficiently different that we used only the value for the pre-test.[18]

Table 2

Statistics for accuracy, response time, and score for 1st and 2nd-year medical students completing the PALM pre and post-tests and for the Fall 2011 cohort with the delayed post-test 6-7 weeks later

RESULTS

Table 1 shows the time and number of trials needed to complete the PALM for students completing it in 2011 and 2012. Of the 261 images, 213 were available in the PALM, the remaining 48 being relegated to the three tests. Statistics for the time taken to complete the PALM and number of trials required to do so are presented in Table 2. The raw data showed a skewed distribution, with a median time of approximately 17.5 min for the 2011 students and 11.5 min for the 2012 students, but with a few students taking considerably longer. Performance for these parameters did not differ significantly between 1st and 2nd year students in 2011; however, the faster performance of year 1 students in 2012 may have resulted with more familiarity with the learning module approach since they had completed a different type of PALM (based on dermatology lesion morphology) earlier in the course. The fact that the number of available images was greater than the median number of images viewed plus twice the interquartile range for each group of students suggests that seeing a particular image more than once was a rare event. The minimum number of trials required to complete the learning module was 3 times the number of categories since a category must be correctly identified for three consecutive image presentations in order to be retired and the learning module was completed when all categories are retired.

Table 1

Time and number of trials needed to complete the PALM learning module for students completing the PALM

We initially assessed performance by comparing Accuracy and RT between pre- and post-tests at the time the PALM was administered and for the Fall 2011 cohort with the delayed post-test [Table 2 and Figure 2a]. Clearly, both Accuracy and RT were significantly improved from the pre-to the post-test, with large effect sizes, as a function of the intervening PALM. Delayed post-tests showed some decline in Accuracy for both 1st-year and 2nd-year students over the intervening 6-7 weeks. The mean scores on delayed post-tests were higher than pre-test values for both 1st-year and 2nd-year students; however, this increased score met our significance statistic only for 2nd-year students [Figure 2 and Table 2; note that we did not have enough statistical power to determine whether the difference in delayed post-test compared to the pre-test for year 1 students in 2011 met our criterion]. In contrast, RTs for delayed post-tests were significantly improved for both groups [Table 2].

Figure 2

Performance on skin histopathology pre-test, post-test and delayed post-test. Results from 1st-year students completing the perceptual and adaptive learning module and associated tests in the Fall of 2011 and 2012 (left and middle groups, respectively) and 2nd-year students completing these in Fall, 2012 (right group). The delayed post-test was optional and was only administered in 2011. Pre-test and post-test performance of students completing the delayed post-test (not shown) was not significantly different from those who did not complete this test. a) Means and standard deviations using answer Accuracy, independent of RT; b) Means and standard deviations using Scores, for which responses occurring after the target RT of 12 s are scored as incorrect. A single asterisk (*) indicates P < 0.0001 and a double asterisk (**) indicates P < 0.0005

In evaluating enhanced pattern recognition (as opposed to analytic, reasoning processes), however, the focus should be on changes in accurate responses made “quickly” (i.e., within the designated target RT). Thus, we re-analyzed the pre-, post- and delayed post-test data, excluding responses that were not made within the 12 s target RT. For these results, labeled “Score” in Figure 2b and Table 2, we observed a much greater retention of information, relative to the pre-test values, 6-7 weeks after the PALM was taken, with P < 0.0001 and effect sizes ranging from 0.8 to 1.6 for 1st and 2nd-year students when comparing either the post-test or delayed post-test values with those of the pre-test. The major source of this change appears to be a reduction in the number of items scored as correct in the pre-test, presumably because prior to completing the learning module, many of the accurate responses were derived from an “analytical” process rather than from the faster pattern recognition.

Student evaluations indicated a very favorable response to this exercise as illustrated in Figure 3 in which students indicated that the PALM improved their confidence and that they found it useful. Support for expanding this approach to material in the rest of the curriculum was also strong. The unusually high response rate for the 2nd-year students most likely derived from the fact that the survey was included along with a mandatory formative assessment whereas, it was included with an optional formative assessment for the 1st-year students, for whom response rate on the survey was similar to that on the formative assessment.

Figure 3

Ratings of the Histopathology perceptual and adaptive learning module by 1st and 2nd year students in Fall, 2011. Survey responses were received from 121 (75%) year 1 students and 159 (98%) year 2 students

CONCLUSIONS

The primary reason for teaching histopathology at our institution is to reinforce the understanding of disease pathogenesis and to develop an appreciation of the role of diagnostic pathology in clinical care. From that perspective, the traditional, analytical, feature-oriented approach to image interpretation remains important. However, the ability to quickly recognize basic pathologic processes present in a histopathologic image is an important secondary goal, to enable students to effectively utilize images for learning when presented in lectures, conferences, textbooks or articles, and for USMLE Step 1 preparation. Achieving this second goal is difficult given available class time and the primary focus on understanding pathogenesis. Recognition of a given pathologic process in a new image is an instance of “transfer” in the cognitive and learning sciences, which is problematic throughout medical education and other domains.[19] The PALM approach was explored as a potentially useful novel adjunct for students that would both provide additional examples of pathologies on which to practice their analytical diagnostic skills and a resource to develop skill in rapidly recognizing pathologic processes in histopathologic images. Therefore, our study was designed to assess whether pattern recognition improved or not following the exercise, which all students took, rather than whether this approach was better than any particular aspect of the existing curriculum, which would have required case and control groups.

PALMs incorporate key elements that are typically lacking in the current undergraduate medical setting for teaching histopathology – presentation of many independent examples of each disease process in short classification episodes, adaptive learning and incorporation of RT as a relevant learning parameter. Expert physicians in all areas of medicine engage in pattern recognition when interpreting clinical data as varied as pathology, radiographs, electrocardiograms and psychiatric presentations.[1520] This “pattern recognition” is manifested as the ability to correctly and rapidly categorize new examples of a diagnosis across a broadly varying range of detailed presentations. As this requires exposure to a large number of relevant examples (“experience”), trainees have limited opportunities to engage in structured and sustained practice in pattern recognition.[2122]

We have demonstrated in the present work that PALMs have the potential to provide a resource that helps students move from the slow process of analyzing pathologic features in tissues based on explicit rules to the much faster process of pattern recognition.

Given the short period required to complete the PALM, improvement was impressive. However, for a given PALM, even more improvement is likely to occur by varying one or more of the parameters of the algorithm that underlies category sequencing and retirement. Most obvious would be to increase the number of successive images from a given category that must be answered accurately and within the target RT in order to retire the category. Therefore, we intend to increase the number of images in the system and explore variations in parameter settings as possible ways to improve the effectiveness of the exercise. Additionally, for having an impact across the organ systems, additional PALM exercises would be needed. As a given exercise does not take much time and can be completed at the students’ discretion, these would also serve as useful refresher exercises to maintain skills. The vast majority of students evaluated the exercise positively and expressed a desire to have these available for other organ systems.

At a practical level, development of a histopathology PALM exercise entails a moderate amount of effort from, at minimum, a content expert, an educator and a skilled computer support person with some knowledge of programming, all of whom are typically present at academic centers teaching histopathology. Furthermore, in terms of total “cost,” this particular application of perceptual learning techniques utilized a commercially available software platform from Insight Learning Technology (http://insightlt.com/#home). From the content side in particular, what distinguishes preparation for this approach from the normal teaching situation is the need for multiple unique images of the same process versus the traditional setting where having several good examples is sufficient. Therefore case identification, slide retrieval and review and image capture and annotation may be time consuming and best dealt with over an extended period of time to capture cases prospectively to the extent possible.

A related aspect of development of a PALM that is not initially obvious is the practical limitation posed by the number of categories one wants students to be able to distinguish. We had 4 diagnostic categories (normal, inflammation, cell injury, neoplasia) and two recognition categories for each (“low” and “high” power), yielding eight categories in total, with a minimum of 30 unique image instances for each in the learning module and two additional instances for each category for each of the three test forms. If this were to be further refined, to have acute and chronic inflammation categories for example, the number of images needed would scale proportionately. Furthermore, there would be five versus four diagnostic categories to choose from. With even more categories, there will be a limit to the number of answer choices that is reasonable.

Besides medical student education, the PALM approach could be useful in other settings where proficiency at image interpretation is necessary but where experience and time are limited. One such is for maintaining and testing proficiency of medical technologists for gram stain and peripheral blood cell identification, especially for critical but uncommonly encountered settings. Additionally, throughout medical education, PALM exercises could be useful where the exposure to particular categories of pathology cases is limited.