Teaching Medical Students About Attention Deficit Hyperactivity Disorder (ADHD): The Design And Development Of An E-Learning Resource.

In this paper, we describe how an e-learning resource on Attention Deficit Hyperactivity Disorder (ADHD) for medical students was designed and developed. The aim of the resource was to provide students with essential knowledge and understanding about ADHD prior to their attendance at a classroom teaching session as well as to serve as a revision tool. The paper focuses on the way in which instructional design, educational and multimedia principles were used to inform the development of the resource. It also reports results of a small-scale evaluation of students' satisfaction with the resource and the way in which they believed it impacted knowledge acquisition of ADHD related concepts and principles. In addition, we consider ways in which the resource could be further utilized and evaluated, for example as part of a flipped classroom approach to learning, and whether this would be a useful model to use when teaching other aspects of Child and Adolescent Psychiatry.

Introduction

The increased complexity of medicine requires medical schools to review and revise the undergraduate medical curriculum, often resulting in addition of new topics.1 However, the time available for teaching is limited and educators may struggle to cover important topics within an increasingly constrained timeframe.2 In an attempt to address this issue, educators have looked for teaching techniques that will help them maximize the use of teaching time to ensure the ever-expanding curriculum is covered. E-learning resources are being increasingly used as teaching aids in medical education. Such resources have many advantages for medical students such as enabling them to engage in independent and individualised learning,3,4 as well as offering students the flexibility to study at a time that suits them and at their own pace.5 High rates of student satisfaction with e-learning have been reported, with many finding it an effective way of learning.6,7 It is interesting to note however, that many medical students do not anticipate e-learning completely replacing traditional teaching methods,8 viewing it more as a tool to complement them in a “blended” or “flipped classroom” approach.9 Moreover, e-learning has been shown to be at least as effective as traditional lecturing in many medical disciplines, and across many stages of medical education.4,5,10

The importance of using well established and robust educational principles when designing e-learning resources has been discussed extensively in the literature.7 However, newcomers into this area may find it difficult to decide which theory or framework to use. This paper adds to the current literature on e-learning and the teaching of undergraduate Child and Adolescent psychiatry (CAP) by describing how instructional design, educational, and multimedia principles were used in grounding the design and development of an e-learning resource () to teach the principles of ADHD to medical students. It also reports preliminary evaluation by considering students’ satisfaction and the impact of the online resource on their knowledge of ADHD. Consideration is given to how the resource could be evaluated more extensively and rigorously in order to identify its educational value and impact on knowledge and skill acquisition.

Background

The amount of time devoted to teaching CAP to undergraduate medical students in medical schools across the world is, on average, small and there is limited agreement about appropriate curriculum content.11 Given the increase in prevalence in Child and Adolescent mental health problems, calls have been made to increase the amount of teaching of associated topics and to prioritise teaching CAP clinical skills over CAP knowledge.12–14 For example, in a recent Delphi study of undergraduate curriculum content in CAP, participants gave a high priority to application of knowledge and to clinical skills such as interviewing parents and taking a child and adolescent psychiatric history.15 In an attempt to help educators maximize the use of available CAP teaching time, an e-learning resource for medical students on ADHD was developed as it was thought this might be a useful adjunct to classroom approaches to teaching ADHD knowledge or related clinical skills.

Overview Of Design And Development Principles

The design and development of the ADHD e-learning resource were based upon a range of instructional design, educational and multimedia principles. A literature search revealed the ADDIE model (Analysis; Design; Development; Implementation and Evaluation) for designing and implementing training to be particularly popular amongst educators, instructional designers and training developers.16,17 This is because the clearly defined stages help organize teaching content and aid the implementation of teaching tools. However, ADDIE is an instructional design methodology that is not specific to e-learning. This is where Overbaugh’s18 guidelines for development of computer-based instruction can prove particularly useful, as they address more e-specific issues that ADDIE may miss.

In the ADDIE model, the development of instructional materials usually starts by analyzing the requirements of the resource.16,17 Developers of e-learning resources need to understand the People the resource is targeting, what they need to know, what their level of pre-existing knowledge and skills are, their preferred way of learning, what the learning outcomes of the resource will be and how the resource will be delivered.19 A useful framework which can be used at this stage is the PACT (People; Activities; Contexts and Technologies).20 This framework aids the development of clearly defined learning objectives that reflect what the students should be able to do after completing the resource; the conditions under which they need to perform them and the acceptable standard,16,21 before relevant learning activities and assessments are designed.22 Learning activities are defined by Beetham23 as “a specific interaction of learners(s) with others using specific tools and resources, oriented towards a specific outcome” (p. 28) and are considered to be of primary importance with supporting materials taking a secondary place.23,24 Context refers to the physical and social environments for the learning activities as well as the support available and the learner’s personal circumstances. Finally, technologies need to be considered with the internet clearly having a central role in providing information and ways for e-learners to interact.

In the design phase of the ADDIE model, a systematic approach should be taken by considering lesson planning, instructional methods, learning objectives, learning activities, content, use of multimedia and the assessment methods to be used.17 Resources should be designed with learning theories in mind, viewing learning as a behavior, as a construction of knowledge and as a social practice25 with cognitive aspects playing a central role, given that the focus on e-learning is usually on individual learners.19

In the development phase, instructional materials and proposed learning content are then further developed and assembled and additional details such as the font to be used, colour and use of graphics are considered.16,17 If using illustrations or pictures obtained off the internet it is important that the correct permissions have been obtained or to those available under a creative commons license have been used. The resource then needs to be tested and amended in response to the feedback.

Useful to consider during the development phase of the ADDIE model are Overbaugh’s18 guidelines for the development of computer-based learning, which are based on Gagné’s26 nine events of instruction framework. Overbaugh grouped the nine events into three domains and added a fourth domain for issues unique to computer-based teaching. The first domain in Overbaugh’s18 guidelines for the development of computer-based learning is known as instructional set and aims to help the learner to engage with the information that is going to be presented. This domain draws on the first three of Gagné’s nine events of instruction and includes gaining learner attention, informing learners of the learning objectives, and stimulating learner recall.26,27 The second of Overbaugh’s18 domains considers teaching strategies and draws on Gagné’s fourth, fifth and ninth events of instruction, that is, presenting the content, providing learner guidance and eliciting performance.26,27 This includes ensuring that the way the teaching stimulus is presented that is the text, the colour and the graphics are distinctive, providing learner guidance and considering the use of strategies which will facilitate the retention and transfer of new knowledge. The third domain considers eliciting student performance, providing feedback and assessing performance which are also stages six, seven and eight of Gagné’s nine events of instruction18,26,27 whereas the fourth domain considers other design issues such as learner control and teaching tools, e.g., to reduce anxiety and direct learner attention.18

The development phase in the ADDIE model is followed by implementation.17 At this stage, any training required to use the e-learning resource is prepared and checks are made to ensure that it is fully functional before it is shared with the learners. In order to reduce the impact of any problems at this stage, an initial pilot run with a small number of learners is advisable so that any teething issues can be addressed before the resource is made available to the wider intended audience. Evaluation is central to the ADDIE process and should be considered at every stage.17 For example, in the early stages, it can help determine if the prototype is fit for purpose.28 It is also important to test for both usefulness and usability as these are the main components of effective learning.29

Pedagogical theories appropriate to e-learning should also be considered and e-resources for adult learners should be based on the principles of adult learning theory or andragogy.30 The overarching assumption is that learners are self-directed and independent in their approach to learning, with a preference for flexibility and autonomy in their learning environment. Of relevance to the development of e-learning resources are Mayer’s principles of multimedia instructional design.31,32 These principles were developed from learning theory and based on evidence and have been shown to improve students’ retention of knowledge33 and their ability to apply what they have learnt in new situations.34 The way in which the underpinning educational and design principles, including Mayer’s multimedia principles32 were applied when developing the resource is described in detail.

Putting Theory Into Practice During The Design And Development Of The Resource

A PACT analysis20 was conducted to analyse the requirements of the ADHD e-learning resource. The people that the resource was being developed for are medical students. As undergraduates, they will be at least aged 18 when they start the medical course, and are likely to be both intelligent as well as computer literate. Being of the “net generation” they are likely to have a preference for learning environments that are multimedia rich and to enjoy a self-directed inquiry approach.35 The learning outcomes for the ADHD resource are shown in Box 1. These represent the lower levels of knowledge, comprehension and application in the cognitive domain of Bloom’s taxonomy, which is used to consider learning outcomes in terms of their complexity and specificity.36

Box 1

Learning Outcomes Of The Online ADHD Resource

After completing this resource you should: Know the clinical features and DSM-V diagnostic criteria of ADHD in children and adolescents Know how ADHD in children and adolescents is assessed Know the appropriate evidence-based treatments for ADHD according to a child or young person’s age and clinical presentation

The learning activities in the ADHD resource all relate to the learning objectives in Box 1. Additional materials on ADHD were sourced from books, clinical guidelines, reputable websites, online ADHD teaching resources and YouTube. It was envisaged that the context of ADHD resource was that it would be able to be used anywhere and at any time the student had access to a computer and the internet and would be accessible to people with different operating systems and browsers. The resource takes approximately 15 mins to complete, more if a number of the hyperlinks are used. It is suitable for individual study, for example, as preparation for a teaching session about ADHD but also for individuals wishing to acquire knowledge about ADHD as well as to aid revision of the topic. It is for continuous use but can also be stopped and restarted at any point. It does not require any tuition/demonstration or for a manual to be read before use. The technology used by ADHD resource is Xerte Online Toolkits (XOT), which is a free open source tool developed for educational purposes by the University of Nottingham.37 Xerte allows the designer to achieve a high level of interactivity and is also user friendly and suitable for use by non-programmers and those without any specialist IT knowledge or skills. The designer does, however, need to have expertise in relation to the page content and the underlying educational theory behind the lesson design. Once published, the learning resource can be amended either by the author, the learners, or by others as it is a shared resource. XOT is accessed via a web browser (Firefox is recommended) but does need Adobe Flash Player to run.

Using the second stage of the ADDIE model,16,17 the resource was then designed to meet the needs of the learners. After the learning outcomes had been determined (see Box 1), the instructional methods and media as well as the strategy for instruction were chosen. Tutorials, examples and page templates available on the Xerte Project website were then examined.37 The “flexible” XOT page template was chosen as this allowed for different content on the slides. A colleague had given permission for cartoons he had drawn to be used to illustrate some of the slides. To enhance student learning, a combination of conceptual scaffolding as well as procedural scaffolding was used.38 The former helps to focus the students on particular aspects of ADHD (e.g. symptoms/diagnosis, assessment and treatment) whilst the latter helps them use the resource. Thus, in the resource, symbols were used to differentiate tasks which need to be completed (§) from additional information/resources that students may wish to access (⌘).

Continuing to follow the ADDIE model,16,17 during the development phase of the ADHD resource, Overbaugh’s18 guidelines were used in conjunction with Meyer’s32 multimedia principles. Student attention is gained through the title “Naughty Callum”, which also indicates the intention to use a case study. Strategies to facilitate maximum sensation are utilized by placing the title in the centre of the slide and using a cartoon picture as a background. By presenting them close together, Mayer’s32 spatial contiguity principle is used, which helps reduce cognitive load (see Figure 1).

Figure 1

ADHD resource Slide 1 showing use of Mayer’s spatial contiguity principle.

The orientation slide at the start of the ADHD resource states the learning outcomes (see Box 1). Mayer’s32 pre-training principle is used to ensure that students have the necessary prior knowledge about the names and characteristics of the main concepts they are to be taught about by offering them hyperlinks to pre-requisite knowledge about ADHD. The provision of anonymous access to remedial materials is also known to relieve information anxiety, that is the gap between what the student knows and what they think they should know.39 An overview slide (see Figure 2) is then provided using a case study of a child (“Callum”) presenting with symptoms suggestive of ADHD. A Case-Based Learning approach (CBL) is used as this presents the learning in an authentic “story-like” manner and enables learners to remember concepts, events, and processes and to recall information.40 Maier and Warren41 suggest that the cases selected are either historically important, unusual or, as chosen for the ADHD resource, are representative of the condition. The case study is used as an advance organizer,42 introducing the topic of ADHD, providing a structure for student thinking and acting as a bridge between their prior knowledge about ADHD and the new information they are about to learn which will help it to be more easily remembered. Meyer’s signaling principle32 is used to highlight important words in bold where it is not possible to delete extraneous material. Signaling helps guide the student’s attention toward the essential material and reduces the processing of unnecessary information. After the ADHD case study, a hyperlink is then offered to a video clip of a similar case. Using Meyer’s32 multimedia principle that explanations are better presented in words and pictures than just words, alone a cartoon picture is then used to illustrate the text (see Figure 2).

Figure 2

ADHD resource Slide 3 showing use of Mayer’s multimedia principle.

The first few slides of the ADHD resource stimulate recall of the student’s prior knowledge by using pre-questions and supplying missing prerequisites via hyperlinks to additional resources. This acts to bring the students up to the same level, as well as catering for their different needs. Interactive formats, e.g., quiz, drag and drop answers (see Figure 3) or writing free text are used.

Figure 3

ADHD resource Slide 7 showing use of Interactive formats to stimulate student recall of prior knowledge.

The different learning styles of the students are catered for by providing similar information using a variety of media, e.g., diagrams or video links on some of the slides. Cartoon graphics, interactive exercises and links to multimedia resources are used throughout the resource to maintain the student’s interest. Learner guidance is offered by returning to the case study at different points throughout the resource and elaborating on Callum’s progress. The material covered is then reviewed at the end of the resource as this has the effect of enhancing retention, particularly as it is done prior to the final assessment and also aids transfer to practice43 (see Figure 4).

Figure 4

ADHD resource Slide 18 showing review of learning materials to enhance retention.

Interactive exercises are provided throughout the resource to elicit performance and provide feedback which is immediate, as this is the most effective, and explanatory, with hyperlinks to remedial information to increase motivation.44 Assessments that are designed to measure student performance against the learning outcomes occur throughout the ADHD resource. Students are provided with a pre-test before the main learning section, as well as interactive learning activities and self-test items throughout. A final quiz at the end of the resource assesses performance and whether the learning outcomes have been met. This consists of ten questions. The first seven questions test the student’s memory of what they have learnt (i.e. their recall) and the final three questions test their understanding (i.e. whether they can apply their new knowledge to a different situation). A hyperlink is then given to a humorous video on ADHD.

Throughout the resource, Meyer’s personalization principle32 is adopted, where a conversational rather than a formal style is used to present the material as this is thought to create a sense of social partnership and result in students trying harder to understand. Other principles of multimedia instructional design32 were also considered during the development phase of the ADHD resource. For example, Meyer’s segmenting principle32 that offers learner-paced segments in a logical and organized structure, allowing students to proceed when they are ready, thus reducing the risk of cognitive overload and Mayer’s coherence principle32 keeping the content of the ADHD resource simple whilst still being meaningful, relevant and interesting, and avoiding the use of unnecessary words, pictures or sounds.

Keller’s45 ARCS (Attention, Relevance, Confidence, Satisfaction) model to motivate students to learn was also considered during the development phase of the resource. The students’ Attention is captured by the use of the case study and illustrative cartoons. The Relevance of the resource is made clear as students are informed at the beginning that “This is a teaching resource on ADHD for medical students”. Student Confidence is raised by informing them of the session outcome and student Satisfaction is increased by providing opportunities for self-testing with feedback, enabling students to check their progress.

Using the ADDIE model, during the implementation phase of the e-learning resource, it was important to ensure that the resource that had been designed and developed was actually functional. A number of limitations were encountered with XOT. For example, although advised it would run on Firefox, this was not the case, and another browser (Chrome) had to be installed. Using Chrome, XOT was still unstable, and kept losing connection, having to be frequently restarted with the loss of any unsaved work. Text size had to be the same on all slides which led to problems with “drag and drop” tasks. The background colour is published as, and reverts back, to blue. The ADHD resource is however still accessible to those with visual impairment or colour blindness i.e. difficulty distinguishing between certain colours, as users can alter the text size or background colour themselves.

The ADHD resource needed to be “published” either as Flash or Html to create the necessary hyperlink for it to become operational. The former was chosen, which unfortunately does mean that it cannot be run on a tablet or smartphone as both lead to major changes in the formatting, e.g., loss of bullet points, and bold type, and changes in background colour and slide layout which is clearly a limitation.

The final phase of the ADDIE model is evaluation although in practice, this should occur throughout the development of a learning resource, and not just at the end. With this in mind, the ADHD e-learning resource was evaluated at a number of stages. For example, informal feedback on the format and content was initially sought from an academic child and adolescent psychiatry colleague who had a special interest in ADHD. A brief 12-item “usability” questionnaire was then developed taking one item from each of the areas in Zaharias and Poylymenakou’s46 64-item usability questionnaire for e-learning applications. These areas are: navigation, learnability, accessibility, consistency, visual design, interactivity, content and resources, media use, learning strategies design, instructional feedback and assessment and learner guidance and support. Although this approach would be inadequate for a research study, given that this was a small-scale evaluation and not research, it was considered to represent a compromise and could be replicated by others wishing to develop similar online resources for use within their own institutions.

In the first pilot of the resource, ten medical students were offered teaching on ADHD using the resource whilst attending a teaching day on CAP. They were then asked to complete the 12-item “usability” questionnaire. Even though the sample size was small, this was considered sufficient as it has been found that 95% of usability problems can be discovered in a sample of just five to six people.47 Further revisions to the ADHD resource were made in accordance with the students’ feedback at this stage. For example, a new instruction was written for users on how to change the font size on a drag and drop exercise to facilitate completion of the exercise and then how to change it back before continuing (as font size has to be the same throughout the resource when using XOT), some information was re-presented as bullet points rather than a paragraph of text and additional information was given in the quiz section about why the answers given were correct/incorrect.

The ADHD resource is designed to collect ongoing user feedback via a SurveyMonkey link on the last XOT slide. This comprises of 5 questions from the 12 questions in the original usability questionnaire asking users to indicate their agreement with a number of statements relating to the resource, one question asking if students would recommend the resource to other medical students and an additional free text question asking for suggestions on how to improve the resource (see Figure 5).

Figure 5

Survey Monkey Feedback Questionnaire At End Of ADHD E-Learning Resource.

In the second pilot, medical students were given a link to the ADHD e-learning resource and asked to look at it prior to attending a small group teaching session on ADHD. Fifteen students also completed the SurveyMonkey feedback questionnaire at the end of the resource. As can be seen in Figure 6, for all 6 questions, at least 75% of the students agreed or strongly agreed with the statements related to the ADHD resource. Notably, of the 15 students who completed the feedback, 14 students indicated that they agreed or strongly agreed that the resource covers the subject (ADHD) in sufficient breadth and depth to meet the learning objectives. 87% indicated that they would be likely to recommend the online resource to another medical student. Free text comments made by students were positive, also indicating areas that could benefit from further development. Some useful comments on how the resource could be further improved were also offered, e.g., offering additional explanation when an incorrect answer was selected.

Figure 6

ADHD online resource: Results of the feedback questionnaire.

Conclusion And Future Directions

This paper describes the design and development of an e-learning resource for teaching ADHD to undergraduate medical students and demonstrates how educational theory, instructional models and principles can benefit the development of e-learning resources.16–18 Using well-established theories and models aided in the organization of content and in designing a learner-centered resource. Moreover, preliminary evaluation revealed that students were satisfied with the resource and felt it helped them acquire knowledge prior to the teaching session. With this in mind, the resource developed may be suitable for use in a flipped classroom approach to teaching ADHD knowledge to medical students. The flipped classroom approach to education is being increasingly considered to be a useful way to deliver essential content to students48 Students are provided with material to gain a baseline level of understanding prior to attending a teaching session. This facilitates deeper understanding of concepts and allows in-class teaching time for addressing questions and concerns.49 This method is gaining momentum in undergraduate teaching as it relieves classroom time of purely didactic teaching, and allows for the information gained prior to attending the teaching session to be applied (e.g. in a role play clinical scenario).50 This moves students away from passive learning towards more active learning methods, which have been shown to increase the performance of students as well as giving them autonomy over their own learning.51 Materials used prior to attendance at flipped classroom sessions can take a number of forms but increasingly, e-learning resources are being used.35 Further evaluation of use of the ADHD resource in that context would now be helpful.

The resource could also provide pre-requisite information on ADHD prior to a clinical skills session for example, where students practice interviewing a parent concerned that their child might have ADHD. In addition, it could act as a revision aid as the interactive sessions can give real-time feedback, through quizzes and end of topic tests with use of active recall. Given the shortage of available teaching hours for undergraduate medical students in CAP, the development of similar e-learning resources may also offer a way to increase time teaching CAP clinical skills related to other disorders whilst providing students with access to self-directed teaching resources to increase their associated knowledge. For those who may consider developing such resources, this paper demonstrates the value of using well-established educational theories and design principles to guide the process.

It is important to note that the evaluation presented in this paper forms part of routine teaching evaluation and cannot be regarded as evaluation research. At present we can only reflect upon some elements of Kirkpatrick’s52 lower level of evaluation, but this could be expanded upon by considering students’ perceptions of their engagement and their interaction with content. In addition, further evaluation could expand to examine learning, changes in behavior and long-term impact through longitudinal studies. This will provide a more robust and comprehensive evaluation of the educational value and the impact of this teaching intervention on knowledge and skill acquisition.