Newer teaching-learning methods and assessment modules in anaesthesia education.

The National Medical Commission in its recent guidelines has stated that postgraduate training in anaesthesiology should have clear objectives and be competency based. This means that the existing teaching-learning (TL) methods need a major face-lift. E-learning and blended learning including learning management systems, virtual classrooms, app-based learning, flipped classroom, podcasts, webinars, web-based collaborative education, reflective feedback, problem-based discussions and mentorship are some newer TL methods that can be adopted. Simulation can help teach technical and non-technical skills such as leadership, teamwork and communication. In line with TL methods, newer assessment methods have to be applied to revolutionise postgraduate anaesthesia education. Formative assessment and assessment of clinical skills are important and to do this, workplace-based assessment methods such as mini-clinical evaluation exercise, direct observation of procedural skills, multisource feedback, logbook and E-portfolio can be applied. Objective structured clinical examination, simulation-based assessment and E-assessment are other useful evaluation methods. Copyright:

INTRODUCTION

Anaesthesia is unique among medical specialities as mastering it requires considerable expertise in three major learning domains-cognitive (knowledge), psychomotor (procedural skills), and affective (non-technical skills). The National Medical Commission (NMC) in its competency- based postgraduate (PG) training programme for anaesthesiology has emphasised the need for newer interactive teaching–learning (TL) methods to teach competencies that cover the cognitive, affective as well as the psychomotor domains.[1] Newer interactive TL methods are student-centric, with pre-decided objectives to meet the goals of achieving the stated competencies. They promote a higher level of thinking and facilitate problem-solving and decision-making. They allow teachers to receive feedback, and encourage self-directed learning. Adopting these methods in the competency-based PG training will promote active involvement and stimulate interest of the postgraduates. Nevertheless, the coronavirus disease (COVID)-19 pandemic has enabled us to understand the merits and demerits of various online TL methods and led to a revolution in the education process.

Newer TL methods

E-learning and blended learning

Learning management system (LMS), classroom in the cloud (virtual classroom)

E-learning can increase learner engagement and satisfaction.[2] LMS uses organised learning resources. LMS can be an extremely useful platform to provide structured e-learning courses and educators should incorporate it for synchronous or asynchronous learning.[3] Blended learning uses both didactics and e-learning techniques and is a common entity in the current system of medical education after the pandemic. It is now possible to teach, learn, discuss, and converge the globe into the classroom.[4] However, competency to create and deliver digital resources must be made an essential part of “teaching the trainers” programmes to acquire these mandatory skills. As we evolve into the e-learning platform, making the sessions interactive becomes imperative or else there is increased possibility of losing the attention of the learners. Hence, each learning objective should be followed by a formal assessment to encourage the interaction and participation of the learners before the facilitator moves on to the next learning objective.

Modular Object-Oriented Dynamic Learning Environment (MOODLE) is a free and open-source LMS that allows learners to study a topic to the depth that they desire and at the pace that suits them. Massive Open Online Courses (MOOCs) are online courses aimed at wide participation and open access via the web. MOODLE and MOOCs are virtual learning environments that can be used for distance education and training. However, good content is a prerequisite to creating MOOCs and has the advantage of dynamic building up and evolution of content and can be made up of several modules. A module is a set of learning opportunities on a well-defined topic and contains specific objectives, TL activities and self-assessment tools.[5] Modules with presentations, videos with added recorded explanations, and uploaded handouts and worksheets can be accessed by the students along with interactive quizzes and multiple-choice questions and these modules act as an invaluable TL tool.

App-based learning

Use of smartphones amongst health professionals is extremely common. According to the French data, more than 50% use it for medical applications.[6] Use of apps to solve complex clinical scenarios and interpret blood gases serves as an extremely useful aid. Smart wristwatches can now be used to interpret the electrocardiogram and offer immediate treatment through telemedicine.[7] App-based module can be used as an alternative to teach basic life support, instead of instructor-based education.[8] Such applications are now being increasingly developed and might soon replace the regular instructor-based training.

Flipped classroom

Flipped classroom is being increasingly implemented in medical education including anaesthesia education as a new teaching model.[9] It is a model of blended learning in which online pre-recorded lectures/videos of average 20 minutes duration/medium length reading assignments are viewed by the students before the scheduled class so that basic concepts related to the topic are understood by them before the class at their own pace and time. This is followed by live, active classroom sessions wherein teachers and peers engage the students in discussions around clinical applications and problem-based learning. The knowledge gained prior to the class can stimulate student interest, and facilitate face-to-face learning. Several studies including a meta-analysis have suggested that the flipped classroom approach in clinical anaesthesia education and basic sciences in anaesthesia yields a significant improvement in student training and learning compared with traditional teaching methods and promotes student-centred, self-directed, active learning.[101112]

Podcasts, vodcasts

Podcast or vodcasts are audio- or visually-enhanced broadcasts distributed through the internet and are able to be viewed on various platforms such as web pages and on hand-held devices. Use of podcasts is on the rise for continuing medical education, especially in the fields of critical care and emergency medicine. The ease of production, rapid dissemination, and appeal to the millennials has made their use common. Podcasts should be kept relatively concise and should meet quality standards with regard to attribution, evidence, and accuracy.[13]

Webinars

Webinars are a powerful online tool to disseminate knowledge from experts to postgraduates and provide accessibility to students over a wide geographical area. However, they are teacher-centric and interaction with students is limited.

Social media and web-based collaborative education

Electronic forums in the form of social media are a powerful tool for teaching and learning. They provide an excellent educational resource to postgraduates. Clinical problems and unusual scenarios are discussed and educational videos and images encountered in day-to-day practice are shared in these forums.[14] They can be used for networking in medical education including anaesthesia education. This means the conscious creation of interconnected relationships among various students and educators such that they can communicate instantly and directly with experts in their field.[15] There are some social networking forums and sites for medical professionals including anaesthesiologists such as YouTube, SlideShare, DailyRounds, Sermo, Doximity, QuantiaMD, WeMed Up, Figure 1, Doc2Doc, DoctorsHangout, Student Doctors Network, ResearchGate, Linkedin, MEU India, etc. Online social networking can create interpersonal bonds, promote collective learning, and remodel and enhance teaching.[16]

Faculty and postgraduates need to be aware of the numerous shortcomings and must use social media responsibly. Following etiquettes, adopting a code of conduct, and a high sense of professionalism including doing thorough literature review from proper databases and using evidence-based guidelines is expected while posting and learning on social media.

Simulation-based teaching

Simulation mimics a clinical encounter and is defined as a technique that replaces or amplifies real experiences with guided experiences that replicate aspects of the real world in an interactive manner.[17] The benefits of simulation-based teaching include avoiding risks to patients and students and is essential for those competencies that are considered core but feasibility of learning on patients is low. Simulation-based teaching can be tailored to individual student needs and the learning curve for many procedures can be shortened. It is useful for rehearsal and repeated practice of psychomotor skills that form an essential component of anaesthesiology.

Simulation-based teaching is not only beneficial for teaching technical skills, but also non-technical soft skills such as leadership, teamwork, and communication. It enhances patient safety by providing a safe simulated environment for repeated practice and learning by committing errors. Simulation-based training is effective for improving crisis management while ensuring the adherence to protocols. The COVID-19 pandemic has also led to fewer opportunities for postgraduates to “Show How” or “Perform” psychomotor skills and simulator-based training has helped to learn and maintain psychomotor skills. Complex skills and competencies can also be deconstructed into multiple skill stations and suitably tailored to the learning objective.

Simulators may be low fidelity low-cost part trainers, full-body task trainers, high fidelity simulators or virtual reality simulators. High fidelity simulators are not always required to provide effective simulation training and part-task trainers are useful for simulating and learning individual skills. The chosen simulation modality and fidelity should be suited to the learning outcomes for effective teaching. High fidelity simulators are computer-aided mannequins that interact with the learners and are to be utilised when the participant ability is high and are effective for teaching complete scenarios, crisis management and teamwork. Scenarios can be changed, poor outcomes can be depicted, differential diagnosis can be analysed, and comprehensive patient management can be taught. Basic life support, advanced trauma life support, neonatal resuscitation and performance of regional anaesthesia techniques can be taught using low fidelity whole-body mannequins. Emergency procedures such as chest drain insertion and surgical cricothyrotomy can be rehearsed using part trainers. High-fidelity whole-body mannequins can be used for situations like anaphylaxis scenario or the management of an anaesthetised patient developing a severe arrhythmia over a period of few minutes.[18]

Standardised patients and role-plays are human-based simulation methods. Standardised patients are individuals who are trained to realistically portray characteristics of a real patient, thereby affording the student an opportunity to learn in a simulated clinical environment. Initial patient assessment and management, communication skills, professionalism and ethics can be taught very effectively with the help of standardised patients and role-plays.

Simulation-based teaching gives an excellent opportunity for feedback to learners via video recording and debriefing sessions, which allow for reflective learning as well as evaluation. A systematic review and meta-analysis of anaesthesiology simulation-based training found that when compared to no intervention, simulation-based training was associated with statistically significant effects for satisfaction and skills, with large effects for behaviours, and smaller effects for time, knowledge, and patient outcomes.[19] PG training in anaesthesia in simulation labs is currently low (20%) in some states in our country as per the results of a recent survey;[20] nevertheless, the new competency-based PG training programme of the NMC has made simulation-based training mandatory for several skills.[1]

Reflective feedback

Feedback is the cornerstone of effective teaching and learning.[21] Engaging the student in a “reflective feedback conversation”, wherein the student is asked to reflect upon his/her own performance during a debrief after a clinical encounter helps improve upon the performance. The teacher provides actionable guidance, which indicates a path for improvement.[22]

Problem-based discussions

Starting the discussion of a topic with a clinical vignette or scenario is an excellent way to put the topic in perspective. Problem-based discussions are learner-centric and enable students to analyse data in order to reach a conclusion. They promote deep learning, enhance self-directed skills, provide a stimulating learning environment, and promote interaction between students and faculty members. Teamwork, self-directed learning, and reflection are the three important components of problem-based discussions.

Mentorship

Mentorship programmes generate a safe environment to provide crucial support in teaching, research participation, and productivity, and most importantly, create a social bonding among faculty members, students, and residents during their professional training.[23] Mentorship programmes for Anaesthesiology residents have been shown to result in improved performance in both professional skills and theoretical assessments.[24] Resident-managed peer-mentoring programme is a productive method of engaging students and residents in clinically focussed training and can help to foster a laddered mentoring system.[25] However, the autonomy given to residents of good standing in such a programme should be supervised to ascertain that bad habits are not inculcated in the mentees by the peer mentors.

Newer assessment methods

Assessment and feedback are necessary for teachers and learners to gauge progress and for continuous refinement of the curriculum. A single summative assessment at the end of residency does not provide any chance for the student to improve upon or overcome inadequacies. Regular formative assessment throughout residency and meaningful feedback would help the residents to reflect upon their strengths and weakness and adapt accordingly.[26] Assessment should be both - summative and formative as per the recent NMC guidelines and should include the assessment of both clinical and non-clinical professionally useful skills.[1] Simulation-based and Workplace-based assessment (WPBA) tools can be used for formative assessment and the assessment of multiple domains of learning including clinical competence [Table 1].[27] They have been shown to improve trainee scores and also help in identifying underperformers early.[26] Simulation can help assess problem-solving skills in acute care.[18] Some common WPBA methods include mini-clinical evaluation exercise (mini-CEX) and direct observation of procedural skills (DOPS). Other WPBA methods include multisource feedback, logbook, and E-portfolio. Objective Structured Clinical Examination (OSCE), though not novel is another assessment method recommended in the NMC curriculum.[1]

Table 1

Assessment methods and capabilities assessed

Generic Professional Capabilities	CEX	MSF	DOPS	Simulation	CBD	OSCE
Professional values and behaviour	✓	✓	✓	✓	✓	✓
Professional skills	✓	✓	✓	✓	✓	✓
Professional knowledge	✓	✓	✓	✓	✓	✓
Capabilities in leadership and team working		✓	✓	✓	✓
Capabilities in patient safety and quality improvement				✓	✓	✓
Clinical skills	✓	✓		✓	✓	✓
Technical skills			✓	✓		✓
Capabilities in practice
Managing an operating list			✓
Managing multidisciplinary working		✓		✓	✓
Managing outpatient clinic	✓				✓

CEX: Clinical evaluation exercise; MSF: Multi-source feedback; DOPS: Direct Observation Procedural Skills; CBD: Case Based Discussion; OSCE: Objective Structured Clinical Examination

Objective Structured Clinical Examination

OSCE can be a better evaluation tool than conventional methods, especially in terms of objectivity, uniformity, and versatility of clinical scenarios that can be assessed. The OSCE assesses domains that are difficult to evaluate, namely, communication, professionalism, as well as technical skills that relate to patient care [Table 1]. It provides feedback to the students on their progress or performance and allows the assessor to measure its effectiveness. The examiner variability is ruled out because the standards of competence are preset and agreed checklists are used for scoring. The time provided, questions asked, and marking are uniform for the student. Other advantages of OSCE are that it can be designed according to the local needs, departmental policies, and availability of resources. A limitation of OSCE is that it can lead to observer fatigue if he/she has to record the performance of several candidates on lengthy checklists.[28] Some other disadvantages of OSCE include patients’ non-cooperation, and time required for preparation to set OSCE.

Direct Observation of Procedural Skills

DOPS provides an opportunity for direct observation of trainees in the clinical workplace against a structured checklist [Annexure 1] and thereby, offers an authentic measure of performance and clinical competence.[2930] The strengths of this evaluation method are quick feedback to participants, promotion of their practical skills, autonomy during evaluation, acceptability by participants, and formative nature. A systematic review concluded that DOPS can be used as an effective and efficient evaluation method to assess medical students because of its appropriate validity and reliability, positive impact on learning, and high student satisfaction level.[31] DOPS has been perceived as an effective assessment and TL tool by postgraduates and faculty in Anaesthesiology.[32]

Annexure 1

Direct Observation of Procedural Skills (DOPS) Assessment Form

Trainee [Post Graduate (PG)student]					Evaluator (Faculty)
Name: R1/R2/R3:					Name:
Date:
ASSESSOR FEEDBACK:
Verbal and written feedback is a mandatory component of this assessment.
General
Strengths
Development needs
Recommended actions
TRAINEE SELF-ASSESSMENT
I need the assessor in the theatre suite			I need the assessor in the hospital;			I could manage this procedure independently and do not require direct supervision
1	2	3	4	5	6	7	8	9
RATINGS
Ratings should be judged against the standard laid out in the syllabus for the trainee’s stage of training.
N=Not observed D=Development required, S=Satisfactory (no prompting or intervention required) O=Outstanding
Domain						Rating	Comments
1: Describes indications, anatomy, procedure and complications to assessor
2: Obtains consent, after explaining procedure and possible complications to patient
3: Prepares for procedure according to an agreed protocol
4: Demonstrates situational awareness through constant clinical monitoring
5: Administers effective analgesia or safe sedation
6: Demonstrates good asepsis and safe use of instruments and sharps
7: Performs the technical aspects in line with the guidance notes
8: Provides reassurance and checks for discomfort, concerns and complications
9: Deals with any unexpected event or seeks help when appropriate
10: Completes required documentation (written or dictated)
11: Communicates clearly with patient and staff throughout the procedure
12: Demonstrates professional behaviour throughout the procedure
DOPS DETAILS
Name of Procedure:
No. times procedure previously done			Emergency/Elective
Performed in a simulated setting:			Description of the simulation:
Difficulty of procedure: Easier than usual/Average difficulty/More difficult than usual
Trainee’s signature:			Assessor’s signature:

R-1 – first year PG, R-2 – second year PG, R-3 – third year PG Note: Global summary which is a part of the assessment form is not included in this table

Some of the main weaknesses are time constraints, assessor bias, lack of training, and subjectivity.

Mini-Clinical Evaluation Exercise

Mini-CEX is a well perceived, effective, and efficient workplace-based formative assessment tool in anaesthesia PG education.[333435] A number of pre-planned trainee-patient encounters including history taking, physical examination, and informed decision-making take place over 15 minutes in different clinical settings such as the pre-anaesthesia examination/palliative care/intensive care/post-anaesthesia care clinic. Examples of these encounters include preoperative assessment of a patient for emergency caesarean section, assessment of a cancer patient with pain, preoperative assessment of a high risk case, assessment of a postoperative case in the recovery area, etc. The faculty member directly observes the student during the encounter, rates the performance in clinical judgement, counselling skills, organisation, efficiency, professionalism and overall clinical competence in the assessment form [Annexure 2] and provides an immediate feedback for about five minutes.[36] Improvement in student communication and clinical skills, and improvement in learning because of faculty feedback are some positive outcomes of mini-CEX.[3738] Some studies have reported that the exercise can create anxiety, nervousness, and hesitancy in the trainee about being observed.[35] Mini-CEX has been adopted into anaesthesia training programmes in some countries and has been used for evaluation of clinical skills in the Pathway 6 of Educational commission for foreign medical graduates (ECFMG)certification.[33343940]

Annexure 2

Clinical Evaluation Exercise (CEX) Assessment Form

Trainee [Post Graduate (PG)student]	Evaluator (Faculty)
Name: R-1/R-2/R-3: Date:	Name:
Assessor’s (Teacher’s) comments
What did the trainee do well?
Areas for improvement
Agreed action
Feedback
Nil Neutral High
Trainee satisfaction with mini-CEXAssessor satisfaction with mini-CEX	1	2	3	4	5	6	7	8	9	10
Time taken (in minutes) For observation of trainee ' For feedback Interval between observation and feedback
RATINGS
Ratings to be judged against the standard laid out in the syllabus for the trainee’s stage of training.
How do you rate this trainee:
	Unsatisfactory			Satisfactory			Superior			Unable to comment
	1	2	3	4	5	6	7	8	9
Medical interviewing skills
Physical examination skills
Humanistic qualities/Professionalism
Diagnostic skills and underlying knowledge base
Management and follow-up planning
Clinical judgement
Communication and listening skills
Counselling skills
Organisation/efficiency/time management skills
Overall clinical care
CEX DETAILS
Clinical setting: PAE room/Pain and Palliative clinic/OT/ICU	Emergency/Elective
Performed in a simulated setting Description of the simulation:
Summary of the clinical problem:
Focus of encounter:	History	Exam	Diagnosis	Management			Explanation			Consent
Trainee’s signature:	Assessor’s signature:

R-1 – first year PG, R-2 – second year PG, R-3 – third year PG; PAE: preanaesthesia examination; OT: operation theatre; ICU: intensive care unit Note: Global summary which is a part of the assessment form is not included in this table

Multisource feedback

Multisource feedback, also known as ‘360-degree assessment’ provides formative feedback to the trainees, especially concerning ‘generic’ or ‘soft’ competencies assessment in which the trainees are assessed by various assessors who are the people who see them work in real-life; for example, the student can be asked to give a subarachnoid block for a difficult case in the operation theatre. He/she will be scored on several constructs such as communication skills, professionalism, teamwork, organisation and continuing professional development interest, etc., using a checklist [Annexure 3], by the operating surgeons, co-workers including assisting nurses and anaesthesia/operation theatre technicians and patient.[4142]

Annexure 3

Multi-Source Feedback Assessment Form

Trainee [Post Graduate (PG)student]	Rater (Faculty)
Name:	Name:
R-1/R-2/R-3
Clinical setting: PAE room/Pain and Palliative Clinic/OT/ICU	Emergency/Elective
How do you rate this trainee in:	Outstanding	Satisfactory	Development required	Not observed by me
Clinical Care
1. History taking and examination skills
2. Relevant knowledge and diagnostic skills
3. Ability to formulate appropriate management plans
4. Procedural (technical) skills
5. Record keeping (timely, accurate, legible)
Maintaining good medical practice
6. Ability to manage time and work under pressure
7. Decision making and implementation skills
8. Awareness of own limitations (willing to ask for help)
9. Initiative and leadership skills
10. Focus on patient safety (clinical governance)
Learning and teaching
11. Willingness to ask for feedback and to learn from it
12. Teaching (enthusiasm and effectiveness)
Relationships with patients and colleagues
13. Communication with patients and their relatives
14. Communication with colleagues
15. Active involvement with the team
16. Accessibility and reliability
Summary
Overall, how do you rate this doctor compared to other doctors at the same level with whom you have worked?
Please give specific examples relating to any area in which you feel this trainee is outstanding:	Please give specific examples relating to any area in which you feel that this trainee requires development:
Signature:	Date:

R-1 – first year PG, R-2 – second year PG, R-3 – third year PG, PAE: preanaesthesia examination; OT: operation theatre; ICU: intensive care unit

Logbook and e-portfolio

Portfolio assessments that focus on attaining knowledge and skills over time are desirable than traditional examinations from the perspective of student learning. Electronic portfolio is a web-based, centralised electronic system that comprises of personal information, educational and professional achievements, and also career development plans.[43] It is maintained by trainees to record their daily activities comprising of entrustable professional activity assessment, log book of patient care activities, research activities (dissertation, project, audit, papers presented), academic activities (seminars, case discussions, journal clubs, departmental and institute review meetings), credit points and awards (conferences, workshops, continuous medical education sessions), teaching assignments and any other extra or co-curricular activities. Behind any such compilation, there are rich and complex processes of planning, synthesising, sharing, discussing, reflecting, giving, receiving, and responding to feedback. E-portfolios are being developed to improve teaching, assessing, and tracking the development of competence.

Electronic logbooks permit data entry from remote sites, for example operation theatre, ward or intensive care units. Compliance with e-logbooks is seen to be better as they offer an intuitive interface, are less labour intensive and require a mobile device. The residents can edit and customise headings, menus, and other data sets according to their needs.[44] E-logbook can be compiled at the end of the assessment period and submitted as a proof of competency for a given qualification. Online analysis of resident logbook may also help to assess and standardise the curriculum in future. E-logbook is a very useful concept and regulatory agencies like the NMC or Indian Society of Anaesthesiologists can come up with a specific logbook for anaesthesia students.

Other E-assessment techniques

Multiple choice questions, short answer questions, online polls, picture-based questions, electronic patient management problems, discussions, projects, case studies, self-assessment and peer assessment are some other forms of E-assessment. E-assessment is a part of every LMS. E-assessment may have to be used in the near future. Objective structured video examination, though not novel, is a competency-based, valid, reliable method of evaluation of complex psychomotor skills. The student is instructed before the evaluation regarding the specific skill e.g. central venous cannulation to be performed on the mannequin as he/she would perform on the patient and the time limit to complete the task. The procedure is video-recorded. Confidentiality in the identity of the student is maintained and his/her face/voice is not revealed. A close-up view of the performer's hands is obtained. This is followed by a performance-based evaluation of the video-recordings using a checklist. The video-tapes will be subsequently observed by some more faculty members who can rewind the videos if necessary to confirm the presence or absence of each component of the checklist. The student performance is then graded using a global rating score. The online learning and assessment module of the NMC recommends the use of these online assessment formats which have become popular after the coronavirus disease pandemic.[45]

Challenges and barriers in the implementation of newer TL methods and assessment methods

Acceptance of newer TL methods by teachers (especially senior faculty members) can be a challenge and their mind-set has to be gradually changed. Also, faculty members should be adept with using e-technology and the administration should provide the required facilities and encouragement for implementation of the new TL methods. Implementing changes and the newer methods in a stepwise manner can help overcome these challenges. No TL method is complete; for example, online teaching has limitations for teaching psychomotor skills. Several disadvantages are associated with E-learning. These include increased screen time leading to onset and progression of myopia, computer-vision syndrome with dry eyes, blurred vision, headaches, neck and shoulder pain due to faulty posture, sleep disturbances, weight gain and obesity related diseases, and a negative impact on brain function and emotional well-being.[46] However, the use of innovative blended TL methods can help overcome these obstacles. WPBA is not a replacement for conventional assessment but should be used in judicious combinations. Simulation also has its own limitations, especially the extent to which a simulator can reproduce real-life situations.[18] Simulator availability and funding are other common limitations to simulation use.[47] Also, training of faculty as well as extensive preparation is required for effective simulation-based training. Hence, it is always better to use a combination of TL and assessment methods.

SUMMARY

There are several newer TL and assessment methods and many of them can be applied to PG anaesthesia education. The subjects of anaesthesiology and medical education are rapidly growing and a revolution in TL and assessment methods is now taking place.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.