Addressing assessment in libyan medical education.

ASSESSMENT IS A POWERFUL DRIVER OF STUDENT LEARNING: it gives a message to learners about what they should be learning, what the learning organisation believes to be important, and how they should go about learning. Assessment tools allow measurement of student achievement and thereby give teachers insight into their students' learning, and enable teachers to make systematic judgements about progress and achievement. It is vital then that assessment tools drive students to learn the right things as well as measure student learning appropriately. Any attempts to reform curricula and teaching methods must consider the role of assessment in the learning process.Libyan doctors and medical students have been calling for changes to teaching and assessment methods at undergraduate and postgraduate levels. A team from the Academic Centre for Medical Education at University College, London have been running workshops in conjunction with the Libyan Board of Medical Specialties since 2006 to discuss strategic aims of assessment in medical education in Libya for the 21st century and to deliver an assessment skills course to Libyan educators. This article outlines the course and the outcomes of preliminary discussions between academics from the UK, participants in the assessment courses and representatives from the Libyan Board of Medical Specialties. As a result of these discussions it was agreed by all that Libyan Medical School assessment methods need updating and, despite significant challenges, changes in assessment must be made as soon as possible. There is a real need for support in both addressing these changes and for practical training for assessors in contemporary assessment methods.

‘Examinations are formidable even to the best prepared, for the greatest fool may ask more than the wisest man can answer’

Charles Caleb Colton, 1780 – 1832

Introduction

Current context

In both the UK and the US, medical education, including assessment, has been transformed over the last 30 years, driven by changes to the medical environment in which graduates will work. There has been a shift away from the traditional apprenticeship model of learning in medicine: the student is no longer considered an empty vessel to be filled by those who have obtained the desired competencies [1] instead it is now widely recognised that medical education is preparation for the multitude of demands of professional practice [2]. Traditionally, clinical competence has been assessed by measuring students' ability to recall and apply knowledge. This is no longer considered a sufficient guide to ability and subsequent performance. Instead before they embark on careers as medical professionals, students must demonstrate their competence in terms of professional attitudes and skills as well as knowledge [2]. This means medical schools need to have the tools and understanding to systematically, appropriately and reliably assess the competence of medical students.

Assessment is a term that refers to a spectrum of activities that take place in the learning environment to measure the learning that has taken place. In its broadest sense it can be considered as ‘about getting to know our students and the quality of their learning’ [3]. It can be formal, informal, formative or summative, objective or subjective, learning can be measured against peer or against a pre defined criteria and conducted by the learner, his peer or the teacher [4]. Every assessment that is provided gives a clear message to the learner about what they should learn and how they should go about that learning. Learners can, to some extent, escape from the effects of poor teaching, by creating their own learning strategies and opportunities; however, if they wish to graduate and practice, they cannot escape the effects of inadequate assessment [5]. Students will often learn only what is going to be assessed. It is a powerful driver of student learning and so it is important to get it right (see supplementary material for the qualities of a “good assessment”).

Libyan doctors and medical students are calling for changes to teaching and assessment methods at both undergraduate and postgraduate levels [6,7,8]. In December 2006 staff from University College London's Academic Centre for Medical Education (ACME) were invited by the Libyan Board of Medical Specialties (LBMS) in Tripoli to facilitate the first of a series of one-day workshops on assessment and to initiate the discussions about developing a contemporary assessment strategy.

This article outlines the course and the outcomes of preliminary discussions between academics from the UK, participants in the assessment courses and representatives from the Libyan Board of Medical Specialties regarding the future of assessment in Libyan medical education.

Assessment course aims and objectives

The one day course aimed to increase participants' knowledge of up-to-date assessment theory and methods and to promote discussion about the future for assessment in Libyan medical education.

The objectives of the course were that by the end of the training day participants would:

Have explored the definition and purpose of assessment

Be familiar with key concepts in assessment

Have a framework for selecting and using appropriate assessments in their own setting

Have considered their role as an assessor

Be familiar with the Objective Structured Clinical Examination (OSCE) assessment method

Have practiced three contemporary assessment techniques

Course content and format of the day

The day consisted of a morning of interactive sessions facilitated by ACME academics and practical workshops to develop skills in the afternoon, comprising the following elements:

An overview of assessment

Selecting appropriate assessments

Appropriate assessor behaviour

An introduction to the OSCE

A Multiple Choice Questions (MCQ) writing workshop. Participants practiced writing useful, reliable and valid written examination questions

A Mock OSCE workshop: participants played roles of students and examiners on a 4 station OSCE. Libyan medical students and junior doctors acted as simulated patients.

The facilitated sessions and practical workshops were well evaluated by participants.

The day was rounded off with discussions between UK and Libyan academics and medical educations on how assessment should be taken forward in Libyan medical education. The group discussed:

How assessment should change to fit in with contemporary medical education in Libya

What assessment ‘toolkit’ might work

How assessors and candidates should be prepared for assessments

Result of the discussion regarding the future of undergraduate assessment in Libya

The discussion for taking assessment forward produced some interesting ideas. The Deans of the Libyan medical schools have agreed that there is a need for change in assessment practices. The LMBS endorsed this need for change and a lively discussion produced agreement on the following principles:

Libya needs to update assessment methods

Planning is key to successful developments

Those in positions of responsibility for assessment must commit to the process of change

OSCEs should be used throughout medical education (undergraduate, postgraduate and education of professions allied to medicine)

Training and support to write and validate MCQs are required

A single comprehensive question bank should be created and used by all medical schools. For each MCQ, a standard should be set when it is entered into the bank and stored in the question bank as well, in order to identify hard and easy questions

Assessors should be trained in assessment behaviors, particularly to help them behave in an objective and standardized manner. Crucially they need to understand the curriculum being tested and need to be able to give effective feedback.

Candidates should be given opportunities to learn about and practice assessments so that they understand assessments that they are expected to take, and have access to learning tools and resources to prepare them

Assessments must be fair

Resources are necessary to implement change on a wide scale

All key assessors must attend the ACME assessment course

Despite the challenges to be faced, changes in assessment must be made as soon as possible

It was agreed that concrete steps need to be taken and that the Faculty of Libyan medical schools need to identify:

the time frame for change and key personnel who will take a lead in the change process

resources and requirements to implement these changes

a time frame for piloting the OSCE at one Medical School

Conclusion

Medical education is changing the world over and it is essential for Libyan medical educators to keep abreast of local and international demands that are made of today's healthcare providers. Medical educators have a responsibility to train professionals who are able to provide excellent clinical care. Today this means they need to understand and integrate clinical knowledge, skills and attitudes in order to be able to diagnose effectively and perform safely in the context of developing good relationships with patients and colleagues. Assessment is an integral part of this process.

Participants on this pilot course overwhelmingly agreed that there was much to be done and that change needed to happen. Participants debated how to take things forward and agreed that Libya is ready to make changes to assessment. This will require tools, equipment, time to change, training for all involved and, above all, it needs commitment to change.