Managing Obstetric Emergencies and Trauma (MOET) structured skills training in Armenia, utilising models and reality based scenarios.

BACKGROUND: Mortality rates in Western Europe have fallen significantly over the last 50 years. Maternal mortality now averages 10 maternal deaths per 100,000 live births but in some of the Newly Independent States of the former Soviet Union, the ratio is nearly 4 times higher. The availability of skilled attendants to prevent, detect and manage major obstetric complications may be the single most important factor in preventing maternal deaths. A modern, multidisciplinary, scenario and model based training programme has been established in the UK (Managing Obstetric Emergencies and Trauma (MOET)) and allows specialist obstetricians to learn or revise the undertaking of procedures using models, and to have their skills tested in scenarios.
METHODS: Given the success of the MOET course in the UK, the organisers were keen to evaluate it in another setting (Armenia). Pre-course knowledge and practice questionnaires were administered. In an exploratory analysis, post-course results were compared to pre-course answers obtained by the same interviewer.
RESULTS: All candidates showed an improvement in post-course scores. The range was far narrower afterwards (167-188) than before (85-129.5). In the individual score analysis only two scenarios showed a non-significant change (cord prolapse and breech delivery).
CONCLUSION: This paper demonstrates the reliability of the model based scenarios, with a highly significant improvement in obstetric emergency management. However, clinical audit will be required to measure the full impact of training by longer term follow up. Audit of delays, specific obstetric complications, referrals and near misses may all be amenable to review.

Background

The Newly Independent States (NIS) of the former Soviet Union have struggled with the Soviet legacy since their transition to open societies, particularly in the realm of health care [1]. Maternal mortality rates in the Western Europe have fallen significantly over the last 50 years and the maternal mortality ratio now averages around 10 maternal deaths per 100,000 live births. However, in some of the NIS the ratio is nearly 4 times higher, at 40.2/100,000 live births [2] and it could be even higher in some regions, but there are no reliable statistics to confirm this at present. As 15% of all births are complicated by a potentially fatal condition, emergency services must be made available. Indeed, it has been suggested that 'providing skilled attendants able to prevent, detect and manage the major obstetric complications, together with the equipment, drugs, and other supplies essential for their effective management, is the single most important factor in preventing maternal deaths'.[3]

Ensuring that birth attendants are skilled, is therefore a key remit of all agencies committed to the Safe Motherhood Initiative. In the UK, the Confidential Enquiries into Maternal Death have contributed substantially to better emergency management and they continue to be a source of advice for further improvements in many areas eg. hypertensive disease of pregnancy and obstetric haemorrhage. In relation to deaths from hypertensive disease of pregnancy, the assessors recently recommended that appropriate protocols should be available, 'to prevent junior staff being exposed to potentially dangerous clinical situations of which they have little experience'. Obstetric haemorrhage remains a leading cause of maternal death worldwide and in the majority of the cases in the UK, care remains 'substandard'.[4] The enquiries have repeatedly encouraged units to organise 'fire-drills', so that when emergencies occur all members of staff know exactly what to do. In addition, it remains essential for all health professionals to be familiar with the fundamentals of cardiopulmonary resuscitation [5], including airway management. Resuscitation of the pregnant patient requires awareness of specific guidance in relation to the tilted position for cardiac massage and the urgency of peri-mortem caesarean delivery.[6]

At the same time as endorsing every effort to make motherhood 'safer for mothers', it is important to keep a parallel focus on the fetus and neonate. Similar lessons from past mistakes can be learnt. In the UK every year about 600 normally-formed, mature babies die unexpectedly during labour. In July 1997, CESDI [7] published a detailed report on intrapartum stillbirths. It identified sub-optimal care in nearly 80% of these cases, and estimated that in at least 50% of cases better care 'would reasonably be expected to have made a difference'. Specific recurring intrapartum complications include shoulder dystocia, where inexperience can cost minutes and make the difference between life and death, assisted vaginal deliveries of breech pregnancies and instrumental deliveries. An established prevalence of poor technique in instrumental delivery was apparent in a CESDI review of cases where babies died.[8] In this review it was clear that many basic rules with regard to instrumental delivery had not been adhered to, as some of the deaths were related to multiple attempts at delivery with different instruments. Alternative methods of training need to be utilised, including models and scenario teaching.

Whilst it is important for clinicians to learn common skills well before practising on labour ward, it is also essential that rare complications are rehearsed with models. For example, the RCOG has recommended that consideration should be given to training operators to be able to undertake a symphysiotomy should the after-coming head of a breech be entrapped.[9] Given the rarity of symphysiotomy, this clearly requires model usage. Other rare procedures required in obstetrics which need to be taught using models include internal podalic version of the second twin, management of inverted uterus, decompression craniotomy and decapitation.[10] In the Confidential Enquiries there has also been increasing recognition of the contribution of maternal trauma, resulting from accidents, burns and from domestic violence. These issues are of relevance in every country of the world.

It was for these reasons that we established a modern, multidisciplinary, scenario and model based training programme in the UK; Managing Obstetric Emergencies and Trauma (MOET). This course allows specialist obstetricians to learn, or revise, how to undertake these and other procedures on models, and then to have their skills tested in scenarios.[11] The number of courses held each year in the UK is rapidly expanding and feedback from course attendees continues to be positive. In November 2000 a modified MOET course was run in Bangladesh. Given its success, we were keen to evaluate this programme in another different setting. An opportunity arose in Armenia, where 'Family Care' had already been working for some time in Nagorno Karabach.

Decreased neonatal mortality rates, rising attendance rates of women for prenatal care and positive changes in health practices are among some of the successful outcomes of Family Care's three year support programme to maternal health in Nagorno Karabach. "Family Care", has built its programme based on 8 years of experience in support to the health sector in the region. In order to produce sustainable outcomes the organisation stresses the need to implement comprehensive programmes which upgrade the services from central referral level out to the primary peripheral facilities. Given the poor conditions resulting from economic collapse in countries of the NIS, this entails structural rehabilitation with proper equipping and furnishing of facilities accompanied by intensive training activities and development of appropriate health information systems. Professional development is inclusive of all disciplines of staff including management, clinical and auxiliary workers. Training is conducted both by local consultants and experienced expatriate health specialists and emphasises the care of the normal prior to addressing complications and pathology. Priority is also given to activities addressing the needs of families which includes introduction of prenatal education for parents, their involvement in care of sick newborn and the dissemination of information both on health issues as well as hospital services.

Having completed the rebuilding of the Maternity Unit in Stepanakert, the focus of the Family Care skills programme was to focus on 'promoting normality', supported by a midwife from the UK (EB). Thereafter the charity was keen to introduce modern training methods for management of emergencies.

The objectives of the Family Care sponsored MOET course were:

a. to build on the existing improvements in the Safe Motherhood infrastructure in Nagorno Karabach.

b. to revise existing obstetric skills and develop new skills for obstetric specialists from the central and district hospitals.

c. to validate a course which could be 'rolled-out' out to doctors and midwives in other hospitals and areas in the NIS.

A MOET manual and the standard MOET scenarios were sent to Armenia where they were translated into Russian. Each candidate was given the course material in advance so that baseline knowledge and skills could be fairly assessed.

Objectives of this paper

• To summarise quantitative and qualitative immediate assessments of the course.

• To examine individual changes in score obtained for each of the scenarios.

Methods

Baseline knowledge was assessed with a series of scenario based structured questions administered by one of the instructors (EB) before the first session. The course itself consisted of two days in which Adult Life Support, the management of trauma, obstetric, neonatal and anaesthetic emergencies were covered. A variety of teaching methods were used on both these days. Each topic was introduced by a short summary lecture, followed by skills practice, using different sorts of models; adult volunteers, neonatal resuscitation mannequin, purpose made obstetric model and a symphysiotomy model. Where adult volunteers (eg, SD) were being used the participant was expected to communicate clearly, with the "patient" with regard to planned action and examination. A team approach was encouraged by having anaesthetic and midwifery observers (whose assistance could be requested) for each scenario. The 24 obstetric scenarios were all 'reality based' having arisen from the MOET course organisers' personal experience. When the scenarios are presented to the candidates this is done in the form of a 'problem'. Although a clinical skill is taught (eg breech delivery), prior to that the training focus is on 'problem solving', with an emphasis on behaviours that value and respond to the patient's emotional and psychological, as well as physical needs [12]. The third morning was used to assess the candidates, who each had to manage an obstetric emergency simulation chosen by the instructing examiners (a "moulage"). The moulage performance was judged by a pair of instructors, who used a closed marking system (0,1 or 2; these scores could be modified with +/-). Finally, each candidate was then required to answer the same series of scenario based structured questions, as they had done prior to commencing the course (administered again by EB). Participants had individualised feed-back on their performances as all stages, and they, in turn, were asked to complete an evaluation proforma on each component of the course. The course was run in February 2001, for 8 candidates, with four instructors (RJ,VM,EB,KS). This paper summarises the changes in knowledge achieved as measured by the scores obtained as well as the individualised feed-back.

At the end of the course all delegates were given an evaluation form. The evaluation questionnaire consisted of six statement questions, utilising a Likert scale of 0–5 (completely disagree – completely agree, for responses). The qualitative free text assessments were categorised as 'positive' or 'negative' and collected under broad themes.

The Wilcoxon signed ranks matched pairs test was used to compare the difference between the before and after scores for each scenario. No adjustment was made for multiple testing, as this was an exploratory study, hence a p-value of <0.05 was considered significant. The software used was StatXact Turbo (CYTEL), Cambridge, Massachusetts (PJ).

Results

There were 8 candidates on the course, all obstetrician and gynaecologists. All eight candidates showed an improvement in post-course, compared to pre-course, scores (Table 1), with the range of scores being far narrower afterwards (167–188) than it was beforehand (85–129.5). In the individual score analysis only two scenarios showed a non-significant change (cord prolapse and breech delivery). The majority of other scenarios show highly significant improvements between the two scores (Table 2). In the moulage, 2 candidates performed at a very high level and 2 at a high level. None of the other 4 candidates had an unsatisfactory practical score (0). The 4 top candidates had also done well in the knowledge scores and were invited to be instructors on the next MOET course. The overall best candidate was a junior consultant. The overall ratings in the anonymous course evaluations were good (Table 3). A summary of the free text comments made by the participants is shown in Table 4. The majority of comments were complimentary and positive. Specific negative feed-back related principally to a sense that the course had been 'rushed'.

Table 1

Overall scores

Code	Score before	Score after	% increase
A	85	174	+ 105%
B	99	186	+ 88%
C	105.5	185	+ 73%
D	87.5	167	+ 91%
E	103.5	180	+ 74%
F	106	176	+ 66%
G	116	182	+ 57%
H	129.5	188	+ 45%

Table 2

Difference between scores after course compared to before course

No	Name of Scenario	Median	Range	Statistical Significance (p=)
1	Cardiac arrest and perimortem caesarean section	4.5	2–6	0.014
2	Resuscitation of the neonate	6	4–10	0.014
3	Trauma in pregnancy	5	3–6	0.013
4	Sepsis	4.5	1–8	0.014
5	Pulmonary embolism	1.5	1–2	0.012
6	Amniotic fluid embolism	1	0–4	0.035
7	Preeclampsia with HELLP* & pulmonary oedema	2	1–4	0.013
8	Magnesium sulphate cardiac arrest	1	0–2	0.033
9	Recurrent convulsions	1	0–3	0.034
10	Eclampsia with respiratory depression (diazepam overdose)	1	0–2	0.034
11	Shoulder dystocia	2.5	0–5	0.016
12	Breech delivery	1.5	0–4	0.057 (not significant)
13	Cord prolapse with live baby	0	0–2	0.38 (not significant)
14	Uterine inversion	2	0–3	0.022
15	Massive obstetric haemorrhage during caesarean section	1	1–2	0.011
16	Assisted delivery – ventouse	9	1–10	0.013
17	Forceps delivery for mento anterior position	3	1–7	0.014
18	Internal podalic version of second twin	1.5	1–3	0.013
19	Antepartum haemorrhage – abruptio placenta	4	1–8	0.014
20	Postpartum haemorrhage – trauma	3	1–5	0.014
21	Postpartum haemorrhage – atonic uterus	4	2–6	0.014
22	Decapitation	3	1–5	0.014
23	Craniotomy	2	1–4	0.014
24	Symphysiotomy	6	5–6	0.011
25	Ruptured uterus	2	0–10	0.016

*Haemolysis, Elevated Liver enzymes and Low Platelets

Table 3

Anonymous course evaluation

	Question		No.	%
1	The course was interesting	(MOET good) 5	8	100%
		4
		3
		2
		(MOET bad) 1
2	The course taught me new procedures	(MOET good) 5	7	87.5%
		4	1	12.5%
		3
		2
		(MOET bad) 1
3	The course was relevant to my practice	(MOET good) 5	3	37.5%
		4	2	25%
		3	3	37.5%
		2
		(MOET bad) 1
4	I would recommend the course to friends	(MOET good) 5	7	87.5%
		4	1	12.5%
		3
		2
		(MOET bad) 1
5	I feel more confident to practice	(MOET good) 5	5	62.5%
		4	3	37.5%
		3
		2
		(MOET bad) 1
6	I feel more confident to teach others about emergencies in obstetrics	(MOET good) 5	5	62.5%
		4	3	37.5%
		3
		2
		(MOET bad) 1

Table 4

Course feedback

What did you like most about the course?

The way and manner the materials were given and presented

Care in case of amniotic fluid embolism

The game-like form of the training scenarios and the way they were done – understanding between lecturer and audience

Consistency in the work

The issues of resuscitation of pregnant women in obstetrics

Practical recommendations

What did you like least about the course?

Few situations offered to one candidate

Squeezed presentation of all topics into a short frame of time

All was done at one level

Too short course, no practice with real patients

Duration (too short)

Was there any emergency missing from the course?

No

Sepsis

Yes – the instructors should have a psychological test as well, the translation of materials includes too few words, in the process of teaching instructors to get them to act automatically

No, but would like more

No

More on sepsis

How could we make the course better for next time?

For the solution of definite issues of emergencies to give some time to each candidate

Topics of the training in advance to get acquainted to them.

Prolong the course, more materials

Involve more doctors

More materials, work in the hospital

More days and time

To prolong it and to add more visual study aids

Discussion

This paper summarises the development of the first MOET course to be run in Europe, outside the UK. In keeping with previous experience it demonstrates the reliability of the model based scenarios [11], with a highly significant improvement in knowledge about obstetric emergency management [13]. All candidates performed at a satisfactory, or greater, level in the practical moulage. From the participant feed-back that we have had, the balance of curriculum and teaching methods appears to be about right. Current educational theory provides good evidence to support a multifaceted approach (pre-course reading, lectures, skill stations and knowledge assessments). With fewer candidates on this course, and with the local commitment of one of the instructors (EB), it was possible to allocate time for an in-depth pre-course and similar post-course assessment of each participant. This assessment demonstrated significant improvements in all areas, apart from two. The degree of improvement was highly significant in some of the scenarios, suggesting that the subject material was new (and interesting) and that the scenarios made the necessary impact. Further evaluation of longer-term practice changes is now required. According to independent evaluators, an audit of practice following a short term training based on protocols for normal birthing, life saving skills and family-centred maternity care, and requiring demonstrated competency, improved practices up to 3 years post training in Bolivia, Guatemala, Indonesia, Vietnam, Morocco and Ukraine, and even up to 8 years in Nigeria [14].

Given the limited resources for equipment we used candidates and instructors as patients. In the UK we have used actors for the Day 3 moulage. One of the most revealing aspects of MOET courses in the UK has been how much the candidates enjoyed working with actors. This has also been the experience of other groups. In a paper on the overall experiences at the Calgary Medical Skills centre [15], the use of volunteer patients and trained actors for all levels of medical education and evaluation was found to help in achieving the aim of providing a predictable learning environment and standardised teaching methods. Improving the performance of skilled birth attendants requires incorporation of ways to increase their communication and counselling skills.[12] Using actors and actresses allows a greater sense of 'reality' and makes scenarios more demanding in that they require genuine communication with the 'patient' and 'supporter'. It would appear that using course participants can be equally satisfactory.

Since the first MOET course was run two instructor candidates (EK and VO) have run a second MOET course (mainly for the hospital midwives and at their request). This was very successful. Local leadership which aims to upgrade training based on evidence of effectiveness and on providers' concerns is much needed.[12] Building relationships among different cadres is important. It has previously been recognised that active, hands-on training opportunities that demonstrate the experience, skills and roles of various providers working together are most effective [14]. Further progress in MOET in Armenia will depend on the success of local teaching cascades. Whilst it was not within the responsibilities of the authors to identify longer-term barriers to change, there no doubt exist. It is therefore important that the work environment is supportive of the improvements; all levels of the hierarchy should be involved in planning and implementation.[14]

Conclusion

This paper demonstrates the reliability of the model based scenarios, with a highly significant improvement in obstetric emergency management. The full impact of training needs to be evaluated in longer term follow-up. This will require the use of clinical audit. Audit of delays, specific obstetric complications, referrals and near misses may all be amenable to review, in a positive spirit.[12,14]

Competing interests

None declared.

Author's Contributions

Author 1 (RJ) conceived of the study and participated in its design, coordination, facilitation and the writing of the first draft but died before it could be completed. Author 2 (VM) participated in the design, coordination and facilitation of the study, and is now the corresponding author and guarantor for the manuscript. Author 3 (EB) assisted with design, facilitation and assessment. Author 4 (EK) and Author 5 (VO) performed exceptionally well and emerged as the lead candidates. Author 6 (MI) and Author 7 (KS) facilitated during the course. Author 8 (SD) as project manager of Family Care, Armenia supported the application for the course to be held, to the Minister for Health in Nagorno Karabagh. Author 9 (PJ) analysed the data and reported on statistical significances.

Pre-publication history

The pre-publication history for this paper can be accessed here: