Competence of health workers in emergency obstetric care: an assessment using clinical vignettes in Brong Ahafo region, Ghana.

OBJECTIVES: To assess health worker competence in emergency obstetric care using clinical vignettes, to link competence to availability of infrastructure in facilities, and to average annual delivery workload in facilities.
DESIGN: Cross-sectional Health Facility Assessment linked to population-based surveillance data.
SETTING: 7 districts in Brong Ahafo region, Ghana. PARTICIPANTS: Most experienced delivery care providers in all 64 delivery facilities in the 7 districts. PRIMARY OUTCOME MEASURES: Health worker competence in clinical vignette actions by cadre of delivery care provider and by type of facility. Competence was also compared with availability of relevant drugs and equipment, and to average annual workload per skilled birth attendant.
RESULTS: Vignette scores were moderate overall, and differed significantly by respondent cadre ranging from a median of 70% correct among doctors, via 55% among midwives, to 25% among other cadres such as health assistants and health extension workers (p<0.001). Competence varied significantly by facility type: hospital respondents, who were mainly doctors and midwives, achieved highest scores (70% correct) and clinic respondents scored lowest (45% correct). There was a lack of inexpensive key drugs and equipment to carry out vignette actions, and more often, lack of competence to use available items in clinical situations. The average annual workload was very unevenly distributed among facilities, ranging from 0 to 184 deliveries per skilled birth attendant, with higher workload associated with higher vignette scores.
CONCLUSIONS: Lack of competence might limit clinical practice even more than lack of relevant drugs and equipment. Cadres other than midwives and doctors might not be able to diagnose and manage delivery complications. Checking clinical competence through vignettes in addition to checklist items could contribute to a more comprehensive approach to evaluate quality of care. TRIAL REGISTRATION NUMBER: NCT00623337. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

We used clinical vignettes to assess health worker competence in rural Ghana, choosing two major causes of maternal mortality that independently working delivery care professionals should be able to diagnose and manage.

While not nationally or longitudinally representative, our health facility assessment included all 64 delivery facilities in seven districts of Brong Ahafo region and captured the best competence available at the time of interview.

Despite limited sample size and risk of social desirability bias in vignettes, we were able to clearly identify serious shortcomings in health worker competence in the area.

We used available surveillance data in the districts to estimate births per facility and found an association between competence and workload.

Introduction

Skilled birth attendants are crucial to reducing the 289 000 maternal deaths, 2.8 million neonatal deaths and 2.6 million stillbirths that still occur every year.1–4 Shortage of midwives, especially in remote locations, has led to training of healthcare professionals other than midwives to manage deliveries.5 Presence of a skilled attendant at birth is one of the main indicators of progress towards improving outcomes related to pregnancy and childbirth, however, the approach of increasing coverage with birth attendants without assessing their skills has been criticised for ignoring quality.6

Measuring quality of care is challenging because quality is a multidimensional concept without a universal definition.7 8 Quality of emergency obstetric care (EmOC) in low-income and middle-income settings is often evaluated with signal functions that indicate the capacity of a facility to perform certain life-saving interventions.9 Usually, performance of a signal function within the past 3 months is verified using patient chart abstracts. Facilities performing a set of six basic signal functions are classified as providing basic emergency obstetric care, and facilities performing emergency surgery (eg, caesarean delivery) and blood transfusion in addition to the six basic functions are classified as providing comprehensive emergency obstetric care. This well-defined set of key actions targeting the main causes of maternal mortality enables monitoring, evaluating and comparing obstetric care within and between countries.

However, availability of skilled attendants or signal functions may not sufficiently reflect quality of care.6 10 In a multicountry survey including 29 countries and 357 large delivery facilities (over 1000 annual deliveries per facility), high coverage with three key signal functions did not translate into reduced maternal mortality.10 The authors suggest that this might be due to delayed management of emergencies and lack of comprehensive patient care. For example, septic shock requires comprehensive shock management in addition to antibiotic treatment measured by the signal function. Indeed, quality classification based on signal functions assumes that provision of certain functions indicates that complications were recognised correctly and managed comprehensively without explicitly measuring the overall quality of clinical practice or competence of health workers.

In high-income settings, clinical vignettes have been shown to better reflect physicians' practice than medical record abstracts.11 12 They can provide information about procedural changes needed to improve health outcomes and are an inexpensive way of assessing competence, in particular, when chart documentation is incomplete.13 Therefore, vignettes could be an appealing and feasible way to study clinical practice and assess quality of obstetric care in low-income and middle-income countries.

In this paper, we assess competence of health workers in delivery facilities in Brong Ahafo region in Ghana using clinical vignettes. We compare competence between health worker cadres and between health facility types. To assess whether clinical practice was limited rather by facility infrastructure or by health worker competence, we compare competence in vignette actions with availability of necessary drugs and equipment. Finally, as a minimum workload and, thus, experience, may be necessary to maintain competence, we study the association between respondent competence and average workload at facilities.

Methods

Study design

A cross-sectional health facility assessment was conducted to study health worker competence in emergency obstetric care, and linked to population-based surveillance data on place of birth.

Setting

The study was conducted in all 86 health facilities providing care for mothers and newborns in seven districts in the central Brong Ahafo region of Ghana in October and November 2010: Kintampo North and South, Nkoranza North and South, Tain, Techiman and Wenchi. Several large trials conducted in the area provided reliable health and demographic data relating to mothers and babies.14–17 The maternal mortality ratio in the area is estimated at 377 per 100 000 pregnancies,15 and neonatal mortality at 31 deaths per 1000 live births.18

Health facilities

In this analysis, we focused on the 64 facilities in the area providing delivery care as we aimed to study emergency obstetric care. These facilities included one public regional hospital, 10 public, quasi-public (ie, mission) and private hospitals (level C), 34 public health centres (level B), 11 private maternity homes (level A) and eight small public facilities to which we refer collectively as ‘clinics’ (level A). Among the hospitals, the regional hospital functioned as the referral facility for the area, four were main district hospitals, four were other district hospitals and two were private hospitals. Fourteen facilities were located in urban areas: seven hospitals, one health centre and six maternity homes. The health facilities in the study area have previously been classified in terms of their routine and emergency obstetric care and emergency newborn care performance,19 and in terms of quality of care provided for newborns.20 The Newhints trial14 surveillance system collected information on every pregnancy in the study area between November 2008 and December 2009, including the specific delivery facility for facility births.

Participants

The most experienced provider managing deliveries and newborns present at each facility at the time of the visit was interviewed. These 64 respondents included three doctors, 38 midwives, five medical assistants, nine nurses and nine belonging to ‘other’ cadres. The group ‘nurses’ included community health nurses, enrolled nurses, public health nurses and staff nurses. The group ‘others’ included health assistants, health extension workers, trained traditional birth attendants and ward assistants. The distribution of respondents in different facility types is presented in table 1.

Table 1

Respondent cadres by facility type (n=64)

Facility type	Doctors	Medical assistants	Midwives	Nurses	Others
Hospitals; n=11 (%)	2 (18)	0	8 (73)	1 (9)	0
Health centres; n=34 (%)	1 (3)	5 (15)	18 (53)	4 (12)	6 (18)
Clinics; n=8 (%)	0	0	4 (50)	2 (25)	2 (25)
Maternity homes; n=11 (%)	0	0	8 (73)	2 (18)	1 (9)
Total; n=64 (%)	3 (5)	5 (8)	38 (59)	9 (14)	9 (14)

Data collection

The health facility assessment conducted by a physician and a research assistant, included questions regarding antenatal care, routine delivery care, emergency obstetric and neonatal care, referral practices, infrastructure and equipment (including observation of tracer items). Detailed information on staffing was also collected, including numbers of healthcare professionals managing deliveries, complications and newborns. Clinical vignettes were used to capture competence in managing critical maternal and newborn emergencies.

Vignettes and scores

The first vignette (case A) describes a pregnant woman with signs and symptoms of pre-eclampsia, and the second vignette (case B) represents a case of severe antepartum haemorrhage. Each vignette is divided into a section on diagnosis and a section on management that were read to the respondents separately. Respondents provided open responses and were asked if they would do anything else until they explicitly said no. The interviewer then marked which items from a list of actions were mentioned. The list of vignette actions included a number of best-practice actions related to diagnosis and initial management of obstetric emergencies according to WHO Pregnancy, Childbirth, Postpartum and Newborn Care guidelines21 for primary level of care, and some additional actions, such as supplemental oxygen administration, that can be performed at a referral-level facility or in a very well-functioning first-line facility (table 2).

Table 2

Vignette cases and construction of vignette score

Vignette action	Vignette score
Case A. Section 1. A 26-year-old woman who is 7 months pregnant comes in complaining of headaches, blurred vision and epigastric pain and her face looks swollen. In this facility, what would you usually do to establish a diagnosis?
Measure the woman’s blood pressure*	+1 point if mentioned
Check her urine for protein*	+1 point if mentioned
Check her reflexes*	+1 point if mentioned
Check fetal heart rate*	+1 point if mentioned
Refer to other health facility*, or	+1 point if at least one out of two mentioned
Call specialist or respondent is a specialist (ie, a doctor)
	Section maximum: 5 points
Case A. Section 2. On examination, she had a blood pressure of 170/120 mm Hg, 3+ protein in her urine and brisk reflexes. How would she be managed at this facility?
Giving antihypertensive drug, eg, hydralazine, labetalol or nifedipine*	+1 point if mentioned
Give magnesium sulfate or, if not available, diazepam*	+1 point if mentioned
Have somebody stay with her all the time in case she starts having seizures*	+1 point if mentioned
Plan for delivery within the next 24 hours, or	+1 point if at least one out of two mentioned
Refer to other health facility*
	Section maximum: 4 points
Case B. Section 1. A 35-year-old woman who is 8 months pregnant comes to this facility because she has started to bleed heavily vaginally. She has no contractions and does not complain of any pain. In this facility, what would you usually do to establish a diagnosis?
Check the woman′s vital signs*	+1 point if mentioned
Check fetal heart rate	+1 point if mentioned
Perform abdominal examination	+1 point if mentioned
Do not perform vaginal examination*	+1 point if mentioned
Refer to a facility where caesarean delivery can be performed*, or	+1 point if at least one out of two mentioned
Call specialist or respondent is a specialist (ie, a doctor)
	Section maximum: 5 points
Case B. Section 2. The woman has a feeble pulse at 120/min, her systolic blood pressure is 85 and she is pale, sweating and breathing rapidly at 30 breaths per minute. Fetal heart sound is normal. There is no pain on abdominal examination. She is still bleeding bright red blood vaginally. You suspect placenta praevia and, therefore, do not perform a vaginal examination. How would such a patient be managed now?
Elevate legs to increase return of blood to the heart*	+1 point if mentioned
Give intravenous fluids rapidly*	+1 point if mentioned
Give oxygen by mask or nasal cannula, or	+1 point if at least one out of two mentioned
If oxygen unavailable, refer to a facility where caesarean delivery can be performed
Perform ultrasound to confirm diagnosis, or	+1 point if at least one out of two mentioned
If ultrasound unavailable, refer to a facility where caesarean delivery can be performed
Give blood transfusion, or	+1 point if at least one out of two mentioned
If blood transfusion unavailable, refer to a facility where caesarean delivery can be performed
Prepare for caesarean delivery, or	+1 point if at least one out of two mentioned
Refer to a facility where caesarean delivery can be performed*
	Section maximum: 6 points
Cases A and Case B	Total maximum: 20 points

*Vignette actions mentioned in WHO Pregnancy, Childbirth, Postpartum and Newborn care.21

The vignette score was created by assigning one point to each action that could be performed in any kind of facility (ie, both a first-line facility and a referral facility). For actions performed only or mainly at referral facilities, specifically ultrasound, blood transfusion, supplemental oxygen and caesarean delivery, respondents got one point if the action was not available at the respondent's facility and the respondent said they would refer the woman to a facility capable of performing a caesarean delivery, even if the action was not specifically mentioned (table 2). Combining both cases A and B, the scores for the diagnostic section and management section were 10 points each adding up to an overall maximum score of 20 points. Respondent competence was classified as ‘high’ for >15 points (>75%), ‘moderate’ for 10–15 points (50–75%) and ‘low’ for <10 points (<50%).

As a sensitivity analysis, two alternative vignette scores were created, one that weighed the actions by their clinical importance, and one for which mentioning life-saving key interventions was a requirement. As there were no major differences in results with these alternative scores, we present the simpler approach of assigning one point per action.

Data analysis

We compared median overall vignette scores by respondent cadre using Kruskal-Wallis tests. If the vignette scores measure clinical competence, higher cadres should have higher scores. We also compared median vignette scores by facility type (hospitals, maternity homes, health centres and clinics) using Kruskal-Wallis tests.

To identify whether availability of drugs and equipment, or clinical competence, were the limiting factor in providing EmOC through executing vignette actions, we compared six vignette answers with the availability of corresponding items: for example, administering intravenous fluids was compared with availability of infusion sets.

The association between vignette score and workload was analysed using linear regression. Workload was defined as the average number of annual deliveries in a facility in 2009 divided by the number of skilled birth attendants working in that facility. There were altogether 13 692 deliveries in the study area in 2009. We defined a skilled birth attendant (SBA) as a doctor, medical assistant, midwife or nurse trained in managing deliveries. For this analysis, stillbirths were counted as deliveries and multiple births were counted as one delivery. All analyses were performed using Stata V.12 (Statacorp College Station, USA).

Ethical approval and informed consent

Ethical approval for this study was obtained from the London School of Hygiene and Tropical Medicine in the UK, and from the Kintampo Health Research Centre in Ghana. Health professionals included in the health facility assessment signed a written informed consent before the start of the interview. For the Newhints trial, consent was obtained from all women of reproductive age living in the surveillance area (Newhints14—clinicaltrials.gov, NCT00623337).

Results

Of the 86 facilities in the study area, 64 (74%) provided delivery care and were included in this analysis. Sixty-nine per cent of the 13 692 deliveries in the area in the year 2009 took place in a health facility.

Competence by health worker cadre

The median total vignette score was 55% of actions mentioned correctly with a range from 5% to 75%. For 28% of all respondents, competence was classified as low (<50%) and none of the respondents achieved a high score (>75%). The vignette score differed significantly between respondent cadres, showing a clear trend (figure 1A). The three doctors had the highest scores with a median of 70% (range 60–75%), the 38 midwives scored a median of 55%, the nine nurses 50%, five medical assistants 45% and the nine other cadres 25%. A doctor, one public health nurse and two midwives scored the highest points overall (75%). Doctors performed equally well in the sections relating to diagnosis (70%) and management (67%), and they performed better in both sections compared with other cadres (figure 1B). Midwives were the second best cadre with 67% correct in diagnosis and 48% in management. Medical assistants had a moderate competence in diagnosis (52%) but low competence in management (40%). Their results were similar when compared with nurses who had 49% in diagnosis and 41% in management. The other respondents scored a low percentage in diagnosis (33%) and in management (18%).

Figure 1

(A) Per cent correct in vignettes by respondent cadre in delivery facilities (n=64). The boxes show the middle 50% of the scores, the vertical lines show the range of scores, the central horizontal line represents the median score, and outliers are represented by the dots. Group ‘others’ includes health assistants, health extension workers, traditional birth attendants and ward assistants. (B) Mean per cent correct in vignettes by respondent cadre (n=64) and by vignette section. Group ‘others’ includes health assistants, health extension workers, trained traditional birth attendants and ward assistants.

Competence by type of delivery facility

Higher vignette scores were strongly associated with the type of delivery facility (p<0.001, figure 2). Respondents in the 11 hospitals achieved the highest score with a median of 70% (range 55–75%); respondents were eight midwives, one nurse and two doctors. Maternity home and health centre respondents were mainly midwives and both had a similar level of vignette competence with a median of 53% in health centres and 50% in maternity homes. Health workers in clinics, also mainly midwives, scored lowest in the vignettes with a median score of 45% (range 5–55%) (figure 2, table 1).

Figure 2

Per cent correct in vignettes by facility type (n=64). The boxes show the middle 50% of the scores, the vertical lines show the range of scores, the central horizontal line represents the median score, and outliers are represented by the dots.

Respondents in quasi-public facilities had a higher vignette competence with a median of 75% compared to private and public facilities where the median score was 55% (p=0.01). Competence varied more in public facilities (range 5–70%) and in private facilities (10–75%) compared to quasi-public facilities (60–75%). Health workers in urban facilities were more competent as judged by vignette scores (median 60%, range 45–75%) compared to rural respondents (median 55%, range 5–75%) (p=0.01).

Infrastructure and competence

Both lack of necessary drugs and equipment, and lack of knowledge on how and when to use them in practice, would limit diagnosis and management of a woman presenting with pre-eclampsia or antepartum haemorrhage in facilities. Our comparison revealed that the limiting factor in the majority of cases was competence (figure 3). In all facility types, required drugs and equipment were more frequently available than the actions were mentioned in the vignettes, with one exception; administering antihypertensive drugs (case A) was mentioned more frequently in health centres (44%) than they were available (35%). All hospitals had all six items available, and hospital respondents were more likely to mention the corresponding actions in the vignettes compared with respondents of other facility types.

Figure 3

Vignette actions with corresponding health facility assessment checklist items for all facilities combined and by facility type (n=64).

Group ‘clinics’ includes clinics, health posts and Community-based Health Planning and Service compounds.

*Administering parenteral anticonvulsants (magnesium sulfate or diazepam) or parenteral antihypertensive drugs (hydralazine, nifedipine or labetalol).

Missing part of bar is proportion of facilities where neither item was available nor action mentioned.

Of the drugs and equipment studied with their corresponding vignette actions, the sphygmomanometer to measure blood pressure, intravenous fluids with infusion sets, and fetoscope or an electronic fetal heart monitor were the most frequently available equipment items present in nearly all facilities (≥97%). Blood pressure measurement and administering intravenous fluids were also mentioned frequently, but surprisingly, only 22% of facility respondents mentioned assessing fetal distress in the vignette featuring a pre-eclamptic woman (case A). In hospitals, monitoring fetal heart sounds in case A was mentioned approximately twice as often (55%) as in maternity homes (27%), and nearly four times as often as in health centres (15%). Assessing fetal distress of a woman with antepartum haemorrhage was slightly more common and mentioned by 34% of all respondents. Although all clinics had a fetoscope available, none of the clinic staff mentioned during the vignettes that they would apply it in practice.

Anticonvulsants were available in 94% of all facilities, but in the pre-eclampsia vignette, they were only mentioned by 50% of respondents in facilities where it was available. While 91% of hospital respondents reported they would administer anticonvulsants, only 41% of respondents in health centres, and 45% in maternity homes, answered that they would have administered these to the pre-eclamptic woman. Half of all clinics and 45% of maternity homes had antihypertensive drugs available and their use was mentioned in the vignette by 25% of clinics and 18% of maternity homes where these items were available. Oxygen was available in 48% of all facilities, but only 5% of all respondents mentioned administering supplemental oxygen to a woman presenting with severe antepartum haemorrhage. No clinic had oxygen available, but three-quarters of maternity homes, a third of health centres and all hospitals, had oxygen cylinders. However, none of the health workers interviewed in maternity homes and health centres, and only 27% of hospital respondents would have administered supplemental oxygen to a woman presenting with antepartum haemorrhage.

Workload and competence

The overall number of SBAs working in facilities ranged from 0 to 2 in clinics, 0 to 4 in maternity homes, 0 to 8 in health centres and 3 to 53 in hospitals. The overall median frequency of deliveries per SBA per year was 26 (range 0–184), and the mean was 39 deliveries per SBA.

The highest workload was in hospitals with a median of 52 deliveries annually per SBA. The facility that managed the highest workload was a hospital with a total of 2398 deliveries in the year 2009 and 13 SBAs, resulting in 184 deliveries per SBA. The workload was unevenly distributed between hospitals. In 4 of the 11 hospitals, each SBA attended, on average, to <2 deliveries per year, and in three hospitals the delivery workload was >100 deliveries per SBA per year. Maternity homes had the second highest workload with a median of 45 deliveries annually per SBA, followed by health centres (21 deliveries per SBA) and clinics (13 deliveries per SBA). Workload in clinics ranged from 0 to 42 deliveries per SBA, with one outlier, a rural clinic with one SBA and 104 deliveries in 2009. The second highest delivery workload overall was in a health centre with one SBA only and 135 deliveries. A higher delivery workload in a facility was associated with a higher competence as measured with the vignettes (p=0.03, figure 4).

Figure 4

Association between the vignette score and delivery facility workload (n=60). Three facilities without skilled birth attendants and one facility without skilled birth attendants and deliveries are excluded from the analysis.

Five of the 64 delivery facilities employed SBAs but did not manage deliveries during the surveillance period, and three delivery facilities employed no SBAs but managed altogether 57 deliveries. If all 13 692 deliveries in the surveillance data in 2009 had taken place in the 64 delivery facilities managed by the 273 SBAs (or by the 189 midwives) employed there, the average delivery workload would have been 50 deliveries per SBA (or 72 deliveries per midwife).

Discussion

We found that competence in first-line management of obstetric emergencies as measured by two clinical vignettes varied markedly by cadre and facility type. The two cadres with most training in obstetric care—doctors and midwives—scored highest in the vignettes, as expected. However, even doctors and midwives were classified as only moderately competent. The association of higher vignette score with facility type likely reflects the availability of more experienced and skilled staff in hospitals compared to smaller facilities. The better competence of health workers in urban areas and in quasi-public facilities is in line with most urban facilities and all four quasi-public facilities being hospitals. Despite the moderate overall performance of health professionals, these expected findings support the validity of the two vignettes in assessing competence of delivery staff.

It has been suggested that clinical vignettes are a particularly useful way of assessing quality of clinical practice in low-income and middle-income countries because chart abstraction is time-consuming and can be unreliable even in high-income countries.22 23 Clinical vignettes have been used in low-income and middle-income settings in a variety of ways.20 23–27 For instance, quality of care for diarrhoea, tuberculosis and prenatal care was evaluated using vignettes in a cross-country study including 300 physicians in five middle-income countries—China, the Philippines, El Salvador, India and Mexico.26 In that study, the average quality of care was similarly low across countries, and there was high variation of performance within countries with some exceptionally well-performing physicians. The authors concluded that availability of resources is not the only or even an important predictor of competence, and that improvements in quality could be achieved by targeting poor performers.

Comparison of vignette actions and necessary drugs and equipment in our study illustrated the lack of these inexpensive key items and lack of competence to use these in clinical situations. In many cases when the items were available, they were still not mentioned in the vignettes when they should have been, suggesting that competence was the limiting factor. For example, while nearly all facilities had equipment to monitor fetal distress, it was rarely mentioned in the vignettes despite the fact that use of a fetoscope is relatively straightforward, there are no additional costs associated with its use, and assessing fetal distress should be among the first examinations of a woman presenting with an obstetric emergency. Another striking finding was how infrequently oxygen was mentioned in the vignettes even when it was available. Oxygen was available in 48% of all facilities, but only 5% of respondents mentioned administering oxygen to the woman in bleeding shock. This example represents a double gap: a resource gap and a competence gap. In most instances, the larger gap was in competence, which reveals a major missed opportunity.

Human resource shortages have been identified as a large barrier to delivery care in Ghana.5 The mean yearly workload found in this study (39 deliveries per SBA) is similar to what has been reported for other sub-Saharan African countries.28 Our estimates for workload are lower than the numbers reported by Witter et al29 from the central and Volta regions of Ghana in 2005 where midwives working in public facilities were reported to carry out on average 19 deliveries weekly (corresponding to 988 annually), and midwives in private facilities attended to four deliveries weekly (208 annually). It is possible that there are regional variations in workload within Ghana. Alternatively, the discrepancy may be due to methodological differences; the study by Witter et al used self-reported data, whereas, we calculated the number of births per attendant by dividing the number of facility births from a surveillance system through the number of SBAs taken from a health facility assessment. If several providers attend to one delivery, the first method is going to give higher workloads.

The workload was unevenly distributed between facilities in our study area, even among hospitals. Except for one outlier with 104 yearly deliveries, the workload was very low in clinics; SBAs in clinics managed only a median of one delivery every 28 days. In five facilities, health professionals trained to manage deliveries did not attend to any deliveries during the 1-year period. On the other extreme, there was a facility with 184 births per SBA, or 343 births per midwife in a year, and three facilities—a maternity home, a clinic and a health centre—with only one SBA and >100 deliveries in a year. This uneven distribution of deliveries among SBAs is likely to lead to worse quality of care as some SBAs are overburdened while others manage too few deliveries to maintain their professional competence. At the same time, nearly one in three women in the study area still delivered without SBA outside a health facility.

This study has several strengths and limitations. It was conducted in the Brong Ahafo Region in central Ghana, and results can thus not be easily generalised to the whole of Ghana. However, the study area is large, comprising seven districts, and has a very similar maternal mortality ratio and facility delivery rates as in Ghana as a whole.30

Each of the facilities was visited only once during daytime, interviewing the most experienced member of staff present at the time. Therefore, the facility assessment is a snapshot of the quality of care provided in each facility, and it may have been better or worse at other points in time. In fact, in 21 facilities (33%) there were other employees registered belonging to a cadre with higher average vignette score than the respondent cadre. However, had the interviewer been a woman in labour, the respondent would have attended to her, and this would have been the quality of care available for her. Our evaluation thus captured the best competence available in each facility at the time of the interview.

While they do not provide information about routine care or other emergencies, our vignettes represent two major causes of maternal mortality that all health workers in charge of deliveries and all delivery facilities should be capable of diagnosing and managing or referring. It seems likely that higher competence as measured with these vignettes would result in health benefits, but as we did not study the association with a health outcome, we cannot be sure.

A major shortcoming of vignettes, which they share with chart abstraction, is that they provide little information on personal interaction skills13 that are arguably an important dimension of quality of care.28 There is, furthermore, some debate in the literature on whether vignettes might capture knowledge instead of clinical practice and lead to overestimation of actual practice due to social desirability bias.13 26 31 Despite this, we found a very low performance in the two vignettes, which raises serious concerns about the quality of basic emergency care in the area.

It is a strength of this study that all facilities with births in the surveillance area, and all facilities offering delivery care, were included in the health facility assessment. Knowing the number of births per facility from the surveillance system enabled us to calculate facility-level workloads, even though information on deliveries for individual health workers was not available. Not all health workers trained in deliveries necessarily manage them, and SBAs are likely to have other tasks besides managing deliveries.5

Only three respondents were doctors, and we are thus limited in our ability to draw conclusions about their competence. In contexts with a shortage of doctors, these may focus on and thus be more familiar with more complicated management of obstetric emergencies, such as a caesarean delivery, rather than initial management of obstetric emergencies as assessed in our vignettes.

The health facility data and the surveillance data were collected during different, subsequent years, in 2010 and 2009, respectively. While the situation regarding health facilities and births may have changed in the area after data collection, our findings should still be valid for this time period. It is, furthermore, possible that the number of births and/or the number of health workers was different in the following year, affecting the workload estimates, though this effect is unlikely to have been substantial. It is also possible that a few deliveries in the study area were missed by the surveillance. Some facilities managed deliveries of mothers living outside of the surveillance area. A regional hospital included in the facility assessment is, in particular, very likely to have attended deliveries from outside that were not included in the surveillance. Therefore, the workload estimates presented here are likely to be underestimates. This regional hospital was included in the analysis because it attends to all referred patients needing tertiary care from the study area, and excluding the regional hospital from the analysis did not change the overall results. Finally, interviewer communication style and respondent perceptions about the assessment might affect competence evaluation. Consistency in interviewing style was maintained throughout the facility assessment as all interviews were conducted by the same person.

Conclusion

Our results suggest that health workers, other than midwives and doctors, might not be able to provide sufficient quality of EmOC. In parallel with increasing the availability of equipment and drugs, it must be ensured that health workers are able to use them correctly in practice. An appropriate workload translates into experience and competence; it must be high enough for the maintenance of professional skills without being so high that staff are overburdened. Collecting facility-level data on workload could allow for a more efficient distribution of limited human resources within a health system and improve quality of care. Vignettes provide an opportunity to evaluate whether health worker practice is deficient due to infrastructure or competence,13 to identify poorly performing health worker cadres and target training to them,26 or reconsider task-shifting, and to understand the reasons when increased coverage of interventions does not translate into improved health outcomes. Evaluating competence with vignettes in addition to checklist items or chart abstraction could be part of a more comprehensive approach to measuring quality of care.