Evaluation of a mental health training project in the Republic of the Sudan using the Mental Health Gap Action Programme curriculum.

This paper reports on the training of primary care physicians in the family medicine programme at the University of Gezira, Sudan, using the World Health Organization's Mental Health Gap Action Programme Intervention Guide (mhGAP-IG). The training had a positive impact on their knowledge of and attitudes to mental disorder. More field tests of the mhGAP-IG would be useful to make further recommendations on its cultural relevance and its adaptation for use in low- and middle-income countries. Distance supervision of training of primary care physicians by internal facilitators is seen as critical for the sustainability of the intervention.

Globally, mental, neurological and substance use disorders (MNS) are a major cause of disability, accounting for more disability-adjusted life-years (DALYs) than any other type of non-communicable disease (Prince et al, 2007). In low- and middle-income countries (LAMICs), MNS are largely unrecognised and untreated, in part due to lack of mental health services, lack of trained personnel and lack of capacity of the primary healthcare (PHC) system to provide the care required (WHO Regional Office for the Eastern Mediterranean, 2010).

One of the recommendations of the World Health Organization (WHO) in its 2001 World Health Report was that mental healthcare should be integrated into primary care and community care through the training of PHC personnel (WHO, 2001). The WHO thereafter launched its Mental Health Gap Action Programme (mhGAP) (WHO, 2008) and the mhGAP Intervention Guide (mhGAP-IG) (WHO, 2010) as a mental health curriculum to be used in training.

The population of Sudan is 36 233 000. It is classified as a low-income country in World Bank income categories. The adult literacy rate is 59%. The total expenditure on health is 4.3% of gross domestic product. The proportion spent on mental health is undetermined. There are 114 health providers, 0.09 psychiatrists and 0.2 psychiatric nurses per 100 000 people. There are 0.2 mental health beds per 10 000, of which 90% are hospital-based (WHO & AIMS Sudan, 2009). Most mental health services are concentrated in urban areas; 17 of Sudan’s 25 states have no psychiatric services, and their populations rely almost exclusively on services provided by traditional/faith healers.

The WHO recommendations underpin the directions enshrined in the Health Policy of Sudan. This paper reports on the training of physicians in the family medicine programme at the University of Gezira, Sudan, using the mhGAP-IG. The objectives were to train PHC physicians to assess, manage and refer individuals with MNS, to evaluate the outcome of the training in terms of PHC physicians’ knowledge about and attitudes towards MNS and to field test the mhGAP-IG in terms of its usefulness and applicability.

Method

Volunteer psychiatrists from the Royal College of Psychiatrists and others in an international volunteering network were recruited as external facilitators for the training. The project was developed by the collaborative efforts of the College, the Federal and State Ministries of Health for Sudan and Gezira state, and the WHO Regional Office for the Eastern Mediterranean. It began with a 1-week training-of-trainers course in Khartoum, where 30 Sudanese psychiatrists and psychologists invited by the Federal Ministry of Health were trained. There was an explicit commitment before induction in the training-of-trainers course that participants would later travel to the University of Gezira with the external facilitators to co-facilitate the training of PHC physicians.

A 1-week PHC training course was designed using the mhGAP-IG. One cohort of PHC physicians was trained every week over 7 weeks, to include all the PHC physicians enrolled in the family medicine programme at the University of Gezira. Except for the first 2 weeks, there were two different external facilitators per cohort per week. Teaching methods included small-group problem-based learning, role-plays, mini-lectures, quizzes and clinical interviews with out-patients and in-patients at two local hospitals.

Outcome measures included pre- and postintervention attitude surveys and knowledge and aptitude (KAP) tests, and an end-of-course evaluation. The KAP test consisted of 11 clinical cases, each followed by a series of questions in a short-answer format. The cases were designed by the Regional Advisor for the WHO Regional Office for the Eastern Mediterranean (author K.S.) and were based on the mhGAP-IG. The same KAP test was administered before and after the training. Data obtained from the test are presented as mean test scores, and mean score per case. Results on attitude surveys and course evaluations are presented as descriptive data.

Results

A total of 150 PHC physicians were trained. Data from the pre- and post-intervention surveys showed a positive attitude towards mental illness before the training and little change after.

The proportions of trainees passing the test pre- and post-intervention was 15% (7/48) and 80% (37/46) respectively. The mean total KAP test score pre- and post-intervention was 19.32% (n =91) and 44.75% (n = 89) respectively.

The percentage improvement was calculated as:

[(post-test mean – pre-test mean)/pre-test mean] × 100

For the total KAP test score, the percentage improvement was 131.6%; for the number of trainees passing the KAP test, the improvement was much higher – 450%. Figure 1 shows the mean scores per case. The most improvement was in delirium, followed by school refusal and postpartum psychosis, with mean score increases of 5.0, 4.3 and 3.9 respectively.

Fig. 1

Pre- and post-intervention results from the knowledge and aptitude (KAP) tests, by case.

The course evaluation showed an overwhelmingly positive response, with 95% of responders finding the course good or better. No one rated the course poor or very poor. The response rate was 46% (60/130).

The narrative comments were overwhelmingly positive. A few themes emerged:

enhanced confidence in dealing with patients with MNS

the course being too short

the need for an annual refresher course.

Trainees best enjoyed the patient encounters at the hospitals. Some comments reflected more positive attitudes: ‘I used to hate psychiatry and now I love it and am interested’; and ‘before I started I was afraid of mental patients, and now I feel confident in treating them and not afraid’.

Discussion

To our knowledge, this is the first published study using mhGAP-IG to train PHC physicians in LAMICs. A literature search found three studies (Edgell, 1970; Schmidt, 1972; Swift, 1972) where training was conducted in parts of India and Africa. Only one (Schmidt, 1972) commented on the potential efficacy of this approach, but it was not demonstrated or formally evaluated. A fourth study (Srinavasa Murthy & Wig, 1983) highlighted the need for culturally relevant training materials with basic and simplified diagnostic and treatment interventions.

The results showed a positive impact of the teaching on the physicians. There was an overall trend to improvement in scores. There was no change in attitudes toward MNS, which were in any case positive before the intervention.

The course evaluations were overwhelmingly positive. According to scholarly models for training evaluation (Kirkpatrick, 1994), the improvements in KAP test scores represent a positive impact at Kirkpatrick levels 1 (the trainees enjoyed the training) and 2 (increase in knowledge and capacity). Although improvement in clinical application of knowledge (Kirkpatrick level 3) was observed during clinical teaching at the hospitals, this level of impact was not formally measured. This may involve ongoing evaluation of trainees’ clinical skills and performance by clinical supervisors throughout their training programme. Level 4 impact (improved patient outcome) is more difficult to measure and may be evaluated by trainees keeping a log of cases, diagnoses, treatment and disposition. There are inherent difficulties in getting this type of data in LAMICs due to resource limitations in terms of local supervisory capacity and medical record-keeping. Evaluation at this level would also be unrealistic in this small pilot study exploring the use of the mhGAP-IG.

The authors’ experience with the mhGAP-IG was that it was easy to use and a helpful guide for planning lectures. Trainees found it easy to follow the treatment algorithms. The content was mostly familiar in terms of terminology and medications. For the purposes for which it was launched by the WHO it was meant to avoid Western disease constructs and focus on locally available treatment resources. However, its cultural relevance has not been empirically tested. Consideration could be given to modifying the mhGAP-IG to focus on more locally common clinical presentations, such as enuresis, child-rearing problems, domestic violence and suicide without diagnosable MNS, and less on epilepsy-specific syndromes, which seemed too advanced for its intended purpose.

The study’s limitations included its small sample size, missing data for three cohorts, inconsistencies in test scoring (some external facilitators allowing the trainees to score the test themselves), uncertain validity of the KAP test (did it measure improved knowledge or simply detect rote knowledge?) and concerns about the cultural relevance of the curriculum and assessment tools.

Modifications to the training programme may include formal guidance on cultural expressions of psychological conflict for external facilitators, for example about suicide. Gender patterns, methods and triggers for suicide differ in Sudan and other LAMICs and suicide often occurs in the absence of a diagnosable MNS (Vijayakumar & Rajkumar, 1999). Management is therefore quite different. Other modifications may include translation of the mhGAP-IG into Arabic; integration of religion and spirituality into the training; adding an observed interview to the final evaluation; fewer and shorter cases on the KAP test, with a simpler and more standardised scoring system; two consistent external facilitators for the entirety of the training, rather than two per week per cohort, to avoid inconsistent scoring; and teaching in Arabic via a translator.

The next step of this project includes ongoing training of PHC physicians by internal facilitators with internet-based support from external facilitators. This issue is seen as critical for the sustainability of the intervention.