An educational program for mental health nurses and community health workers from pacific island countries: results from a pilot study.

Delivery of mental health care relies upon professionals with the latest evidence upon which to base their care. This research reports on a pre-test/post-test evaluation of a four-week education program delivered to Pacific Island participants (n = 18) to enhance knowledge, skills, and attitudes (KSAs). The education program used a combination of formal lectures, tutorials, clinical visits, simulations, and laboratory sessions. The measure used was the Nurse Self Report (NSR) questionnaire. Results indicate an education intervention can be an effective tool for improving the knowledge, skills, and attitudes of Pacific Island people who care for persons experiencing mental health problems.

Despite ongoing calls to improve mental health services across the globe (Prince, Patel, Saxena et al., 2007; World Bank, 1993; World Health Organisation [WHO], 2001; WHO Atlas, 2005), progress towards meeting this challenge has been slow in middle- and low-income countries (Saraceno, van Ommeren, Batniji, Cohen, Gureje, Maloney, Sridhar & Underhill, 2007). Barriers to the improvement of mental health service delivery include the existing public health priorities, resistance to decentralisation of services, challenges to the delivery of mental health services in primary health care settings, low numbers of trained professionals, and the lack of effective leadership (Saraceno et al., 2007). The last two challenges identified are exacerbated in middle- and low-income countries by the lack of both adequate training facilities and opportunities for ongoing professional development (Saxena, Thornicroft, Knapp, & Whiteford, 2007).

Mental health nurses, generalist nurses, and community health workers are fundamental frontline personnel who provide mental health services in the Pacific for vulnerable populations residing in rural and remote areas. Nurses and/or community health workers are often the only available mental health staff. Thus, the need for improved training and ongoing professional development opportunities is critical. The delivery of mental health care in the Pacific relies upon the human resources of professionals rather than technology (Saxena, Thornicroft, Knapp, & Whiteford, 2007). Saxena et al. (2007) argue that the current shortages of trained professionals including mental health nurses, psychiatrists, psychologists, and social workers, hinders the delivery of mental health care in middle- and low-income countries.

At the 2012 Creating Futures Conference, co-hosted by the University of Papua New Guinea (UPNG), Queensland Health, the National Department of Health PNG, and the Cairns Institute, James Cook University (JCU), the poor state of mental health in PNG and across the Pacific was discussed (Hunter, 2012). The JCU WHO Collaborating Centre for Nursing and Midwifery Education and Research Capacity Building were asked to assist with the development of programs to upskill current staff responsible for the delivery of mental health care across the Pacific. As a result, a mental health theory and skills program designed for nurses and community health workers from the Pacific was conducted. The four-week program provided theoretical and practical skills to a cohort of mental health workers to prepare them to deliver up-to-date, evidence-based care to people with a mental illness and to educate and lead others working in the area of mental health in the future.

In order to determine the efficacy of the program, a pilot study using a pre-test/post-test evaluation was conducted. This article presents the results of the evaluation.

METHODS

Research Design

The pilot study used a pre-test/post-test design that compared 18 workshop participants’ responses at two different times. The initial test was to determine a baseline of participants’ self-reported mental health knowledge, skills, and attitudes (KSAs). The second test was to examine any changes in participants’ KSAs as a result of attending the four-week mental health education program. It was hypothesized that participants’ knowledge, skills, and attitudes would improve after the delivery of the education program.

Study Context

The study was conducted at James Cook University in North Queensland, Australia. The University is small in comparison with others in Queensland and is located in a regional setting.

Participants

A convenience sample of 18 participants (all participants who attended the education program) was recruited into the study. The participants were from the following Pacific Island countries: Cook Islands; Solomon Islands; Palau; Kiribati; PNG; Nuie; and Fiji (see Figure 1). An AusAID Australian Fellowship Award provided funds for participants to attend the four-week course.

Figure 1.

Map of Countries of Participants. Source: Wikimedia Commons under Creative Commons Licence CC0

Procedures

After ethical clearance by the University’s Human Research Ethics Committee (H5208), participants attending the four-week educational program were invited to participate in the study. Prior to distribution of the questionnaires a verbal explanation supported by an Information Sheet was disseminated to all program attendees. All attendees at the program voluntarily agreed to participate and signed a consent form. In order to avoid the possibility of coercion, the invitation to participate and the delivery of the pre-test and post-test measures were given by two student interns (EC and AG), who were neither members of the research team nor teachers in the education program. The participants were evaluated twice: once at baseline (the first day of the education program) and at the completion of the education program (the last day).

Instrument

The pre-test questionnaire collected demographic and personal information from respondents including age, gender, qualifications, previous work experience in mental health settings, and previous personal interaction with a person with a mental illness. The questionnaire used to evaluate the participants’ KSAs in both the pre- and post-test was the Nurses Self Report (NSR) questionnaire, as modified by Henderson, Happell, and Martin (2007) to measure the impact of theory and clinical placement on undergraduate students’ KSAs. This instrument was originally modified from the tool developed by Gillette et al. (1996) to evaluate emergency nurses’ perceptions of caring for patients with mental health problems. Workshop participants were asked to identify how accurately each statement on the NSR reflected their current KSAs in working with people experiencing mental health problems. This was accomplished by circling the most appropriate response on an adapted Likert-type four-category response scale (1 = Very true, 2 = Fairly true, 3 = Somewhat true, 4 = Not at all true). The NSR has previously been found to be a reliable instrument, with a reported Cronbach’s alpha of 0.85 (Henderson et al., 2007). The tool was found to be reliable in the current study with a Cronbach’s alpha of 0.98. Participants took approximately 15 minutes to complete the questionnaire. The questionnaire was completed anonymously, and anonymity was further preserved by having the interns collect the completed questionnaires and place them into a sealed box.

The Education Program

The education program was developed following the general principles of the World Health Organization Mental Health Gap Action Program (WHO, 2010): Communication with people seeking care and their carers; assessment; treatment and monitoring; mobilizing and providing social support; protection of human rights; attention to overall well-being; and four of the six goals of Grand Challenges in Global Mental Health (Table 1) (Collins, Patel, Joestl, March, Insel, & Daar, 2011).

TABLE 1 

Summary of Grand Challenges Goals Prioritized by Participants and Incorporated into the Program

Goal B	Goal C	Goal D	Goal E
Advance prevention and implementation of early interventions	Improve treatments and expand access to care	Raise awareness of the global burden	Build human resource capacity

The program was based on the best available evidence related to mental health, identified as important for the conduct of such programs (Patel, Garrison, de Jesus Mari, Minas, Prince & Saxena, 2008; Collins, Patel et al., 2011), and integrated existing priorities for mental health care (WHO, 2010; Collins, Patel et al., 2011). The program design also took into account other factors, including cultural context, English language proficiency of participants, various education/experience levels, and needs identified at the earlier workshop in PNG. These factors were similar to those taken into account by the developers of a mental health training course delivered in Haiti after the 2010 earthquake (Cianelli et al., 2013).

The program was conducted for four weeks, excluding weekend days. Lectures and invited presentations were conducted in the mornings followed by workshops, tutorials, simulated exercises, and laboratory-based activities in the afternoon. Additionally, specialist lecturers, clinicians from local mental health services, consumers, and consumer advocates delivered sessions (see Table 2).

TABLE 2 

Program Summary

Workshops	Mental Health Simulation	Visits/Visitors
• Teaching theories & styles• Education pedagogy & philosophy• Mental health promotion• Leadership & governance• Reflective practice• Clinical supervision• Evidence-based practice• Resilience• Perinatal mental health• Mental health first aid• Mental health promotion poster presentation	• Mental health nursing roles• Mental health assessment• Mental state assessment• Psychosocial assessment• Mental health disorders	• Consumer consultant• Local mental health nurses• Local non-government services• James Cook University library & computer rooms• Private mental health hospital• Public mental health unit• Aboriginal & Torres Strait Islander cultural centres

Data Analysis

Data generated by the questionnaires were analysed using SPSS (Statistical Package for the Social Sciences), version 20 (IBM SPSS Inc., Chicago, IL, USA). Frequencies, percentages, means, modes, and standard deviations were calculated to determine patterns in the data. Wilcoxon Signed-Rank tests were used to identify differences between the baseline pre-test and final post-test NSR scores. Spearman’s Rank Order correlation was performed to determine differences between continuous variables. Mann-Whitney tests were used to analyse relationships between nominal and continuous variables. Results were calculated for the sample as a whole then calculated for participants stratified by age, gender, nurse type, and mental health qualifications.

RESULTS

Demographic Characteristics

The majority of workshop participants were female (61.1%), were aged under 45 years (72.2%), had previously worked in a mental health setting as a mental health nurse (77.8%), and all (100%) had a previous personal interaction with a person with a mental illness. The most common formal nursing or health care qualification was a Bachelor degree (33.3%) followed by an Advanced Diploma (25%). Other highest qualifications/training included a certificate (8.3%), a post-graduate certificate (16.8%), a diploma (8.3%), and a Masters degree (8.3%). Overall, 44.4% of participants had specific mental health qualifications as well as nursing or health worker qualifications.

Knowledge, Skills, and Attitudes

Stage 1: Baseline Data

The mean for each item on the NSR stage 1 (NSR1) was calculated for the study sample and ranked from 1 (indicating highest confidence), to 22 (indicating lowest confidence) (Table 3). The highest ranked KSAs were: referral of mental health clients; respectful communication; working with those with self-inflicted injuries; and performing assessments of psychotic symptoms and mental status. Participants also were reasonably confident about their ability to triage a mental health problem, differentiate between delirium and dementia, and assess potential for violence. The lowest ranking KSAs suggest that the participants were less confident in their ability to understand victims of repeated domestic violence, identify legal parameters, and assess suicide risk. Of the 22 KSAs, the participants reported being least confident in their ability to understand clients who injure themselves or others while driving under the influence of substances.

TABLE 3 

NSR1: Knowledge, Skills, and Attitudes Ranked from Highest to Lowest Confidence at Baseline

Rank	Mean	NSR1 Questions
1	1.77	I am able to refer mental health clients to appropriate services.
2	1.88	My communication with mental health clients is respectful.
3	1.94	I always know when to refer a client for psychiatric consultation.
4	2.05	I work well with patients with self-inflicted injuries.
5	2.11	I am able to communicate effectively with people who are suspicious or paranoid.
6	2.16	I am skilful in assessing psychotic symptoms.
7	2.16	I am skilful in performing a mental status assessment.
8	2.16	It is as easy for me to triage a mental health problem as it is to triage a physical health problem.
9	2.22	I am skilful in differentiating between delirium and dementia.
10	2.22	I am skilful in assessing potential for violence.
11	2.22	I am able to detect side-effects of psychoactive drugs.
12	2.22	My communication with clients with self-inflicted injuries is respectful.
13	2.23	My communication with clients affected by alcohol is respectful.
14	2.25	I utilize psychiatric terminology appropriately.
15	2.27	My communication skills enable me to calm anxious people.
16	2.29	I find it easy to understand clients who harm themselves.
17	2.33	I am skilful in differentiating substance intoxication and psychosis.
18	2.33	I understand the therapeutic actions of psychoactive drugs.
19	2.38	I find it easy to understand clients who are victims of repeated domestic violence.
20	2.50	I am able to identify the legal parameters that relate to mental health clients.
21	2.50	I am skilful in assessing suicide risk.
22	2.70	I find it easy to understand clients who injure themselves or others while driving under the influence of substances.

1 = Very true, 2 = Fairly true, 3 = Somewhat true, 4 = Not at all true.

Lower mean scores indicate higher confidence

There was no significant difference in self-reported KSAs between males and females nor according to age. Mental health nurses reported higher confidence compared with community health workers in assessing potential for violence (p = 0.025) and assessing psychotic symptoms (p = 0.046), but the aggregated KSA scores were not significantly different between the two groups.

Stage 2: Post-Test upon Completion of the Education Program

The mean score for each NSR stage 2 (NSR2) item was calculated for the entire sample and ranked from 1 to 22, from most to least confidence. These rankings were then compared to the NSR1 rankings from baseline to determine any changes (Table 4). Mean scores on all 22 items on the NSR2 indicated higher KSAs for participants after the mental health education program, and all scores, apart from the lowest two, indicated high confidence (≥2.0). The difference between the two aggregated mean scores (NSR1 and NSR2) was not statistically significant.

TABLE 4 

NSR2: Knowledge, Skills, and Attitudes Ranked from Highest to Least Confidence from Baseline to Post Program

NSR 2 Rank	NSR 2 Mean	NSR2 Questions	NSR 1 Rank	NSR 1 Mean
1	1.39	I am able to refer mental health clients to appropriate services.	1	1.77
2	1.50	I always know when to refer a client for psychiatric consultation.	3	1.94
3	1.50	I am skilful in assessing psychotic symptoms.	6	2.16
4	1.50	It is as easy for me to triage a mental health problem as it is to triage a physical health problem.	8	2.16
5	1.55	My communication with mental health clients is respectful.	2	1.88
6	1.61	I am skilful in performing a mental status assessment.	7	2.16
7	1.61	My communication with clients affected by alcohol is respectful.	13	2.23
8	1.66	I find it easy to understand clients who are victims of repeated domestic violence.	19	2.38
9	1.66	I am able to communicate effectively with people who are suspicious or paranoid.	5	2.11
10	1.66	My communication with clients with self-inflicted injuries is respectful.	12	2.22
11	1.66	My communication skills enable me to calm anxious people.	15	2.27
12	1.77	I am able to detect side-effects of psychoactive drugs.	11	2.22
13	1.77	I work well with patients with self-inflicted injuries.	4	2.05
14	1.77	I understand the therapeutic actions of psychoactive drugs.	18	2.33
15	1.83	I utilize psychiatric terminology appropriately.	14	2.25
16	1.83	I am skilful in differentiating substance intoxication and psychosis.	17	2.33
17	1.88	I am skilful in assessing potential for violence.	10	2.22
18	2.00	I find it easy to understand clients who injure themselves or others while driving under the influence of substances.	22	2.70
19	2.00	I find it easy to understand clients who harm themselves.	16	2.29
20	2.00	I am able to identify the legal parameters that relate to mental health clients.	20	2.50
21	2.05	I am skilful in assessing suicide risk.	21	2.50
22	2.06	I am skilful in differentiating between delirium and dementia.	9	2.22

1 = Very true, 2 = Fairly true, 3 = Somewhat true, 4 = Not at all true

Lower mean scores indicate higher confidence

A key outcome from this study is that after attending the mental health education program, mean scores on all items, apart from two, reduced to ≤2, which indicates confidence in 20 KSA items. These results show an improvement on pre-test scores in which only three items scored ≤2.

The KSAs ranked highest in the NSR1 were consistent with the KSAs ranked highest in the NSR2: Referral of mental health clients, performing assessments of psychotic symptoms, triaging mental health problems, and respectful communication. Participants reported a large increase in confidence in respectfully communicating with clients affected by alcohol and understanding clients who are victims of repeated domestic violence. Participants also were reasonably confident about their communication skills with mental health clients in situations as detailed in Table 4. The lowest three ranking KSAs did not change greatly apart from participants’ confidence in their ability to differentiate between delirium and dementia, which reduced from a ranking of 9 to the lowest ranking item of 22.

There was no difference in self-reported KSAs between males and females, mental health nurses and community health workers, according to age, or those with and without mental health qualifications.

Pre-test and post-test aggregated Knowledge scores (5 questions), Skills scores (9 questions), and Attitudes scores (8 questions) were compared. Figure 2 shows all three aggregated scores reduced indicating higher confidence in KSAs post-test compared with pre-test. The difference in pre- and post-test scores for Knowledge (p = 0.048), Skills (p = 0.021), and Attitudes (p = 0.047) was significant.

Figure 2.

Changes in Knowledge, Skills, and Attitudes: Pre- and Post-Test Mean Scores

DISCUSSION

This research was undertaken to measure the efficacy of a mental health education program designed specifically for participants from Pacific island countries with the aim to improve knowledge, skills and attitudes. The program was based on the best available evidence, developed following the general principles of the World Health Organization Mental Health Gap Action Program (WHO, 2010) and four of the six goals of Grand Challenges in Global Mental Health (Collins, Patel et al., 2011), and using culturally and contextually appropriate information to ensure its specificity and applicability for the participant group. In addition, the educational intervention has been designed to incorporate a variety of pedagogical techniques to enhance the teaching and learning experience for mental health workers from Pacific islands, and it incorporated practical sessions with local clinicians, consumers, and carers to increase social awareness and related issues. This is similar to the findings of Cianelli et al. (2013), who also found that a short, specifically designed mental health training program has the ability to improve nurses and other health workers’ mental health knowledge, attitudes, and skills. However, in this project, the educational intervention differs from those that focus primarily on content delivery. The inclusion of mental health consumers and carers in the education program, alongside visits to local clinical facilities, proved to be an effective strategy to enhance change in participant attitudes.

This study is also unique as it measures the influence of an education intervention on the KSAs of nurses and community health workers from Pacific island countries who care for persons experiencing mental health issues. There is a scarcity of research that reports the professional development needs of Pacific island nurses. Given the lack of specialized professionals in the area of mental health and the recent World Health Organization (2013) call for an increase in, and more equitable distribution of, mental health professionals in low- and middle-income countries, the findings of this research provides important support for the need for ongoing professional development for mental health workers in Pacific island countries, as suggested by Saxena et al. (2007).

Many of the participants of the education program are leaders or hold important management positions in their respective countries. The improvement in the knowledge of the participants in this study may assist to improve the provision of mental health care and leadership in represented Pacific island countries, given people who hold positions of power within ministries of health and departments of public health often have low levels of essential mental health knowledge (Chisholm et al., 2007).

Limitations

While the research indicates promising results, it has a number of limitations. First, the small sample size means the results are not generalizable. A larger sample size would allow for examination of the relationships between variables, especially the effect of the training program for participants from different countries. As this was a pilot test of a newly developed education intervention program, a larger sample will be available after the course is rolled out on a larger scale.

Second, the use of the convenience sample from one delivery of the education program also limits the generalizability of the findings. However, as a pilot study of the first delivery of the education program, the results are extremely valuable.

Third, the questionnaire was validated by Henderson, Happell, and Martin (2007) and amended by the authors (KU and TP), but not tested on the cohort prior to the study. Therefore, the reliability and validity of the tool with this cohort is currently unclear.

Fourth, the study was limited by self-report data and by the social desirability inherent in all self-report measures. Self-report questionnaires may contain bias because individuals tend to endorse socially desirable knowledge, skills, and attitudes and under-report socially undesirable knowledge, skills, and attitudes. However, the pre-test/post-test study design is more effective than a single post-test program evaluation because each participant serves as his or her own control (Nimon, Zigarmi & Allen, 2011).

Implications for Practice and Research

This research indicates that the education program developed and delivered to the 18 participants from Pacific island countries was beneficial. Participants reported the education program had a meaningful impact on their knowledge, skills, and attitudes for the delivery of mental health care. This research extends previous research that reports effective outcomes from education-based training programs. This research is seminal to the field as there is nothing currently available to inform the development of educational interventions for mental health workers in the Pacific islands. Research should be undertaken after the next delivery of the education program to further evaluate the effectiveness of the program, explore the need for new content to be introduced into the program, and explore the outcomes of the program in participants’ countries of origin.

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.