Applying a mixed methods design for needs assessment of an international fellowship training program for Mongolian health professions.

PURPOSE: Continuing professional development is essential for improving health care services, especially in developing countries. Most of the training programs in Mongolia were not based on a targeted needs assessment. Therefore, we aimed to apply a mixed methods design to assess the training needs of Mongolian health professions.
METHODS: We conducted a needs assessment using a convergent parallel mixed methods design in two steps. The survey and interview questions were developed to identify priority areas, targeted trainees, and effective training methods. A survey on 60 respondents, 15 individual interviews, and a focus group interview with 14 participants were conducted in the first step. In the second step, 12 representatives of key stakeholders were invited to a second focus group interview.
RESULTS: Current health policy areas, areas related to future national plans, and areas not currently receiving governmental or international support were suggested as the main priorities. The stakeholder suggested that trainees should be selected based on their professional experience and language level, as well as each hospital's needs. Building teams including various professions, such as nurses, technicians, and biomedical engineers, was recommended as a way to exchange ideas with each other and to build teamwork for future collaboration.
CONCLUSION: Medical training needs are dynamic and complex; therefore, a deep understanding of the context and setting is necessary. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design, which could be an effective way to conduct needs assessments for training programs.

Introduction

Continuing professional development is essential for improving health outcomes and health care services. The linkage between the competency of health professions and the improvement of health outcomes is a major challenge in developing countries [1]. Without active learning, health professions could not maintain their competencies in a sustained manner after graduation [2]. Developing an effective program for continuing professional development is a four-step process that includes targeting the audience, identifying training needs, developing educational strategies, and determining the expected training outcomes. In order to identify training needs, it is important for health professions to reflect on their own knowledge and skills [3]. Their thoughts about what situations they find challenging and feedback from colleagues could be useful for identifying training needs [4].

Training programs that are effective at changing health professions’ behavior are based on well-prepared needs assessments [5]. Needs assessment is a process that helps program developers to understand trainees and training environments [6]. Training needs for targeted audiences can be investigated by surveys, focus groups, regional analyses, and evaluations of current programs. The results of a targeted needs assessment could be used by regulatory bodies to set standards and to establish requirements for training programs. It is also crucial for justifying the agenda and grant requests for the program [7].

New strategies for needs assessment are needed [2]. In an increasingly dynamic and complex world, quantitative and qualitative approaches should be combined in the form of a mixed methods design [8]. The major advantage of using a mixed methods design is that doing so combines the strengths and non-overlapping weaknesses of both approaches [9]. Even though a mixed methods design is more costly and time-consuming than using a single approach [9], other researchers highlighted that it enables the assessment of multiple factors which are crucial for successful training implementation [10]. Various ways exist to combine quantitative and qualitative approaches [11,12], including the use of a convergent parallel design where both methods are performed independently and the results are merged to extend the scope, breadth, and range of the components [13].

In developing countries, building the capacity of health professions to meet local needs is challenging [1]. Mongolia is a developing country located in East Asia, the population of which reached 3 million by the end of 2016 [14]. Approximately 70% of the population live in the cities, and two-thirds of all medical doctors work in Ulaanbaatar, the capital city [15]. Various international training programs have been implemented to improve health professions’ capacity in Mongolia. However, there are not any continuing professional development programs which are formally delivered by the academic societies or the Ministry of Health in Mongolia. A few international training programs provided by Korea and Japan used a survey for assessing the training needs of primary and secondary level hospitals. Both program developers organized the training of trainer workshops and several training programs for pediatric diseases, emergency care, ultrasonography, laboratory, X-ray, electrocardiography, and obstetric and gynecologic diseases areas. The program from Korea was implemented in the capital city and one province, while the program from Japan covered the two provinces in Mongolia [16,17]. There is lack of literature of the targeted needs assessment of tertiary level hospitals. Therefore, we aimed to apply a convergent parallel mixed methods design for targeting training needs and to develop a comprehensive training strategy for higher level training in Mongolia.

Methods

1. Setting and participants

Our research scope was to assess training needs of higher level health professions in Mongolia. Important stakeholders including doctors, professors, and high level officers from all the central hospitals and two government organizations which are responsible for post-graduate training were selected as targets for the study. We conducted the needs assessment in two steps. A convergent parallel mixed methods design was used for the first step of the needs assessment. A survey, individual interviews, and a focus group interview were conducted in parallel and analyzed separately. In the second step of the needs assessment, representatives of key stakeholders were invited to the second focus group interview. The aim was to develop a consensus framework based on the results of the first step. Some participants in the survey also participated in the interviews in both steps (Fig. 1).

Fig. 1.

Flow Chart of the Study Design

2. Quantitative approach

We administered the survey mainly to the medical doctors at tertiary level hospitals. However, we also included some hospital directors and government officers in the survey. Based on the health report of the Ministry of Health in Mongolia [15] and studies in Mongolia and other developing countries [16,18], the survey focused on four points: the major needs of health professions, the major obstacles to health professions’ development, priority areas, and ways to target trainees for further training. Respondents chose two options out of five for major needs, three out of seven for major obstacles, five out of 14 for priority areas, and three out of eight for trainees to be targeted. At the end of each item, an answer option for “other” was added. Thus, the respondents were able to add their own responses (Appendix 1).

3. Qualitative approach

The qualitative assessment of the first step included individual interviews and the first focus group interview. Individual interviews were conducted with medical doctors who had participated in previous fellowship training and stakeholders from three central hospitals and four specialized centers (cancer, maternal and child care, trauma and orthopedics, and infectious diseases). The first focus group interview was conducted with members of the two main organizations that are responsible for postgraduate training: The Center for Health Development under the Ministry of Health and the Mongolian National University of Medical Sciences under the Ministry of Education. The interview was focused on current health policies and international projects, the major needs and obstacles of health professions, the targeting of trainees, and the selection process of the fellowship training program (Appendix 2). As the second step of the needs assessment, a second focus group interview was conducted with the 12 representative key stakeholders from the individual interviews and first focus group interview. The aim of the second focus group interview was to share the results of the first step and to develop a consensus about the final needs.

4. Data analysis

All quantitative data were analyzed by Windows Excel 2016.Ink (Microsoft Corp., Redmond, USA). To ensure reliability, second researcher analyzed the qualitative data as an inter-rater. In order to test the validity, research team constantly compared the data of both methods.

5. Ethics approval

This study was funded by the Korean Foundation for International Health. The Institutional Review Board of Seoul National University College of Medicine approved this study (C-1803-024-927). All stakeholders participating in this study provided informed consent and contributed of their free will.

Results

1. Survey results

A total of 60% of the respondents were women, and the plurality of respondents were 40–49 years old. In average seven specialized doctors from each hospital responded to the survey. Over one-third of the respondents (36.6%) had more than 20-years of work experience (Table 1). Improving skills (24%) and the incentive system (23%) were identified as major needs, followed by improving the attitudes, human resources policy, and knowledge of health professions (Table 2). Poor undergraduate medical education (21.5%), postgraduate education (19.4%), and continuing professional development training (15.3%) were identified as major obstacles to health professions’ development, followed by poor medical facilities and equipment (Table 3). Pediatric diseases, cancer, cardiovascular diseases, surgery, and respiratory diseases were priority areas for future training (Table 4). The question about trainees to be targeted was used to identify the primary candidates for future training. We divided doctors into three categories based on their work experience: those with less than 10 years of experience were classified as junior, those with 10–20 years of experience as mid-career, and those with more than 20 years of experience as senior. Junior doctors (30.7%), nurses (21.9%), and mid-career doctors (17.5%) were most commonly suggested to be targeted as trainees (Table 5).

Table 1.

Descriptive Statistics for the Respondents

Characteristic	No (%)
Gender
Female	40 (66.7)
Male	20 (33.3)
Age (yr)
≤29	9 (15.0)
30–39	14 (23.3)
40–49	26 (43.3)
50–29	10 (16.7)
≥60	1 (1.7)
Work experience (yr)
≤5	9 (15.0)
6–10	10 (16.7)
11–20	19 (31.7)
≥21	22 (36.6)

Table 2.

Major Needs of Health Professions

Rank	Major needs	% of responses
1	Improving skills	24.0
2	Improving the incentive system	23.0
3	Improving attitudes	21.2
4	Improving human resources policies	21.2
5	Improving knowledge	9.7
6	Others	0.9

Table 3.

Major Obstacles to Health Professions’ Development

Rank	Major needs	% of responses
1	Poor undergraduate education	21.5
2	Poor postgraduate education	19.4
3	Poor continuing professional development training	15.3
4	Poor medical facilities	15.3
5	Poor medical equipment	15.3
6	Lack of incentives	11.8
7	Lack of government support	1.4

Table 4.

Priority Areas for Future Training

Rank	Priority areas	% of responses
1	Pediatric diseases	15.1
2	Cancer	11.9
3	Cardiovascular diseases	11.5
4	Surgery	7.9
5	Respiratory diseases	7.3
6	Hospital management	7.3
7	Nursing	6.0
8	Emergency care and trauma	5.5
9	Obstetrics and gynecology	5.5
10	Infectious diseases	5.0
11	Public health	5.0
12	Diagnostic laboratory	3.2
13	Medical education	3.2
14	Medical research	2.8
15	Others	2.8

Table 5.

Targeted Trainees for Future Training

Rank	Targeted trainees	% of responses
1	Junior doctors	30.7
2	Nurses	21.9
3	Mid-career doctors	17.5
4	Health administrators	9.5
5	Technicians	6.6
6	Senior doctors	5.8
7	Professors	4.4
8	Other	2.2
9	Directors and heads	1.4

2. Results of the interview in the first step

The interviewees emphasized cardiovascular diseases, cancer, research, emergency care, and pediatric as priority areas. They stated that team training for cardiovascular diseases is the current main direction of national health policy. Newly established centers, such as the center for orthopedics, the diagnostic center, and the university hospital were also mentioned. Participants recommended building and training new teams for the new centers and for areas that are lacking support from the government and international organizations. The other highlighted area of the interview was research. Improving research capacity is an important factor in health professions’ development. The interviewees mentioned that university faculty members and health professions at the hospitals could form research teams for further research development.

The interviewees suggested three main directions for selecting the trainees targeted for international training. First, team training was highly recommended. Building a team including members of various professions, such as nurses, technicians, and biomedical engineers, is important to promote collaborative practice. Second, the selection criteria should be based on trainees’ professional experience and language level. Some interviewees recommended including some members with a high level of English competency to help other members with communication. Third, the trainees’ selection process should be based on the hospital’s needs. A government organization should not recommend trainees whose background is not related to the hospital’s goals and needs.

3. Results of the focus group interview in the second step

During the second step of the needs assessment, we shared the results of the survey and the first focus group interview with the representative key stakeholders. The key stakeholders agreed that cardiovascular and pediatric diseases should be considered the most important areas. The stakeholders also mentioned that changes in infrastructure and national health policy should be considered as important priority areas. Some other important areas such as obstetrics and gynecology and infectious diseases were found to be already covered by governmental and international support. Therefore, the key stakeholders suggested basing future initiatives on unmet needs and avoiding overlap with other programs. The key stakeholders suggested targeting mid-career doctors as trainees. They pointed out that junior doctors would be advantageous as trainees because of their English level and active learning style. However, they could have some difficulties in recognizing crucial information in various learning situations. In contrast, mid-career doctors have more professional experience, meaning that they could more effectively learn in any situation. Thus, the key stakeholders highlighted that professional experience is the most important factor in effective training. Lastly, they agreed that training for members of other professions, such as nurses and technicians, is crucial for collaborative practice in health care.

Team training that would include members of various professions, such as nurses and technicians, was highly recommended. There was disagreement about whether teams should be formed within a hospital or organized across hospitals. The key stakeholders stated that it might be difficult to build a team across hospitals for some specialized areas, such as cancer and trauma. Nonetheless, they agreed that training teams at the same time would be more appropriate than dividing teams to be trained in different periods. In this way, the team members will be able to exchange ideas with each other and naturally build teamwork to support future collaboration. Six months was suggested as an appropriate period for both basic sciences and clinical training.

Discussion

Most of the studies which used a mixed methods design were conducted in the developed countries. There are lack of existing literature of training needs assessment in the developing countries. One study applied a mixed methods design for assessing the training needs of pediatric oncology nurses in some African and Asian countries [19]. In our study, we applied a convergent parallel mixed methods design to assess the training needs of Mongolian health professions in the tertiary level hospitals. Our team developed survey questionnaires based on a review of the local literature. We found that one of the major needs of Mongolian health professions was to improve their skills. It could be supported by the fact that most of the previous international training programs were mainly based on lectures and discussions in order to improve the knowledge [16,17]. The results of the survey provided us a general overview of the priority areas and the trainees to be targeted. However, other researchers have stated that developing a survey based on a literature review [6] and applying the categories created by researchers could be too general for specific contexts [8]. According to the WHO, Mongolia has the highest burden of cardiovascular diseases and liver cancer which were also shown in our survey result. However, the survey respondents also chose pediatric diseases as one of the priority of topics which were declining in the past years [14].

Qualitative approaches incorporate the categories identified by participants themselves, and are responsive to local situations [8]. Thus, we conducted interviews in order to obtain a deeper and broader understanding of the situation. We found three main directions for priority areas and targeting trainees. Current health policy areas, areas relating to future national plans, and areas not covered by the government were suggested as the main priorities. For example, non-communicable diseases were already covered by the various training programs supported by the government and international aids. However, there was a lack of support to the areas which are covered by the newly established centers in the Ulaanbaatar city such as orthopedics center and diagnostic center [16,17]. The stakeholders suggested selecting trainees for international fellowship training based on their professional experience and language level, as well as each hospital’s needs. A high level of English language competency was also mentioned as a crucial factor for effective international training in a previous study [20].

Other important issues were the team training and research areas which were highly recommended in the interview and not included in the literature review and survey results. Other researchers have reported that improving teamwork in medical teams is challenging, meaning that a systematic approach is needed for course development. They suggested including not only medical doctors, but also all types of health professions in team training [21]. Inter-professional education improves patient care services and enhances the skills and experience of health professions [22]. Several factors influence team training performance, such as the combination of team members and the scope of the organizations [21]. During our focus group interview with stakeholders, there was disagreement on whether teams should be formed within a hospital or organized across hospitals. However, the participants agreed that the decision should depend on the training topic.

Even though, there were some different results from each method, most of the results from both methods were similar in our study (Table 6). Various researchers stated that if the results from two approaches have similar conclusions, the results have greater credibility. Also, there are other studies emphasize the importance of utilizing the local stakeholders for the construct validity of the mixed methods [23]. Our study included the representatives of the main government organizations, the main central and specialized hospitals, and the single national medical university in Mongolia. Therefore, our results could be used as the basic information on training needs for future programs and research.

Table 6.

Comparison of the Results of Survey and Interviews

	Survey	Individual and focus group interviews
Priority areas	Pediatric diseases, cancer, and cardiovascular diseases	- Emergency care and medical research
		- Focus on current health policies and not to overlap with other government and international training programs
Targeted trainees	Junior doctors, nurses, and mid-career doctors	Mid-career doctors due to their professional experiences
Effective training methods	Not applicable	Inter-professional team training

Limitation of our study is that it was conducted to tertiary level hospitals in the capital city of Mongolia and the number of participants was relatively small. Thus, the results of this study cannot be interpreted as representing the training needs of all levels of the health delivery system in Mongolia. Further research should include health professions at both the regional and national levels in order to obtain wider information about training needs in Mongolia.

In conclusion, medical training needs are dynamic and complex, requiring a deep understanding of the context and setting. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design. The results could be used as a basis for implementing training programs in central and specialized hospitals. This study showed that a mixed methods design could be an effective way to conduct needs assessments for training programs.