Impact of faculty development programs for positive behavioural changes among teachers: a case study.

PURPOSE: Faculty development (FD) is essential to prepare faculty members to become effective teachers to meet the challenges in medical education. Despite the growth of FD programmes, most evaluations were often conducted using short questionnaires to assess participants' satisfaction immediately after they attended a programme. Consequently, there were calls for more rigorous evaluations based on observed changes in participants' behaviours. Hence, this study aims to explore how the FD workshops run by the Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore have impacted behavioural changes in the educators.
METHODS: We followed up with the educators at least half a year after they have attended the workshops. With limited literature as reference, we initiated a small-scale case-study research design involving semi-structured interviews with six educators which was triangulated with three focus group discussions with their students. This allowed us to explore behavioural changes among the educators as well as evaluate the feasibility of this research methodology.
RESULTS: We identified three emerging categories among the educators: ignorance to awareness, from intuition to confirmation and expansion, and from individualism to community of practice.
CONCLUSION: Although FD have placed much emphasis on teaching and learning approaches, we found that the teacher-student interaction or human character components (passionate, willing to sacrifice, are open to feedback) in becoming a good educator are lacking in our FD workshops.

Introduction

The knowledge and clinical skills necessary for safe and effective practice of medicine have increased substantially over the last few decades. The delivery of quality care has also become increasingly complex. With these developments, it is increasingly challenging for clinical teachers to train students and residents for safe and effective practice [1]. Developing clinical and biomedical science faculty to become effective teachers is critical to meet these challenges in medical education. Thus, faculty development (FD), or staff development as it is often called, is essential to prepare faculty members in preparation for the change. FD was initially conceptualised as strategies to improve teaching performance [2]. This definition has evolved to a broader perspective which involves scholarship, leadership, and quality assurance in responding to the challenges of multiple roles and evolving responsibilities [3].

It is well documented that FD promotes and contributes to faculty members’ teaching knowledge, behaviours, and skills, rekindling their motivation to change their attitude towards embracing effective learning strategies [4]. However, a self-reported questionnaire remained the most widely used tool to assess these improvements immediately after they attended a FD programme [4,5]. Despite the growth in FD research, many studies still employ weak designs and quantitative studies are still dominating research in this field [3]. Only two studies employed qualitative method in understanding the impact of FD programme [6,7] as mentioned in the research article by Leslie et al. [3], while Steinert et al. [5] reported seven studies. Additionally, these studies mainly focused on an interpretive description of an intervention. Self-reported changes in specific teaching methods remained as the most popular method in assessing educator’s behavioural changes [5]. However, the validity of using this type of assessment in understanding teachers’ behaviour is debatable. Do teachers’ behaviour and skills really improve if they said so? And, how do teachers know they have improved without applying the skills learned in workplace? Therefore, there were calls for more rigorous method to understand changes in participants’ behaviours after attending FD programme.

To address the gap mentioned above, we employed a different research approach which involved using a modified case study method to explore how the FD workshops run by the Centre for Medical Education (CenMED), Yong Loo Lin School of Medicine (NUS Medicine) have impacted behavioural changes among the teachers. This small-scale study was conducted as a pilot to explore the feasibility of conducting such evaluation in a research intensive medical school and in a high service based academic medical centre. The study involved following up with the teachers at least half a year after they have attended the workshops. This period allowed participants to assimilate, plan, and implement what they have learned into their teaching practices. In addition, we also take into account students’ perspectives on their teacher’s teaching for two reasons: (1) to triangulate the teachers’ qualitative data and (2) to understand from students’ perspectives the roles expected of a teacher. The findings we obtained not only helps us to design and implement a more effective FD programme but also provide researchers with another feasible method in collecting data besides the usual quantitative and exploratory qualitative methods in FD.

Methods

We employ specific single case variety as the research methodology for this article as multiple units of analysis were taken into account and measured [8]. Case studies are utilised because it is suitable for answering “how” and “why” questions to generate and build theory with little data or theory exist [8,9]. In addition, it allows the researchers to use “controlled opportunism” to discover new or existing concepts by collecting flexible responses from the respondents [10]. There are many types of case studies methodology namely descriptive, explanatory, and explorative. An explanatory approach was applied in this case study because this approach is useful to assess how interventions are working and why. Furthermore, this methodology is appropriate in understanding the root cause of a problem, further modification to the intervention and, attempts to explain the causal effects (if found) [9]. A comparative analysis was carried out to compare and contrast the data collected.

Construct validity, internal validity, external validity, and reliability suggested by Yin [9] were carefully considered in this research. The construct validity was dealt with during the study by using multiple sources of evidence such as semi-structured interviews with tutors, focus group discussions (FGDs) with students and scholarly work evidence from the interviewed tutor. Then, the members of the group reviewed the draft of the case study report when the chains of evidence were established. The internal and external validity of the findings were also attended by applying time-series analysis and replication logic respectively. Finally, we use a case study protocol and develop a case study protocol to ensure the reliability of the results. Ethical approval was obtained from the Institutional Review Board at National University of Singapore (reference no., 13-392; approval no., 2002). Interviewees read the participant information sheet and the investigators answered their queries before they signed on the consent form to take part in the study, as approved by the Institutional Review Board.

1. Interview

For teachers’ interview, invitation emails were sent by a research assistant to participants who had attended at least one FD workshop offered by the Centre in 2013 and 2014. Participants included healthcare professionals, basic scientists as well as support staff and administrators working in this field and convenient sampling was used. Six participants voluntarily took part in the interview, including three physicians, a basic scientist, an allied health educator, and a dental educator. Three of them were from NUS Medicine, the rest were from off-campus clinical training sites (Khoo Teck Puat Hospital and Alexandra Hospital) as well as another tertiary institution (Nanyang Polytechnic). The focus of CenMED FD programmes, apart from imparting technical pedagogical skills to participants so that they can facilitate student- centred and outcome-driven learning sessions, is to professionalise the teaching among the clinicians and basic sciences educators. The workshops offered by CenMED in 2013 and 2014 are included in Appendix 1.

For student FGDs, we planned to have six FGDs with students taught by the six interviewees initially. However, due to inability to gain access to certain groups of students, only three FGDs were recruited with six to seven students per group. Students who attended at least one lesson taught by the interviewees were eligible to participate and participation is voluntary. Thirteen clinical and six pre-clinical students from NUS Medicine took part in the FGDs. To capitalise on students’ shared experience, homogeneity is crucial. Hence, students taught by the same interviewee were grouped together and hence separated into clinical/pre-clinical.

The interviews were conducted between February to November 2014 at the participants’ workplace while the FGDs were done from March to October 14 at CenMED’s office. Each interview lasted from 45 minutes to 1 hour. A set of semi-structured interview questions and FGDs guide were used. The questions were finalised after discussions between the investigators following a literature review. The interview questions were centred on the following: (1) changes made to teaching after attending workshops; (2) reasons to attend the workshops; and (3) changes to students’ feedback, learning, and participation to their teaching. For the FGDs, we focused on: (1) students’ views on teachers’ teaching activities, (2) level of engagement in class, and (3) satisfaction with teachers’ teaching. Please see Appendix 2 for the full questions and prompts. A semi-structured format was adopted so that we could get responses relevant to the study aims and provide participants the freedom to share their thoughts. Field notes were taken and all the sessions were audio-recorded for accuracy.

To maintain consistency in the data collected, one of the study investigators who had prior experience in qualitative research conducted the interview for all sessions. Before the start of each session, the interviewer explained the study purpose and the process to the participants. Additionally, participants were informed that the information collected would be kept confidential and their names will not be used in any future publications. Participants signed the consent form. For the FGDs, each student was assigned a unique code and reminded to say the number before participating in the discussion. This ensured the confidentiality of the participants as well as the accuracy of the subsequent transcript.

2. Data analysis

Two research assistants listened to the recording, verified the content with the field notes, and typed the transcript. No identifiers were included and participants were identified by a unique code (number for student and ‘I’ for teachers). Member checking was done to ensure that the data was not amended or misinterpreted. The completed transcript was sent back and participants were given the opportunity to add, remove or modify the content.

The content analysis method was used to analyse the data. Three members from the research team read the transcripts several times to understand fully the data, before coding and organising them into categories and sub-categories. Keywords or phrases that relate closely to the aims of the study were identified and coded based on the meaning of the sentences. Next, codes that appeared consistently and were associated with similar content were grouped into sub-categories, and similar sub-categories were combined into a category. To ensure the congruency of the data as well as rigor, another independent researcher also coded the data. During occasions when there were differing views on the coding, they re-examined the data meaning and discussed before agreeing on a code. A qualitative analysis software was used (Atlas.ti, version 7; Scientific Software Development, Berlin, Germany).

Results

Prior to attending FD programme offered by CenMED, some of the teachers have attended several teaching courses while the rest did not. Hence, the teachers attended FD programme with different goals and expectations which later influenced their development after attending the programme. There are three emerging themes which are summarised in Table 1 and the themes were elaborated in the following.

Table 1.

Example of the Three Themes Emerged

Emerging theme	Teachers’ findings	Students’ findings
From ignorance to awareness	"I was given a teaching time slot, and I just satisfy that requirement… I didn’t really appreciate to understand the bigger picture of the whole situation."	"He actually has a couple of different sessions in which he uses multiple methods to teach…"
From intuition to confirmation and expansion	"After a few workshops I have attended, I think I developed my method that is question-based learning. In question- based learning I tried to solve everything in the tutorial with questions. I start from a specific question and widen the spectrum of things."	"I think DR Y managed to strive a good balance between preparing us in the skills we need to be a good doctor as well as the skills we need for exams… Another thing that is very good about him is that he is genuinely interested in teaching the students…He is also receptive to feedback."
		"I mean, to know a lot of things is one thing. To put the effort, like to structure, organise everything, that requires a lot of effort."
From individualism to community of practice	"So when I come to the workshop, I wanted to find out in terms of other disciplines facing the same challenge and how they have been managing specific challenge in their own discipline, it is kind (of) like a peer network, peer support group…mutually sharing of knowledge."

1. From ignorance to awareness

One respondent narrated that he has not gone through any courses about teaching and learning previously and did not take the initiative to understand them. He teaches because he was given this responsibility and this was mentioned several times in his transcript. For example,

“I was given a teaching time slot, and I just satisfy that requirement… I didn’t really appreciate to understand the bigger picture of the whole situation.”

“When the students come around (for) my tuition, what you are doing is you are teaching your specialty.”

The respondent realised the importance of having an overview of the curriculum after attending the FD programme and began to make inquiry about academic matters such as an overview of the medical curriculum and students’ information as stated below:

“So in the past, I didn’t pay much attention (to) which year of students are coming… The only objective is to provide teaching to that particular slot, now I am more aware. I make sure whatever I do cater (to) their needs.”

The respondent showed heightened awareness of the learning process and an increased reflection on his own teaching and students learning. He also started taking into consideration his colleague’s teaching to avoid overlapping in terms of content and using this to activate students’ prior knowledge. This is evident when the respondent commented about workplace-based assessment:

“In the past, when (a) student approach(ed) me, we randomly select a case that we have, or sometimes we just do a discussion over the case, and then I score them and return the sheet to them. Having attended the workshop, I made additional effort to try identifying (an) appropriate case, and then, make an effort to actually observe for mini clinical evaluation exercise (CEX) and verbal feedback to them.”

Students’ findings were positive and consistent with the data from the teacher. For example,

“I think his way of teaching is doing it situationally.”

“He actually has a couple of different sessions in which he uses multiple methods to teach…”

2. From intuition to confirmation and expansion

Several teachers in this category have been developing their teaching approaches and experimenting new approaches based on their experience before attending the FD workshops. The respondents in this category have certain pre-conceptions about teaching and learning as described in the following excerpt:

“I was looking for idea (on) how to interact with student … I would like to have more interaction with the students and the reason that I couldn’t have a good teaching (was) maybe I am not very effective in interaction.”

“We don’t really have a structured methodology as to how we teach, a lot of time, it is based on our anecdotal experience- what we learnt from our previous teachers, we try to apply and modify accordingly.”

After attending the workshop, some of them had their concepts enhanced while some received affirmation that what they have done was on the right track. For example, one participant commented as below:

“Since last year that (workshop) I attended, I believe I am in the right direction.”

“I am always aware of entrustable professional activities (EPAs)… but right now, I’m always mindful to make sure what I am teaching is in accordance with EPA.”

After the respondents’ concepts about teaching and learning have been further developed, they tended to be more innovative in planning and designing new approaches. For example,

“After a few workshops I have attended, I think I developed my method that is question-based learning. In question-based learning I tried to solve everything in the tutorial with questions. I start from a specific question and widen the spectrum of things.”

“We try doing one mini-CEX just before we do the ‘Simulated Pathological Eye Case Scenarios’ and we realised the response from the students were overwhelming good… We are trying and hopefully this will replace the formal lecture.”

Based on the teachers’ findings, the major impact of FD programme was improving teaching approaches. This was also shown in the excerpt by the students.

“I think what’s so good about him is that he used a lot of materials to teach different concepts, such as plastic bag about these models… he helps us to visualise something which is very difficult to think.”

“I felt like the best thing about his class is very engaging.”

However, students also observed tutors’ personality and behaviour other than teaching approaches. Inevitably, teaching approaches were important in engaging students; yet, it might not work just to have good teaching approaches. The excerpts below echoed this statement.

Student A: I think Dr A is very attentive to class. Not like other tutors, he would look at each of us and ask us “do you have a question?

Student B: Yes, he noticed about our body language.

Student C: It’s effort. I mean, to know a lot of things is one thing. To put the effort, like to structure, organise everything, that requires a lot of effort.

Student B: Yeah, he is actually willing to extend the part of teaching for us.

Student A: If you don’t know how to answer, he doesn’t embarrass you or anything. He motivates you to learn, we cannot disappoint him. (Excerpt from one of the FGD on tutor A)

From the excerpt above, students mentioned a few good characters of a teacher which motivate them to learn because they were afraid that they might disappoint the teacher. Here is another excerpt indicating the importance of a tutor’s personality and behaviour in influencing student learning.

Student X: I think DR Y managed to strive for a good balance between preparing us in the skills we need to be a good doctor as well as the skills we need for exams… Another thing that is very good about him is that he is genuinely interested in teaching the students… He is also receptive to feedback.

Student Y: Some students feel neglected in other hospitals because they didn’t really spend time for students. But despite of having all these activities, he still can arrange certain focused session with students.

Students Z: I think he is also very encouraging and friendly tutor… we made mistakes, he continued to prompt us to improve and he was truly willing to teach us so that we can improve. Think that was one of the key things about him compared to some doctors. (Excerpt from one of the FGD on tutor Y)

These discussions showed that a good teacher goes beyond having just teaching technique. Although the major aim of FD programme is to have an impact on medical teachers’ practices, it remains unknown whether the FD programme has also influenced the tutor behaviour and personality in teaching.

3. From individualism to community of practice

We discovered that one respondent had some preexposure of medical education knowledge and he sought more than just implementation of certain approaches in his teaching and learning. He also had clear understanding of certain concepts in medical education as described in the following excerpt.

“In the United Kingdom, I was involved in (a) few modules around medical education, so I have done a few assignments based on that as well… Workplace-based assessment interests me, maybe because I used that in the United Kingdom and here, but the way (they are) used is different… Here is more of a summative assessment rather than a formative assessment.”

The respondent elucidated a higher expectation and broader scope from the workshop since he exhibited understanding in the medical educational concepts. For example,

“I try to understand the cognitive side, not just transform of medical knowledge, it’s more of their understanding of a particular concept, so that students could engage more.”

“There’ll always be challenges… for example giving feedback to the students in different settings, different learners…”

The respondent in this category perceived FD programme as a learning community apart from improving his own teaching and learning knowledge.

“… It’s like a symbiosis, it’s sort of a peer group… rather than seeking my own specific challenge, it’s more knowing what is existing around… to see whether (it) can be applied to my own teaching.”

“So when I come to the workshop, I wanted to find out in terms of other disciplines facing the same challenge and how they have been managing specific challenge in their own discipline, it is kind (of) like a peer network, peer support group… mutually sharing of knowledge.”

These were the results from teachers’ perspective regarding the benefits they perceived and changes in teaching after attending the FD programme which were triangulated with students’ data.

Discussion

The results of the present study are consistent with the previous findings in the literature on impact of FD workshop on participants [5,6,11,12]. Although FD encompasses broader components such as educational research, curriculum development, and education leadership, the major impacts most educators elucidated were still on teaching and learning such as changes in teaching approaches and developing an awareness of the learning process. However, our study has in-depth data to demonstrate different participants who have diverse expectations which in turn has different impacts on their teaching. For some participants, it is an awareness creation for them while some sought after confirmation and expansion of their teaching and learning approaches. Innovation in teaching and learning approaches is also mentioned by the participants. Some learners sought for different level objectives such as understanding of theories as well as participating in learning communities to share and exchange knowledge. There are a lot of factors that influence the intent of a person who attended FD programme. It might be linked to a change in healthcare delivery, pursuit of excellence in teaching and medical education [13,14] changing emphasis in healthcare, and desire to offer quality training programmes for students and residents or to network with colleagues [15]. Understanding these factors can be very useful in the planning, designing, and delivery of the FD programme.

However, is it only teaching approaches (technical component) that matters in order to become a good educator? This answer can be found in students’ data. Apart from teachers who understand and apply best practices in teaching and learning, students are also looking for teachers who are passionate, willing to sacrifice, and are open to feedback as shown in their interview data. Many research in higher education described the characteristics of an effective teacher which can be grouped into three ‘core categories’—teacher knowledge (pedagogical knowledge), teacher-student interaction or human characters (enthusiasm, respect, availability, etc.), and teaching skills (classroom management and clarity) [16]. Subject matter knowledge has been the most common expectation of a good teacher, nevertheless, teacher-student interaction as mentioned by the students are equally important in this study. A study of Tam et al. [17] revealed that the attributes of an effective teacher is somehow different to some Western studies as Asian students place more emphasis on ‘care’ and ‘heart’. Similarly, in a research carried out by Kikukawa et al. [18] to explore attributes of a good clinical teacher, medical knowledge and clinical competence of teachers were not emphasized, even though these were the most commonly identified themes in Western countries. Students prefer a teacher who is more accessible to one who has a lot of medical knowledge. Though the students in this study did not weigh which category is more important, teacher-student interaction and human characters inevitably are perceived as important attributes of a good teacher in an Asian context.

However, these teacher-student interactions or human character components are lacking in our FD workshops. This is also common in the FD programmes of other institutions as the goals of FD across countries and disciplines have placed much emphasis on teaching and learning [19]. Existing programmes focus on helping new faculty members with the understanding of roles and responsibilities as a teacher, student-centred teaching skills, and integrating technology into teaching and research. Yet, we have over-looked the teacher-student interactions or human character components as one of the important aspects to be integrated in FD programme.

1. Study limitations

This is a small-scale study to explore the feasibility of the case study method to gauge behavioural changes among the teachers after attending CenMED FD programme. Thus, this have very limited claim to generalisability. With difficulty in gaining access to recruit students for interview, not all teachers’ comments were triangulated. Hence, some teachers’ comments were triangulated with other evidence such as score sheet and portfolio. Besides that, students could only provide their opinion on teachers’ teaching skills although exploring the change in behaviour in other aspects such as leadership, scholarship, and different roles (assessors, curriculum planner, and evaluator) is crucial after attending a FD programme. This will require further planning to investigate the behavioural changes in different aspects after attending FD.

The study employed purposive sampling and hence the findings can only be generalised in places with similar context or conditions. There may be some recall bias as the students were taught by many tutors. Moreover, students’ and tutors’ data was collected sometime after the implementation. Researchers overcome the trustworthiness of the data by constantly comparing students’ and tutors’ verbatim transcript. This could be enhanced in future by using the observation method to triangulate the data.

Although preliminary findings from this research revealed that students are fond of tutors who spend quality time and close contact with them, future research could focus on the students’ assessment performance and results. Additionally, more in-depth studies could be conducted on whether good quality contact with tutor will impact students’ learning, which might influence the structure and organisation of the curriculum as well as specific FD interventions.

Another aspect discovered is conducting a proper evaluation needs good planning and constant reminders to the faculty. Getting the student groups for the interviews was also a challenge due to their busy schedules.

2. Conclusion

In conclusion, our findings have added new insights to the existing scope of FD literature. We have piloted a case study approach to evaluate the impact of FD workshops beyond the usual methods. We also highlighted the needs of teachers from various levels and the gaps in the existing FD programme. Perhaps a general needs assessment survey is required to understand the expectation of the participants before attending the FD programmes. While most FD programmes are focusing on pedagogy in training educators, integrating the affective components in training educators is crucial. However, the challenge will be to incorporate these programs to organizational processes as part of the overall professional development of clinical and biomedical faculty.

Workshops offered by CenMED:
1. 21st century teaching learning strategies
2. Developing learning outcomes and entrustable professional activities
3. Competency based medical/education: mini-clinical evaluation exercise
4. Teaching for effective learning: large group teaching
5. Teaching for effective learning: small group teaching
6. Teaching for effective learning: e-learning
7. Competency based medical/health professional education: providing effective feedback to enhance learning
8. Competency based medical/health professional education: written assessment
9. Competency based medical/health professional education: skills/workplace based assessment
10. Competency based medical/health professional education: multi-station exams
11. Portfolio for learning & assessment
12. Developing context rich multiple choice questions
13. Curriculum design, evaluation and continuous quality improvement
14. Scholarship of teaching & learning

Focus group discussions
The focus group discussions were guided with 6 semi-structured questions, along with probes and follow-ups to guard against superficial responses. These questions were modified depending on whether it is classroom teaching or clinical teaching.
	1. What do you think about the readings that the instructor assigned for you to complete before the teaching-learning activities?
	2. What do you think about the instructor’s teaching activities?
	3. What do you think about the assignments?
	4. What do you think about your performance during the semester?
	5. What do you think about your engagement in the class?
	6. Overall how satisfied are you with the instructor’s teaching?
Further clarification or follow-up questions as within the normal scope of the focus group were asked.
Interview questions
1. What major challenges did you have in your teaching prior to attending the Centre for Medical Education (CenMED) programs?
	- What activities do you do with your students? (Open-ended)
2. What made you attend the CenMED programs?
	- What other similar programs did you attend before?
3. Which CenMED programs have you attended?
4. What changes have you made in your teaching practice after attending the CenMED programs?
	- Syllabus
	- Interaction with students
	- Giving feedback
	- Teaching style
5. What changes have you observed in students’ preparedness in your teaching after attending the CenMED programs?
	- Evidence
	- Examples
6. What changes have you observed in students’ participation in your teaching after attending the CenMED programs?
	- Evidence
	- Examples
7. What changes have you observed in your students’ learning after attending the CenMED programs?
8. Have the CenMED programs helped you in any other ways? Please explain.
9. Identify areas of our programs that need improvement.
	- Why?
	- How?
10. In next 5 years, anything new you would like to see CenMED offers?
	- New programs/topics
	- Formats of delivery
	- Innovations that could help you.