Lecturing skills as predictors of tutoring skills in a problem-based medical curriculum.

PURPOSE: Recruitment of tutors to work in problem-based learning (PBL) programs is challenging, especially in that most of them are graduated from discipline-based programs. Therefore, this study aims at examining whether lecturing skills of faculty could predict their PBL tutoring skills.
METHODS: This study included evaluation of faculty (n=69) who participated in both tutoring and lecturing within particular PBL units at the College of Medicine and Medical Sciences (CMMS), Arabian Gulf University, Bahrain. Each faculty was evaluated by medical students (n=45±8 for lecturing and 8±2 for PBL tutoring) using structured evaluation forms based on a Likert-type scale (poor to excellent). The prediction of tutoring skills using lecturing skills was statistically analyzed using stepwise linear regression.
RESULTS: Among the parameters used to judge lecturing skills, the most important predictor for tutoring skills was subject matter mastery in the lecture by explaining difficult concepts and responding effectively to students' questions. Subject matter mastery in the lecture positively predicted five tutoring skills and accounted for 25% of the variance in overall effectiveness of the PBL tutors (F=22.39, P=0.000). Other important predictors for tutoring skills were providing a relaxed class atmosphere and effective use of audiovisual aids in the lecture.
CONCLUSION: Predicting the tutoring skills based on lecturing skills could have implications for recruiting tutors in PBL medical programs and for tutor training initiatives.

Introduction

The concept of learning rather than teaching was a major breakthrough that challenged the role of traditional teachers in the academic front.1 An effective teacher is the one who stimulates thinking and facilitates student learning.2 Therefore, the role of the teacher goes well beyond providing information, with the teacher having a range of key roles to play in the education process.3 Twelve roles for any medical teacher have been identified and grouped into six domains: information provider, role model, facilitator, student assessor and curriculum evaluator, curriculum and course planner, and resource material creator.3

Problem-based learning (PBL) is one of the learning strategies that enables the students to develop critical thinking skills through posing challenges based on clinical case scenarios.4 The small-group tutorial is one of the corner-stones in PBL5 during which students are put in an active learning situation by providing them with clinical problems.6 The problem comes first without advance readings, lectures, or preparation, serving as a stimulus for the need to know.7 The students then go through active discussion and analysis of problems, develop hypotheses, explain mechanisms, and generate learning issues.8 These student-generated learning needs guide self-directed learning between the tutorial sessions, and then in subsequent sessions, students reapply, synthesize, and appraise their learning.9 Teaching in a PBL curriculum is a much different experience than almost any other instructional method. In the PBL tutorial group, the teacher’s main role is a facilitator for the students learning. On the other hand, she/he may deliver lectures (Resource Sessions) to a large student group with a primary focus on the content expertise as lectures in PBL can also be an important learning resource.10

Therefore, the role of the PBL tutor is of pivotal importance, as student learning would depend on the tutor’s understanding and appreciation of her/his responsibilities in the small group.5 The tutor keeps the group focused on their tasks and guides them to achieve their goals.11 The tutor’s skills used during the PBL tutorial session are: asking open-ended questions, listening to students and intervening only when needed, helping students to reflect on their experience, monitoring progress of the group, promoting the group dynamics, stimulating critical thinking, and encouraging a warm tutorial atmosphere.12 The tutor must not dominate a session with content-specific questions and answers that convert it into a tutor-led seminar.9

Most of the faculty members in PBL medical programs are graduated from discipline-based programs with good familiarity in lecturing and little experience in small-group facilitation. Although faculty development programs are routinely held to train PBL tutors to better practice their role in the small group tutorials, the performance of faculty members as tutors in PBL is unpredictable. Furthermore, students’ rating of the effectiveness of faculty on PBL tutoring is influenced by their teaching styles and other dimensions such as content expertise and interpersonal attributes.13 On the other hand, lecturing styles can vary from being didactic, in line with discipline-based programs, to being interactive, which is more in line with the notions of knowledge and learning in PBL.14 However, there is no current body of evidence on whether aspects of lecturing skills can predict the tutoring skills of faculty members in PBL programs. This study was, therefore, designed to answer the following research question: What aspects of the lecturing skills of a faculty member can predict his/her tutoring skills in a PBL medical school?

Methods

Study context and subjects

This study is a cross-sectional survey conducted at the College of Medicine and Medical Sciences (CMMS), Arabian Gulf University, Bahrain. The study included faculty members (n=69) who participated in both tutoring and teaching a particular PBL unit (block) in the pre-clerkship phase (years 2, 3, and 4) during the academic years 2010 to 2012. At the end of each PBL unit, lecturing skills of faculty members were evaluated by students (n=45±8 per faculty), while tutoring skills of the same faculty members were evaluated by students in their tutorial groups (n=8±2 per faculty). For each faculty member, evaluations of tutoring skills and lecturing skills by students were averaged for the purpose of statistical analysis. All faculty members involved in PBL tutoring have been involved in a comprehensive faculty development program before being inducted and certified as tutors. Lecturers are the subject matter experts of disciplines related to the problems of each PBL unit. The research protocol was approved by the Curriculum Committee and Research and Ethics Committee at CMMS.

Study instruments

Two types of instruments were used in this study for evaluation of the faculty members by students. To ensure content validity of both instruments, they were developed based on a comprehensive literature review and refined by input from a group of medical education experts. The first instrument evaluates the PBL tutoring skills and includes ten items addressing the tutor’s role in the PBL tutorials. These items are: enthusiastic about educational role, stimulates group interactions, provides effective feedback, facilitates discussion of learning needs, helps students to achieve unit objectives, encourages students’ self-directed learning, communicates clearly with students, facilitates integration of knowledge, and behaves as a role model. In the last item, students were asked to provide an overall rating of the effectiveness of the tutor. The second instrument evaluated the lecturing skills of faculty members in Resource/Review Sessions. This instrument is modified from a previously validated form.15 The evaluation form included six areas: lecture organization and content, interactivity and clarity of presentation, subject matter mastery, class atmosphere, effective use of audiovisual aids, and time management in the lecture. In both study instruments, each parameter is evaluated based on a five-point Likert scale (1= poor, 2= fair, 3= good, 4= very good, 5= excellent).

Statistical analysis

The data were entered and analyzed using the Statistical Package for Social Sciences (SPSS) version 22 (IBM Corporation, Armonk, NY, USA). Data were presented as mean ± standard deviation (SD) for each parameter. Internal consistency reliability for the study instruments was measured using Cronbach alpha statistics. In order to determine the effectiveness of different lecturing skills (independent variables) in predicting PBL tutoring skills (dependent variables), stepwise multiple regression was used to estimate how much variance in tutoring skills was accounted for by items of lecturing skills. A P-value <0.05 was considered statistically significant.

Results

Descriptive statistics

The response rate from the students in PBL tutorials was 78%, while the response rate for students who evaluated the lecturing skills was 38%. However, the absolute number of responses for evaluating PBL tutoring skills of faculty was 552 and for lecturing skills of faculty was 3, 105 students. Internal consistency reliability coefficient for the tutoring skills evaluation form was 0.94 and for lecturing skills was 0.93. Table 1 shows the students’ ratings for 69 faculty members regarding their tutoring and lecturing skills. The mean scores of both tutoring and lecturing skills are considered to be relatively high. Among the tutoring skills, the ability to communicate well was rated with the highest mean scores (4.55±0.51), while providing effective feedback was the lowest (4.29±0.59). Among the lecturing skills, subject matter mastery was rated with the highest mean score (4.01±0.50), while clarity and interactivity was the lowest (3.89±0.53).

Table 1

Student-rated scores of the faculty members regarding their tutoring and lecturing skills

Tutoring and lecturing skills	Mean±SD
Tutoring skills
Enthusiastic about his role	4.50±0.54
Stimulates students’ interaction	4.46±0.52
Provides effective feedback	4.29±0.59
Facilitates generation and discussion of learning needs	4.41±0.57
Helps students achieve learning objectives	4.44±0.55
Stimulates students’ self-directed learning	4.35±0.53
Communicates clearly	4.55±0.51
Facilitates integration of knowledge	4.45±0.54
Behaves as a role model	4.49±0.56
Overall effectiveness	4.49±0.55
Lecturing skills
Presentation is organized with appropriate content	3.96±0.51
Interactivity and clarity	3.89±0.53
Subject matter mastery	4.01±0.50
Relaxed class atmosphere	3.96±0.43
Effective use of audiovisual aids	3.94±0.49
Time management	3.93±0.48

Note: Data scores are expressed as mean ± SD.

Abbreviation: SD, standard deviation.

Do lecturing skills predict tutoring skills in a PBL program?

The six items of lecturing skills (lecture content and organization, interactivity and clarity, subject matter mastery, relaxed class atmosphere, effective use of audiovisual aids, and time management) were used in a stepwise multiple regression analysis to predict each of the items in tutoring skills. The prediction models indicated that three lecturing skills, namely, subject matter mastery, relaxed class atmosphere, and effective use of media of audiovisual aids were significant predictors of PBL tutoring skills. Table 2 shows the raw and standardized regression coefficients of the predictors, their squared correlations (ΔR2) to show how much of the variance in each outcome variable can be accounted by each predictor, and the analysis of variance (ANOVA) results (F and P-values) to indicate the degree of significance of the model in predicting the outcome variables. Among the lecturing skills, subject matter mastery was the most important predictor of overall effectiveness in tutoring skills and accounted for 25% of the variance in overall effectiveness of PBL tutors (F=22.39, ΔR2=0.25, β=0.50, P=0.000). In addition, subject matter mastery positively predicted the following five PBL tutoring skills: 1) enthusiastic about his/her role; 2) helps students achieve unit objectives; 3) stimulates students’ self-directed learning; 4) facilitates integration of knowledge; and 5) behaves as a role model.

Table 2

Stepwise linear regression analysis of the relationship between students’ rated scores of lecturing skills of faculty as predictors for their PBL tutoring skills

Lecturing skills (predictors)	Tutoring skills (outcomes)	Unstandardized coefficients		β	ΔR2	ANOVA
		b	SE b			F	P-value
Subject matter mastery	Enthusiastic about her/his role	0.49	0.12	0.46	0.21	17.59	0.000
	Helps students to achieve objectives	0.46	0.12	0.42	0.17	13.99	0.004
	Encourages students’ self-directed learning	0.48	0.12	0.46	0.21	17.54	0.000
	Facilitates knowledge integration	0.53	0.11	0.49	0.24	21.53	0.000
	Behaves as a role model	0.58	0.12	0.51	0.26	23.61	0.000
	Overall effectiveness	0.55	0.12	0.50	0.25	22.39	0.000
Relaxed class atmosphere	Provides effective feedback	0.45	0.16	0.33	0.11	8.11	0.006
	Stimulates group interactions	0.55	0.13	0.45	0.20	16.98	0.000
Effective use of audiovisual aids	Communicates clearly with students	0.55	0.11	0.51	0.26	23.51	0.000
	Facilitates discussion of learning needs	0.48	0.14	0.39	0.16	12.33	0.001

Notes: : A measure of how strongly each predictor variable influences the criterion variable. Δ2: Indicates the proportion of the variance in the criterion variable which is accounted for by our model.

Abbreviations: ANOVA, analysis of variance; PBL, problem-based learning.

The second significant predictor of tutoring skills was relaxed class atmosphere during the lecture, which positively predicted providing effective feedback and stimulating students’ interaction in PBL tutorials. Finally, effective use of audiovisual aids in a lecture positively predicted communicating well with students and facilitating group discussion of learning needs in PBL tutorials.

Discussion

This study examined the faculty lecturing skills as predictors of their tutoring skills in a PBL medical program. The results of the regression analysis indicate that subject matter mastery in a lecture is the most significant predictor of the overall effectiveness of a PBL tutor. Therefore, according to students’ evaluations, the teacher who is able to explain difficult concepts for students and respond satisfactorily to their queries in a lecture is most likely to be a good PBL tutor. These findings add another piece to the controversy regarding the use of content versus non-content experts as PBL tutors. Previous studies indicated that students appreciated the facilitative role of faculty in PBL tutorials.16,17,18,19 Furthermore, tutors stressing on the learning process in the tutorial group were perceived as more effective than content expert tutors.17 In contrast, other studies indicated that students rated highly tutors with content knowledge related to the problem.13,20,21 The contradictive findings in these studies could be due to several variables including the definition of content expertise, cultural differences, level of faculty training, curriculum structure, and academic level of students.

This study demonstrates that subject matter mastery in a lecture predicted the enthusiasm of PBL tutors about their role, promoting knowledge integration, helping students to achieve unit objectives, stimulating student’s self-directed learning, and the tutor as a role model. These findings are supported by previous studies which indicated that students with expert tutors generated significantly more learning issues, which were more congruent with the faculty objectives,22 and spent longer studying time on self-directed learning.22,23 Furthermore, a recent study in Brazil indicated that medical students perceived content expert facilitators to be more effective than their nonexpert counterparts at building knowledge, guiding the learning process, achieving cognitive learning, generating learning goals, and motivating self-study.24 In contrast, PBL tutors with subject-matter expertise were reported to play a directive role in the tutoring process, supply more direct answers to questions posed by students, and suggest more points for discussion with less emphasis on group interactions.15,25,26 However, a key issue is how the faculty uses her/his content expertise in promoting students learning in both lectures and in PBL tutorials. Therefore, it is conceivable that faculty in the current study use their content expertise in promoting students’ learning in PBL tutorials through identifying gaps of knowledge to generate learning needs and demonstrating the relevance of knowledge to promote integration, which motivate the students’ self-directed learning.

The finding that subject matter mastery explained only 25% of the variance in PBL tutor effectiveness indicates that there are other important factors which could explain tutor effectiveness. A previous study indicated that students identified three main roles for effective PBL tutors: facilitative expertise, knowledge expertise, and clinical reasoning expertise.27 In addition, Maudsley9 claimed that effective PBL tutors promote student learning by creating a supportive environment that encourages active participation by all members of the group, by monitoring the quality of learning through questions and feedback and by encouraging the development of students’ metacognitive skills. Furthermore, one of the attributes of an effective PBL tutor was establishing rapport with students, through establishing good relationships with students, respecting students’ opinions, understanding students’ feelings, appreciating students’ performance, and being friendly and kind with students.13 These studies are corroborated by our finding that providing a relaxed class atmosphere significantly predicted stimulating students’ interactions and providing effective feedback in PBL tutorials. A positive learning environment is essential for feedback to be maximally effective.28 In addition, feedback was termed effective when it is conducted in a private setting using a considerate tone and requires good interpersonal skills on the part of the teacher.29 Similarly, climate setting was identified as one of the key elements on how the prospective PBL tutor might prepare for her/his role as a facilitator.12 The teacher can establish a relaxed class atmosphere by demonstrating respect for the learners and their needs, praise, encouragement of participation, and lack of threat to personal integrity.30 An interesting finding in the present study is that effective use of audiovisual aids by the lecturer predicted both her/his communication clearly with students and helping them to discuss learning needs in PBL tutorials. This finding indicates that faculty who encourage the use of audiovisual aids in lectures are likely to encourage the students to use them in PBL tutorial groups. A recent study indicated that students perceived that the use of diagrams as an audiovisual aid in PBL tutorials helped them to structure knowledge, to develop an overview of topics, and stimulated them to find relationships between topics.31 Furthermore, PBL tutors emphasized that diagramming increased interaction and enhanced the focus and detail of the discussion.31 Overall, this study emphasized that effective PBL tutoring requires multiple skills from faculty members that are positively related to their role in large group lectures.

There are some limitations of this study which need to be explored. The study is conducted in one PBL medical school and evaluation of lecturing and tutoring skills of faculty is done only through students’ ratings. The way PBL is implemented in that school and how students see a good PBL tutor could, therefore, affect the relationships between lecturing and tutoring skills. A larger scale study conducted on different PBL medical schools and triangulating sources of faculty evaluation will be required to ensure generalizations of the study findings. Although only two measures were taken which supported the content-validity evidence and internal consistency reliability for both instruments, a comprehensive study to evaluate the other types of validity evidence for the scores emanating from these instruments will be required.

Conclusion

Subject matter mastery, providing a relaxed class atmosphere, and effective use of audiovisual aids by faculty in lectures significantly predicted their PBL tutoring skills. However, subject matter mastery in a lecture appears to be the most important predictor of the overall effectiveness of PBL tutors. These findings could have implications for recruiting PBL tutors in medical programs and for tutor training workshops.