Relationships between the quality of blended learning experience, self-regulated learning, and academic achievement of medical students: a path analysis.

PURPOSE: This study examined the relationships between the different aspects of students' course experience, self-regulated learning, and academic achievement of medical students in a blended learning curriculum.
METHODS: Perceptions of medical students (n=171) from the Royal College of Surgeons in Ireland, Medical University of Bahrain (RCSI Bahrain), on the blended learning experience were measured using the Student Course Experience Questionnaire (SCEQ), with an added e-Learning scale. In addition, self-regulated learning was measured using the Motivated Strategies for Learning Questionnaire (MSLQ). Academic achievement was measured by the scores of the students at the end of the course. A path analysis was created to test the relationships between the different study variables.
RESULTS: Path analysis indicated that the perceived quality of the face-to-face component of the blended experience directly affected the motivation of students. The SCEQ scale "quality of teaching" directly affected two aspects of motivation: control of learning and intrinsic goal orientation. Furthermore, appropriate course workload directly affected the self-efficacy of students. Moreover, the e-Learning scale directly affected students' peer learning and critical thinking but indirectly affected metacognitive regulation. The resource management regulation strategies, time and study environment, and effort regulation directly affected students' examination scores (17% of the variance explained). However, there were no significant direct relationships between the SCEQ scales and cognitive learning strategies or examination scores.
CONCLUSION: The results of this study will have important implications for designing blended learning courses in medical schools.

Introduction

From a social-cognitive perspective, self-regulated learning (SRL) has been defined as an active, constructive process by which the learners set goals, monitor their learning, and control their motivation, behavior, and cognition.1 This process is guided and constrained by the students’ goals and the contextual factors in the learning environment.1 Self-regulation of motivation involves controlling motivational beliefs, such as self-efficacy, goal orientation, and task value. Self-regulation of behavior involves the active control or use of various resources available for students, such as time, study environment, effort, and peers. Self-regulation of cognition involves the use of different cognitive strategies, such as rehearsal, organization, elaboration, and critical thinking as well as metacognitive regulation strategies. Metacognitive learning strategies are divided into three components: monitoring of learning processes, knowledge about cognition, and control of the learning processes.1 Models of SRL more or less share a general assumption that the student’s learning is primarily affected by self-regulatory processes, which mediate the relationship between personal characteristics (eg, intelligence and ability) and context of the learning (eg, good teaching, appropriate assessment). Educational researchers have argued that SRL is not a fixed trait and that students can improve their motivation and learning strategies when effective teaching strategies and environments are provided.2

Online learning environments are characterized by the autonomy of the learner, and therefore, self-regulation becomes a critical factor for students who are to take advantage of the benefits of these environments. To support this assumption, researchers have demonstrated that SRL is a predictor of academic achievement in technology-mediated learning environments.3,4 Recently, “blended” learning has been introduced as an innovative method to integrate e-Learning with face-to-face instruction, to provide meaningful student learning experiences. Blended learning has been defined as the appropriate mix and use of face-to-face instructional methods and various learning technologies to support planned learning and foster subsequent learning outcomes.5

The Student Course Experience Questionnaire (SCEQ) has been one of the most widely used instruments for evaluating the learning experiences of students.6 The questionnaire is composed of five scales, namely, good teaching, clear goals and objectives, appropriate assessment, appropriate workload, and generic skills. With the aim of evaluating the quality of the e-Learning component of blended learning environments, a previous study developed a 32-item e-Learning Experience Questionnaire.7 The questionnaire was composed of four scales: Good e-Teaching, Good e-Resources, Appropriate Workload, and Student Interaction. The researchers found significant correlations between the e-Learning scale scores and an overall rating of the quality of online materials and activities and statistically reliable correlations between good e-teaching scores, good e-resources scores, a deep approach to learning scores, and students’ grades.7 The same Australian research group examined the construct validity of an e-Learning scale that was added to the SCEQ to make it suitable for evaluating blended learning courses, in 3,602 students at a university degree level. Their results indicated that the SCEQ exhibited strong psychometric properties, including good construct and criterion validity, and high internal consistency and interrater reliability.8 Although these studies provided a step forward for understanding how the online part of the blended experience of student learning contributes to the quality of student learning in higher education, the effects of the quality of blended experience on the students’ SRL and their academic achievement is still an area which requires further research.

This study was therefore designed to address the following question: Do students’ perceptions of the quality of blended learning experience influence their SRL and academic achievement? If so, which aspects of the blended learning experience mediate these effects?

Methods

Study context

The target and sampling population in this study were the medical students at the Royal College of Surgeons in Ireland, Medical University of Bahrain (RCSI Bahrain), studying in the Junior Cycle of the medical curriculum. The curriculum is a 6-year program composed of a foundation year and 5 years of study divided into three cycles (Junior Cycle, Intermediate Cycle, and Senior Cycle). The Junior Cycle is delivered over three semesters (JC1, JC2 and JC3), and each semester is composed of six modules. The examinations during the Junior Cycle take place at the end of each semester. The assessment process involves continuous assessment; practical sessions; a written component, in the form of multiple choice questions and short open-ended questions; and reflective assignments. The overall score of the students at the end of the semester was used as a measure of academic achievement in this study. The curriculum in this phase was an integrated course in the clinical application of basic medical sciences. The study questionnaires were distributed to all the students at the end of the lectures, during the last week of modules 7 and 8 (Cardiovascular and Respiratory systems and Upper Limb) for JC2 students and modules 13 and 14 (Neuroscience) for JC3 students. The students who did not attend the lectures were personally invited to complete the remaining questionnaires to achieve maximum response rate. All participants signed a consent form and were informed that their responses would be kept confidential. The study protocol was approved by the Research Ethics Committee at the RCSI Bahrain.

Blended learning at RCSI Bahrain

The curriculum at RCSI Bahrain is delivered through face-to-face teaching that is complemented by online learning through a fully customized Virtual Learning Environment (VLE). Moodle™ serves as the RCSI Bahrain’s VLE. Moodle is an integral tool for students, facilitating both online learning as well as providing all program-related information, such as lectures, course notes, examination timetables, and assessment details. The VLE supports both student-to-student and student-to-lecturer communication through discussion forums that provide threaded, asynchronous, discussions, where students can post a query to which other students or a faculty member can respond. The VLE also provides many additional electronic learning resources, such as online tutorials and formative assessments, in order that students can learn and monitor their own progress.

Study instruments

Measurement of self-regulated learning

This study evaluated SRL of medical students using the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich et al.9 The MSLQ is made up of 81 items and consists of a motivation section and a learning strategies section. The motivation section comprises 31 items that assess students’ goal orientation, task value, self-efficacy, control of learning beliefs, and test anxiety. The learning strategy section contains 31 items assessing students’ use of different cognitive and metacognitive strategies. The cognitive strategies include rehearsal, elaboration, organization, critical thinking, and metacognition. In addition, there are 19 items assessing student management of different resources. The resource management scale includes four subscales, namely: time and study environment, effort regulation, peer learning, and help-seeking. Items are scored based on a seven-point Likert-type scale, from 1 (“not at all true of me”) to 7 (“very true of me”). Scale scores are constructed by taking the mean scores of the items that make up each scale.

Measurement of the quality of blended learning experience

The quality of the blended learning experience was measured by a questionnaire originally devised by Ramsden10 and modified for blended learning environments by adding a new e-Learning scale.8 The questionnaire is made up of two components: one measures the overall experience of the students in the course and is derived from the SCEQ developed by Ginns et al,6 while the other component measures the online experience of the students during the course.8 The questionnaire is composed of 22 items, each scored with a five-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”), and includes five core scales: good teaching scale (six items), clarity of goals and standards (four items), appropriate assessment (three items), appropriate workload (four items), and e-Learning scale (five items). Previous studies examining the psychometric properties of the SCEQ indicated that three of the scales (good teaching, clarity of goals and standards, and appropriate assessment) yield a second-order construct (called teaching quality).11,12 The e-Learning scale in the SCEQ includes five items related to the effects of online activities on learning: online learning resources, communications with peers, communications with teaching staff, integration between online and face-to-face learning, and impact of online experience on learning engagement.

Statistical analysis

Data analysis was conducted using Statistical Package for the Social Sciences (SPSS®) version 20 software and IBM SPSS Amos™ version 20. Data were presented as mean ± standard deviation (SD) of each parameter. A P-value of <0.05 was considered to be statistically significant. Internal consistency reliability for each scale was analyzed using Cronbach’s α statistic. To examine the effects of the SCEQ scales on the SRL scales and examination scores, a conceptual path model was created, using structural equation modeling as suggested by the literature. The goodness-of-fit measures of the path analysis included the chi-square (χ2), comparative fit index (CFI), Tucker–Lewis index (TLI), and the root mean square error of approximation (RMSEA). The χ2 test indicates the amount of difference between expected and observed covariance matrices. A χ2 value close to zero and a χ2 test P-value greater than 0.05 indicate that there is little difference between the expected and observed covariance matrices. Therefore, the χ2 value should preferably not be significant. However, because it is often unrealistic to find models in which the χ2 value approximates the degrees of freedom, a χ2/df below 2 is considered as an appropriate cutoff criterion.13 CFI assesses the agreement between the empirical results and the results expected by the model, and a good fit is indicated by a CFI value of 0.90 or greater.14 The RMSEA estimates how well the model would fit the sample if optimal parameters were available, and RMSEA below 0.05 indicates good fit, but values below 0.08 indicate reasonable fit.15 TLI, or Nonnormed Fit Index (NNFI), is another indicator that is commonly used to measure model fitness.16

Results

Descriptive statistics

Out of the 222 questionnaires distributed to the students, 171 completed forms were obtained (response rate =77%). For JC1 students, 88 students out of 121 responded (72.1%), and for JC3 students, 83 out of 101 students responded (82.2%). Table 1 shows the mean responses (± SD) of students and the α of each scale of the MSLQ and SCEQ. Among the five SCEQ scales, the e-Learning scale had the highest α (0.87), while self-efficacy had the highest α (0.86) among the MSLQ scales.

Table 1

Mean scores (± SD) of student responses (n=171) and Cronbach’s alpha (α) of the different scales of the SCEQ and MSLQ

Scale	Number of items	Scores (mean ± SD)	α
SCEQ scales (Likert scale 1 to 5)
Teaching quality	13	3.00±0.55	0.86
Good teaching	6	2.88±0.74	0.80
Clear goals and standards	4	3.40±0.74	0.76
Appropriate assessment	3	2.60±0.81	0.60
Appropriate workload	4	2.71±0.74	0.62
e-learning	5	3.10±0.89	0.87
MSLQ scales (Likert scale 1 to 7)
Motivation beliefs
Intrinsic goal orientation	4	5.16±1.30	0.60
Extrinsic goal orientation	4	5.69±0.99	0.64
Task value	6	5.77±0.77	0.67
Control of learning beliefs	4	5.36± 1.02	0.60
Self-effcacy for learning	8	5.20±0.84	0.86
Test anxiety	5	4.78±1.16	0.67
Cognitive and metacognitive strategies
Rehearsal	4	5.01±1.13	0.60
Elaboration	6	4.95±1.05	0.78
Organization	4	5.13±1.22	0.71
Critical thinking	5	4.30±1.14	0.73
Metacognition	6	4.74±0.84	0.60
Resource management strategies
Time and study environment	8	4.80±0.93	0.63
Help seeking	4	4.10±1.21	0.67
Effort regulation	4	4.90±1.10	0.60
Peer learning	3	4.30±1.41	0.58

Notes: Quality of course experience was measured by the SCEQ, and self-regulated learning was measured by the MSLQ, composed of motivation beliefs, cognitive/metacognitive strategies, and resource management strategies.

Abbreviations: MSLQ, Motivated Strategies for Learning Questionnaire; SCEQ, Student Course Evaluation Questionnaire; SD, standard deviation.

Path analysis for the relationships between quality of blended learning experience, students’ self-regulated learning, and examination scores

Figure 1 illustrates a summary of the conceptual path model created between the different study variables. In order to produce a more parsimonious model (ie, a model that accomplishes the desired level of prediction with as few predictor variables as possible), SRL scales with no significant relations with the other variables were omitted from the path analysis. The omitted SRL scales included task value, test anxiety, extrinsic goal orientation, and rehearsal strategies. By dropping nonsignificant paths and using model fit indices, we found a good fit between the tested model and the data (χ2=98.1, df =67.0, χ2/df =1.46, CFI =0.95, TLI =0.93, RMSEA =0.05). All path coefficients (ie, standardized regression weights) were statistically significant at the 0.05, 0.01, or 0.001 levels.

Figure 1

Path analysis of the different scales of the quality of learning experience, self-regulated learning, and academic achievement in a blended learning environment.

Notes: Quality of course experience was measured by the Student Course Experience Questionnaire, and self-regulated learning was measured by the Motivated Strategies for Learning Questionnaire, composed of motivation beliefs, cognitive/metacognitive strategies, and resource management strategies. Numbers on the arrows represent the estimates of standardized regression weights between the independent and dependent study variables. The error terms (e) inside the small circles reflect the unexplained variance and measurement errors. Interactions between academic achievement and both effort regulation, and time and study environment were statistically significant, at P<0.05. The rest of interactions were statistically significant, at P<0.01 or P<0.001.

Relation between SCEQ score and self-regulated learning

The face-to-face component of the blended experience affected students’ motivation as follows: 1) control of learning beliefs were directly influenced by the teaching quality and appropriate workload (12% of the variance explained); 2) self-efficacy was directly affected by appropriate workload (β=0.25) and indirectly by teaching quality (β=0.16); and 3) intrinsic goal orientation was directly influenced by teaching quality (β=0.25). e-Learning directly affected students’ peer learning (β=0.31) and indirectly affected critical thinking and metacognition, through peer learning (β=0.13 and =14, respectively).

Interactions between self-regulated learning scales

Students’ elaboration and organization strategies were directly affected by metacognitive regulation (β=0.66) and indirectly affected by peer learning (β=0.25) and critical thinking (β=0.24). Metacognitive regulation was directly affected by self-efficacy, critical thinking, and peer learning (37% of the variance explained). Time and study environment regulation was directly affected by intrinsic goal orientation and metacognitive strategies (22% of the variance explained). Help-seeking was directly affected by metacognitive strategies and peer learning (25% of the variance explained). Effort regulation was directly affected by time and study environment, and metacognitive regulation, and was indirectly affected by peer learning and intrinsic goal orientation (40% of the variance explained).

Relations with academic achievement

Examination scores were directly influenced by effort regulation (total effect: β=0.23), and directly and indirectly affected by time and study environment (total effect: β=0.33), and were indirectly affected by metacognitive regulation (β=0.19). These variables explained 17% of the variance in students, examination scores. However, neither the SCEQ scales nor motivation and cognitive strategies directly affected students’ examination scores.

Discussion

The primary goal of this study was to examine the relationship between factors measuring blended learning experience, SRL, and the academic achievement of medical students. The study expands on the previous research in order to explain the contribution of the e-Learning scale to the total blended experience.8 One of the main findings in this study is the effect of teaching quality and appropriate workload on students’ motivational aspect of self-regulation, specifically the expectancy and value components. Thus, students who perceived high quality of teaching had high beliefs that outcomes are contingent on one’s own effort, rather than external factors (high control of learning), and were more motivated to learn because of internal reasons, such as curiosity or mastering the content (high intrinsic goal orientation). Furthermore, students who experienced inappropriate workload were likely to have low confidence in their ability to perform a learning task (low self-efficacy). A recent study demonstrated significant correlations between the SCEQ scales and goal orientation, self-efficacy, and control of learning in 368 postgraduate students, in five different subject areas.17 Another study of 3,165 students was conducted in online and blended courses from 42 institutions and examined the relationship between learner self-efficacy and their ratings of the quality of their learning. They demonstrated that the relationship between teaching presence (a construct of teaching quality) and self-efficacy is stronger for students in blended learning environments.18 Taken together, these findings could represent a message to course designers about the impact of teaching quality and of balancing the course depth and breadth on students’ motivation. Although the findings in this study indicated a unidirectional relationship between the aspects of quality of course experience and motivation beliefs, future studies should examine the complex reciprocal interactions between these constructs.

The path model in this study indicated that the e-Learning component of blended experience had a direct effect on peer learning, which consequently affected help-seeking, critical thinking, and metacognitive regulation. The online interaction with peers and faculty is expected to stimulate students to be more proactive and help them to clarify learning material and reach insights that may not have attained on their own.9 The learning management system (LMS) used in this study provides communication tools such as discussion forums, which allow students to communicate with each other synchronously and asynchronously, and to work collaboratively online. For students to effectively monitor their understanding during online discussion, they require metacognitive skills to clarify any misconceptions and to actively decide to seek help if necessary. Peer learning mediated students’ help-seeking behavior because it is a type of social interaction where students take the initiative and request help from peers and teachers when they face difficulties in learning. These findings are consistent with a recent study demonstrating that the degree of cooperative work during a blended learning course is positively related to learning-oriented help-seeking behaviour.19 Similarly, a previous study indicated that computer-based learning technologies support students’ cooperation and help-seeking because of reduced threat and effort, higher quality, anonymity, convenience, and easing of temporal demands.20 Further studies that explore the relationship between the different characteristics of the LMS and the SRL of students can provide critical information regarding the impact of these tools on students’ self-regulation.

Students who scored high on three of the SRL scales, namely, time and study environment, effort regulation, and metacognitive regulation, were the high achievers in this study. Furthermore, metacognitive regulation played a pivotal role in directly mediating core of SRL activities, including cognitive strategies (elaboration and organization) and resource management strategies (help-seeking, time and study environment, and effort regulation). Therefore, examination scores were related to the level of planning study time, regulating the general study environment, and exhibiting commitment when facing difficult academic tasks. Overall, research evidence shows that organizational and time management strategies are strong predictors of academic achievement.21,22 Similarly, time and study environment, and effort regulation were significantly related to performance in online courses in liberal arts23 and Gerontology.4 In the current study, the modest effect of these variables on academic achievement means that a number of other variables that were not part of the model could exert an influence on academic achievement.

There are a number of statistically nonsignificant relationships between the study variables that require explanations. The finding that the quality of teaching and appropriate workload did not explain any significant variance in student cognitive or metacognitive self-regulation could ostensibly appear at odds with many previous studies that demonstrated a direct relationship between students ratings of course experience and their approaches to learning.11,24–26 This conflict could be partly resolved given that these studies conceptualized learning approaches into more generic learning styles (surface, deep, and strategic), which fused motivation and cognitive aspects of learning strategies.27 In contrast, measuring SRL using the MSLQ in the current study conceptualizes and assesses the five cognitive strategies separately from any motivational components.1 Regarding academic achievement, neither the quality of the course experience nor the motivation beliefs contribute to academic achievement. These data appear at odds with previous studies, which demonstrated that academic achievement is directly affected by the quality of course experience11,26,28 and student motivation, especially self-efficacy.4,29,30 Future studies using other measures of assessing academic achievement, such as acquisition of generic skills or core competencies, could provide a clearer picture of the effects of these variables on academic achievement.

Although the research design does not lack rigor, this study has some limitations. The first was the self-reported nature of the instruments used for measuring SRL by the MSLQ and the SCEQ. Second, students were asked to reflect on their experience after they had studied different subjects; therefore, the possible lack of consistency in their perceptions across different subjects could have affected the validity of the study findings. Third, the relatively small sample size used in the study could limit the generalizability of the study findings.

Conclusion

In conclusion, this study highlights the importance of the perceived quality of blended learning courses on student’s motivation and SRL strategies. Furthermore, adopting self-regulation of effort, time and study environment, and metacognition can positively influence the academic achievement of medical students. However, since this study was based on correlations, experimental research design is required to support cause–effect relationships between the study variables. Furthermore, similar studies in other medical schools are required, to test the generalizability of these study findings.