The Impact of a Revised Curriculum on Academic Motivation, Burnout, and Quality of Life Among Medical Students.

OBJECTIVE: The purpose of this study was to determine the impact of a revised curriculum on medical student academic motivation, burnout, and quality of life.
METHODS: This cross-sectional comparative study involved 2 medical school cohorts of second year and fourth year medical students at The University of Auckland: a cohort under a traditional curriculum (n = 437) and a cohort under a revised curriculum (n = 446). Participants completed self-reported questionnaires measuring academic motivation, burnout, and quality of life. Two multivariate analyses of covariance (MANCOVAs) were conducted.
RESULTS: The response rate was 48%. No statistically significant differences were found between curriculum cohorts for mean scores of academic motivation, personal burnout, and quality of life. However, differences were found when comparing preclinical medical students and students in their clinical years of training. In comparison with Year 2 medical students, the MANCOVA for Year 4 students showed a significant main effect for the revised curriculum with respect to both physical and environmental quality of life.
CONCLUSIONS: A revised medical curriculum had a differential effect on quality of life for Year 4 students in the latter years of medical school who are based in a clinical learning environment.

Introduction

There is increasing recognition of the need to maintain medical students’ motivation for learning and well-being if they are to successfully complete medical school and continue on into their postgraduate specialty training.[1,2] However, the intensity of the medical training process may have unintended negative consequences with a high incidence of complaints of burnout and poor quality of life (QOL) among medical students.[3-5] Such distress can have important repercussions for student learning and academic motivation.[3,6]

Although calls for changes to medical curricula to enhance student motivation and address psychological distress have been made, little is known about how curricula should be reformed and what effects can be expected.[4] The most common curriculum reforms described and evaluated in the medical education literature are changes to assessment methods, which have been shown to have implications for both medical student motivation and well-being.[7-10] Moving from numerical or letter-grade hierarchical systems to pass/fail grading is one such example.[7,8] A further method of assessment that has been hypothesised to affect student motivation and well-being is progress tests, a form of longitudinal assessment that assesses the end objectives of a curriculum.[11] Often, a number of tests are set during an academic year, each consisting of a large number of questions assessing graduate-level knowledge. Because progress tests are longitudinal measurements, it is assumed that students will be less anxious about passing or failing a single test, as a cross-sectional failure has little impact on a series of good results.[11,12] However, these hypothesised benefits remain untested and the extent to which progress testing affects student distress remains unknown.

Similarly, it is unclear from the existing literature whether or not progress testing fosters a deep approach to learning as theorised. For example, a study by Blake et al[13] showed that the overall effect of progress testing on students’ approach to learning was minimal. Van Berkel et al[14] also investigated the influence of progress tests on study behaviours and could not conclude whether progress tests promote intrinsically motivated learning.

Research aim

Progress tests were implemented into the University of Auckland’s Bachelor of Medicine and Bachelor of Surgery (MBChB) programme in 2013 as part of a revised medical curriculum. The aim of this study was to evaluate the effect of these changes on academic motivation, burnout, and QOL among medical students.

Method

Study setting and participants

The University of Auckland medical curriculum consists of 5 years of study that follows on from a premedical year of health sciences or biomedical sciences (‘Year 1’). The 5 years of study are structured across 3 phases. The first phase (‘Year 2’ and ‘Year 3’) is considered the ‘preclinical’ years and has a focus on science within clinical medicine; this is followed by the second phase, which is clinically oriented (‘Year 4 and Year 5’); and the third phase, which prepares the student for the medical workforce (‘Year 6’).

The revised curriculum was progressively rolled-out over a 3-year period: first, Year 2 and Year 4 students in 2013; second, Year 3 and Year 5 students in 2014; and finally, Year 6 students in 2016. This research involved the comparison of students in the first phase of implementation of the revised curriculum. Year 2 and Year 4 students within the revised curriculum in 2013 (n = 446) were compared with Year 2 and Year 4 students under the traditional curriculum in 2012 (n = 437). Table 1 provides a summary of the traditional and revised curricula at The University of Auckland.

Table 1.

Summary of a revised curriculum implemented at the University of Auckland.[15,16]

	Traditional curriculum	Curriculum revisions
Teaching and learning
Year 2 and Year 4	No formalised clinical scenarios database	Clinical scenarios database – used to effectively define the core curriculum of the Bachelor of Medicine and Bachelor of Surgery (MBChB) programmeGreater emphasis on delivering content in an interdisciplinary manner
Year 2	Lecture-based delivery of content	Increased opportunities for small group learningReduction in the proportion of lecture-based delivery of core material and increased clinical content
Year 4	Didactic campus-based lectures in addition to clinical rotations	Reduction in didactic campus-based lectures and greater emphasis on symposia for interdisciplinary learningIntroduction of formal learning sessions at hospital-based clinical schools (synchronous learning)
Assessment
All year levels	No single exam for all year levels	Progress testing for all year levels
Year 2	Final exams	End-of-module tests with module grades and online formative self-tests. These are in parallel with the progress tests
Year 4	No end-of-year clinical OSCE	End-of-year clinical OSCE
Curriculum domains
	Acquisition and application of medical knowledgeProfessional, clinical, and research skillsHauora Māori (indigenous health)Population health and primary care	Aspects of the previous domain of clinical, professional, and research skills were integrated to create a new personal and professional skills domainAddition of health and well-being’ subtheme, and inter professional learning themes
Clinical attachments
Year 4	Rotation through medical and surgical specialties during the course of the academic year	Half year of medical specialty rotations and a half year of surgical specialty rotations
Year 4	Students rotated throughout different clinical campuses	Students remain at one clinical campus
Year 4	General practice teaching on campus and in general practice clinics or in rural locations	General practice teaching on campus only

Abbreviation: OSCE, Objective Structured Clinical Examinations.

Procedures

Written informed consent was obtained from all participating students, and ethics approval was obtained from the University of Auckland Human Participants Ethics Committee (reference 8467). Self-selected students were distributed a self-report paper-based questionnaire composed of validated measures of academic motivation, burnout, and QOL.[17-20] Students in the traditional curriculum completed the questionnaire in September 2012, whereas students in the revised curriculum completed it in July 2013. All questionnaires were completed at the end of a lecture. Data from all questions that were answered were included in the analysis. Data that were missing from each questionnaire were excluded from the analysis.

The questionnaire was composed of the Academic Motivation Scale (AMS), which measures intrinsic, extrinsic, and amotivation.[17,21] Intrinsic motivation refers to motivation derived out of genuine interest in or enjoyment of an activity, whereas extrinsic motivation is for personal gain or to avoid punishment.[22] Amotivation is an absence of motivation. The questionnaire also contained subscales of Motivated Strategies for Learning Questionnaire (MSLQ) to measure self-efficacy and test anxiety.[17,23] The AMS and MSLQ have been previously used in medical education research both internationally and in New Zealand.[24-26] In a study by Mitchell[26], the construct validity of the AMS has also been investigated among a University of Auckland medical student population and suggested that the AMS was appropriate for the assessment of motivation among medical students.

Consistent with a study by Kusurkar et al,[27] the AMS was adapted so that it could be applied to medical students. The AMS has 28 items across 7 subscales. For the purposes of this study, each item is scored on a Likert scale of 1 to 5. Intrinsic motivation scores were calculated from the AMS as an average of the intrinsic motivation scores on the 3 subscales.[27] In comparison, extrinsic motivation scores were calculated by taking an average of introjected regulation and external regulation scores.[27] The identified regulation subscale of the AMS was not included within calculations and subsequent data analysis as the items on this subscale are such that most students in professional education would answer positively.[27]

The questionnaire also contained the World Health Organization Quality of Life-BREF (WHOQOL-BREF) and the ‘personal burnout’ scale from the Copenhagen Burnout Inventory (CBI) to measure QOL and burnout, respectively.[19,20] Both the WHOQOL-BREF and the CBI have been used in a number of QOL and burnout studies during medical training, and the WHOQOL-BREF has been previously validated among the University of Auckland medical student population.[25,28,29]

The WHOQOL-BREF questionnaire included 24 items that encompass 4 QOL domains (physical, psychological, social, and environmental), which measures an individuals’ perceptions of their physical and psychological state, their social relationships, and their living environment.[20] The scores for each domain were calculated using the WHOQOL-BREF syntax with scoring between 4 and 20.[30] The higher the score, the higher the QOL. The personal burnout subscale of the CBI contains 6 items with scoring from 0 to 100 for each item with higher scores indicating higher levels of burnout. The total score on the scale was the mean of the scores on the items.[19]

Statistical analyses

All statistical analyses were performed using IBM SPSS 22.0 internal reliability measures, and Cronbach α coefficients were calculated to assess scale reliability for each section of the questionnaire.[31] The χ2 analyses were conducted to determine any significant demographic differences between cohorts within each curriculum. Any significant demographic differences were then included in a multivariate analysis of covariance (MANCOVA) model as covariates.

A series of MANCOVAs were conducted: the first MANCOVA included measures of burnout and QOL as dependent variables and the second MANCOVA included measures of academic motivation and self-efficacy and test anxiety as the dependent variables. The independent variable was curriculum cohort (traditional or revised), and covariates were included based on the findings of the χ2 analysis. A subgroup analysis of curriculum cohorts by year level was then conducted. The effect size was calculated from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Results

The Cronbach α scores for each of the WHOQOL domains, CBI, AMS, and MSLQ subscales were within acceptable limits (Tables 2 and 3).[33]

Table 2.

Cronbach α values for motivation subscales within the study questionnaire.

Intrinsic motivation	Extrinsic motivation	Amotivation	Self-efficacy	Test anxiety
0.86	0.88	0.88	0.89	0.84

Table 3.

Cronbach α values for burnout and quality of life subscales within the study questionnaire.

Personal burnout	Physical QOL	Psychological QOL	Social QOL	Environmental QOL
0.84	0.73	0.75	0.66	0.77

Abbreviation: QOL, quality of life.

The response rate was 48%. The mean age of the sample was 22 years. Men represented 184 of the 426 participants. No statistically significant differences in age, sex, ethnicity, or admission criteria were noted between curriculum cohorts. However, there were differences in response rates between year levels, and therefore, year level was included in the MANCOVA model as a covariate. Participant characteristics are outlined in Table 4.

Table 4.

Study participant characteristics.

Student characteristics	All students, N = 426	Students in traditional curriculum, N = 212	Students in revised curriculum, N = 214
Male, No. (%)	N=184 (43%)	93 (44)	91 (43)
Age, mean (SD)	21.8 (3.0)	22.0 (3.1)	21.5 (2.9)
Year 2	207	87	125
Year 4	N=214	N=120	94

The MANCOVA showed no statistically significant differences in mean scores of burnout, QOL, and academic motivation between students in the traditional and revised curricula (Tables 5 and 6). When comparing differences by year level, the MANCOVA for Year 2 students also showed that there were no significant differences between curricula with respect to burnout and QOL, academic motivation, self-efficacy, and test anxiety (Tables 7 and 8).

Table 5.

Personal burnout and QOL LSM scores and SEs by curriculum.

Well-being LSM[a] (SE)	Students in traditional curriculum	Students in revised curriculum	P value**	Effect size – partial eta squared[b]
Personal burnout[c]	39.9 (1.1)	39.2 (1.1)	.673	.000
Physical QOL[d]	15.5 (0.1)	15.7 (0.1)	.462	.006
Psychological QOL[d]	14.5 (0.2)	14.5 (0.2)	.905	.000
Social QOL[d]	15.0 (0.2)	14.7 (0.2)	.358	.002
Environmental QOL[d]	14.8 (0.2)	15.2 (0.2)	.095	.007

Abbreviations: LSM, least square mean; MANCOVA, multivariate analysis of covariance; QOL, quality of life; SE, standard error; WHOQOL-BREF, World Health Organization Quality of Life-BREF.

The reporting LSMs are adjusted for year level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by the Copenhagen Burnout Inventory – personal burnout subscale; range of scores = 0 to 100.

Measured by the WHOQOL-BREF; range of scores = 4 to 20.

P values from MANCOVA.

Table 6.

Academic motivation LSM scores and SEs by curriculum.

Academic motivation LSM[a] (SE)	Traditional curriculum	Revised curriculum	P value**	Effect size – partial eta squared[b]
Intrinsic[c]	3.5 (0.04)	3.5 (0.04)	.258	.003
Extrinsic[c]	3.0 (0.1)	3.0 (0.1)	.719	.000
Amotivation[c]	1.4 (0.04)	1.3 (0.04)	.080	.007
Self-efficacy[d]	3.4 (0.1)	3.3 (0.1)	.295	.003
Test anxiety[d]	2.9 (0.1)	2.9 (0.1)	.781	.000

Abbreviations: AMS, Academic Motivation Scale; LSM, least square mean; MANCOVA, multivariate analysis of covariance; MSLQ, Motivated Strategies for Learning Questionnaire; QOL, quality of life; SE, standard error.

The reporting LSMs are adjusted for year level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by a modified version of the AMS; range of scores = 1 to 5.

Measured by the MSLQ; range of scores = 1 to 5.

P values from MANCOVA.

Table 7.

Quality of life LSM scores and SEs by curriculum among Year 2 students.

Well-being LSM[a] (SE)	Year 2 traditional curriculum	Year 2 revised curriculum	P value**	Effect size – partial eta squared[b]
Personal burnout[c]	41.8 (1.7)	41.9 (1.4)	.953	.000
Physical QOL[d]	15.2 (0.2)	15.2 (0.2)	.977	.000
Psychological QOL[d]	14.2 (0.3)	13.9 (0.2)	.405	.003
Social QOL[d]	14.8 (0.3)	14.2 (0.3)	.192	.008
Environmental QOL[d]	14.8 (0.2)	14.7 (0.2)	.689	.000

Abbreviations: LSM, least square mean; MANCOVA, multivariate analysis of covariance; QOL, quality of life; SE, standard error; WHOQOL-BREF, World Health Organization Quality of Life-BREF.

Is significant at the .05 level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by the Copenhagen Burnout Inventory – Personal burnout subscale; range of scores = 0 to 100.

Measured by the WHOQOL-BREF; range of scores = 4 to 20.

P values from MANCOVA.

Table 8.

Academic motivation LSM scores and SEs by curriculum among Year 2 students.

Academic motivation LSM[a] (SE)	Year 2 traditional curriculum	Year 2 revised curriculum	P value**	Effect size – partial eta squared[b]
Intrinsic[c]	3.5 (0.1)	3.6 (0.1)	.158	.010
Extrinsic[c]	2.8 (0.1)	2.9 (0.1)	.336	.005
Amotivation[c]	1.3 (0.1)	1.3 (0.1)	.775	.000
Self-efficacy[d]	3.3 (0.1)	3.2 (0.1)	.325	.005
Test anxiety[d]	3.0 (0.1)	3.1 (0.1)	.427	.003

Abbreviations: AMS, Academic Motivation Scale; LSM, least square mean; MANCOVA, multivariate analysis of covariance; MSLQ, Motivated Strategies for Learning Questionnaire; QOL, quality of life; SE, standard error.

Is significant at the .05 level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by a modified version of the AMS; range of scores = 1 to 5.

Measured by the MSLQ; range of scores = 1 to 5.

P values from MANCOVA.

However, the MANCOVA findings for Year 4 students showed a significant difference in physical and environmental QOL scores between the 2 curricula (Table 9). Year 4 students in the revised curriculum scored higher on measures of physical and environmental QOL than students in the traditional curriculum, with a difference in mean scores of 0.6 (σM = 0.2) and 1.1 (σM = 0.2), respectively. There was also a strong trend towards improvement in psychological QOL of marginal significance with a difference in mean score of 0.6 (σM = 0.2) and a P value of .052. There were no significant associations between the type of curriculum, with burnout and social QOL.

Table 9.

Burnout and quality of life LSM scores and SEs by curriculum among Year 4 students.

Well-being LSM[a] (SE)	Year 4 traditional curriculum	Year 4 revised curriculum	P value**	Effect size – partial eta squared[b]
Personal burnout[c]	38.9 (1.4)	34.9 (1.6)	.071	.015
Physical QOL[d]	15.8 (0.2)	16.4 (0.2)	.022	.024
Psychological QOL[d]	14.7 (0.2)	15.3 (0.2)	.052	.017
Social QOL[d]	15.1 (0.3)	15.5 (0.3)	.355	.004
Environmental QOL[d]	14.8 (0.2)	16.0 (0.2)	<.001	.059

Abbreviations: LSM, least square mean; MANCOVA, multivariate analysis of covariance; QOL, quality of life; SE, standard error; WHOQOL-BREF, World Health Organization Quality of Life-BREF.

Is significant at the .05 level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by the Copenhagen Burnout Inventory – personal burnout subscale; range of scores = 0 to 100.

Measured by the WHOQOL-BREF; range of scores = 4 to 20.

P values from MANCOVA.

In relation to academic motivation, the MANCOVA results for Year 4 students showed no significant differences between the traditional and revised curricula (Table 10).

Table 10.

Academic motivation LSM scores and SEs by curriculum among Year 4 students.

Academic motivation LSM[a] (SE)	Year 4 traditional curriculum	Year 4 revised curriculum	P value**	Effect size – partial eta squared[b]
Intrinsic[c]	3.5 (0.1)	3.5 (0.1)	.685	.001
Extrinsic[c]	3.2 (0.1)	3.1 (0.1)	.292	.005
Amotivation[c]	1.5 (0.1)	1.3 (0.1)	.057	.017
Self-efficacy[d]	3.5 (0.1)	3.5 (0.1)	.658	.001
Test anxiety[d]	2.9 (0.1)	2.7 (0.1)	.102	.012

Abbreviations: AMS, Academic Motivation Scale; LSM, least square mean; MANCOVA, multivariate analysis of covariance; MSLQ, Motivated Strategies for Learning Questionnaire; QOL, quality of life; SE, standard error.

Is significant at the .05 level.

Effect sizes from partial eta squared: small = .01 to .06, medium = .06 to .138, large >.138.[32]

Measured by a modified version of the AMS; range of scores = 1 to 5.

Measured by the MSLQ; range of scores = 1 to 5.

P values from MANCOVA.

Discussion

The findings from this study showed that there were no statistically significant differences between cohorts of medical students under traditional and revised curricula in relation to academic motivation, personal burnout, and QOL. However, differences were found among students in their clinical years of training. Curriculum factors were associated with small but statistically significant differences in physical and environmental QOL. These findings suggest that changes in a curriculum may have had a differential effect on medical student QOL for those students in the latter years of medical school who are based in a clinical learning environment.

Although there have been a multitude of relatively minor revisions to the medical curriculum at the University of Auckland, this article focuses particularly on the effect of progress testing because it is often stated that assessment drives learning and because progress testing was arguably the biggest change to the curriculum.[16,34,35] It has also been established in studies of medical curricula reform that the way students are evaluated has a greater impact on their well-being than other aspects of curriculum structure.[36]

Taken as a whole, the observations that no significant differences were found in scores between curriculum cohorts is surprising, especially when considered alongside other studies that have demonstrated that changes in curricula often have an impact.[7-9,36]

A possible explanation for differences in findings between the Year 2 (preclinical) and Year 4 (clinical) students relates to the differences in the implementation of progress testing across these year levels. Progress testing for Year 2 students is used alongside traditional block testing as part of the overall evaluation of student performance. Previous findings from Van Berkel et al[14] have demonstrated the detrimental effects of assessment systems on student learning when both progress tests and block tests are used. Block testing often reinforces a reproduction and performance-oriented approach to learning, which is the antithesis of progress testing, which aims to foster a deep approach to learning.[13]

Similarly, the potential benefits of progress testing for student motivation and QOL may not have been realised for Year 2 students in this study because the use of both progress testing and block testing could have had diverging influences on the quality of study behaviour. This influence was also observed following the implementation of progress testing at Maastricht University.[11] When block tests were made formative, students changed their focus to continuous self-directed learning, but when the block test was made summative again, many students reverted to short-term memorisation despite the progress test remaining unchanged.[11]

Another explanation could be that junior medical students at this stage are still unsure about their study behaviour, which was found to be one of the main stressors among medical students in one study.[37] When this uncertainty is taken in the context of newly introduced progress testing, it may have undermined its potential benefits because of the need to adapt study behaviours. This uncertainty was likely exacerbated by limited availability of learning resources specific to progress testing to aid student learning.[37]

For the Year 4 students in this study, progress testing was implemented alongside clinical and professional skills assessments. Following the introduction of progress testing as part of the revised curriculum, there were no significant changes in academic motivation scores observed. These findings are in contrast with results from Chen et al,[35] who also conducted a study of Year 4 students in the revised medical curriculum at the University of Auckland. Their study aimed to determine the effect of progress testing on medical students’ approaches to learning and stress and found a decrease in surface approaches to learning over time but no corresponding increase in a deep approach to learning. Similarly, Blake et al[13] also observed a decline in superficial learning strategies but no increase in a deep approach to learning, following the implementation of progress testing to the curriculum at McMasters University. These findings suggest that progress testing may play a role in moving students away from extrinsic and superficial learning approaches, but not necessarily towards an intrinsic and deep approach to learning.

Cognitive evaluation theory could provide a possible explanation for this shift in learning behaviour.[38] This theory implies that based on how rewards are interpreted, extrinsic motivation can have a negative effect on intrinsic motivation. For example, if medical students perceive progress testing as a source of extrinsic motivation, their intrinsic motivation is likely to reduce; if it is perceived as providing positive information about their level of knowledge and competence, then intrinsic motivation is likely to increase.[39]

Following the introduction of the revised curriculum, a small but significant increase in physical and environmental QOL scores was observed for Year 4 students. The improvement in environmental QOL scores may be attributable to student ‘cohorting’ under the new curriculum in which students remain at one clinical campus for an academic year, therefore avoiding the need for travel between campuses that was a feature of the traditional curriculum. Another difference was the introduction of general practice teaching on campus, rather than within general practice clinics off-campus or in rural locations. These two initiatives may have alleviated problems with transportation, which is associated with improving environmental QOL among medical students.[40]

The improvement in physical QOL scores corresponds to the timing of the implementation of the personal and professional skills domain within the revised curriculum. Key topics of the health and well-being component of this domain include stress management, exercise and nutrition, healthy thinking, and improving health-seeking behaviours. It may be that initiatives such as these are having a positive impact on students’ physical health, reflected by the increasing physical QOL scores seen in this study. However, further research is needed to clarify the effectiveness of such initiatives.

Limitations

First, the response rate was 48%, and therefore, there is a risk of self-selection and nonresponse biases which was not controlled for in this study. Second, this study was conducted at a single academic institution and therefore may not be generalisable to the wider medical student population. Future research is needed across multiple institutions to determine whether the findings described in the current research are also found in other settings.

This research encompassed students in the first phase of implementation of the revised curriculum, that is, students within the revised curriculum in 2013. Further research is needed on future cohorts of medical students to determine the long-term impact of the revised curriculum and progress testing on student motivation and well-being.

Conclusions

The findings of this study demonstrate that the implementation of a revised curriculum may have had a differential effect on the QOL of Year 4 students who are based in a clinical learning environment. Medical schools should consider optimising curriculum structure and assessment methods to reduce student distress and promote motivation for learning and QOL.