Trajectories of learning approaches during a full medical curriculum: impact on clinical learning outcomes.

BACKGROUND: No consensus exists on whether medical students develop towards more deep (DA) or surface learning approaches (SA) during medical training and how this impacts learning outcomes. We investigated whether subgroups with different trajectories of learning approaches in a medical students' population show different long-term learning outcomes.
METHODS: Person-oriented growth curve analyses on a prospective cohort of 269 medical students (Mage=21years, 59 % females) traced subgroups according to their longitudinal DA/SA profile across academic years 1, 2, 3 and 5. Post-hoc analyses tested differences in academic performance between subgroups throughout the 6-year curriculum until the national high-stakes licensing exam certifying the undergraduate medical training.
RESULTS: Two longitudinal trajectories emerged: surface-oriented (n = 157; 58 %), with higher and increasing levels of SA and lower and decreasing levels of DA; and deep-oriented (n = 112; 42 %), with lower and stable levels of SA and higher but slightly decreasing levels of DA. Post hoc analyses showed that from the beginning of clinical training, deep-oriented students diverged towards better learning outcomes in comparison with surface-oriented students.
CONCLUSIONS: Medical students follow different trajectories of learning approaches during a 6-year medical curriculum. Deep-oriented students are likely to achieve better clinical learning outcomes than surface-oriented students.

Background

Seminal work by Marton and Säljö [1] and Biggs [2] described learning approaches as motivations and strategies students adopt to gain new knowledge, and distinguished between deep (DA) and surface learning approaches (SA). DA underlie intrinsic motivation and interest during learning, understanding the meaning of what is learnt, relating information to prior knowledge, looking for underlying principles and critically evaluating knowledge and conclusions drawn. SA relate to instrumental motivation for learning, reproducing content, memorising and rote learning in order to pass the tests. Learning approaches are central during higher education since they impact several learning outcomes [3-5]. In particular, they are determinant for academic achievement even after taking into account time spent in learning, gender and intellectual ability [3-7].

Training medical students to become competent doctors require them to develop, among others, competencies in reasoning skills, clinical problem solving and critical analysis [8], all skills strongly related to the use of DA. Indeed, learning approaches are significant predictors of academic performance for medical learners [9] and the amount of clinical knowledge acquired during clinical training as well as the success in final examination are positively associated with the use of DA during medical studies [10, 11]. Even more, the way students use learning approaches during medical school can predict their approaches to work when they will become practicing doctors [12]. It is thus crucial that medical schools create the conditions to promote and sustain deep learning during medical training. Therefore, pedagogical interventions need to be informed by empirical evidence from observational studies in order to maximise chances of producing tangible results. In addition, it remains unclear whether and how learning approaches change over time and impact students’ clinical learning outcomes.

How to encourage students to learn in-depth has been the subject of a myriad of studies and the emerging picture is contrasted and even contradictory [13, 14]. Explanations for these findings rely on the complexity of factors impacting learning approaches. On one hand, students can modulate their use of SA and DA depending on the educational context in which they are learning [14, 15]. However, the efforts put into promoting the use of DA through learner-centered teaching methods have been rather unsuccessful [14, 16]. Even worse, students seem more readily to shift from DA to SA, than the opposite [14, 16, 17]. On the other hand, specific student-related factors seem to influence how they use learning approaches and how they modulate their strategies in response to a given educational environment. Students might have personal predispositions towards using either SA or DA [18] and their initial use of learning approaches when entering university could be more predictive of the subsequent use of learning approaches than the educational context itself [19-21]. Moreover, some students might be stable, whereas others in the same learning context might change their use of learning approaches, revealing that even the most carefully designed learner-centered teaching environment may induce or prevent the use of DA [22]. Finally it appears that how students perceive their educational environment is even more influential than the context itself, thus questioning the efficiency of learner-centered environments [13, 23].

As it is essential to follow and support students’ strategies to good learning practices, authors recently called for a person-centered approach with a longitudinal perspective [20]. Indeed, major limitations of previous studies include measurements with only short-time intervals and analyses at the group rather than at the individual level [17]. To our best knowledge, no prospective long-term cohort study has evaluated how learning approaches evolve and influence the learning outcomes along an entire medical curriculum. Accordingly, the current study combined variable- and person-centered longitudinal analyses [24] to address two unanswered research questions: (1) Do subgroups with different trajectories of learning approaches exist in a given population of medical students? (2) If yes, do these subgroups have different learning outcomes?

Based on previous research, we hypothesised the presence of two groups of medical students with different trajectories of learning approaches and learning outcomes along the curriculum. More specifically, students more oriented in using DA (deep-oriented students) at the onset of their studies would keep or strengthen this predominant learning approach whereas students initially more oriented in using SA (surface-oriented students) would tend to reinforce their use of SA during medical studies. Deep-oriented students would then report better learning outcomes than surface-oriented students.

Method

Sample and procedures

Participants were recruited among medical students enrolled during their first academic year at the University of Geneva, Switzerland, in 2011, 2012 and 2013. Data on learning approaches used in this study derived from students’ self-reported answers at an annual survey and student performance from examinations scores recorded by the Geneva Faculty of Medicine from Year 1 to 5 and by the Institute for Medical Education for the federal licensing exam (FLE) in Year 6. Participants provided their student ID in order to be matched throughout the duration of the study and to merge their self-reported answers with their examinations scores. Data were anonymised as student IDs were managed by a technical administrator. For the current analyses, we selected students that delivered at least three self-reports across four data collections: first/pre-selection study year (Year 1), beginning of pre-clinical training (Year 2), end of pre-clinical training (Year 3), and end of clinical training (Year 5). In total, 269 students (mean age at Year 1 = 20.85 years, SD = 1.92, range = 18–38, 59 % females) were included, of which 197 (73 %) participated in all four data collections. We reported further details on sample selection and data acquisition procedures in the Additional file 1. Informed consent was signed by the students prior to the participation to the study.

Description of the educational context

Table 1 summarises the teaching and evaluation formats for each study year as well as the learning outcomes measures in this study. A full description is available in the Additional file 1. Briefly the pre-graduate medical curriculum has a duration of six years and is designed to provide a student-centred and integrated approach to students’ acquisition of theoretical knowledge and clinical competencies. The first study year is taught by lectures and assessed by a high-stake computer-based exam (CBE) constituted of multiple-choice-questions (MCQ) (factual knowledge). The second and third preclinical years are taught mainly by problem-based learning, assessed by a CBE and an oral examinations consisting of problem-solving questions (applied medical knowledge) in parallel with a clinical skills training, assessed by Objective-Structured-Clinical-Examination (OSCE). The fourth and fifth years are devoted to clinical training, through Problem-Solving activities and rotations in clinical clerkships, assessed by a CBE constituted of clinical vignettes and structured oral examination (applied clinical knowledge) and OSCEs (clinical skills). The sixth elective year finishes with the Swiss FLE comprising a high-stake written exam constituted of MCQs and a clinical skills part made of OSCEs.

Table 1

Description of the teaching and evaluation formats by study year

Study Year–Teaching unit	Main teaching format	Evaluation format	Measured learning outcomes
Year 1-Modules A/B	Integrated lectures	High-stakes CBE (MCQ)	Factual medical knowledge
Year 2-Modules 1/2	Small group PBL tutorials Clinical skills training	CBE (vignettes) Oral examination	Applied medical knowledge
Year 3-Modules 3/4	Small group PBL tutorials Clinical skills training	CBE (vignettes) Oral examination, OSCEs	Applied medical knowledge Clinical skills
Year 4-ICRU	Small group problem solving	CBE (clinical vignettes)	Applied clinical knowledge
Year 4-LCE-I	Rotations through clerkships	CBE (clinical vignettes) Oral examination OSCEs	Applied clinical knowledge Clinical skills
Year 5-LCE-I	Rotations through clerkships	CBE (clinical vignettes) Oral examination OSCEs	Applied clinical knowledge Clinical skills
Year 5-LCE-II	Rotations through clerkships	Oral examination	Applied clinical knowledge
Year 6-Elective year	Elective clerkships	High-stakes written exam (clinical vignettes) High-stakes OSCEs	Applied clinical knowledge Clinical skills

Notes: ICRU Introduction to clinical reasoning unit, LCE-I Learning in the clinical environment – first part, LCE-II Learning in the clinical environment – second part, CBE computer-based exam, MCQ multiple-choice questions, PBL problem-based learning, OSCE objective structured clinical examination

Measures

Learning approaches

A French version of the Revised two-factor Study Process Questionnaire (R2-SPQ) [25] was created by translation and back-translation of two independent reviewers to measure students’ learning approaches across four assessments (i.e. Years 1, 2, 3 and 5) [26]. The R2-SPQ consists of 20 items scored on a 5-point Likert scale (from 1 = this item is never or only rarely true of me to 5 = this item is always or almost always true of me) with 10 items measuring DA and 10 items measuring SA. Examples of items tapping into DA and SA dimensions respectively are “I test myself on important topics until I understand them completely”, and “I find the best way to pass examinations is to try to remember answers to likely questions”. Total scores of DA and SA were calculated by summing up the scores of all the questions grouped under each dimension. Reliability results for DA (Cronbach’s α coefficients ranging from 0.74 to 0.84 across assessments) and SA (α coefficients ranging from 0.64 to 0.78) were aligned with previous studies [25, 27].

Learning outcomes

Learning outcomes were estimated from the exam scores for eight consecutive parts of the curriculum as described in Table 1. Exam scores were standardized so to have a mean of 0 and a standard deviation of 1, and were then summed and averaged by exam session to obtain repeated performance scores from pre-clinical throughout clinical training years [28]. Bivariate correlations between different exams within identical sessions ranged from 0.10 to 0.62 and were all significant at p < 0.01. Bivariate correlations between standardized exam scores across sessions ranged from 0.18 to 0.68 and were all significant at p < 0.01. These results support the analytical approach to combine scores from these different examinations into unique performance scores across academic years.

Covariates

Reading from previous research [29, 30], the following variables were adopted as covariates in the analyses: age (in years), gender (0 = males, 1 = females), and repeater1 status (0 = non repeater, 1 = repeater). We also controlled for differences according to enrolment year (1 = 2011, 2 = 2012, 3 = 2013).

Data analyses

An extended description of the analytical procedure adopted in the current study is presented in the Additional file 1. Preliminary analyses assessed accuracy of data entry, missing data and normality. Longitudinal analyses were divided into four steps. (1) First, latent growth modeling (LGM) [31], estimated the average initial level (henceforth intercept) and rate of change (slope) of DA and SA across assessments (from Year 2 to 5). (2) Second, to examine whether different trajectories of learning emerged from the total sample, group-based trajectory modeling (GBTM) [32] analysed the heterogeneity in the development of both DA and SA. GBTM is an exploratory approach assuming that the observed population is composed by a mixture of underlying trajectory subgroups. When testing GBTM models, different subgroup solutions are specified. The best-fitting model is then selected based on the goodness-of-fit indexes and theoretical considerations. (3) Third, after having identified different trajectories of DA and SA, Poisson regression with robust standard estimation method determined whether any covariate explained the likelihood of trajectory membership. (4) Finally, repeated measures analysis of covariance was conducted to assess the effect of time and trajectory membership on standardized exam scores across study years controlling for age, gender, enrolment year and repeater status. Stata 15 (StataCorp. 2015. Stata Statistical Software: Release 15. College Station, TX: StataCorp LP) was used for data analyses.

Results

Preliminary analyses

Table 2 reports descriptive statistics for all variables included in the analyses. The initial check of data for careless responding patterns across R2-SPQ’s items [33] revealed one case in Year 1 and two cases in Year 5 who were therefore coded as missing. Overall missing rates on learning approaches scores were 19.3 % in Year 1, 11.9 % in Year 2, 5.6 % in Year 3 and 11.2 % in Year 5. Little’s test was not significant (χ2 = 41.20, df = 40, p = 0.418), confirming that data were missing completely at random. Full information on maximum likelihood estimation method was thus used in the LGM and GBTM analyses. List-wise deletion method was adopted in the Poisson regression model and repeated measures analysis of covariance. This decision was further supported by the absence of multivariate outliers [34] and by absolute values of skewness and kurtosis below 1 and 6 respectively, thus suggesting that learning approaches and standardized exam scores were reasonably normally distributed [35].

Table 2

Descriptive statistics for all variables included in the analyses (N = 269). Standardized exam scores are reported

Variable	Range	Mean	SD
Age at Year 1	18–38	20.85	1.92
Female (n = 158, 59 %)
Repeater (n = 141, 53 %)
DA
Year 1	17–50	33.46	5.69
Year 2	16–49	34.21	5.46
Year 3	17–46	32.62	6.11
Year 5	11–47	29.43	6.59
SA
Year 1	10–37	22.24	4.78
Year 2	11–40	21.55	5.10
Year 3	11–41	23.65	5.84
Year 5	11–39	23.02	5.91
Exam sessionsa
Year 1-Modules A/B	0–100	62.17	12.74
Year 2-Modules 1/2	0–100	67.29	14.98
Year 3-Modules 3/4	0–100	50.93	17.94
Year 4-ICRU	0–100	60.54	17.70
Year 4-LCE-I	0–100	53.72	18.60
Year 5-LCE-I	0–100	61.98	16.13
Year 5-LCE-II	0–100	49.22	18.05
Year 6-Elective year	0–100	55.91	18.89

Notes. DA Deep learning approaches, SA Surface learning approaches, ICRU Introduction to clinical reasoning unit, LCE-I Learning in the clinical environment – first part, LCE-II Learning in the clinical environment – second part

aStandardized performance scores per exam session were expressed on a scale from 0 to 100 to ease interpretation

Identifying trajectories of learning approaches

Examination of fit indexes resulting from the LGM analysis (Table A) confirmed that DA declined (Slope mean = -1.10, SE = 0.23, p < 0.001) while SA increased across assessments (Slope mean = 0.66, SE = 0.22, p = 0.002). GBTM analyses indicated that the two-group solution was the best one according to differences in fit indexes between different solutions, interpretability and parsimonious considerations (Table B). The trajectories for both groups are illustrated in Fig. 1. Mean differences in learning approaches between trajectory groups were all significant at p < 0.001 at each assessment year. The deep-oriented group included students (n = 112; 42 %) predominantly using DA and little SA. The surface-oriented group included students (n = 157; 58 %) using more SA and less DA than their deep-oriented peers. According to post hoc analyses (Table C), deep-oriented students were fairly stable in their use of DA and SA but reported a significant decrease in DA between Year 3 and 5 and a small significant increase in SA between Years 2 and 3. Conversely, surface-oriented students reported a steady decline in DA from Year 2 to Year 5 and a significant increment in SA between Year 2 and 3, then stable between Years 3 and 5. Reading from Poisson regression analyses, trajectory subgroup membership was not explained by age, gender, repeater status or enrolment year.

Fig. 1

Expected scores in deep and surface learning approaches across assessments by trajectory groups with 95 % confidence intervals

Learning outcomes by trajectory groups

Repeated measures analysis of covariance tested the effect of time and trajectory membership on standardized exam scores across study years while controlling for age, gender, enrolment year and repeater status. Trajectory membership had a significant direct effect on performance [F(1, 1760) = 6.88, p = 0.009] and a significant interaction effect with time [F(7, 1760) = 2.53, p = 0.014], meaning that learning outcomes differed across time in a different way according to trajectory groups. In order to ease the interpretation of these results, linear predictions for student standardized performance scores were graphically plotted for both groups across study years (Fig. 2). Examinations of contrasts between the two trajectory groups on standardized performance scores along study years (Table D) showed that deep-oriented students diverged toward better learning outcomes in comparison with their surface-oriented peers from Year 4 onwards. Unconditional of age, gender, enrolment year and repeater status, deep-oriented students were 21 % more likely to report higher performance scores than their surface-oriented peers. During Years 5 and 6, the odds for a better performance outcome were again in favour of deep-oriented students by more than 30 %. Further post hoc analyses by the means of independent sample t-tests with Bonferroni adjustments showed that performance scores were significantly different at p < 0.005 between trajectory sub-groups in Year 5 – LCE-II and Year 6 (Table D).

Fig. 2

Expected standardized exam scores across years by trajectory groups with 95 % confidence intervals. Notes. ICRU: Introduction to clinical reasoning unit. LCE-I: Learning in the clinical environment – first part. LCE-II: Learning in the clinical environment – second part

Discussion

In this study, we modelized subgroups of medical students who evolve differently in their use of learning approaches along the curriculum, and – most importantly – who have different long-term clinical learning outcomes. To our best knowledge, evidencing these trajectories as well as their outcomes is a novel finding. In addition, whereas a lot of work has been done on other samples of university students, essentially concerning their academic performance, our results extend previous findings by showing that learning approaches also have an impact on the acquisition of clinical knowledge and skills. In our student sample, we found two groups with different trajectories of learning approaches. Students starting higher in SA and lower in DA at the onset of their studies tended to increase their use of SA across study years, while decreasing in DA. On the other hand, students starting higher in DA and lower in SA remained more stable on their use of DA and SA across study years, but still reported a slight decrease in DA during clinical years. We identified these two trajectory groups as surface-oriented and deep-oriented students, respectively. Even if medical students are naturally oriented to use DA at the beginning of their studies [36], more than half of our sample tended to use SA and even slightly reinforced this tendency especially during pre-clinical training. Thus, like other studies adopting person-oriented analytical approaches in higher education research settings [21, 37, 38], we observed in this prospective long-term cohort of medical students, subgroups exhibiting different initial profiles in learning approaches (i.e. their spontaneous use of DA vs. SA at the onset of studies) and specific trajectories. Whereas Balasooriya et al. [22] described contradictory trajectories of learning approaches in subgroups of students after a single course unit, the trajectories observed in this study remained fairly parallel during the full medical curriculum.

Thus, if deep learning approaches are important predictors of academic performance, the essential question is how to promote students’ deep learning. An important amount of studies leads to contradictory results about the potential effect of the teaching context [23]. On the other side, several authors suggested that students could have a predisposition towards using either surface or deep learning, or that they might adapt or not their learning approaches depending on their personal profiles [18-21]. Therefore, evidence suggests that depending on various personal attributes, in particular their preferred learning approaches, students might not react the same to a given learning environment [13]. Our results are consistent with this hypothesis and confirm that students’ use of learning approaches at the onset of their studies is more predictive of the subsequent use of learning approaches than the educational context itself [19, 21]. Students constitute in fact a heterogeneous population whose individuals react and adapt differently in identical educational environments. This corroborates previous reports showing that even the most carefully designed learner-centered teaching environment might induce or not the use of DA [22]. An underlying reason for this heterogeneity might be connected to how students perceive their educational context [14, 39, 40]. It has been shown that surface-oriented medical students struggle to recognize the relevance of the educational context with regard to their future practice, have pre-existing beliefs that learning is linear, and feel confused with learner-centred methods [41]. On the other hand, the environment provided by medical studies may be stressful since it is characterized by a high workload, a competitive climate, complexity and uncertainty of knowledge, and high stakes assessment requirements. Taken together, these conditions could be experienced very differently among students, and create for the more fragile and less resilient of them, stress and anxiety pushing them towards using SA and impacting negatively their performance [42-44].

This study also demonstrates that deep-oriented students have better long-term clinical learning outcomes, thus confirming and extending at a longitudinal cohort level previous research [10, 11, 45]. These findings suggest that “one size does not fit all” and that medical educators should tailor the educational context to address this heterogeneous population of students. Nevertheless, several educational strategies might help surface-oriented students to reinforce the use of deep approaches. First, enhancing internal motivation [46] could potentially increase the initial use of DA [7, 29, 47]. Second, working on the beliefs and preconceptions that students have on learning may improve their perception of the educational context and foster the use of DA [41, 48]. Third, students’ social identity has been associated with how they perceive their academic environment and its relevance for their future practice, and is linked to an increased use of DA and a higher academic performance [49]. Finally, helping students to cope with stress will also promote use of DA [43, 44]. Identifying some of the factors that positively impact on the adoption of DA might contribute to define targeted strategies directed to students in need for guidance on their approach to learning. Furthermore, setting goals for reflective learning and writing with appropriate feedback have proved useful for promoting learners’ self-regulation strategies [50] and strengthening deep learning [51].

Strengths, limitations and suggestions for future research

The main strength of this study is its person-oriented longitudinal design in which students have been followed during an entire curriculum and surveyed at different time points. Whereas the methodology in itself is not novel in the field of higher education research, to the best of our knowledge, no previous study has combined variable- and person-centered longitudinal approaches in the field of the medical education for research questions applied to learning approaches and academic performance. It certainly has limitations. First of all, these findings relate to a single institution and may not generalize to other contexts, even more since the number of students of this cohort is relatively modest. Second, although beyond the scope of the current study, several determinants of students’ learning profiles were not included in the current analyses although they may contribute to explain both initial levels and longitudinal changes in DA and SA. Future studies may thus extend the person-oriented model adopted here by investigating the individual characteristics that are inherent to a given trajectory. Moreover, as reminded in our analyses, such analytical approach is exploratory by definition [52]. More studies are thus needed to replicate and eventually validate the two trajectories of learning approaches observed here across different contexts and samples of medical students. Finally, future research may evaluate whether and how identify surface-oriented students as early as possible so to accompany and help them coping with their learning difficulties. In this perspective, making students aware of their predispositions towards surface learning could represent a first step towards changing. In addition, interventions to reinforce coping, motivation, beliefs and professional identity, are also able to promote DA. In our own practice as educators, we identified that favouring a reflective practice could be a way of addressing these issues.

Conclusions

Despite the above limitations, this study contributes to the understanding of how medical students adopt learning approaches throughout their studies and how this impacts their learning outcomes. Our results suggest that students’ longitudinal trajectories of deep and surface learning approaches reflect their initial levels and that this is significantly associated with their clinical learning outcomes. Next steps should focus on testing the effectiveness of follow-up programs intended for medical students in need for guidance, to provide them with the best possible learning experience and equipping them to meet the challenges of the medical practice.

Additional file 1: Table A. Comparison of fitted latent growth models for deep and surface learning approaches (N = 269). Table B. Group frequencies and fit indices based on estimated posterior probabilities for group-based trajectory modeling analysis of deep and surface learning approaches with different numbers of latent trajectory groups (N = 269). Table C. Comparisons of average scores in DA and SA between assessments and by trajectory groups based on paired sample t-tests. Table D. Differences in performance scores at each exam session between trajectory subgroups and contrasts between trajectory subgroups on standardized performance scores along exam sessions. Values are mean (standard deviations) unless otherwise stated (N = 269)