How does preclinical laboratory training impact physical examination skills during the first clinical year? A retrospective analysis of routinely collected objective structured clinical examination scores among the first two matriculating classes of a reformed curriculum in one Polish medical school.

OBJECTIVE: As a result of a curriculum reform launched in 2012 at our institution, preclinical training was shortened to 2 years instead of the traditional 3 years, creating additional incentives to optimise teaching methods. In accordance with the new curriculum, a semester-long preclinical module of clinical skills (CS) laboratory training takes place in the second year of study, while an introductory clinical course (ie, brief introductory clerkships) is scheduled for the Fall semester of the third year. Objective structured clinical examinations (OSCEs) are carried out at the conclusion of both the preclinical module and the introductory clinical course. Our aim was to compare the scores at physical examination stations between the first and second matriculating classes of a newly reformed curriculum on preclinical second-year OSCEs and early clinical third-year OSCEs.
DESIGN: Analysis of routinely collected data.
SETTING: One Polish medical school. PARTICIPANTS: Complete OSCE records for 462 second-year students and 445 third-year students. OUTCOME MEASURES: OSCE scores by matriculation year.
RESULTS: In comparison to the first class of the newly reformed curriculum, significantly higher (ie, better) OSCE scores were observed for those students who matriculated in 2013, a year after implementing the reformed curriculum. This finding was consistent for both second-year and third-year cohorts. Additionally, the magnitude of the improvement in median third-year OSCE scores was proportional to the corresponding advancement in preceding second-year preclinical OSCE scores for each of two different sets of physical examination tasks. In contrast, no significant difference was noted between the academic years in the ability to interpret laboratory data or ECG - tasks which had not been included in the second-year preclinical training.
CONCLUSION: Our results suggest the importance of preclinical training in a CS laboratory to improve students' competence in physical examination at the completion of introductory clinical clerkships during the first clinical year. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

We retrospectively compared objective structured clinical examination (OSCE) scores at physical examination stations between the first and second matriculating classes of a reformed undergraduate curriculum on preclinical 462 second-year OSCEs and 445 early clinical third-year OSCEs at the completion of introductory clinical clerkships.

Stations in both OSCEs were highly standardised and identical checklists were used throughout the analysed period.

That we analysed OSCE records from only one medical school limits the generalisability of our findings.

Extension of the observation period into later clinical years and a longitudinal assessment of individual students’ performance are lacking.

Introduction

Recent decades have witnessed a well-recognised international decline in physical examination skills among medical students and residents.1–6 This has largely been ascribed to an increasing reliance on advanced imaging technologies and laboratory markers. Notably, the inability to properly perform and interpret a physical examination can expose the patient to redundant and costly procedures and, more importantly, may lead to a missed or delayed diagnosis with potential deadly consequences.3 Therefore, in order to prevent the physical examination from becoming merely a lost art, a remedial intervention is necessary. This intervention should be planned early, preferentially already at undergraduate level,7 keeping in mind that junior doctors—engaged in administrative tasks and paperwork—spend three to five times more time in front of a computer screen than in direct contact with patients.8 9

‘To resuscitate clinical skills among clinicians’, Ramani7 proposed—among ‘Twelve tips for excellent physical examination teaching’—integration of simulation with bedside learning as well as systematic assessment of clinical skills (CS), the latter elegantly summarised in a lapidary phrase ‘assessment drives curriculum’. Objective structured clinical examinations (OSCEs) are a recognised assessment tool in medical education. OSCEs are increasingly valued for their ability to predict students’ future performance in the clinical setting.10–15 The approach of using OSCEs has practical implications, providing a basis for the optimisation of clinical education and offering insight into remedial strategies to improve students’ poor clinical performance.10 16 17

Of note, although scores on OSCEs done in the second and third years of study were related to performance on the US Medical Licensing Examination (USMLE) Step 2 CS component,14 this association was not strong, and the OSCE scores in years 2 and 3 were only weakly inter-related.12 Additionally, USMLE Step 2 CS scores and second-year OSCE scores correlated moderately with each other, but this relationship lost significance in a multivariate analysis.11

On the other hand, of the OSCE components taken at the end of the first clinical year (year 3), skills in physical examination and data interpretation exhibited the highest ability to predict students’ performance in five subsequent clinical examinations during the fourth and fifth years of study.10 Scores on an OSCE in the first clinical year have a unique property: they can be linked to future clinical competence and may be used to estimate the contributions of preclinical training in a CS laboratory and subsequent bedside teaching to early clinical competence. Surprisingly, there is limited data available comparing second-year and third-year OSCE scores between various academic years. Chima and Dallaghan15 recently compared OSCE scores for graduates of 2013 and 2014 classes and described a discordance between class-to-class variation in scores obtained during second-year preclinical OSCEs and OSCEs completed at the conclusion of the third-year internal medicine clerkship.

In 2012, a new curriculum was launched at our institution, where the preclinical course is scheduled for a period of 2 years, instead of the traditional 3 years. Our final year of the medical curriculum (year 6) is dedicated to internships in teaching hospitals during which final-year students assist junior doctors by performing similar tasks under direct clinical supervision. The new curriculum includes a preclinical module of CS laboratory training in year 2, supplemented with bedside teaching of basic CS in the Fall semester of year 3, as an introduction to further clinical exposure. The curricular reform has created an additional incentive to make the best possible use of existing educational resources within a limited timeframe. To reach our ultimate goal of maximising early clinical proficiency, continuous optimisation of teaching methods based on an ongoing assessment of the effects of our curriculum reform is necessary.

Our aim was to compare the scores obtained by medical students at physical examination stations between the first and second matriculating classes of the reformed curriculum on preclinical second-year OSCEs and third-year OSCEs at the completion of an introductory clinical course. We hypothesised that differences in the performance between classes on preclinical OSCEs may be reflected in the results of early clinical OSCEs.

Methods

Characteristics of the redesigned curriculum

Within the new curriculum, a 30 hours preclinical module of training in a CS laboratory takes place in the Department of Medical Education of our university in either the Fall or Spring semester of year 2 (15 weeks; 2 hours per week) (table 1). This module includes practical exercises with simulated patients and manikin-based learning. In the Fall semester of year 3, students enter a 12-week module in bedside teaching of basic CS (ie, mini-clerkships in the departments of Internal Medicine, Surgery, Paediatrics and Obstetrics/Gynaecology for 3 weeks each) as an introduction to the core clinical rotations in years 3–6 (table 1).

Table 1

Traditional and reformed medical curriculum at our university

Type of curriculum	Year of study
	1	2	3	4	5	6
Previous curriculum
Preclinical courses	x	x	x
Clinical skills laboratory training			x
Introductory clinical course			x
Core clinical clerkships				x	x	x
Reformed curriculum
Preclinical courses	x	x
Clinical skills laboratory training		x
Introductory clinical course			x
Core clinical clerkships			x	x	x
Internship						x

An OSCE was carried out at the conclusion of both teaching modules, starting from the academic year 2013–2014 and onwards. Each OSCE was composed of several stations covering history taking, physical examination and students’ skills in cardiac/pulmonary auscultation. Additionally, the third-year OSCE included stations assessing students’ ability to interpret laboratory data and a typical ECG, as well as two surgical stations (assessing suturing skills). Our highly-standardised physical examination stations did not differ between the second-year and third-year OSCEs, and they remained unchanged throughout the analysed period, including all tasks randomly chosen from a set of 19 (stations set I) and those from a different set of 16 tasks (stations set II).

Data analysis

We analysed previously collected examination data from second-year OSCEs (February/June 2014 and February/June 2015 examination sessions) and third-year OSCEs (February 2015 and February 2016 examination sessions). As a data source, we used examination records stored in the Department of Medical Education at our university using existing institutional protocols. For the purpose of our analysis, fully anonymised data sets were used in order to ensure personal data protection. Because data sets were anonymous, we were not able to longitudinally estimate individual student performance on the second-year and third-year OSCEs. An individual OSCE score for each physical examination station and data station was calculated from OSCE grades as a relative value, with the reference being an optimal result for the given task, assumed to be 100%.

The accordance of OSCE scores with a normal distribution was estimated by means of the Shapiro-Wilk test. Owing to the non-normal distribution, the data were presented as medians and IQRs. Then, OSCE scores were compared separately between the classes who matriculated in 2012 and 2013 for preclinical second-year OSCEs and third-year early clinical OSCEs, respectively. Between-class differences in OSCE scores were assessed by the Mann-Whitney U test. In order to deal with missing data, the analysis was first performed for OSCE records with complete data points and then repeated including also incomplete OSCE records with at least one available data point. A p value below 0.05 was considered significant. The analysis was performed using STATISTICA (data analysis software system), V.12 (StatSoft, Tulsa, Oklahoma, USA).

Results

Out of potentially eligible 513 second-year and 466 third-year OSCE records, we had excluded 51 and 21 incomplete records, respectively, due to missing data. OSCE records with complete data points were available for 462 second-year students and 445 third-year students from the first two matriculating classes of the reformed curriculum, for a total of 907 OSCEs that entered our final analysis.

Compared with the first class of the new curriculum who matriculated in 2012, higher (ie, better) OSCE scores in physical examination skills were observed for students who matriculated 1 year later in 2013. Improved OSCE scores were noted during both the second year of study (February/June 2015 vs February/June 2014 examination sessions) and the third year (February 2016 vs February 2015 examination sessions) (table 2).

Table 2

Comparison of OSCE scores (%) between the classes who matriculated into the new curriculum in 2012 and 2013

Year of study	Year of matriculation		Between-class comparison of OSCE scores, p value*
	2012	2013
Year 2—preclinical OSCE	February/June 2014	February/June 2015
Physical examination (stations set I)	86 (67–100)	89 (78–100)	0.007
Physical examination (stations set II)	82 (60–92)	90 (83–100)	<0.001
Cardiac/pulmonary auscultation	100 (75–100)	100 (75–100)	0.5
Year 3—early clinical OSCE	February 2015	February 2016
Physical examination (stations set I)	82 (67–90)	86 (78–100)	<0.001
Physical examination (stations set II)	81 (67–100)	90 (83–100)	<0.001
ECG interpretation (basics)	100 (80–100)	100 (80–100)	0.7
Interpretation of laboratory data	88 (75–100)	100 (75–100)	0.8
Cardiac auscultation	80 (60–80)	80 (60–80)	0.6
Pulmonary auscultation	60 (60–80)	100 (80–100)	<0.001

*Data obtained by Mann-Whitney U test.

OSCE scores (%) are shown as median and IQR.

OSCE, objective structured clinical examination.

Additionally, the magnitude of the improvement in median third-year OSCE scores was proportional to the corresponding changes between academic years in the preceding second-year preclinical OSCE for each of two different sets of physical examination tasks (stations set I: 4% and 3%; stations set II: 9% and 8%; for third-year OSCEs and second-year OSCEs, respectively) (table 2). In contrast, no significant changes between academic years were found for the ability to interpret laboratory data or ECGs (ie, tasks which had not been included in preclinical teaching during the second year of the curriculum) (table 2). In regards to auscultation skills, the only significant between-class change was an improved competence in pulmonary auscultation for the second matriculating class of the new curriculum during their third year of study (table 2).

The results were substantially unchanged either on adjustment for different timings of second-year OSCEs in the academic year (ie, separate analyses for OSCEs scheduled after the Fall or Spring semester) or after inclusion of incomplete OSCE records with one or more available data points.

Discussion

Our most salient finding was that OSCE scores at physical examination stations were higher for students matriculating into the newly reformed curriculum in 2013 compared with those matriculating in 2012. A proportional improvement was noticed between 2012 and 2013 cohorts in both preclinical second-year OSCE scores and early clinical third-year OSCE scores. Additionally, the magnitude of the improvements in physical examination competence between classes during the early clinical year correlated with the differences in scores attained by students in 2012 and 2013 matriculating classes during the preclinical second-year OSCE for each of two different sets of physical examination tasks.

The observed association differs from the results of a recent study reporting significantly higher internal medicine clerkship OSCE scores in the first clinical year (year 3 of study) despite a trend of lower second-year preclinical OSCE scores for graduates of the class of 2014 compared with the class of 2013.15 Additionally, the authors observed no association between student performance on preclinical OSCEs and OSCEs completed after an internal medicine clerkship.15 Admittedly, similar to the previously mentioned report,15 it would be appropriate to estimate the effects of preclinical OSCE scores on the results of early clinical OSCEs. However, since our data sets were anonymised, we were not able to analyse individual students’ performance; therefore, a longitudinal assessment of student performance was not possible.

Our observation has several potential explanations. First, inconsistencies in OSCE administration and grading between academic years could account for the observed differences in OSCE scores, as suggested previously.15 However, stations in both OSCEs were highly standardised and identical checklists were used throughout the analysed period. Second, the OSCEs were monitored and supervised by different teams of faculty members affiliated with either the Department of Medical Education (second-year OSCE) or the departments supervising the introductory clinical courses (third-year OSCE). Moreover, at equivalent OSCEs, the performance of students matriculating in 2012 and 2013 was assessed by virtually the same teams of examiners, including only lecturers—previously trained by senior teachers in OSCE planning, administration and grading—with a wide and proven experience in the scoring of OSCE stations. Third, even when considering the possibility of non-uniform grading across the study period, hypothetical year-to-year differences in OSCE scores might be expected for all OSCE components. Nevertheless, we observed a significant year-to-year variation exclusively in OSCE scores reflecting physical examination skills. Finally, the previously described influence of the timing of clinical clerkships in the academic year15 could be excluded because the introductory clinical course was scheduled in the Fall semester for both 2012 and 2013 matriculating class.

In conclusion, the association of year-to-year improvements in scores at physical examination stations in preclinical OSCEs and OSCEs in the middle of the first clinical year is suggestive of the importance of preclinical training in a CS laboratory to improve competence in basic physical examination at the completion of early bedside teaching. A preclinical laboratory teaching module appears to be easier to standardise and more responsive to quality-oriented interventions in comparison to the traditional clinical bedside teaching. Additionally, as to second-year students, it was suggested that an incorporation of formal clinical instruction to their training could be easier compared with those who have already begun clinical clerkships and elective rotations.3 Moreover, the effectiveness of clinical bedside teaching is known to depend on multiple factors, and studies on the relationship between clinical exposure and early clinical OSCE scores have brought conflicting results.18–21 Of note, Martin et al 19 reported no correlation between self-reported clinical exposure to patients and students’ performance on an OSCE taken at the end of the first clinical year. Importantly, Kim and Myung21 described a high variation in the number of patients for whom a medical history was taken or physical examination was performed during clerkships, which is probably indicative of a limited efficacy of bedside teaching in some departments. This observation could also be responsible for the differences between 2012 and 2013 matriculating classes in cardiac and pulmonary auscultation skills after the introductory clinical course—probably due to interclinic variation in the characteristics of patients hospitalised in individual internal medicine departments.

Whether the observed trends will be maintained in later clinical years, requires further investigations with a prolonged follow-up. Additionally, that we analysed OSCE records from only one medical school, poses another limitation to the interpretation and generalisability of our results. Nevertheless, even if seems premature to draw any far-going conclusions for the time being, our findings might have practical implications before future data become available. The results of this assessment can serve as a stimulus for further improvements in teaching physical examination skills, OSCE planning and implementing a remedial intervention for low-scoring students. Our curriculum reform offers a promising and realistic opportunity to put these plans into practice as the new curriculum promotes a continuous optimisation of preclinical and clinical education based on an ongoing assessment of teaching effects. Improved undergraduate education is the starting point to interrupt a vicious cycle of undervaluation and underuse of the physical examination in clinical decision-making with regard to real-world patients.