Clerkship maturity: does the idea of training clinical skills work?

BACKGROUND: With the reformed curriculum "4C", the Medical Faculty of the University of Cologne has started to systematically plan practical skills training, for which Clerkship Maturity is the first step. The key guidelines along which the curriculum was development were developed by experts. This approach has now been validated.
MATERIALS AND METHODS: Both students and teachers were asked to fill in a questionnaire regarding preclinical practical skills training to confirm the concept of Clerkship Maturity. RESULTS AND DISCUSSION: The Cologne training program Clerkship Maturity can be validated empirically overall through the activities of the students awaiting the clerkship framework and through the evaluation by the medical staff providing the training. The subjective ratings of the advantages of the training by the students leave room for improvement. Apart from minor improvements to the program, the most likely solution providing sustainable results will involve an over-regional strategy for establishing skills training planned as part of the curriculum.

Introduction

Starting with winter semester 2003/2004, the University of Cologne introduced the new degree course at the medical faculty called 4C (Competence-based Curriculum Concept Cologne) [1], [2], [3]. Apart from fulfilling the goals of the Medical Licensure Act (Approbationsordnung für Ärzte [4]), the specific profile of the 4C model is defined in the so-called General Teaching Principles (Leitbild Lehre) [5], according to which Cologne graduates:

have the necessary knowledge and skills to identify important and common diseases and acutely life-threatening situations and to induce their treatment;

show behavior and attitudes that are conducive to their acceptance by patients and medical staff and to improving the standing of physicians in society;

are willing and capable to engage in independent and science-based CPD in general practice but also in a clinical discipline or a basic subject of their choice.

These goals are primarily achieved through the following new elements in the curriculum [1], [2] which are combined with traditional subject and cross-subject teaching:

interdisciplinary competence areas (88 in total, each with 5-12 hours of lectures on important and common topics of in- and out-patient treatment; competence areas 1-24 are for undergraduate students)

patients treatment in parallel to the course (“Studipat”) in which medical students are brought into contact with one patient each at a GP surgery for the first four years of study

organization of week-long block placement from the 5th semester onwards

offering electives at the end of each semester via compulsory blocks of electives

scientific qualification through two projects in which the students immerse themselves into a scientific subject

skills training in the KISS (Cologne Inter-Professional Skills Lab and Simulation Center).

KISS is a central institution of the deanery of student affairs and serves to provide training and independent practice of practical medical skills in all outcome-areas as defined by Harden [6], [7]. In contrast to the traditional process-based separation of knowledge, skills and attitudes in the German-speaking world, outcome-based trainings starts by defining outcomes (elsewhere also vaguely termed “competence-based” [8], [9]. Three different levels for curriculum development are discussed (which build on each other). The first is “Doing the right thing” which refers to the knowledge level and the ability to handle the daily clinical routine. The second is “Doing the things right”, the level of emotional, analytic and creative intelligence to use knowledge and skills appropriately. The third level focuses on the person and their future development - “The right one doing it”, the professional role in its social and societal reality. Recently more and more curricula have been developed from this point of view and as a result, there are also calls for the adaptation to accreditation criteria from the English-speaking world [10]. This is somewhat surprising, especially in view of the thin to non-existent proofs of efficacy, in particular with regard to the 3rd level “Attitudes” and “Professionalism" [11].

Based on these development criteria, and bearing in mind the requirements of the medical licensure act, clinical skills are grouped into three fields in Cologne: emergency skills, communication skills and technical skills. These elements transcend the entire degree course as a teaching and learning helix, with the educational objective of students reaching Clerkship Maturity after five semesters and the Internship Maturity (”PJ-Reife”) after ten semesters. Table 1 (Tab. 1) shows the teaching content of skills training taught up to Clerkship Maturity divided into semesters.

Table 1

KISs courses and content during the first five semesters

In a subsequent formative OSCE (Objective Structured Clinical Examination, cf Harden & Gleeson [12]) with six stations during the 5th semester, students receive feedback on their practical skills on a German-English bilingual certificate, ”Famulaturreife - Clerkship Maturity”.

The program running from the first semester of study to Clerkship Maturity was developed in 2003/2004 by an expert task force of the Advisory Board of Study Affairs of the Medical Faculty of the University of Cologne. Starting with the winter semester 2003/2004, and with broad participation from many different disciplines (anesthesia, psychosomatic medicine, transfusion medicine, internal medicine) and following the Delphi-model the “Leitbild Lehre” was developed [5]. To validate this procedure retrospectively, two questions were investigated:

What skills are required by the physicians and GPs supervising students on clerkships?

How frequently were students able to perform these skills as part of their clinical clerkship and which advantages did the Clerkship Maturity examination bring them from their point of view?

Materials and methods

To evaluate the opinions of the teaching staff, a semi-standardized, machine-readable questionnaire was designed. Some of the content is shown in Table 2 (Tab. 2)1. The questions were primarily developed on the immediate basis of the earlier KISS planning. In addition, open questions were used for additional information. The questionnaire was tested using ten randomly selected students to ensure its comprehensibility and subsequently sent to the senior physicians of the Cologne Teaching Hospitals, the GP surgeries and the senior physicians at the University Clinic of Cologne by post.

Table 2

Questionnaire for the teaching staff

To evaluate the opinion of the students a semi-standardized, machine-readable questionnaire based on the above described questionnaire was designed to reflect the reality of education. In this context the primary concern was not an evaluation of the importance of the skills training content by the students but data on the frequency of skills used (“empirical relevance”) as part of the clerkships (For Questionnaire Excerpts see Table 3 (Tab. 3))2. The questionnaire was sent to every student of the target group (7th semester).

Table 3

Questionnaire for students

Both questionnaires looked at the teaching/learning on offer in the KISs skills lab and the skills actually required during the clerkships. In addition, the form collected socio-demographic data regarding the age, sex, year of study, number of completed weeks on clerkship and additional medical training that may have been gained.

The completed questionnaires were digitized using Remark®. The data was evaluated statistically using Microsoft Excel® 2003. Mean values and standard deviation are also shown. The graphics were generated using Microsoft Excel® 2003. The open questions were analyzed using qualitative, structured content analysis after Mayring [13].

Results

Results of the teaching staff questionnaire: 36 of 80 (45%) hospital-based physicians and 77 of 140 (55%) GPs returned completed questionnaires. There was no follow-up due to anonymisation. Figure 1 (Fig. 1) shows the evaluation of the necessity/importance of different skills for Clerkship Maturity as viewed by the teaching staff. While frequently needed skills such as disinfecting hands or taking blood pressure and pulse are rated quite high, more rarely needed skills or skills deemed as premature for this stage were rated low (such as placing a stomach tube or a CVL). Significant differences between the hospital-based physicians and GPs were only seen with a few items such as taking blood samples, iv/im injections and suturing techniques (see Figure 2 (Fig. 2)). It became clear that skills desirable for the ward round where rated more highly by hospital-based physicians significantly more frequently.

Figure 1

Comparison of the frequency of the skills used by the students on their clerkships compared to their respective necessity or importance as ranked by the teaching staff (see also Table 2 and 3; the values shown are mean values and standard deviation subdivided into the different units of the skills training; (students (Studierende): 1 very frequently – 6 not at all/teaching staff (Ärzte): 1 strongly agree – 6 strongly disagree). (a) “Communication Training”, (b) ”Emergency Skills”, (c) “Technical Skills”

Figure 2

Ranking of the necessity of different skills as assessed by the hospital-based physicians (Klinik) and GPs (Praxis); mean values and standard deviation

Interpersonal skills were rated differently by hospital-based physicians and GPs too. While empathy with 64% was the most frequently mentioned criterion by GPs, only 36% of hospital-based physicians rated this skill as being important. With 20%, both groups equally mention the “ability to listen/handle patients” or “interest in the patient” as an important skill for students.

Additional suggestions for skills that should be included in the KISS offer regarding Clerkship Maturity are only given sporadically (e.g. computer skills, wound management or team skills) and thus do not allow a consistent overview.

Results of the student questionnaire: The questionnaire was sent to 140 students of which 75 were returned completed, a return rate of (54%). 24 of the 75 students who responded were male, 45 female (6 did not specify). The mean age of the students was 26 years. 59 students were attending the 7th semester, 7 the 8th semester and 3 higher semesters. 7 students listed a qualification in the medical field, including 3 paramedics (most frequent mention). Only 5 students who had returned their questionnaires were studying under the regular degree course. For this reason, there is no comparison between the regular and the model degree course responses. Figure 1 (Fig. 1) contrasts the student responses regarding the frequency of skills needed during the clerkship and their importance as rated by the teaching staff. It is clear that the frequency and necessity (importance) of some skills diverge rather drastically (especially in emergency care). While the teaching staff rate training in examination-skills as very important (1.4±0.9) whereas the students rate this as less important (2.9±1.7).

Students suggested the following in the open questions regarding possible improvements of the skills training prior to Clerkship Maturity:

better knowledge transfer (9 mentions)

more opportunities for practicing (5 mentions)

practicing with real patients (3 mentions)

ability to practicing rather than watching the lecturer (3 mentions)

smaller groups (2 mentions)

Although skills in taking patient histories are rated as relevant and frequently needed, as can be seen in Figure 1 (Fig. 1), (note that 16% of the students stated they did not have the opportunity to take medical histories in their clerkship), students rate the learning effect of the course on taking histories and communication rather moderately (2.9±1.7 to 3.1±1.4, cf question 7b in Table 3 (Tab. 3)). This is consistent with requests by the teaching staff for more training in taking patient histories (1.7±1.1, cf Question 10 in Table 2 (Tab. 2)).

The students also assessed the certificate they received following the OSCE examination during the 5th semester. The feedback on their practical skills provided by the certificate is rated rather modestly (3.4± 1.7 on a 6-point Likert-scale, 1=very helpful and 6=not helpful at all). The certificate was seen as even less helpful when it comes to applying for clerkship placements (4.7±1.8). This corresponds to statements made by the physicians of which only 6% stated to be familiar with the certificate.

Discussion

The aim of this study was to validate the planning of training on practical skills as designed by the experts for the first 5 semesters of study in the new degree course in human medicine at the University of Cologne empirically. As a validation of training goals, both teaching staff and students were questioned and the data of these two groups was compared. The students were also asked for their subjective experience gain through the KISS training.

Generally speaking, it was possible to confirm the validity of the Clerkship Maturity goal based on the data collected from the teaching staff and the students.

Skills explicitly taught in the Skills Lab for achieving Clerkship Maturity are usually indeed frequently needed in clinical clerkships. Even the handling of drips which is not explicitly part of the skills training and which is taught quasi in passing through the teaching of placing winged infusion sets. Interestingly, students are confronted with the topics ultrasound and ECG rather more frequently than suspected. It might be worthwhile considering if the program needs to be amended to reflect demand in the clerkships. On the other hand, this content is taught later on in the clinical section in more detail. This also corresponds to the opinion of the physicians who rate neither as very important skills for the clerkships (ultrasound even less so than ECGs).

These differences that were discovered between skills rated as necessary by the teaching staff on the one hand and the experienced frequency of emergency care components needed during the clerkships as reported by the students on the other hand can be easily explained. These skills extremely important to have in some situations (in some cases life-saving) but in reality these are rarely required of students on clerkships. There is a remarkable difference concerning the divergence between the frequency of applying skills for clinical examination as stated by the students and the perceived necessity as rated by the teaching staff. The teaching physicians rate this item in place 2, whereas the students rank this skill in place 5. The method of comparing the frequency of a clinical skill used by students on the one hand with a ranking of the importance by teaching staff on the other could be seen as problematic. But it is not unlikely that at this stage students (in the first or second clinical semester) only possess a limited ability to evaluate the necessity of a skill but are quite capable of reporting the frequency based on their own experiences. Allowing the frequency of skills used during clerkships to be rated by the teaching physicians is also questionable as such an evaluation might not be valid due to the workload and the fact that students are supervised by different members of staff. However, all physicians are able to rate the necessity of certain skills for the clinical activities of students on clerkships.

The differences between the rated necessity of the skills in “taking blood samples”, ”iv/im injections” and “suture techniques” between hospital-based staff and GPs is not surprising, given the different fields of activity. Nevertheless, it must be borne in mind that the practical training has to prepare students both for clerkships at a hospital and in GP’s surgery.

Although the use of the majority of skills taught in training was confirmed by the questionnaire, a small number of skills at the bottom end of the ranking suggest room for re-organization. For example catheterizing the bladder, which seems to have a rather low relevance in the clinical routine of a student on clerkship (ranked 15th of 18 items of all skills, see Figure 1 (Fig. 1)). The focus of this unit has already been changed to “working with sterile surfaces and material” using catheterizing as an example.

It is interesting that the category “empathy” is given twice as frequently in the open questions when asked about the soft skills students should have with hospital-based doctors citing this item twice as often as GPs. A possible explanation of this is the fact that the qualitative content analysis summed up related words like “humbleness” (in front of the patient) under the umbrella term “empathy”. In any case this should give food for thought further down the line, in particular regarding the teaching of “attitudes,” as they are referred to in the General Teaching Principles (see above), corresponding to the third level of the outcome-based learning (professionalism [6]); because empathy is seen as a prerequisite skill for a successful interaction between the doctor and the patient. Although it was shown in the past that, as a first step to a conscious doctor-patient interaction, the “First Semester Tutorial on Medical Psychology” (now called “Introduction to Bio-psycho-social Medicine” as an introduction to teaching communicative competences) that the thinking of students could be directed towards a multidimensional understanding of health and disease [14], [15], [16], [17]. However, this does not correspond to a complex modulation of attitudes3 as required by the General Teaching Principles [5] (see also [6]). So the comments made by the physicians in the open answers can be interpreted as a sign of gaps in the efforts to date to teach adequate doctor-patient communication. In this context the evaluation of the model degree course by the students that they had not received enough training in doctor-patient communication can also allow the conclusion that being familiar with a procedure and its risks leads to more reflective actions. Especially because the quality of the self-evaluation given by the students is questionable as a valid source, more research is needed in this area. It should not be left unmentioned that there is a possible clue that more intensive practical training can (initially) lead to unexpected uncertainty.

Because a structured curriculum for skills training in Germany is not part of standard education (and not currently required by the ÄAppO), it is not surprising that the answers to the question on the usefulness of the OSCE certificate at the end of the training phase in the 5th semester is subjectively not seems as being advantageous by the students. Based on our data we cannot clarify if this is simply down to the fact that this certificate is not sufficiently well known in hospitals and surgeries to date or if students do not even mention it in their applications. Interestingly, isolated reports of experiences by students applying for international clerkships present a different picture.

Even though there is room for improvement regarding the perceived bonus of the skills training during the pre-clinical phase with the aim of Clerkship Maturity (the usefulness of the OSCE feedback could, if necessary, be increased by skills area relevant evaluations such as hygiene skills, fine motor skills, etc.), the construct itself seems valid and reliable. The success of this initiative overall and a positive view by the students can only come into its forte through the national debate on education and the nascent National Competence-based Catalog of Learning Targets [18]. Only time will tell if the concept of Clerkship Maturity (and the Internship Maturity and Professional Development Maturity based upon it) can be transferred. The development of medical skills training which is adapted to local needs is still in its infancy in the German-speaking countries and must be further developed in many ways. The idea of Clerkship Maturity can at least provide a conceptual framework for this.

Notes

1 In addition the questionnaire for the teaching staff included questions about the general trainability of medical skills and differences in the performance of students from the regular degree course (8th ÄAppO) and the model degree course (9th ÄAppO). These questions are not discussed in this article as it apparently was not possible for the teaching staff to differentiate between students of the regular and the model degree course.

2 The student questionnaire also asked for an evaluation of their own competences in comparison to students of the regular degree course (students following the old curriculum i.e. 8th ÄAppO). The results are not discussed in this article due to the low response rate.

3 See the training goals developed in the Study Guidelines at the Medical Faculty of the University of Cologne, based on the General Teaching Principles (§1 Sect 3 „Attitudes“) [5]:

Readiness to apply medical ethical principles in practice and research

Respect and honesty towards patients and colleagues

Realistic estimation of one’s own skills, abilities and limits and readiness, to draw appropriate conclusions from this;

Readiness to take on responsibility and for accuracy.

Acknowledgement

Special thanks to Mrs Imke Wietoska, Marlen E. Sauer and Charlotte M. Schober who contributed to the creation of the questionnaire databases as part of their scientific project. I would like to sincerely thank PD Dr. Jan Matthes for his support in bringing this Master Thesis to fruition as part of the MME-D.