Effects of clinical practice focusing on level-3 OSCE items.

[Purpose] This study examined the effects of clinical training focusing on level-3 OSCE (analytical and therapeutic skills) items, and compared the achievement levels of physical (PT) and occupational (OT) therapist students. [Subjects] A total of 282 (165 PT and 117 OT) students enrolled at our university between 2008 and 2010 were studied. [Methods] OSCE scores were compared between before and after clinical training focusing on level-3 OSCE items, and between PT and OT students.
[Results] Scores for 5 out of the 6 level-3a items were significantly higher after than before clinical training. Increases in scores of 2 or 3 level-3b and -3c items were also observed after clinical training. There were no marked differences between PT and OT students in scores for level-3a, -3b, and -3c items before clinical training. In contrast, after clinical training, OT students' scores for 3a and 3c items related to dressing were higher than those of PT students, and the latter's scores for 3b items related to transfer were higher than those of the former.
[Conclusion] The results suggest level-3 OSCE items are effectively taught during clinical training.

INTRODUCTION

Physical and occupational therapies are clinical sciences playing an important role in the practice of rehabilitation medicine and services. They are also evidence-based academic methods used in clinical environments, and clinical studies and techniques addressing disabilities through practical activities. In these fields, nurturing human resources, contributing to their development and promotion, and implementing advanced research projects leading to the progress of clinical sciences are regarded as the most important challenges.

In response to increased social demands for rehabilitation, the numbers of physical and occupational therapist training schools have rapidly increased in recent years to 248 and 182, respectively, and the total numbers of certified physical (PT) and occupational (OT) therapists as of 2013 were 110,000 and 65,000, respectively. Considering that more than 10,000 PTs are newly certified every year, the number of these therapists is likely to become large in the future. In addition, in clinical environments, PTs and OTs with 3 years or less experience account for approximately 25% of their total numbers. The rapid increase in the number of therapists has consequently reduced overall years of experience of therapists in clinical environments, suggesting a decrease in the quality of clinical services. Therefore, expanding their ranges of activity is an urgent issue, and in order to prevent their work conditions from further deteriorating, it is crucial to ensure novices have sufficient skills, and to improve the quality and specialty of inexperienced and experienced therapists. In line with this, as future perspectives on appropriate education systems, it is important for faculty members to participate in clinical practice to clarify therapists’ actual skills, and adopt educational approaches based on technical items necessary for clinical training. On the other hand, for clinical supervisors and PTs and OTs in charge of post-graduate education, it is essential to accurately recognize the content students learn at training schools. Therefore, systems to provide standardized and comprehensive skill education throughout the period between pre- and post-graduation, involving faculty members, clinical supervisors, and PTs and OTs in charge of post-graduate education, are needed.

Under these circumstances, we have engaged in specialized education for students, in other words, “education to nurture specialists with clinical skills”, since our department was founded within the study university in 2004, focusing on clinical demands and regarding physical and occupational therapies as a “domain of therapeutics” or a clinical science. In short, the central goal of our education is the development of therapists’ clinical skills. In the current therapist education system, however, students’ clinical skills are assessed by supervisors of clinical training facilities, rather than the faculty members of training schools. In therapist education to nurture “clinical professionals”, this is a critical issue. To address this, our faculty members participate in clinical treatment, and continuously examine the appropriateness methods for training students with therapists engaged in clinical practice. Similarly, the university has adopted the Objective Structured Clinical Examination (OSCE) system, aiming to specify clinical education standards for training schools and standardize methods of assessment among clinical supervisors1). The OSCE is a method of clinical skill assessment, proposed by Harden2) in 1975, and has been reported to be appropriate for the assessment of learning achievement levels in the psychomotor and emotional domains, which are difficult to evaluate with written examinations3). The standardization of assessment and treatment is also likely to improve the quality of clinical rehabilitation services.

In our department, OSCE items have been used as educational content since 2005 to standardize OSCE-based skill education methods, and this educational approach was adopted not only before, but also after clinical training in 2007 to establish an OSCE-based skill education systems. Furthermore, since 2011, in our department the OSCE has been regarded as a course of study (subjective clinical skill training), and the details of OSCE-based learning have been shown to clinical supervisors to increase the consistency of their assessment. Using the OSCE to link school education and clinical training together, we aim to integrate schools and clinical training facilities, which has been difficult for all training schools throughout Japan up to the present.

This study may be of significance in the standardization of methods of assessing clinical skills in therapist education. Our previous studies revealed associations among OSCE scores, academic achievements, and clinical training outcomes4,5,6,7,8). In the present study, the effects of clinical training focusing on level-3 OSCE (analytical and therapeutic skills assistance and guidance) items were examined, and the achievement levels of PT and OT students were compared with a view to providing reference data for appropriate clinical training and education.

SUBJECTS AND METHODS

A total of 282 PT and OT students were studied. They comprised students enrolled at our university: 58 PTs and 40 OTs in 2008, 46 and 35 in 2009 and 61 and 42 in 2010, respectively. The OSCE was conducted in 3 rooms (stations), in each of which a task was presented, and the examinees made the rounds of these stations to implement all tasks, following instructions. In each station, 2 examiners (a physical or occupational therapy faculty member and a clinical supervisor) and 1 simulated patient (another clinical supervisor) were present. The time to implement each task was 5 minutes, and immediately after its completion, 3-minute feedback was provided. Subsequently, the examiners and simulated patient discussed, and assessed the examinees’ performance. The tasks were not previously shown to the examinees.

For statistics, Predictive Analytics Software (PASW) Statistics 18.0 was used. For the comparison of scores for level-3 OSCE items (6 items 3 domains) between before and after clinical training, the paired t-test was conducted, and for comparison of scores between PT and OT students, the unpaired t-test was conducted.

Consent regarding the collection and use of data in this study was obtained from the study students, and this study was approved by the Ethical Review Board for Clinical Research of the Fujita Health University (10-121, 14-260).

RESULTS

At OSCE level 3a, scores of standing, dressing, elevating the trunk, toileting, and transfer were significantly higher after clinical training. Similarly, at level 3b, scores of walking, elevating the trunk, and transfer were markedly higher after clinical training; at level 3c, scores of elevating the trunk and toileting were also significantly higher after clinical training (Table 1).

Table 1.

Comparison of level-3 OSCE scores between before and after clinical training

Level-3a	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Before clinical training	71.5 ± 13.2 % (n=42)	71.2 ± 10.5 % (n=46)	69.9 ± 14.3 % (n=14)	68.2 ± 16.4 % (n=46)	67.7 ± 13.5 % (n=46)	58.0 ± 11.0 % (n=13)
After clinical training	82.0 ± 7.1 % (n=27)	82.0 ± 6.5 % (n=28)	69.8 ± 10.1 % (n=32)	77.5 ± 12.7 % (n=44)	76.3 ± 11.0 % (n=46)	79.9 ± 11.8 % (n=28)
Before clinical training vs. After clinical training	**	**		**	**	**
Level-3b	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Before clinical training	72.3 ± 8.2 % (n=10)	77.1 ± 12.6 % (n=42)	71.8 ± 13.2 % (n=39)	66.6 ± 10.0 % (n=10)	69.2 ± 16.8 % (n=41)	75.8 ± 8.0 % (n=42)
After clinical training	77.3 ± 11.6 % (n=25)	81.3 ± 8.6 % (n=42)	82.1 ± 8.9 % (n=24)	79.6 ± 7.9 % (n=28)	67.1 ± 17.0 % (n=23)	80.8 ± 8.0 % (n=41)
Before clinical training vs. After clinical training			**	**		**
Level-3c	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Before clinical training	80.6 ± 8.5 % (n=38)	58.7 ± 11.1 % (n=7)	68.2 ± 12.7 % (n=40)	73.6 ± 13.1 % (n=40)	30.9 ± 5.2 % (n=10)	76.8 ± 12.6 % (n=40)
After clinical training	78.3 ± 9.2 % (n=40)	71.6 ± 9.3 % (n=26)	67.4 ± 12.4 % (n=40)	75.3 ± 8.9 % (n=23)	59.9 ± 14.2 % (n=26)	79.8 ± 9.6 % (n=21)
Before clinical training vs. After clinical training				*	**

**p<0.01, *p<0.05

There were no marked differences between PT and OT students in the scores of level-3a −3b, and −3c items before clinical training. In contrast, after clinical training, OT students’ scores for dressing at levels 3a and −3c were significantly higher than those of PT students, while the latter’s score for transfer at level 3b item was markedly higher than those of the former (Table 2).

Table 2.

Comparison of level-3 OSCE scores between physical and occupational therapy students

Before clinical training
Level-3a	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	71.3 ± 12.7 % (n=27)	72.7 ± 8.8 % (n=26)	67.7 ± 12.2 % (n=10)	67.2 ± 17.6 % (n=27)	65.5 ± 13.8 % (n=28)	57.3 ± 11.6 % (n=10)
Occupational therapy students	71.8 ± 14.4 % (n=15)	70.2 ± 12.0 % (n=20)	75.4 ± 19.4 % (n=4)	70.2 ± 15.0 % (n=18)	70.9 ± 12.8 % (n=19)	60.1 ± 10.2 % (n=3)
Physical therapy students vs. Occupational therapy students
Level-3b	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	73.7 ± 9.8 % (n=6)	76.7 ± 13.4 % (n=23)	71.6 ± 15.3 % (n=18)	69.8 ± 9.6 % (n=6)	68.3 ± 15.6 % (n=26)	75.2 ± 8.6 % (n=28)
Occupational therapy students	70.2 ± 5.6 % (n=4)	77.6 ± 11.9 % (n=19)	71.0 ± 11.4 % (n=21)	61.7 ± 9.8 % (n=4)	70.7 ± 19.4 % (n=15)	76.2 ± 6.5 % (n=14)
Physical therapy students vs. Occupational therapy students
Level-3c	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	81.8 ± 8.4 % (n=20)	55.4 ± 12.9 % (n=3)	70.2 ± 11.3 % (n=24)	70.9 ± 11.8 % (n=22)	29.4 ± 2.1 % (n=4)	76.6 ± 13.0 % (n=22)
Occupational therapy students	79.2 ± 8.7 % (n=18)	61.2 ± 10.8 % (n=4)	65.2 ± 14.5 % (n=16)	76.8 ± 14.1 % (n=18)	31.9 ± 6.6 % (n=6)	77.1 ± 12.5 % (n=18)
Physical therapy students vs. Occupational therapy students
After clinical training
Level-3a	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	80.6 ± 7.6 % (n=18)	79.9 ± 5.4 % (n=20)	71.8 ± 9.3 % (n=22)	76.3 ± 13.9 % (n=27)	76.4 ± 12.0 % (n=29)	78.4 ± 12.9 % (n=16)
Occupational therapy students	84.7 ± 5.5 % (n=9)	87.2 ± 6.4 % (n=8)	65.5 ± 10.9 % (n=10)	79.5 ± 10.5 % (n=11)	76.3 ± 9.3 % (n=17)	82.0 ± 10.4 % (n=12)
Physical therapy students vs. Occupational therapy students		**
Level-3b	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	77.4 ± 5.5 % (n=11)	80.2 ± 9.3 % (n=21)	83.1 ± 9.2 % (n=12)	80.0 ± 7.2 % (n=17)	65.5 ± 17.0 % (n=13)	81.8 ± 7.7 % (n=30)
Occupational therapy students	77.2 ± 14.9 % (n=14)	82.5 ± 7.9 % (n=21)	81.1 ± 9.0 % (n=12)	79.0 ± 9.2 % (n=11)	69.3 ± 17.8 % (n=10)	77.1 ± 7.7 % (n=11)
Physical therapy students vs. Occupational therapy students						*
Level-3c	Standing up	Dressing activity	Walking	Sitting up on the bed	Toilet activity	Transfer
Physical therapy students	77.8 ± 8.8 % (n=22)	68.8 ± 9.6 % (n=17)	67.3 ± 13.1 % (n=21)	73.5 ± 8.0 % (n=15)	61.9 ± 16.4 % (n=9)	82.0 ± 11.0 % (n=10)
Occupational therapy students	78.9 ± 10.0 % (n=18)	77.0 ± 6.3 % (n=9)	67.5 ± 11.9 % (n=19)	78.7 ± 10.1 % (n=8)	58.8 ± 13.4 % (n=17)	77.7 ± 8.2 % (n=11)
Physical therapy students vs. Occupational therapy students		**

**p<0.01, *p<0.05

DISCUSSION

In recent years, the OSCE has been used as an educational approach for the objective assessment of clinical skills, mainly in the field of medicine9,10,11). Compared to conventional written examinations, the OSCE enables examiners to assess clinical skills in the psychomotor, emotional, and cognitive domains, and clarify items requiring impovement10). Saito et al. reported that it is effective to adopt the OSCE in medical education in order to train medical students, by developing, necessary basic skills in both technical and behavioral aspects, and it enables educators to guide students toward the appropriate integration of knowledge, skills, and behavior1).

Rehabilitation medicine is a practical system, in which medical services are provided based on learning achievements. Therefore, considering ourselves as learning specialists, we have re-examined the significance of OSCE-based skill education from the viewpoint of rehabilitation medicine, and have systematized it through new approaches, rather than simply adopting medical education systems. The OSCE for PTs and OTs consists of 3 levels, aiming to improve students’ clinical skills. level 1 addresses communication and care-giving skills, level 2 addresses examination and measurement skills, level 3 addresses analytical and therapeutic skills. The feedback provided before and after clinical training enables students to subjectively recognize their clinical skills, and develop learning plans to improve inadequate knowledge, behavior, and skills step by step. This practical learning process provides a basis for the application of the OSCE system to PT and OT education.

The present study examined the effects of clinical training focusing on level-3 OSCE items, which are related to analytical and therapeutic skills as basic physical and occupational therapy techniques needed in clinical environments, involving frequent contact with patients, in addition to their difficulty levels.

The comparison of scores between before and after clinical training showed there was general improvement after clinical training. However, for some items related to therapeutic skills (b and c), scores remained unchanged. For example, the scores for toileting in therapeutic skills (b), showed no differences between before and after clinical training, suggesting that the students may have been provided with limited opportunities to perform intervention approaches for toileting during clinical training. This tendency was particularly marked in PT students, as the difference in specific approaches between physical and occupational therapies occasionally prevents them from experiencing therapeutic intervention for this item in some training facilities, explaining the absence of changes in scores for toileting (b). Regarding scores for items related to therapeutic skills (c), there was no difference in the scores for walking between before and after clinical training. In relation to this, it should be noted that during clinical training, students very rarely deal with patients requiring guidance/assistance, which is the most important point of level-3c items. In the OSCE conducted in the present study a patient using an ankle foot orthosis without a cane, requiring approximately 75% assistance, was presented as a case. It is usually rare that students assist patients with high risk of falls during rehabilitation sessions. While students’ levels may also be considered to some extent, the possibility for them to be allowed to treat such patients depends on clinical therapists’ judgments of risks, and in terms of safety. Thus, the majority of students are unlikely to encounter cases similar to the one presented in the OSCE. In this respect, it may be necessary to present a wider variety of items in the OSCE for walking(c), and consider their appropriateness in clinical training.

The comparison of scores of PT and OT students revealed that OT students’ scores for dressing (a) and (c) were markedly higher than those of PT students after clinical training. In the OSCE, dressing is limited to upper garments and consequently more markedly reflects the specificity of occupational therapy than other OSCE items. Today, the majority of clinical training facilities are hospitals specializing in post-acute rehabilitation, in which intervention areas are clearly divided between physical and occupational therapies, and this background may be associated with the improvements in OT students’ scores for dressing. In order to expand the scope of physical and occupational therapies, it may be important to nurture therapists with the ability to cover all clinical areas with appropriate therapeutic intervention approaches. It may also be necessary to add items focusing on lower garments to dressing-related OSCE items. Separately, PT students’ scores for transfer (b) were significantly higher than those of OT students after clinical training. This may be due to their specialty, similar to the case of dressing.

In the OSCE, transfer refers to between a bed and a wheelchair, and, therefore, PT students are likely to provide the intervention more frequently than OT students during clinical practice. In addition to bed-wheelchair transfer, transfers from/to a bathtub or toilet are also important tasks in clinical environments, and the appropriateness of adding items focusing on this point in the second version or those following of the OSCE should also be discussed. After clinical training, the difference between PT and OT students scores was not significant. Furthermore, their scores generally improved after clinical training, indicating the efficacy of the use of level-3 OSCE items during clinical training.

In this study, it was noted that some items do not clearly reflect learning effects due to the difference in specialty between PTs and OTs. Considering the recent expansion of PT and OT activities along with the promotion of community-based rehabilitation, it may be essential to nurture therapists with the ability to cover all ADL areas with appropriate rehabilitation approaches while maintaining their specialties. Accordingly, systems to enable students to experience a wide range of clinical areas should be provided in cooperation with the leaders of training facilities, in addition to establishing those based on the OSCE.