Development of clinical competence assessment tool for novice physical and occupational therapists-a mixed Delphi study.

[Purpose] The aim of this study was to clarify essential abilities of novice physical and occupational therapists for independent execution of their duties and to develop a clinical competence assessment tool. [Subjects] Forty-five experienced therapists participated in this study. [Methods] A two-phase mixed-methods design was used. First, semi structured interviews were conducted on 15 experienced therapists to create a comprehensive list of essential abilities that novice therapists need. Second, 30 experienced therapists participated in a two-round Delphi study to select items for the assessment tool being developed.
[Results] Fifty-five items were extracted and classified into three categories: basic attitudes, therapeutic skills, and clinical practice-related thoughts.
[Conclusion] Present results suggest that not only knowledge of execution of therapy-related duties and therapeutic skills is essential in novice therapist, but also appropriate abilities in social adjustment, self-management, and self-education. The newly developed tool might be useful for postgraduate education in clinical practice.

INTRODUCTION

Demand for improvement of quality of therapists has increased with the recent advancement in medical technologies and the growing understanding of society toward rehabilitation medicine1). However, the basic scholastic ability of students enrolled in training schools is not increasing. Consequently, the quality of therapists has not improved2). In 2005, the minimum attainment level for pre-graduation education was changed from “becoming able to perform basic physical therapy” to “becoming able to perform basic physical therapy with some advice and supervision” by the Physical Therapy Education Guidelines3). A previous study4) reported that most novice physical therapists (PTs) need supervisors’ advice and cannot accomplish their duties independently. Other previous studies5,6,7,8,9) suggest that novice PTs do not have sufficient clinical skill and thus require postgraduate education.

In the current Japanese situation, there are some problems with the postgraduate education system. First, postgraduate education is performed in individual facilities, which have their own characteristics and are thus not homogenized10). Some facilities have developed and implemented education programs independently11). In other facilities, novice PTs rotate among different departments based on the type or stage of disease12). Regarding the relation between the number of therapists and the education system, postgraduate education systems are better established in a facility with a large number of therapists13). Second, many members of the Japanese Physical Therapy Association are young, with ages ranging from 20–30 years3, 4).Thus, lack of supervising therapists is also a crucial problem. Under these circumstances, the postgraduate education systems are not likely to ensure a certain level of quality.

To maintain a level of educational quality, setting an explicit goal is necessary. In addition, goal setting has a second-order effect that promotes self-directed learning based on adult learning theories14). In line with these perspectives, the medical education system has compulsory post graduation clinical training for 2 years that has an explicit goal15). In the nursing education system, postgraduate training goals and guidelines on supervision have been established as well16). However, in the therapist education system, goals for post graduation therapists have not been determined yet.

To achieve a certain level of quality in novice PTs, the first goal is for them to be able to implement their duties independently. Therefore, the education guideline to meet this goal should be determined. The aim of this study was to clarify the essential abilities of novice physical and occupational therapists for independent execution of their duties and to develop an assessment tool.

SUBJECTS AND METHODS

Forty-five experienced therapists participated in this study. We used a two-phase mixed-methods design. First, semi structured interviews were conducted on 15 experienced therapists to create a comprehensive list of essential abilities that novice therapists need17). Second, 30 experienced therapists participated in a two-round Delphi study to select items for the assessment tool being developed for novice therapists. Therapists working in hospitals, training schools, or geriatric health services facilities; those with experience in supervising other therapists; and those with management experience were included in the study. All experienced therapists were provided with oral explanations regarding the details of this study as ethical considerations. Their participation in the study was regarded as their signed consent. The study was conducted with the approval of the Ethics Committee of Fujita Health University (13-254).

Before the interview, the definition of “therapist who is able to implement their duties independently” (Table 1), which was defined based on the definition of doctors’ basic clinical ability18), was presented to participants to ensure that the term has the same meaning to the subjects. All participants were simultaneously interviewed using with focus group semi structured interview methods19) based on an interview guide (Table 1). The participants were asked to imagine supervising other therapists and answer questions such as, “What novice PT behaviors affect your appreciation that therapists under your supervision have the ability to fulfill their professional duties independently?” The researchers encouraged the participants to verbalize their thought and listened with close attention. All interview contents were recorded using an IC recorder and were converted into character data to create narrative records. Then, we extracted the contents related to “therapists who are able to fulfill their professional duties independently” and classified the contents into categories based on similarity of content.

Table 1.

Definition of terms and the interview guide

Definition of terms: ‘Therapists who are able to independently implement their duties’

1) Having basic clinical skills that all therapists should develop regardless of individual domain expertise

2) Not requiring supervisors’ active guide

3) Being able to behave appropriately in workplaces (as members of society)

The details of an interviews guide used in this study:“Please answer the following questions, envisioning what you supervise novice therapists:”

1) When do you feel that therapists under your supervision reach the level enough to routine clinical practice independently?

2) What actions do you expect such novice therapists to achieve under your supervision?

3) When you have been brought up as a therapist, what type of experience did you need to become able to independently implement your duties?

We used a two-round Delphi study. In the first round, all participants were asked to rate the extracted items through the interview. We used 5-point rating method (1=unnecessary; 2=relatively necessary; 3=necessary; 4=absolutely necessary but may not be achieved within 6 months; 5=absolutely necessary and should be achieved within 6 months) focusing on “therapists who are able to fulfill their professional duties independently.”

In the second round, the ratings of each rater and the frequency distribution, median, and interquartile range of all participants’ ratings were presented. Subsequently, all raters were again asked to rate the items extracted from the first round. Using the results of the second round, the items where more than 80% of the total number of experienced therapists scored a rating of 4 or 5 were adopted as evaluation items. Then, these items were classified based on semantic similarity, and each category was named by the abstraction of their semantic contents. To confirm the reliability of the results, the items must also be classified into such categories by other experienced therapists serving as third party. Based on the classifications set by the researchers and third-party therapists, a cross-tabulation table was created and the κ coefficient was calculated20).

RESULTS

One hundred fifty-six items were extracted from data obtained through interview with 15 experienced participants in relation to abilities necessary for therapists to fulfill their professional duties independently. By unifying similar or overlapping items, these items were aggregated to 86 items. After a considerable discussion using two-round Delphi technique, 55 items were adopted as evaluation items. These items were classified into three categories: basic attitudes, 19 items; therapeutic skills, 20 items; clinical practice-related thoughts, 16 items (Table 2). The κ coefficient, representing the agreement rate between the researcher and third-party experienced therapist, was 0.86.

Table 2.

Outcomes of the development of a clinical ability evaluation table for therapists

Evaluation items:
Basic attitude
	Using appropriate language as a member of society
	Adhering to appointed times and deadlines
	Complying with rules in the workplace
	Understanding the role and duties of the therapist as a team member
	Adopting appropriate actions in consideration of the role of the therapist as a team member
	Contributing to the improvement of coordination as a team member
	Efficiently implementing duties so as to be completed within working hours
	Appropriately understanding and considering confidentiality and personal information management
	Performing appropriate infection control measures (including hand wash)
	Performing appropriate equipment management (before and after use)
	Performing treatment, with a sense of responsibility
	Appropriately managing the therapist’s own physical condition and schedule and avoiding interfering with his/her duties
	Appropriately implementing reporting, communication, and consultation procedures (developing and expressing the therapist’s own thoughts) in all times
	Identifying problems which are difficult to independently address
	Consulting problems which are difficult to independently address with appropriate persons in appropriate situations
	Seriously accepting and addressing issues noted by the supervisor or the therapist’s own failures
	Developing positive attitudes and making efforts to achieve knowledge and skills
	Performing treatment and implement duties based on learning outcomes and experience
Therapeutic skills
	Adopting appropriate measures, such as life-saving techniques, to manage sudden changes in patients’ conditions
	Appropriately dealing with individual patients in consideration of their symptoms
	Using appropriate verbal or non-verbal communication methods for individual patients
	Showing empathy when communicating with patients in consideration of their psychological conditions
	Appropriately listening to patients and their families to clarify their needs
	Having medical knowledge necessary for a therapist
	Selecting appropriate evaluation items for individual patients
	Performing vital (blood pressure and heart rate) measurement, according to each situation
	Appropriately (and also accurately, efficiently) conducting medical interviews with patients
	Appropriately (and also accurately, efficiently) examining reflexes
	Appropriately (and also accurately, efficiently) conducting orthopedic examination
	Appropriately (and also accurately, efficiently) evaluating pain
	Appropriately (and also accurately, efficiently) evaluating coordination
	Appropriately (and also accurately, efficiently) evaluating muscle tone
	Appropriately (and also accurately, efficiently) measuring the range of motion
	Appropriately (and also accurately, efficiently) evaluating the muscle strength
	Appropriately (and also accurately, efficiently) conducting sensory examination
	Appropriately (and also accurately, efficiently) performing morphometry
	Appropriately (and also accurately, efficiently) evaluating the motor function of patients with paralysis (using the SIAS and Brunnstrom Stage Test)
	Appropriately (and also accurately, efficiently) evaluating ADL (using instruments, such as the FIM and Barthel index)
Clinical practice-related thoughts
	Clarifying individual patients’ general characteristics
	Identifying individual patients’ possible risks based on the results of examination
	Logically examining the causes of problems in movements or activities of daily living
	Developing treatment programs to achieve goals (also referring to literature)
	Safely implementing treatment programs
	Safely handling treatment devices
	Appropriately managing risks related to medical accidents, such as tube removal and bleeding
	Appropriately managing risks related to falls
	Providing appropriate range-of-motion training
	Providing appropriate muscle-strengthening training
	Providing appropriate assistance and guidance for the maintenance of sitting positions
	Providing appropriate assistance and guidance for the maintenance of standing positions
	Providing appropriate assistance and guidance for standing from a seat
	Providing appropriate assistance and guidance for transfer
	Providing appropriate assistance and guidance for gait training
	Continuously evaluating (and observing) patients in the progress of treatment

Criteria: Ratings. 4=Being able to accurately understand and adopt appropriate actions without supervision. 3=Being able to accurately understand and adopt appropriate actions under monitoring and supervision. 2=Being able to understand and adopt appropriate actions to a certain extent under monitoring and supervision. 1=Being unable to understand or adopt appropriate actions even under monitoring or supervision. 0=Being inappropriate for implementation.

DISCUSSION

This study adopted the sequential exploratory strategy performed from a qualitative approach to a quantitative approach in a phased manner21, 22). In other words, the data were collected qualitatively and examined quantitatively using the Delphi technique, which is a questionnaire method with repetitive feedback23,24,25). As a result, 55 items were adopted by two-round Delphi technique. The focus group interview used in the present study might be effective in facilitating the collection of a broad range of opinions from experienced therapists because similar experiences about supervision assists in causing a sense of empathy and stimulates the discussion. The items were classified into three categories: basic attitudes, therapeutic skills, and clinical practice-related thoughts. The high κ coefficient between classifications by the researcher and a third-party therapist suggest high reliability of the classification.

The basic attitudes category consisted of items associated with the ability for continuing self-education, as mentioned in the Japanese Physical Therapy Association’s code of ethics26). In addition, an attitude of humility toward each patient and a cooperative attitude toward other professionals were included in this category. These items might reflect the critical importance of social nature. Thus, the results suggest that knowledge of these social skills is also important in the postgraduate education of novice therapists. The therapeutic skills category consisted of abilities associated with collection of medical information such as communication with patients and assessment technique. These items suggest that novice therapists should have a certain amount of communication skill in addition to medical knowledge. Thus, optimal communicational education methods have to be developed for the postgraduate education of novice therapists. The clinical practice-related thoughts category included items associated with integration of patient information obtained from medical assessment, determination of disabilities (impairment, activity limitation, participation restriction), planning of therapeutic program, re-evaluation, and corresponding plan revision. These items are substantially coincident with the clinical reasoning model proposed by Edwards, which includes a set of processes such as the recognition and interpretation of medical information, development and revision of hypothesis, determination of intent, and re-evaluation after intervention27).

In the present study, experienced therapists were encouraged to consider not only knowledge of implementing therapy-related duties, clinical practice-related thoughts, or therapeutic skills, but also appropriate social skills and attitudes, self-management, and self-education, for continuous self-improvement. The 55 evaluation items may cover all domains defined in the taxonomy (cognitive, emotional, and psychomotor), which is a concept often used in medical education and services28). Therefore, the preset items might accurately represent the abilities needed by therapists to implement their duties independently and should be the appropriate goals of novice therapists.