Are physical therapy interns competent in patient management skills? Assessment of the views of clinical and academic physical therapists.

[Purpose] This study aimed at evaluating PT interns' performance from the perspectives of clinical and academic physical therapists (PTs). [Methods] An online questionnaire based on a modified version of the Clinical Internship Evaluation Tool (CIET) was used to evaluate PT interns' performance in patient management skills. Assessors comprised clinical and academic PTs from Saudi Arabia. The survey aimed at assessing the competency of interns in 25 patient management skills comprising four major domains: clinical examination, patient evaluation, diagnosis and prognosis, and intervention. Assessors were also asked to rank the importance of possessing each skill and evaluating the internship programs at their facilities. A multivariate logistic regression analysis assessed whether the demographic variables between the two groups influenced their views.
[Results] A total of 148 participants (112 clinical PTs and 36 academic PTs) responded to the survey. The majority of the participants agreed that interns were competent in all 25 skills. Differences between the two groups of assessors were observed for five out of the 25 skills. Interestingly, skills with the highest means were perceived as the most important skills by both groups of assessors. There was no association between sociodemographic variables and evaluation scores.
[Conclusion] There were no differences observed between clinical and academic PTs in their evaluation of PT interns in 20 of the 25 clinical skills. There were indications of a need for further improvement in certain competencies and skills such as clinical examination, evaluation, diagnosis and prognosis, and intervention.

INTRODUCTION

In Saudi Arabia, all academic physical therapy (PT) programs require completion of a mandatory 1-year clinical internship, as is the case in many other professional health programs1). Graduates from PT programs are not required to sit any licensing examination because completion of internship is considered sufficient for registration with the Saudi Commission for Health Specialties (SCFHS), a national licensing agency for all health professionals2). Over the past decade, there has been an expansion in the number of academic PT programs—from one program offered by King Saud University (KSU) in Riyadh, to 12 programs offered by universities in the five geographical regions of Saudi Arabia.

Although all PT programs necessitate the completion of a 1-year clinical internship, there are no studies that have examined the effectiveness of these internship programs at providing the required clinical competencies, as suggested by different PT organizations such as the World Confederation for Physical Therapy (WCPT)3). According to the WCPT guidelines on entry-level PT education, competent physical therapists (PTs) should possess the following four major patient management skills: examination, evaluation, diagnosis and prognosis, and intervention3). During the transition from a PT student to a competent therapist, an intern learns to combine and integrate the knowledge, skills, attitudes, and values as well as the philosophies of the profession under the supervision of a clinical teacher4, 5). However, research has indicated that completing a clinical internship does not necessarily equip an intern with the clinical competencies required in PT or any health profession6,7,8,9,10).

Although there are several tools to assess PT interns’ competencies and how they perform in the clinical setting, the most widely used tool is the American Physical Therapy Association (APTA) Clinical Performance Instrument (CPI)11). However, a number of studies have raised some concerns regarding CPI11). For example, in a longitudinal analysis of 1,039 Canadian clinical placements between 2001 and 2008, Proctor et al.12) found that certain items in CPI had a lower completion rate. Straube and Campbell13) concurred with Proctor et al.12) and reported a low response rate among 182 PT students in Chicago. This concern along with others, such as the problem of calculating the rating scale and the length of the instrument, were addressed in the amended CPI version14).

Another validated assessment tool for interns’ performance is the Clinical Internship Evaluation Tool (CIET)15). Whereas CPI is designed to compare the interns’ performance against “entry-level PTs”14), CIET is designed to compare the interns performance against a higher level benchmark, namely “level of competent PTs”15). CIET was developed at the Department of Physical Therapy, University of Pittsburgh and includes two major sections. The first section deals with professional behaviors (i.e., safety, ethics, initiative, and communication skills). The second section deals with patient management and evaluates interns’ ability to efficiently manage patient examination, evaluation, diagnosis/prognosis, and intervention15).

Considering the number of PT programs in Saudi Arabia and the importance of the opinions of clinical and academic PTs on the ability of interns to meet the basic requirements as competent practitioners according to the WCPT guidelines, this study had two goals. The first was to evaluate the performance of PT interns in 25 patient management skills using a modified version of CIET15) and the second was to evaluate the internship programs from the clinical and academic PT perspectives. We hypothesized that there would be no differences between clinical and academic PTs in their evaluation of PT interns.

SUBJECTS AND METHODS

A quantitative cross-sectional study which used a validated web survey was delivered via Fluid Surveys. Before launching the study website (http://ptstudy.ksu.edu.sa), two pilot studies were conducted to examine the content validity and the test–retest reliability of the questionnaire, which helped in its further amendment. The final questionnaire was tested for internal consistency and reliability using Cronbach’s alpha, which yielded a value of 0.67.

Interns are PTs graduates who have to complete a 12-month internship program in a clinical setting before being accepted for registration by SCFHS2). The target participants included clinical and academic PTs from five geographical regions of Saudi Arabia. To differentiate between the two participant groups, we asked if they were primarily working (more than 50% of their time) as clinical or academic physical therapists.

A recent report by the Saudi Ministry of Health (MOH) estimated that there were about 415 PTs in Saudi Arabia; about 80% of them (331 PTs) worked in 99 different hospitals under the MOH umbrella16). From December 2010 to March 2011, an invitation to participate in the study was uploaded to the Saudi Physical Therapy Association (SPTA) website along with an individual invitation e-mailed to each member of the association. E-mails were also sent to PT faculty members affiliated to institutions throughout Saudi Arabia inviting them to participate in the study. Moreover, three reminder e-mails were sent on the 7th, 14th, and 21st day to increase the number of participants, which was followed by a generic invitation sent by fax to all clinical and academic PT departments in institutions throughout Saudi Arabia. A total of 148 PTs responded and fully completed the survey (Appendix 1: online only). The estimated overall response rate was 27%.

The survey comprised three main sections: sociodemographic variables (9 items), opinions regarding the PT internship performance in four major clinical skills (25 items), and the overall assessment (1 item). Sociodemographic variables included age, gender, nationality, number of years after receiving PT degree, highest degree earned, area of PT expertise, registration with the SCFHS, employment and clinical work settings, as well as geographical regional location.

The second section of the survey had 25 items about the interns’ performance in clinical skills in four major domains: examination (8 skills), evaluation (3 skills), diagnosis and prognosis (5 skills), and intervention (9 skills). Participants were asked to rank each of these 25 items on a five-point Likert scale (1 = poor, 2 = fair, 3 = good, 4 = very good, and 5 = excellent). The skills were adopted from CIET15). Furthermore, participants were asked to rank the importance of each of these skills to be mastered on completion of the internship year (1 = not at all important, 2 = somewhat unimportant, 3 = neutral, 4 = important, and 5 = very important).

The final section of the survey dealt with the overall evaluation of the internship programs at the participants’ facilities. PTs were asked how they would assess the current internship programs at their academic or clinical facilities (1 = poor, 2 = fair, 3 = good, 4 = very good, and 5 = excellent). This study was approved by the ethical research committee of the College of Applied Medical Sciences, King Saud University.

Data were analyzed using the chi-square test and/or t-test to describe the respondents’ sociodemographic variables and professional characteristics17). The independent two-sample Student’s t-test was used to compare the means of the scores of clinical and academic PTs17). Because the data were derived from a Likert scale, it was reanalyzed using the Wilcoxon-Mann-Whitney test (a nonparametric test analogous to the t-test) to further assess the results18, 19).

A multivariate logistic regression was conducted to examine the association between being an academic PT and reporting good or higher scores for the four domains and the overall assessment of the internship programs (Excellent/Very good/Good = 1 vs. Fair/Poor = 0) (acceptable vs. not acceptable level) because some differences were observed between the two assessor groups (Table 1). All analyses were performed using SAS, version 9.1.3 (SAS Institute, Cary, NC, USA). Significance was accepted for values of p < 0.05.

Table 1.

Subject characteristics of clinical and academic physical therapists (PTs)

Variables	Clinical PTs (N=112)	Academic PTs (N=36)
Age*
20–29	51 (45.5%)	14 (48.9%)
30–40	41 (36.6%)	8 (2.22%)
Over 40	20 (17.9%)	14 (48.9%)
Gender
Female	62 (55.3%)	14 (38.9%)
Male	50 (44.6%)	22 (61.1%)
Nationality
Saudi	73 (65.2%)	24 (66.7%)
Non-Saudi	39 (34.8%)	12 (33.3%)
Highest degree earned**
Bachelor’s	69 (61.6%)	3 (8.3%)
Other graduate degree	11 (9.8%)	2 (5.6%)
Master’s	32 (28.6%)	18 (50%)
PhD	0 (0%)	13 (36.1%)
Specialty
Orthopedics	42 (37.5%)	12 (33.3%)
Pediatrics	16 (14.3%)	10 (27.8%)
Neurologic	35 (31.2%)	5 (13.9%)
Other	19 (17%)	9 (25%)
SCFHS**
Yes	102 (91.1%)	18 (50%)
No	10 (8.9%)	18 (50%)
Average years of clinical experience
Mean (SD)	8.43 (6.75)	9.97 (5.91)
Employment**
MOH	47 (42%)	3 (8.3%)
Other governmental	37 (33%)	5 (13.9%)
University/College	3 (2.7%)	26 (72.2%)
Private	25 (22.3%)	2 (5.6%)
Geographical regional
Central	42 (37.5%)	12 (33.3%)
Western	25 (22.3%)	10 (27.8%)
Eastern	26 (23.2%)	6 (16.7%)
Southern	13 (11.6%)	2 (5.5%)
Northern	6 (5.4%)	6 (16.7%)

* p < 0.05; **p <0.001: independent t-test and χ2 analyses for unadjusted comparisons between clinical and academic PTs.

RESULTS

Of the total number of PTs in Saudi Arabia, 148 completed the web survey. The characteristics of the clinical and academic PT respondents are shown in Table 1. One hundred and twelve respondents (75%) were clinical PTs and only 36 (25%) were academic PTs. Approximately 50% of respondents in both groups were between 20 and 29 years of age. Of the academic PTs, 14 (39%) were females and 22 (61%) were males. The majority of the respondents in both groups (about 65%) were Saudis. Of the clinical PTs, 69 (61%) had a bachelor’s degree and 11 (10%) had other graduate degrees (such as a graduate certificate, diploma, or clinical doctorate); none of the respondent clinical PTs had a PhD. Approximately 67% of clinical and academic PTs felt that 1 year is the optimum duration required for clinical internship (data not shown).

Nearly 50% of academic PTs who responded had a valid registration with SCFHS as compared with 90% of clinical PTs. Academic PTs were more likely to have a clinical affiliation with university/college hospitals or clinics as compared with 47 (42%) of clinical PTs working in one of the MOH hospitals. The average number of years after graduation (receiving the PT degree) for clinical and academic PTs were 8.4 years and 10 years, respectively. The clinical and academic PTs respondents were more likely to be from the Central and Western regions of Saudi Arabia. However, χ2 tests of proportion indicated that respondents were proportionally representative of the five regions of Saudi Arabia.

Examination Domain: Table 2 summarizes the ranking of competencies demonstrated by PT interns in eight examination skills, as evaluated by the clinical and academic PTs. In seven skills, clinical PTs gave higher scores of evaluation compared with academic PTs. A statistical difference was observed between the two groups in perceiving skill 8, which covers “demonstrating appropriate psychomotor skills when performing tests and measures.” From the clinical and academic PT perspectives, the mean scores of the examination skills were 2.8 (0.9) and 2.7 (0.7) (p > 0.05), respectively. The most important examination skill to be mastered by the new graduates was skill 1, “obtaining an accurate history of the current problem.” Interestingly, this particular skill had the highest mean score (3.32, SD = 1) as viewed by both groups.

Table 2.

Examination skills

Item	ALL(N=148)mean (SD)		Clinical PT (N=112) mean (SD)		Academic PT(N=36) mean (SD)		Importance Ranking
1. Obtains an accurate history of current problem	3.32	(1.00)	3.34	(0.99)	3.28	(1.03)	1
2. Identifies problems related to activity limitations and participation restrictions using standardized outcome instruments when available	3.09	(1.14)	3.16	(1.13)	2.86	(1.15)	5
3. Performs systems review and incorporates relevant past medical history	2.78	(1.13)	2.80	(1.18)	2.69	(0.98)	6
4. Generates an initial hypothesis	2.70	(0.99)	2.70	(1.05)	2.69	(0.79)	8
5. Generates alternative hypotheses (list of differential diagnoses)	2.49	(1.05)	2.57	(1.11)	2.25	(0.81)	7
6. Selects evidence-based tests and measures to confirm or disconfirm hypotheses	2.49	(1.25)	2.51	(1.29)	2.42	(1.11)	3
7. Recognizes contraindications for further tests and measures	2.89	(1.20)	2.85	(1.22)	3.00	(1.12)	2
8. Demonstrates appropriate psychomotor skills when performing tests and measures*	2.58	(1.14)	2.68	(1.17)	2.28	(0.97)	4
Summative Examination Skill Score	2.79	(0.85)	2.83	(0.90)	2.68	(0.68)

Scale: 5 = excellent; 1 = poor

* p < 0.05

Evaluation Domain: Table 3 summarizes the assessment of competencies of PT interns in three evaluation skills. Participants were asked to give their opinion regarding three different evaluation skills and rate the importance of each to be mastered by new graduates. Clinical PTs gave higher scores than academic PTs for each of the three skills. A statistical difference was observed between the two groups with regard to skills 1 and 3 (p < 0.05 for each). The difference for skill 2 was not statistically significant. In addition, a statistical difference was noted between the two groups in the mean score of the evaluation skills [3.1 (1.1) vs. 2.6 (0.8); p < 0.05]. Both clinical and academic PTs agreed that the skill of identifying impairment is the most important skill to be mastered by the interns (skill 2). This particular skill had the highest mean score compared with other skills [3.2 (1.16)].

Table 3.

Evaluation skills

Item	ALL (N=148) mean (SD)		Clinical PT (N=112) mean (SD)		Academic PT (N=36) mean (SD)		Importance Ranking
1. Makes correct clinical decisions based on the data gathered in the examination (confirms/disconfirms initial and alternative hypotheses)*	3.01	(1.11)	3.16	(1.14)	2.53	(0.88)	2
2. Identifies impairments in body structure and function; activity limitations; and participation restrictions	3.20	(1.16)	3.25	(1.17)	3.03	(1.13)	1
3. Administers further tests and measures as needed for appropriate clinical decision making*	2.73	(1.26)	2.85	(1.36)	2.36	(0.83)	3
Summative Evaluation Skill Score*	2.98	(1.04)	3.09	(1.09)	2.64	(0.77)

Scale: 5 = excellent; 1 = poor

* p < 0.05

Diagnosis and Prognosis Domain: No statistical differences were observed between the clinical and academic PTs in evaluating the five skills included under diagnosis and prognosis (Table 4). The skill of selecting appropriate PT interventions or making appropriate consultations or referrals (skill 3) had the highest score compared with all other skills [2.76 (1.19)]. Interestingly, this particular skill was seen as the most important skill to be mastered by any new graduate from the perspectives of both groups.

Table 4.

Diagnosis and prognosis skills

Item	ALL (N=148) mean (SD)		Clinical PT (N=112) mean (SD)		Academic PT (N=36) mean (SD)		Importance Ranking
1. Determines a diagnosis for physical therapy management of the patient	2.74	(1.10)	2.76	(1.20)	2.69	(0.71)	3
2. Determines expected outcomes (using standardized indices of activity limitations and participation restrictions where applicable) of physical therapy interventions (goals)	2.64	(1.13)	2.71	(1.20)	2.39	(0.80)	5
3. Selects appropriate physical therapy interventions or makes appropriate consultations or referrals	2.76	(1.19)	2.81	(1.29)	2.61	(0.77)	1
4. Determines appropriate duration and frequency of intervention; considers cost effectiveness	2.67	(1.16)	2.72	(1.24)	2.50	(0.88)	2
5. Determines criteria for discharge	2.67	(1.23)	2.74	(1.28)	2.44	(1.05)	4
Summative Diagnosis and Prognosis Skill Score	2.70	(1.02)	2.75	(1.12)	2.53	(0.61)

Scale: 5 = excellent; 1 = poor

Intervention Domain: Statistical differences were observed between the clinical and academic PTs in evaluating skills 4 and 5 (Table 5). Clinical PTs gave higher scores for these skills than academic PTs (p < 0.01). In terms of importance, the skill of assessing patient progress using appropriate measures was ranked most important and the skill of incorporating discharge planning into treatment (skill 4) was ranked least important.

Table 5.

Intervention skills

Item	ALL (N=148) mean (SD)		Clinical PT (N=112) mean (SD)		Academic PT (N=36) mean (SD)		Importance Ranking
1. Adheres to evidence during treatment selection	2.70	(1.26)	2.77	(1.34)	2.47	(0.97)	8
2. Applies effective treatment using appropriate psychomotor skills	2.89	(1.24)	3.00	(1.26)	2.56	(1.13)	3
3. Incorporates patient/family education into treatment	3.12	(1.25)	3.13	(1.30)	3.08	(1.11)	2
4. Incorporates discharge planning into treatment *	2.85	(1.19)	2.98	(1.19)	2.44	(1.08)	5
5. Assesses progress of patient using appropriate measures *	3.00	(1.19)	3.18	(1.19)	2.44	(1.00)	1
6. Modifies intervention according to patient/client’s response to treatment	3.00	(1.24)	3.05	(1.25)	2.83	(1.21)	6
7. Recognizes when expected outcome has been reached and makes appropriate recommendations	2.89	(1.18)	2.96	(1.24)	2.69	(0.95)	4
8. Recognizes psychosocial influences on patient management	2.79	(1.18)	2.80	(1.16)	2.75	(1.25)	7
9. Appropriately utilizes the PTA/support staff for interventions	2.81	(1.25)	2.79	(1.29)	2.89	(1.14)	9
Summative Intervention Skill Score*	2.89	(1.02)	2.96	(1.09)	2.69	(0.75)

Scale: 5 = excellent; 1 = poor

* p < 0.01

Overall evaluation of the interns and the internship programs: We computed the overall evaluation scores for the 25 skills were 2.9 (0.9) for clinical PTs and 2.6 (0.6) for academic PTs (p = 0.14) (data not shown). Approximately 50% of participants from both groups (49% clinical PTs and 55% academic PTs) rated the internship programs at their clinical or academic facilities as “good,” whereas 12% clinical PTs rated their internship program as “excellent” compared with 6% academic PTs. The Fisher’s exact test indicated no statistical differences between the ratings of the two groups (p = 0.83).

Effect of evaluator background: In examining the association between being an academic PT and the PT intern evaluation in the four domains of skills (examination, evaluation, diagnosis and prognosis, and intervention) as well as with the whole internship program, we created new two variables for each of the five outcomes: “acceptable” and “not acceptable.” For the “acceptable” variable, excellent, very good, and good were combined and coded as “1.” The “not acceptable” variable was coded as “0” and combined both the fair and poor ratings. In the multivariate logistic regression analyses, no association was observed between the evaluator background (being a clinical or academic PT) and the PT intern performance in the five clinical skills and the overall performance of the internship program. In the adjusted model, we used controls for all of the sociodemographic variables, and no significant associations were observed (data are not shown).

DISCUSSION

In the present study, we investigated how clinical and academic PTs in Saudi Arabia perceived the performance of interns with regard to patient management skills and the internship programs at their facilities. Both the clinical and academic PTs agreed that the interns’ performance in patient management was good overall. However, clinical PTs gave higher evaluation scores than the academic PTs.

The study shows that only five (20%) out of the 25 skills examined by the clinical and academic PTs were ranked significantly different by the two groups of assessors, with the clinical PTs giving higher rankings. These five skills were: “demonstrates appropriate psychomotor skills when performing tests and measures”; “makes correct clinical decisions based on the data gathered in the examination”, “administers further tests and measures as needed for appropriate clinical decision making”; “incorporates discharge planning into treatment”; and “assesses progress of patient using appropriate measures.” No significant differences were noted between the two assessors with regard to the other 20 skills.

The summative scores of the four major domains showed that the PT interns’ performance in the evaluation domain had the highest mean score compared with the other domains [2.98, standard deviation (SD) = 1.04] followed by the intervention, examination, and diagnosis/prognosis domains. Furthermore, in each domain, the single skill that had the highest mean was also ranked the most important skill to be mastered by the interns before completing the internship year; this applied to all domains except the domain on intervention.

In the second part of this study, no significant differences were observed between the two groups of assessors in their overall evaluation of the internship programs. Such agreement between clinical and academic PTs was also noted by other researchers, such as Brudvig and Colbeck20) and Crenshaw et al.21), who studied curricular development in the physical therapy and medical professions, respectively.

Nonetheless, the few differences in the evaluation scores observed between the clinical and academic PTs with regard to the five skills mentioned earlier may reflect the needs of both assessors in PT curriculum development and evaluation of outcomes particularly with regard to skills and competencies. Furthermore, such differences between the two groups of assessors may provide a foundation for reviewing, refocusing, and updating materials, especially since PT is moving to a doctoral entry-level degree20).

The official number of PTs in Saudi Arabia is estimated to be about 50016). In this study, the total number of assessors was only 148, and the representation of clinical and academic PTs was 112 and 36, respectively. These numbers approximately mirror the distribution of both groups in Saudi Arabia, which has a ratio of 3:1. Although we used a number of techniques as recommended by Burns et al.22) to increase the number of assessors recruited from both groups across the five regions of Saudi Arabia, those who responded were only about one-third of the PTs in Saudi Arabia. The techniques included the use of the SPTA website to advertise for the study, using e-mail group lists, social networking websites (Twitter and Facebook), and sending individualized invitations to all major hospitals in the five geographical regions.

We also encouraged participation by organizing a raffle draw with mobile phones as prizes for two random participants. Although the official number of PTs in Saudi Arabia is 500, the actual numbers may be lower, considering the fact that some travel overseas for further study as part of the continuing education/fellowship programs organised by the government and universities. We were unable to determine the exact number of clinical and academic PTs enrolled in such studies at the time of the study. This uncertainty made it difficult to know the exact number of PTs working in Saudi Arabia, and limited out ability to make a decision on the adequacy of the number of assessors included in the study.

To the best of our knowledge, this study evaluating the internship programs for PTs is the first of its kind in Saudi Arabia. As part of the quality assurance and benchmarking PT education and clinical practice, developing reliable and valid assessment tools for internships that can be used nationwide is necessary to ensure that the required skills/competencies are achieved by PT interns before they join the workforce. Most international PT programs use similar tools for assessment. For example, the PT programs in the United States and Canada use CPI14) or CIET15) and those in Australia and New Zealand widely use the Assessment of Physiotherapy Practice (APP)23). Because our focus was on intern’s patient management skills, we decided to use the CIET as it includes more items and has been validated.

One of the limitations of this study was our reliance on a web survey to collect data. This method has not been commonly used in Saudi Arabia and we are uncertain about the proportion of PTs in Saudi Arabia who use the internet for their day-to-day communication activities. However, several studies in Saudi Arabia have recently been conducted using web-based questionnaires such as Survey Monkey24), and there is evidence that health care professionals use the web in their workplace25). Overall, this method has been found to be more efficient at protecting data and preventing it loss, and more convenient for respondents than any other method26).

This study was descriptive and exploratory, and it highlights areas of strengths and weaknesses in the PT internship programs in Saudi Arabia and clinical competencies of the PT interns. The survey did not measure actual intern performance, and this is an important distinction that should be made. The survey was based on the CIET, which is an indirect measure used to gather the perceptions of clinicians and academicians about student intern’s performance which is supposed to be benchmarked against a competent PT, as we discussed earlier.

We believe that this study can be a foundation for future studies for the following reasons. First, it groups the competencies and skills of different PT internship programs in Saudi Arabia. Second, it highlights the views of academic and clinical PTs regarding these competencies and the performance of interns. Third, it highlights areas that need improvement in PT education and internship programs. Further studies should explore the views of PT interns and other stakeholders, including clients, providers, and healthcare systems in Saudi Arabia.

In conclusion, there were no differences observed between clinical and academic PTs in their evaluation of PT interns in 20 of the 25 clinical skills. There were indications for the need of further improvement in certain competencies and skills such as clinical examination, evaluation, diagnosis and prognosis, and intervention.