Development and preliminary reliability testing of an assessment of patient independence in performing a treatment program: standardized scenarios.

BACKGROUND: Physical therapists often assess patient independence through observation; however, it is not known if therapists make these judgments reliably. We have developed a standardized method to assess a patient's ability to perform his or her treatment program independently.
OBJECTIVES: To develop a standardized assessment of patient independence in performance of a treatment program and examine the intra- and inter-rater reliability decisions made by two physical therapists.
DESIGN: Test-retest.
METHODS: An assessment of patient independence in performance was developed. Standardized patient scenarios were used to assess the intra- and inter-tester reliability of two physical therapists. Percentage of agreement (%) and kappa's coefficient (k and k(w)) indexed rater reliability.
RESULTS: Intra-rater reliability of therapist 1 was as follows: knowledge: % = 95, k = 0.90; performance: % = 95, k(w) = 0.82. Intra-rater reliability of therapist 2 was as follows: knowledge: % = 85, k = 0.68; performance: % = 94, k(w) = 0.80. Inter-rater reliability for knowledge was % = 91 and k = 0.79 and for performance was % = 91 and k(w) = 0.72.
CONCLUSION: Trained therapists displayed substantial to excellent intra-rater reliability and substantial inter-rater reliability in assessing a patient's independence in a treatment program. :