Development of a clinical assessment test of 180-degree standing turn strategy (CAT-STS) and investigation of its reliability and validity.

[Purpose] To develop a clinical assessment test of 180-degree standing turn strategy (CAT-STS) and quantify its reliability and construct validity. [Subjects] Outpatients with stroke that occurred at least 6 months previously (N = 27) who could walk 10 m without physical assistance were included. [Methods] The CAT-STS was based on the literature and discussion with four physical therapists. The final version of the CAT-STS includes seven items: direction, use of space, foot movement, initiation, termination, instability, and non-fluidity. Patients were videotaped performing a 180-degree turn while standing. The Motricity Index, gait speed and Functional Ambulation Category were also evaluated. Two raters evaluated the turn on two occasions, and inter- and intra-rater reliability were calculated. Construct validity was also calculated.
[Results] Inter-rater reliability was fair or moderate for many items (kappa = 0.221-0.746). Intra-rater reliability was good-to-excellent for all items (kappa = 0.681-0.846) except direction and termination. Inter- and intra-rater reliability of the total CAT-STS score were substantial and excellent, respectively (intraclass correlation coefficient = 0.725 and 0.865, respectively). The total CAT-STS score was associated with walking ability and the time and number of steps taken to turn.
[Conclusion] The total CAT-STS score is a reliable and valid measure.

INTRODUCTION

Turning while standing is a common daily activity1) because it is necessary to turn while standing when moving in a small space, such as a restroom. In community-dwelling elderly individuals, hip fractures are eight-fold more likely to result from falls sustained while turning than from falls sustained while walking2). Walking and turning are the most common causes of falls in recently discharged patients with stroke and in elderly people residing in long-term care facilities3, 4). Thus, turning increases the risk of falls5).

Turning can be evaluated quantitatively and qualitatively. Quantitative evaluations of turning have been reported in elderly individuals6,7,8,9), patients with stroke10), and patients with Parkinson’s disease11,12,13). Qualitative evaluations of turning have also been reported in elderly individuals, patients with Parkinson’s disease, and patients with stroke14,15,16,17,18). These previous studies have quantitatively evaluated turning using the Timed Up and Go test (TUG) or gait initiation. In contrast, qualitative evaluations of turning while standing have not been studied sufficiently. Turning while standing includes termination of motion, which is evaluated in the Community Balance and Mobility Scale and the Performance-Oriented Mobility Assessment19, 20). Evaluation of termination is important because motion of termination is affected by age and by certain diseases21,22,23). In addition, the qualitative evaluation of turning is associated with falls, and qualitative evaluation is important for movement analysis14, 15). In the clinical setting, both quantitative and qualitative evaluations are important because physical therapists evaluate turning from many viewpoints. In addition, knowledge of how patients turn is useful for training of turning.

The purpose of this study was to develop a tool to assess standing turn strategy and to investigate the reliability and validity of this new assessment tool. In the first step, we developed items to evaluate standing turn strategy. In the next step, we investigated the reliability and validity of these items using videotape-based assessments of turning in outpatients with stroke.

SUBJECTS AND METHODS

This was a cross-sectional reliability and validity study. We developed a clinical assessment test of 180-degree standing turn strategy (CAT-STS). The Institutional Review Board of the Geriatrics Research Institute and Hospital approved this study and all participants provided their written informed consent to participate.

Reviews of turning in elderly individuals, patients with Parkinson’s disease, and patients with stroke were consulted to generate the items to be included in the CAT-STS14,15,16,17,18, 24,25,26,27,28,29,30,31). One researcher (M.K.) selected 11 items that reflect the characteristics of turning in elderly individuals, hemiparetic individuals, and patients with Parkinson’s disease. The 11 items initially selected were; type of movement direction and space; stability; step length; fluidity; number of steps and time required to turn; weight shift toward the paretic limb; use of a cane; initiation; termination; step direction of the first step; and step direction of the subsequent steps.

These 11 items were reviewed and revised by four physical therapists and one researcher (M.K.). Their mean (standard deviation) duration of clinical experience was 15.1 (7.9) years. The CAT-STS items were discussed in eight meetings, each lasting 30–60 min. In the first and second meetings, the characteristics of turning movements and reviews of the qualitative analysis of turning were discussed. In the third, fourth and fifth meetings, the 11 items were revised according to whether or not each item was considered appropriate for evaluating turning and whether each item could be evaluated in a clinical setting. It was determined that all items should be measured on a two- or three-point scale because the evaluation should be completed within a short period and be easy to perform in a clinical setting. In the sixth and seventh meetings, the CAT-STS was evaluated by the five raters (four physical therapists and one researcher) using videotaped performance of turning in 10 hemiparetic patients with stroke. The agreement rates were calculated. The CAT-STS items were modified when the agreement rate of four raters was below 60%, or when the evaluation of the researcher (M.K.) did not agree with the evaluations of the four physical therapists except the researcher. The items in the CAT-STS were then discussed once again. At the eighth meeting, the following eight items were selected for inclusion in the CAT-STS: direction, type of movement direction and space, use of space, initiation, termination, instability, non-fluidity, and foot movement.

Twenty-seven patients with stroke sustained at least 6 months previously were recruited. The participants were selected by the staff of the Geriatric Health Care Facility. Inclusion criteria were the ability to walk at least 10 m with or without an ankle-foot orthosis and the ability to follow commands. Individuals were excluded if they had a musculoskeletal condition or a neuromuscular disease that affected the performance of turning and walking. Participants with a wide range of walking ability were chosen to ensure that the CAT-STS could be applied to patients of varying ability. However, patients who walked completely normally with no aesthetic anomaly or limp32) were not recruited.

A tripod-mounted video camera was located directly in front of the participant at a distance of about 3 m and a pylon was located about 3 m behind the participant. The video camera was adjusted to allow a full view of the participant performing the turning task.

The turning task was a 180-degree turn. This was selected because it is included in some mobility tests, including the TUG33), the Dynamic Gait Index34) and the Standing-start 180° Turn test16). Participants were given the following instructions: “Turn to the pylon. Pause. Then turn in the other direction to the initial position.” After several practices, measurements were taken three times in each turning direction (paretic direction and non-paretic direction). The time and number of steps taken to complete each turn were measured.

Physical impairment of the lower limb was evaluated using the Motricity Index (MI)35). The MI is a reliable and valid test36, 37). Walking ability was evaluated using the Functional Ambulation Category (FAC), which includes walking on uneven terrain and walking up and down stairs38). Walking ability was also evaluated using gait speed in the 10-m walk test39, 40). For the 10-m walk test, participants walked in a straight line at a comfortable speed for 16 m, including 3-m runways at the start and end of a 10-m test walkway. Gait speed was calculated from the time required to walk across the 10-m walkway. The participants completed these tasks at a comfortable speed and used their usual walking aids and ankle-foot orthoses. All tests were examined by one rater on the same day.

A preliminary reliability study was performed to modify the CAT-STS. This was followed by a reliability study. The patients who were used for the preliminary reliability study and the reliability study were a subset of the 27 participants described above.

In the preliminary reliability study, two physical therapists who were not involved in the development of the CAT-STS evaluated the turning of 10 hemiparetic patients with stroke from videotape recordings. The duration of clinical experience of the two physical therapists was 9.5 years and 16.5 years. The 10 patients with stroke were selected from the total sample of 27 patients. Evaluations were conducted twice with an interval of ≥2 weeks between evaluations. Rating guidelines were modified when the agreement rate between the main researcher and one or both of the physical therapists was <80%, or the agreement rate between the physical therapists was <80%. The former indicated that the concepts of the main researcher were not reflected in the rating guidelines. The item “type of movement direction and space” was removed from the CAT-STS after the preliminary reliability study because the agreement rate was <60%. The seven items included in the final CAT-STS are shown in Appendix 1.

In the reliability study, two other physical therapists evaluated the turning of 10 patients with stroke from videotape recordings. The duration of clinical experience of the two physical therapists was 7.6 years and 8.6 years. Evaluations were conducted twice with an interval of ≥2 weeks between evaluations.

Construct validity was evaluated because a gold standard measurement to evaluate turning strategy has not been reported41). The videos of all 27 participants were used to examine construct validity. One researcher (M.K.) evaluated the CAT-STS, and the time and number of steps required to turn, from the videotape recording. Construct validity was evaluated using the associations between the total CAT-STS score and the time and number of steps required to turn, the MI, the FAC, and the gait speed. In addition, comparisons of turning performance among the CAT-STS items were conducted: forward vs backward in “direction,” more than twice shoulder width vs between one and two shoulder widths vs less than one shoulder width in “use of space,” side step vs cross step in “foot movement,” and yes vs no in “initiation,” “termination,” “instability,” and “non-fluidity.”

Data were statistically analyzed using SPSS version 22.0 J for Windows. Percentage agreement and kappa coefficient were used to evaluate intra- and inter-rater reliability for each item of the CAT-STS. The kappa coefficient was interpreted as follows: <0.2, poor agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, good agreement; and 0.81–1.0, excellent agreement42). Intraclass correlation coefficients (ICC) for model 1,1 and model 2,143), were also used to evaluate the reliability of the total CAT-STS score. The ICC was interpreted as follows: 0.0–0.2, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.0, excellent42). The internal consistency of the CAT-STS was assessed using Cronbach’s alpha44). The associations of the total CAT-STS score with the time and number of steps taken to turn, the MI, the FAC, and the gait speed were assessed using Pearson’s correlation coefficient or Spearman’s rank correlation coefficient. An independent t-test was used to compare the time and number of steps taken to turn among the CAT-STS items: forward vs backward in “direction,” side step vs cross step in “foot movement,” and yes vs no in “initiation,” “termination,” “instability,” and “non-fluidity.” A one-way Analysis of variance (ANOVA) and Tukey’s post hoc test were used to compare the time and number of steps taken to turn across participants with scores of 1, 2 and 3 on the “use of space” item. The level of significance was set at p < 0.05.

RESULTS

Ten of the 27 participants were selected for reliability testing. These participants were selected without bias of performance in turning or walking ability. Their mean (standard deviation) age was 63.8 (8.6) years and time since stroke onset was 1691.5 (848.1) days (Table 1). The agreement rate for intra- and inter-rater reliability was 60–100% (Table 2). The kappa coefficients were <0.6 for “direction” and “termination” items. The ICC (1,1) and ICC (2,1) of the total CAT-STS score was 0.865 and 0.725, respectively. Cronbach’s alpha for turns in the paretic and non-paretic direction was 0.756 and 0.611, respectively.

Table 1.

Characteristics and turning performance of the 10 participants included in the reliability analysis and the 27 subjects included in the validity analysis

			Reliability analysis (n=10)		Validity analysis (n=27)
			mean	SD	mean	SD
Age (years)			63.8	8.6	69.7	11.2
Male / Female (n)			9 / 1		17 / 10
Duration of stroke onset			1,691.5	848.1	2,014.9	1,302.7
Height (cm)			166.1	4.3	161.3	9.3
Weight (kg)			66.9	10.7	58.3	13.7
Type of stroke: Ischemic/ Hemorrhage/ Subarachnoid hemorrhage (n)			3 / 6 / 1		12 / 12 / 3
Motricity index			60.6	20.5	59.0	23.0
Gait speed (m/s)			0.43	0.32	0.39	0.25
Cadence (steps/min)			81.0	34.8	85.1	32.4
Stride length (cm)			57.4	24.7	51.8	21.5
FAC*			4	[2–4]	4	[2–4]
Turning performance	Times (s)	Paretic direction	8.5	5.8	8.9	4.5
		Non-paretic direction	8.0	4.7	8.9	4.2
	Number of steps	Paretic direction	12.0	7.8	12.3	7.4
		Non-paretic direction	9.7	3.2	11.3	5.1

SD: standard deviation, FAC: Functional Ambulation Category. *Data in FAC is median [first-third quartile]

Table 2.

Intra- and inter-rater reliability for each item of the CAT-STS in 10 participants

	Intra-rater reliability		Inter-rater reliability
	Agreement (%)	Kappa	Agreement (%)	Kappa
Direction	0.750	0.472	0.750	0.467
Use of space	0.850	0.754	0.650	0.430
Foot movement	1.000	-	1.000	-
Initiation	0.875	0.695	0.725	0.385
Termination	0.700	0.318	0.700	0.324
Instability	0.850	0.681	0.600	0.221
Non-fluidity	0.925	0.846	0.875	0.746
	ICC (1,1)		ICC (2,1)
Total score	0.865		0.725

All of the 27 participants were used for construct validity analysis (Table 1). The time and number of steps taken to turn in the paretic direction were significantly different in one-way ANOVA for the “use of space” item (F = 5.591, p = 0.01 and F = 3.958, p = 0.033 for time and number of steps, respectively). Similarly, the time taken to turn in the non-paretic direction was significantly different in one-way ANOVA for the “use of space” item (F = 5.609, p = 0.01). The time taken to turn was significantly shorter for participants who were rated ‘Nos’ in the “termination,” “instability,” and “non-fluidity” items than in participants who were rated “Yes” in these items (Table 3). The number of steps taken to turn was significantly fewer for participants who were rated “backward” in the “direction” item than in participants who were rated “forward.” All participants were rated “side step” in the “foot movement” item.

Table 3.

Comparison of turning performance in each item of the CAT-STS for construct validity analysis in 27 participants

		Paretic direction					Non-paretic direction
		n	Time (s)		Number of steps		n	Time (s)		Number of steps
Direction	Forward	19	9.9	(4.8)	14.2	(7.8)*	22	9.6	(4.2)	12.1	(5.2)
	Backward	8	6.6	(2.8)	7.6	(3.0)*	5	5.6	(2.6)	8.0	(3.7)
Use of space	Less than one shoulder width	10	5.7	(2.1)††	7.7	(3.3)	9	5.8	(2.7)††	9.0	(6.3)
	Between one and two shoulder widths	14	10.7	(4.7)††	14.4	(7.6)	12	9.7	(4.4)††	12.8	(4.5)
	More than twice shoulder width	3	11.4	(3.6)	17.3	(10.2)	6	11.8	(3.0)	12.0	(3.8)
Foot movement	Cross	0	-	-	-	-	0	-	-	-	-
	Side	27	-	-	-	-	27	-	-	-	-
Initiation	No	22	8.5	(4.7)	10.6	(5.9)	21	8.4	(3.0)	10.8	(2.3)
	Yes	5	10.7	(3.4)	19.4	(9.8)	6	9.0	(4.6)	11.5	(5.7)
Termination	No	19	7.4	(3.3)**	10.2	(5.6)*	17	7.9	(4.2)	9.9	(4.0)
	Yes	8	12.6	(4.9)**	17.3	(9.0)*	10	10.5	(4.0)	11.5	(6.2)
Instability	No	15	6.1	(2.2)**	8.9	(3.5)*	13	6.3	(2.6)**	9.0	(2.6)*
	Yes	12	12.4	(4.2)**	16.5	(8.8)*	14	11.3	(4.1)**	13.5	(6.1)*
Non-fluidity	No	9	4.7	(1.1)**	7.8	(3.6)*	9	4.5	(0.8)**	7.6	(2.5)**
	Yes	18	11.1	(4.0)**	14.5	(7.8)*	18	11.1	(3.4)**	13.2	(5.1)**

Values are mean (standard deviation).

*Significant difference between “forward” and “backward” or “no” and “yes” (p < 0.05)

**Significant difference between “forward” and “backward” or “no” and “yes” (p < 0.01)

†Significant difference between “less than one shoulder width” and “between one and two shoulder widths” (p < 0.05)

††Significant difference between “less than one shoulder width” and “between one and two shoulder widths” (p < 0.01)

The mean (standard deviation) of the total CAT-STS score was 9.7 (1.8) for turns in the paretic direction and 9.3 (1.7) for turns in the non-paretic direction. The total score was significantly correlated with the time and number of steps taken to turn, the FAC, and the gait speed (Table 4).

Table 4.

Relationships between the total CAT-STS score and measures of physical function in 27 participants

	Turning performance		Motricity index	FAC	Gait speed (m/s)
	Time(s)	Number of steps
Paretic direction	−0.754**	−0.660**	0.261	0.580**	0.695**
Non-paretic direction	−0.724**	−0.495**	0.512**	0.758**	0.820**

FAC: Functional Ambulation Category. *p < 0.05. **p < 0.01

DISCUSSION

The aim of this study was the development of a CAT-STS for evaluating turning while standing. The items included in the CAT-STS were considered to reflect the turning strategies that may be used by elderly individuals, patients with hemiparesis, and patients with Parkinson’s disease. The items entitled “use of space” and “initiation” were designed to the capture characteristics of patients with Parkinson’s disease15, 25). The items entitled “termination,” “instability,” and “non-fluidity” were designed to capture the characteristics of elderly fallers14, 26). Therefore, we believe that the CAT-STS will be widely useful for various populations.

Intra-rater reliability was good to excellent, except for the items “direction” and “termination.” Turning while standing is different from walking, and movement of the center of gravity is limited. Participants with a high level of turning performance were able to turn in a small space. Thus, the assessment of “direction” might be difficult for raters, and intra-rater reliability was moderate for the item “direction.” The kappa coefficient for the item “termination” was 0.318, though agreement was 70%. In previous studies, intra-rater reliabilities of analysis of observational assessment and video-based assessment were poor to moderate45,46,47,48,49). Evaluation of stagger or slight adjustments in foot movement might have been difficult for therapists because these movements were small. However, ICC (1,1) of the total CAT-STS score was 0.865. Therefore, the CAT-STS has sufficient reliability when used in a clinical setting.

Inter-rater reliability for each item was poor. Previous reviews have reported that visual analysis using a videotape has poor-to-moderate inter-rater reliability45,46,47). In this study, agreement ratio was greater than 60% for each item, but kappa coefficients were lower than 0.6 for some items. This may be due to different determination criteria used by each therapist. However, ICC (2,1) of the total CAT-STS score was 0.725. Therefore, the total CAT-STS score has sufficient reliability.

Construct validity was evaluated using the associations of the total CAT-STS score with the MI of the paretic lower limb and with walking ability. The total CAT-STS score was strongly associated with the time taken to turn and the gait speed, and moderately associated with the number of steps taken to turn and the FAC. In addition, the items “use of space,” “termination,” “instability,” and “non-fluidity” discriminated differences in the time and number of steps taken to turn. These results indicate that the CAT-STS evaluates turning performance and has sufficient construct validity.

Turning strategies were categorized into two types: step turn and spin turn50). Step turn involves a change in direction opposite to the stance limb, and spin turn involves a change in direction towards the stance limb. Step turn is convenient and stable because it involves a wide base of support while changing direction51). Taking side steps also involves a wide base of support. In this study, the walking ability of our participants was poor, as reflected by a mean gait speed of 0.38 m/s, and all participants used side steps to turn (quantified in the “foot movement” item).

A limitation of this study is the poor reliability for some items of the CAT-STS. We carefully designed the items included in the CAT-STS after many discussions and revisions. Nevertheless, intra-rater reliability and inter-rater reliability were poor for some items. Evaluations of movement strategy, which could be conducted easily using a two- or three-point rating scale, might have a limitation of reliability for each CAT-STS item. In addition, the use of video-based evaluation may have affected the reliability because raters were able to repeatedly observe the video. As a further limitation, the participants in this study were patients with stroke. Thus, our results might not be generalizable to different subject groups. Further studies are required of the CAT-STS to determine the reliability and validity of this observational analysis tool, to determine the usefulness of this tool for different subject groups such as patients with Parkinson’s disease, and to determine the relationship between turning strategy and fall history.

To conclude, we developed an assessment tool (CAT-STS) and quantified the intra-rater reliability, inter-rater reliability, and construct validity of this tool. The intra- and inter-rater reliabilities of the total CAT-STS score were substantial and excellent, respectively. Construct validity was shown by the associations of the CAT-STS score with turning performance and walking ability. The CAT-STS can be conducted easily in a short time, and this scale will be useful for evaluating the strategy used to execute a standing turn in a clinical setting.