Reliability and validity of transfer assessment instrument version 3.0 in individuals with acute spinal cord injury in early rehabilitation phase.

BACKGROUND: Transfers are very important in functional activities of subjects with spinal cord injury (SCI). The transfer assessment instrument (TAI) was the first tool to standardize the assessment of transfer technique.
OBJECTIVE: The purpose of this study was to establish the reliability and validity of TAI 3.0 in people with SCI in early rehabilitation phase.
METHODS: Thirty subjects with acute traumatic SCI were recruited from a tertiary care center for SCI management. Four raters assessed the quality of transfer using TAI 3.0 and a fifth rater used global assessment of transfer scale (VAS). TAI 3.0's intraclass correlation coefficient (ICC) for intrarater and interrater reliability, standard error of measurement (SEM), minimal detectable change (MDC), limits of agreement and concurrent validity was determined.
RESULTS: The intrarater ICC was 0.93 to 0.98 and interrater ICC was 0.99, indicating high levels of reliability. The SEMs among the raters for TAI 3.0 total was from 0.23 to 0.28. The MDC among the raters TAI 3.0 total was from 0.54 to 0.86. Correlation for different raters between the TAI 3.0 and VAS ranged between 0.88 and 0.90.
CONCLUSION: TAI 3.0 is a reliable and valid tool to assess the transfer skill in individuals with SCI in early rehabilitation phase.

Introduction

Wheelchairs are the primary mode of mobility in individuals with spinal cord injury (SCI). The transfers are vital in performing activities of daily living in wheelchair users. A transfer is considered as a movement of oneself from one surface to another in one direction. If a full-time wheelchair user is unable to transfer, or transfers become more difficult, the individual’s quality of life may be further affected.[1]

Transfers often cause excessive loading of the arms. This may lead to upper extremity pain and injuries, such as rotator cuff tears, elbow pain, and carpal tunnel syndrome.[2] The transfers may also place joints of upper limb in extreme positions and expose to high internal joint forces. Therefore, it seems that prevention may be critical in reducing upper limb pain and overuse injuries. Learning the transfer methods that reduces forces and awkward joint motions is an important strategy for preventing impairments in upper limb of wheelchair users.[3,4]

There is a wide variation in the amount and the type of transfer training and no uniform way to evaluate transfer quality in early stage of rehabilitation after SCI. Observation by a therapist and qualitative assessment was the standard method of evaluating transfers. Therefore, McClure et al. developed a tool called transfer assessment instrument (TAI) to assess the quality of transfer in full-time wheelchair users. It is a safe and easy tool to administer outcome measure to assess transfers in wheelchair users. It was found to have an acceptable interrater and intrarater reliability.[5] Tsai et al., introduced the refined TAI version 3.0. It was found to have high reliability (0.74 to 0.88) among raters of different clinical backgrounds and experience in people using wheelchairs for more than a year.[6]

In early stage of rehabilitation following SCI, transfer evaluations are not done objectively and in a consistent manner. The transfer assessment can be influenced by the subjective experience of the therapists. It may lead to less accurate evaluations and variability in transfer skill assessment. TAI was the first tool to standardize the evaluation of transfer technique. The items included in the TAI were based on clinical practice guidelines, available information on literature, and best clinical practices for transfers.[5] There is a lack of valid assessment tool to evaluate transfers in early stage of rehabilitation following SCI. Therefore, the purpose of this study was to establish the reliability and validity of TAI 3.0 in people with SCI in early stage of rehabilitation.

Methods

Sample

A sample of convenience of 30 subjects with acute traumatic SCI who met the inclusion criteria and were willing to participate in the study was included. The sample was selected from rehabilitation department of Indian spinal injuries center, New Delhi. The subjects were in the early rehabilitation phase. The transfer training was started one week before the date of evaluation. The design was methodological research-repeated measure study. The study was approved by institutional ethical committee where the study was carried out.

Sample size

A sample size of 30 subjects with 2 observations per subject achieves 91% power to detect an intraclass correlation of 0.70 under the alternative hypothesis when the intraclass correlation under the null hypothesis is 0.30 using an F-test with a significance level of 0.05 was required for reliabilty testing. A sample size of 25 achieves 81% power with the alternative hypothesis correlation of 0.60 by using a two-sided hypothesis test with a significance level of 0.05 which was needed for validity testing.[7,8] The sample size was calculated using PASS 2008 software.

Subjects included were first time manual wheelchair users who were potential full-time wheelchair users with tetraplegia, high paraplegia (T2–T7) or low paraplegia (T8–L4). They were able to sit with or without hand support for 30 s. Subjects with following problems were excluded, such as, unhealed pressure ulcers, score on Beck depression inventory-II,[9,10] wheelchair user’s shoulder pain index score ,[11] weight relief raises, musculoskeletal deformities of upper extremities, unstable medical condition (e.g., angina, seizures), respiratory distress, cardiovascular, emotional or psychiatric problems and with significant visual impairments.

Procedure

The subjects were recruited by convenient sampling method who met the inclusion criteria. The purpose of the study was explained to the subjects who gave the consent to participate in the study. Then their age (years), gender, height (cm), weight (kg), level of injury and American spinal injury association (ASIA) impairment level,[12] area of body affected, spinal cord independence-III measure (SCIM) score,[13] modified functional reach test score,[14] type of transfer, duration since injury (months), body mass index (BMI) were collected.

Five physiotherapists who had an experience of 3–4 years in rehabilitation of the people with spinal cord injury (SCI) were the raters for transfers done by the participants. Four raters evaluated the transfer skill of participants on TAI 3.0 while the fifth therapist evaluated the transfer on global assessment of transfer scale (VAS). Before the testing, the raters were explained how to administer TAI 3.0 using the text instruction manual of the assessment tool. The instructions contained the details and how to score each item. In addition, general recommendations were provided including where to stand when observing different aspects of transfer. The raters were also told about the instructions to be given to participants during transfers. The study investigator ensured that each rater was trained properly using TAI 3.0.[6] The raters got individualized instruction on items with difficulty in scoring.

Participants were asked to perform four transfers each in two sessions (sessions 1 and 2). The transfers were done from their own wheelchair, to and fro from a wheelchair level surface bed or a height adjustable hospital bed with their usual way of transfer. The height of the hospital bed was adjusted, depending on the individual’s preference. An attendant was there with subject to prevent any fall or provide assistance during transfers. If participants needed assistance for transfers, they were permitted to use transfer device (e.g., transfer board or lift) or the attendant to provide assistance. In session 1, while participants performed transfer, four raters used TAI 3.0 to score and evaluate their transfer skills. For session 2, all participants were asked to return after 72 h to perform the transfer again. Then the same four raters evaluated them for second time using TAI 3.0. Each session lasted for 30 to 45 min per subject.

TAI scoring (TAI 3.0)

The TAI contains two parts. Part 1 comprises of 15 items and is scored as follows: “yes”, 1 point, “no”, 0 points or “not applicable” (N/A), which means a removed item. Part 1 is completed after each transfer and item scores are averaged to produce a single representative item score. The part 1 is the summation of each item’s score multiplied by 10 and then divided by the number of applicable items, ranging from 0 to 10. The items in part 2 are completed after all transfers have been performed. The 12 items in part 2 are scored on a likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). The part 2 score is the summation of each item’s score multiplied by 2.5 and then divided by the number of applicable items, resulting in a range of scores from 0 to 10. The final score of TAI is the average of the part 1 and part 2. The items of the instrument and what is evaluated during a transfer are given in Table 1. All the recruited subjects completed both the sessions (session 1 and session 2) of the study.[6]

Table 1.

The items and evaluation component of the TAI 3.0.

Item no.			What is being evaluated
Part 1		1	Distance between the wheelchair and object to which he/she is transferring on to. The subject’s wheelchair is within 3 inches of the object to which he is transferring on to.
		2	The angle between the subject’s wheelchair and the surface to which he is transferring.
		3	Whether the subject attempts to place his chair to perform the transfer forward of the rear wheel.
		4	If possible, the subject removes his armrest independently or with assistance.
		5	Level or downhill transfer.
		6	Placement of feet in a stable position.
		7	Scoots to the front edge of the wheelchair seat before he transfers.
		8	Hands’ position.
		9	Handgrip of the leading arm.
		10	Handgrip of the trailing arm.
		11	Control over flight.
		12	Head–hip relationship.
		13	Positioning of the lead arm.
		14	The landing phase of the transfer.
		15	The assistant supporting the subject’s arms during the transfer.
Part 2	Weight bearing arm position	1	The lead arm position.
	Set-up phase	2	Sets up for a safe and easy transfer.
		3	Change the height of the object he is transferring to/from to make the transfer level.
		4	Gets close to the object that he is transferring on to.
		5	Uses handgrips when necessary.
	Conservation	6	Uses a transfer device when necessary.
		7	Alternate the leading/trailing arm over the course of the assessment.
	Quality	8	Transfer is smooth and well controlled.
		9	Clearly communicate his needs in transfer.
		10	Does not allow the assistant to pull on his arms during the transfer.
		11	The subject corrects the assistant.
		12	The subject is able to correctly direct his care in an assertive and polite manner.

Global assessment of transfer scale

This is a likert scale which rates the overall transfer quality on a 10-point scale. The participants’ transfer from a wheelchair was evaluated and graded from poor (0) to excellent (10). The criteria for rating a transfer as poor were if the individual does not make use of equipment when needed, do not make transfer easy and safe and inappropriate placement of the hand and feet. A transfer was rated excellent if the transfer was appropriately done without transfer devices, easy and safe, placing the hand and feet on right places and using human assistance when need. During session 1, while four raters evaluated the transfer skill of the participants, the fifth therapist who has not seen the TAI rated the study participants transfer skill on a VAS. The VAS evaluation was done only in session 1.[5]

Data Analysis

Data analysis was performed using SPSS 21.0 software. Descriptive statistics were calculated for the subject’s demographic data including age, gender, type of transfer, type of disability, BMI. The intraclass correlation coefficients (ICCs) within each rater and between raters in part 1, part 2 and final TAI 3.0 scores were calculated to assess reliability. The limits of agreement (LOA) analysis were done by plotting Bland and Altman (B&A) plots with graph pad prism software (Prism version 6.00).[15] To determine variability of TAI 3.0 scores, standard error of measurement (SEM) and minimal detectable change (MDC) were analyzed. Statistical significance was set at with confidence interval of 95%. To establish convergent validity, Pearson correlation coefficients were calculated for each rater to evaluate the correlation of TAI 3.0 scores (total) with global assessment of transfer scores.

Reliability Testing

Interrater and intrarater reliability

For calculating intrarater and interrater reliability, ICC coefficient value (ICC 3,1 : two-way mixed effect and consistency) was calculated separately for part 1, part 2 and total score of TAI3.0. ICCs higher than 0.80 were considered strong, between 0.60 and 0.79 were acceptable, between 0.40 and 0.59 were moderate, and lower than 0.40 were weak.[16]

Standard error of measurement

The SEM provides a value for measurement error in the same units as the measurement itself, it is a measure of absolute reliability.[17] The SEM was calculated for part 1, part 2 and final TAI scores using the formula: , where SD is the standard deviation of the dataset and r is the reliability coefficient or ICC value. The SEM was calculated for individual rater based on ICC values in case of intrarater reliability analysis.

Minimal detectable change

The MDC was calculated for individual rater for intrarater reliability analysis. It is an estimate of the smallest change in score that can be detected objectively for a subject, it is the amount by which a subject’s score needs to change and ensure that the change is greater than measurement error.[18] MDC was analyzed based on 95% confidence interval ( SEM) for part 1, part 2 and final TAI scores.

Limits of agreement

LOA between session 1 and 2 were determined by B&A method.[19]

Validity Testing

Convergent validity

Pearson correlation coefficient was calculated for each rater by correlating their final TAI 3.0 scores with global assessment of transfer scores (VAS) of session 1.

Results

Demographic characteristic of the subjects, mean SD, such as age (years), height (cm), weight (kg), BMI, SCIM score, modified functional reach test score, and frequency distribution of area of body affected, ASIA impairment level, gender, type of transfer, and duration since injury are shown in Table 2. The mean SD of part 1 (TAI 1), part 2 (TAI 2) and total score (TAI total) of TAI at two different time points for all four raters is given in Table 3. The mean SD global assessment of transfer scores was 7.9 1.48.

Table 2.

Demographic characteristics of the sample.

Variable		Mean ± SD/n (%)
Age (years)		31.9 ± 12.3
Height (cm)		163.78 ± 9.64
Weight (kg)		62.61 ± 13.49
BMI (kg/m2)		23.33 ± 2.04
Duration(months)		1.33 ± 0.47
SCIM-III score		40.32 ± 11.64
mFR (cm)		10.27 ± 3.05
Gender	Male	25 (83.3)
	Female	5 (16.7)
Area of body affected	Tetraplegia	6 (20)
	High paraplegia (T2–T7)	5 (16.7)
	Low paraplegia (T8–L4)	19 (63.3)
ASIA impairment level	A	11 (36.7)
	B	11 (36.7)
	C	5 (16.7)
	D	3 (10)
Type of transfer	Independent sitting pivot	20 (66.7)
	Assisted sitting pivot	10 (33.3)

Notes: SCIM: spinal cord independence measure; mFR: modified functional reach; BMI: body mass index; SD: Standard deviation, : number.

Table 3.

Mean SD of TAI at two different time points.

	Time 1			Time 2
Raters	TAI 1	TAI 2	TAI total	TAI 1	TAI 2	TAI total
1	8.42 ± 1.56	7.62 ± 1.90	7.91 ± 1.68	8.30 ± 1.45	7.764 ± 1.89	8.03 ± 1.61
2	8.05 ± 1.57	7.571 ± 1.81	7.81 ± 1.65	8.23 ± 1.43	7.77 ± 1.85	8.00 ± 1.60
3	8.21 ± 1.45	7.63 ± 1.89	7.92 ± 1.61	8.23 ± 1.52	7.83 ± 1.80	8.03 ± 1.61
4	8.12 ± 1.69	7.71 ± 1.87	7.91 ± 1.71	8.16 ± 1.52	7.96 ± 1.78	8.06 ± 1.60

Notes: TAI 1: Part 1 of TAI; TAI 2: Part 2 of TAI; TAI total: TAI total score.

Reliability

The intrarater ICCs ranged from 0.93 to 0.95 for TAI 1, 0.97 to 0.98 for TAI 2 and TAI total, suggesting high levels of reliability (Table 4). The interrater ICC of TAI 1 at first time was 0.98 and for TAI 2 and TAI total was 0.99 and for the second time, ICC of TAI 1, TAI 2 and TAI total was 0.99 (Table 5).

Table 4.

Intrarater reliability analysis for TAI 3.0.

	ICC (95 CI)			MDC			SEM
Raters	TAI 1 (95 CI)	TAI 2 (95 CI)	TAI total (95 CI)		TAI 2	TAI total		TAI 2	TAI total
1	0.95 (0.89–0.97)	0.98 (0.94–0.99)	0.98 (0.94–0.98)	0.94	0.75	0.64	0.34	0.27	0.23
2	0.94 (0.88–0.97)	0.98 (0.96–0.99)	0.98 (0.94–0.98)	1.02	0.69	0.40	0.37	0.25	0.23
3	0.94 (0.86–0.96)	0.98 (0.96–0.99)	0.98 (0.94–98)	1.02	0.72	0.86	0.37	0.26	0.23
4	0.93 (0.85–0.96)	0.97 (0.94–0.98)	0.97 (0.93–0.98)	1.19	0.86	0.78	0.43	0.23	0.28

Notes: TAI 1: Part 1 of TAI; TAI 2: Part 2 of TAI; TAI total: TAI total score; ICC Intraclass correlation coefficient; MDC: minimum detectable change; SEM: standard error of measurement; CI: confidence interval.

Table 5.

Interrater reliability analysis for TAI 3.0.

	ICC (_95CI)		MDC		SEM
Item	Time 1	Time 2	Time 1	Time 2	Time 1	Time 2
Part 1	0.98 (0.97–0.99)	0.99 (0.94–0.99)	0.44	0.42	0.16	0.15
Part 2	0.99 (0.98–0.99)	0.99 (0.94–0.99)	0.53	0.50	0.19	0.18
Total	0.99 (0.96–0.99)	0.99 (0.96–0.99)	0.47	0.44	0.17	0.16

SEM and MDC

The SEM among the raters for TAI 1 was from 0.34 to 0.43, TAI 2 was from 0.23 to 0.27 and for TAI total was from 0.23 to 0.28. The MDC among the raters for TAI 1 ranged from 1.19 to 0.94, TAI 2 was from 0.69 to 0.86 and for TAI total was from 0.54 to 0.86 (Table 3). Between the raters, SEM and MDC is given in Table 5.

LOA Plots

The LOA plot showed that there was 1 data point for part 1 and part 2 outside 1.96 SD (Figs. 1 and 2) and for total score, there was 2 data points outside 1.96 SD (Fig. 3).

Fig. 1.

B&A plot of agreement between sessions 1 and 2 for part 1 score. The figure reveals that only one data point lies outside 1.96 SD.

Fig. 2.

B&A plot of agreement between sessions 1 and 2 for part 2 score. The figure reveals that only one data point lies outside 1.96 SD.

Fig. 3.

B&A plot of agreement between sessions 1 and 2 for total score. The figure reveals that only two data points lie outside 1.96 SD.

Pearson correlation coefficients ranged from 0.88 to 0.90 among the raters with the VAS score () as given in Table 6.

Table 6.

Correlation of total TAI score with the criterion standard (VAS).

Rater	Pearson correlation coefficient	p
1	0.89	0.001
2	0.89	0.001
3	0.88	0.001
4	0.90	0.001

Discussion

Measurements of functional outcomes are an integral part of any goal-orientated, interdisciplinary rehabilitation program. It is important for quantifying the success of rehabilitation program. A good clinical assessment tool should be both reliable and valid.[20,21] The study results showed that TAI 3.0 has higher levels of intrarater (ICCs ranged from 0.93 to 0.98) and interrater reliability (ICC — 0.99). For convergent validity, correlations ranged between 0.88 and 0.90. Results showed significant correlations between TAI and global assessment of transfer skills. The findings are similar, but psychometric properties appear better than the previous reliability and validity analysis done in wheelchair users with various disabilities with the same scale.[5,6] The reason may be that all the subjects were with SCI and from the same rehabilitation care setup. This might have made the sample more homogenous and another component might be the uniformity in transfer training.

The reason for choosing ICCs to analyze the reliability of the TAI is because the ICC measures the association and agreement. The ICC can be used to assess reliability for more than two raters and can be used to analyze ordinal type of data.[6] The mean difference determined by LOA analysis was very small. Only total of four data points (one each for parts 1 and 2 and 2 for total score) were outside the 95% confidence interval limits. Hence, there was an agreement with the two sessions of measurements.[22] B&A plot is the quantification of the agreement between two measurements by plotting it graphically, the mean difference and constructing LOA. The difference of the two paired measurements is plotted against the average of the two measurements. The LOA recommended by B&A is that 95% of the data points should lie within 2SD of the mean difference. The part 1 and part 2 scores meet the LOA criteria whereas total score is slightly less at 93.33%.[19,23]

Scores may vary, given expected variability of individual performance and measurement error. A measure of absolute variability provides useful information to delineate the “expected” changes from “true” changes in performance. Statistically, absolute reliability is determined by the SEM. Clinically useful mechanism for looking at absolute reliability is the MDC score.[24] Results showed that SEMs ranged from 0.23 to 0.28 within raters and 0.15 to 0.19 between raters. The smaller the SEM, the more accurate are the assessments that are being made. The smaller SEM in this study further indicates the accuracy of measurements with TAI 3.0.[25] This study found that the MDC ranged from 0.64 to 0.86 within raters and 0.47 to 0.44 between raters. Minimum of 0.86 point change would be needed to identify a true difference in transfer skills that is not a measurement error. The MDC is relatively easy to calculate which provides clinically relevant information. The limitation of MDC is that it assumes that detectable changes are uniform throughout the scale, but the measurement error can vary at different points in the scale.[25] The raters were given handouts with an explanation of each item, a description of different scoring scenarios and a short practice session. Instructions administering TAI 3.0 might have improved the consistency among raters.[6]

Currently, no other outcome measure exists to assess transfer quality in population with SCI population in early rehabilitation phase. Hence, global assessment (VAS) was used to evaluate convergent validity. Using a non-validated tool to evaluate convergent validity is not only a preferred option, but also cannot be avoided because of a lack of a comparable criterion standard. The results showed an excellent convergent validity. The VAS was also previously used to establish the concurrent validity of TAI.[5]

The objective evaluation of transfers may help the clinicians to improve the transfer training, identify and correct improper transfer techniques. The identification of improper transfer techniques may prevent musculoskeletal injuries and pain in the upper extremities. The evaluation was done only on participants in early phase of rehabilitation, who performed independent or assisted sitting pivot transfers, so results cannot be generalized to all full-time wheelchair users. Future researches should be done to find out the effect of transfer training program on changes in TAI score in people with SCI.

Conclusion

The TAI is a reliable and valid tool which can be used as an outcome measure to evaluate transfer quality in people with acute SCI in early rehabilitation phase.

Conflict of Interest

We hereby declare that there is no conflict of interest involved in this study in terms of monetary benefits or in any other form.

Funding/Support

The study did not receive any funds and it was supported by Indian Spinal Injuries Center, New Delhi, India, by providing space to carry out the study. We acknowledge the support of Chitra Kataria, Principal, and all the supporting staff of ISIC institute of rehabilitation sciences, New Delhi, India for this study.

Author Contributions

The conception and design of the study, or acquisition of data were made by Preeti Baghel, Shefali Walia and Majumi. The analysis and interpretation of data were carried out by Shefali Walia and Majumi. Drafting the paper or revising it critically for important intellectual content was carried out by Preeti Baghel, Shefali Walia and Majumi. All authors approved the final submitted version of the manuscript.