A study of upper extremity training for patients with stroke using a virtual environment system.

[Purpose] A virtual environment (VE) system was designed to facilitate the retraining of motor control by feedback of movement trajectory to patients with neurological impairments, such as stroke victims or those with an acquired brain injury. In this study, we quantitatively assessed motion trajectory of the upper extremity during VE in order to further understand the effect of paralyzed upper extremity movement in VE for each patient as well as the functional clinical evaluations.
[Subjects and Methods] Six patients with stroke were participated in this study. The VE system consisted of a computer, an electromagnetic motion tracking device, which monitored and displayed patient movement on the computer, and the VE software, which provided the tools for creating training scenes. This system was used to facilitate motor relearning of the upper extremity for six patients with stroke.
[Results] The results showed there were improvements in variability and accuracy of the arm movement in motion trajectory. In addition, the scores of clinical evaluations improved, and VE did not hinder motor relearning of the upper extremity, which is necessary for activities of daily living.
[Conclusion] This VE system might be effective at facilitating motor relearning of the upper extremity for stroke patients.

INTRODUCTION

The application of engineering technology to rehabilitation has been studied in recent years1), and rehabilitation robots2, 3) play a major role in this. While there are reports on the effect of training using rehabilitation robots, the analyses and clinical indications for the utilization of versatile robots are still in the initial stage of research. Among the efforts to apply engineering technology to rehabilitation is a training system using a virtual environment (VE) developed at the Massachusetts Institute of Technology as a prototype system4,5,6,7,8,9). This system promotes motor learning in the paralyzed upper extremities of patients with central nervous system (CNS) dysfunction. The system shows a patient’s movements in real time on a computer display together with trajectories of various sample movements. The sample movements are called teachers, and the patient imitates the movement of the teacher.

The effects of VE training have been reported4, 6,7,8) using clinical evaluations including the Fugl-Meyer Assessment (FMA)9), the Wolf Motor Function Test (WMFT)10), and the Functional Independence Measure (FIM)11) of activities of daily living (ADL). However, the effect of VE training on the trajectory of paralyzed upper extremity movements has been evaluated quantitatively in detail by few investigators7, 12). Therefore, it is necessary to clarify the influence of VE training on paralysis of the upper extremities and to investigate the possibility of its clinical application. As a preliminary study, we investigated:1) the 3-dimensional coordinate data of the trajectory of movement recorded in VE training, to determine whether the effect of this training in promoting motor retraining of paralyzed upper extremities can be shown quantitatively; and 2) whether the possibility of clinical application could be shown by investigating the relationship between the improvement in movement of paralyzed upper extremities realized by VE training, and functional and ADL evaluations.

In this study, the 3-dimensional coordinate data recording the trajectory of movement for a doughnut task, which is a reaching task, during VE training, was analyzed. Comparisons were then made between the results of the 3-dimensional coordinate data analysis and clinical evaluations at the start and completion of VE training.

SUBJECTS AND METHODS

Six patients with stroke with no clear dementia or higher brain dysfunction underwent VE training at Sapporo Shuyukai Hospital. The content of the study was explained in writing to all subjects, and their written consent to participate was obtained. The subjects were 2 males and 4 females with a mean age of 57.6±9.5 years. All were right-handed. The diagnosis was left cerebral hemorrhage in 1 patient and left cerebral infarct in 5 patients. The mean number of days from onset to the start of VE training was 47.8±31.2 (Table 1). This research was part of a study that obtained approval from the ethics committees of the University of Tokyo and Sapporo Shuyukai Hospital.

Table 1.

Subject data

Subject	Sex	Age	diagnosis	Period between VE start and onset (days)	Trials for VE (times)
1	Female	63	Left cerebral infarction	33	12
2	Male	45	Left cerebral infarction	20	16
3	Male	66	Left cerebral infarction	60	42
4	Female	49	Left cerebral infarction	101	22
5	Female	52	Left putaminal hemorrhage	8	15
6	Female	71	Left pontine infarction	65	30

To investigate the relationship between VE training and clinical condition, function was evaluated with the upper extremity portion of the FMA9) and WMFT10), and ADL was evaluated with FIM11), before and after the VE training. The upper extremity portion of the FMA is a comprehensive evaluation of recovery that includes range of joint motion, pain, sensory and motor function tests. Items related to the upper extremities (total of 126 points possible) were used. The WMFT evaluates motor quality and performance of the upper extremities. Fifteen tasks performed by the subjects were scored (0–5 points, total of 75 points) and the time required (sec) was measured. The FIM has 13 motor items and 5 cognitive items (total of 126 points possible), and evaluates the level of independence in ADL.

The VE training system consists of a magnetic transmitter and a receiver, used as 3-dimensional position sensor to measure a subject’s movement, an object the subject held while performing the movement, and software that updates the images shown on the desktop display to match the recorded information. The magnetic transmitter is fixed on the desk, and the magnetic receiver is implanted into the grasped object so that the object’s movement in space corresponds to the movement of the subject’s hand. Subjects sit in a chair and grasp the object associated with the task to perform the VE training task (Fig. 1).

Fig. 1.

VE training system and environment

a. A magnetic transmitter is fixed on a desk. A magnetic receiver is attached to the object. b. A subject sits in front of the desktop display and grasps the object for the training task.

In VE training, for each task, the ideal movement sought by the therapist and the movement of the subject are shown simultaneously on the display. The movement sought by the therapist is recorded in advance, and is called the teacher. The teacher is only a symbol that expresses the ideal movement of the upper extremity which a therapist selects. VE training involves various tasks, including doughnut, mailbox, and pouring, and the teacher is displayed for each task. The subjects are instructed to try to imitate the movement while watching the teacher, and the difference between the subject’s movement and the teacher is displayed as visual feedback.

In one VE training session, several trials of multiple tasks are conducted. The number of VE training sessions, selection of tasks, and number of trials can be adjusted by the therapist as necessary.

The trajectory of movement is recorded as a time series of a finite series of 3-dimensional coordinates. The coordinate data of each time point are drawn in order, so that the trajectory of the movement is displayed dynamically. The coordinate data and depicted images are called frames, and the frames are numbered with integers from the start to the end of the movement. For example, the doughnut task has a total of 148 frames, numbered from 0 to 147.

The coordinates are shown on 3 axes: X (right-left), Y (front-back), and Z (up-down). These are the positions at which the magnetic receiver receives the magnetic signal from the magnetic transmitter, recorded in centimeters. In the VE training system, the trajectory of movement can be seen qualitatively as a line or as consecutive points of the object that the patient is moving. In addition, the scores for differences in position, direction, orientation, speed, and rotation speed between the path of the teacher and the movement of the subject can be calculated and shown for each trial.

A doughnut task, which is a simple movement, was used as the VE training task in this study. In the doughnut task, the target can be easily reached with shoulder flexion and adduction of approximately 30 degrees. Specifically, in this task, the cube showing the subject’s hand position on the display moves from the starting position through the hole of an imaginary doughnut and then returns to the starting point, as if drawing a circle (Fig. 2). The purpose of the task is to begin combining movements using the flexion and extension synergies. Initially the subject uses mainly the extension synergy (shoulder adduction with elbow extension) combined with some flexion synergy (shoulder flexion and supination to neutral) to reach the target (doughnut hole). Then, to exit the hole and return to the start position without ‘hitting’ the doughnut, the subject must now use more flexion synergy combined with extension synergy by moving into shoulder abduction with flexion and external rotation, while still maintaining elbow extension to clear the side of the doughnut, before moving into elbow flexion with shoulder adduction to return to the start position (Fig. 2). The aim of the doughnut task is not passing through the hole according to the size of the hole, rather it is that a subject tries to mimic the teacher’s movement. The subjects in a chair 80 cm from the computer display used by the VE training system and grasp the object with the embedded magnetic receiver with the hand of the paralyzed side. The experimental starting position of the subject’s arm on the paralyzed side when performing VE training was 0 degrees at the shoulder joint, 90 degrees at the elbow joint, and the midpoint of supination and pronation of the forearm. Next, with the subject grasping the object, the therapist set the doughnut shown on the computer display so that it was in the center of the display and adjusted the movement speed of the teacher to match the physical function of the subject. The therapist explained that the subject should watch the teacher and imitate its movement. Then, as practice, the therapist manually guided the arm on the paralyzed side to perform the movement. The therapist decided the number of trials for the task for each VE training session, before VE training was conducted. The target number of trials for the task was at least 12. The average time of one trial was 15 minutes.

Fig.2.

A doughnut task; one of the simplest VE training tasks. The cubic frame corresponds to the starting position. A large doughnut (a) is the target for cubes to reach. Preceding cube (b) shows the teacher’s position, and the other cube (c) is the subject’s position, which moves virtually following the movement of the grasped object. The dotted curve (d) shows the teacher’s trajectory. The teacher starts moving from the starting position in the upper left direction, goes through the hole of the doughnut, and comes back to the staring position in the lower right direction. The dotted curve does not appear on the display during training tasks.

The first VE session was designated initial VE, and the last was designated final VE. Using the coordinate data recorded for each trial in a single VE training session, the change in the coordinates between the initial and final VE for the following 2 items was analyzed for: 1) variability of the trajectory movement of upper extremity for each subject when the teacher passed through the doughnut; and 2) accuracy, indicated by the differences between the movements of each subject and the teacher for the trajectory movement of the upper extremity when the teacher entered the doughnut hole, and when the teacher passed through the hole. The specific analysis procedures for each item are described below. First, the coordinate data at the time of the first and last trials were eliminated from the coordinate data recorded for both 1) and 2). Next, the coordinate data for the subject and teacher corresponding to the frame in which the teacher entered the doughnut and to the frame in which the teacher emerged from the doughnut were obtained by replaying the teacher movement frame by frame on the display and by confirming the position of the teacher. Then, the coordinate data of the subject that corresponded to the target positions of the teacher were obtained.

In 1), the mean value of the coordinate data of the subject at the time the teacher entered the doughnut (mean path) was obtained from the trials. Afterward, using the following Formula (1), the distance between the coordinate data (X, Y, Z) and mean path (X’, Y’, Z’) was obtained for each trial. Then the mean distance from the mean path among the trials was obtained .

In 2), replacing (X’, Y’, Z’) with the coordinates of the teacher (X’’, Y’’, Z’’) in the Formula (1), the distance between the coordinate data for each trial and the coordinates of the teacher (X’, Y’, Z’) was determined. Then, the mean distance from the teacher in the trials was calculated.

The differences in the mean values at the initial and final VE were determined for the FMA score, WMFT score and time (sec), and FIM score for each subject. The paired t-test was performed for WMFT times. The FMA scores, WMFT scores, and FIM scores were compared using Wilcoxon’s rank test. Similarly, the mean distances from the mean path and mean distances from the teacher path were compared for all subjects using the paired t-test. SPSS for Windows version 18 was used for the statistical analysis, with values of p<0.05 accepted as significant.

RESULTS

The results of the clinical evaluation at the initial and final VE are given for each subject in Table 2. For FMA and WMFT, the scores of all subjects rose and the number of seconds decreased. For FIM, the scores increased for all subjects except subject 4. The results averaged over all subjects for the clinical evaluation at the initial and final VE are shown in Table 2. The mean FMA score rose from 106 to 116. The mean WMFT score rose from 56.67 to 66.83, and the mean number of seconds decreased from 77.78 to 45.46. The mean FIM score rose from 110 to 118. Significant differences were found for all clinical evaluation items of FMA score, WMFT score and FIM score (p < 0.05).

Table 2.

Clinical evaluation of all subjects

	First VE			Final VE
Subjects	FMA score (points)	WMFT score (points)/time(seconds)	FIM score (points)	FMA score (points)	WMFT score (points)/time(seconds)	FIM score (points)
1	116	59 / 62.39	120	126	71 / 34.5	122
2	109	58 / 41.95	116	119	70 / 27.11	121
3	105	55 / 34.19	108	114	63 / 26.47	126
4	101	46 / 156	123	109	61 / 47	123
5	108	68 / 37.96	83	118	71 / 29.02	105
6	97	54 / 134.2	112	110	65 / 108.64	113
mean±SD	106±6.06	56.67±6.57 / 77.78±48.83	110.33±13.17	116±5.8	66.83±4.02/45.46±29.10	118.33±7.16

The results of the distance from the mean path and distance from the teacher are shown in Table 3 for each subject. The distance from the mean path decreased in 5 of the 6 subjects (all but subject 2). The distance from the teacher when the teacher entered the doughnut decreased in all subjects. The distance when the teacher emerged from the doughnut also decreased in 5 subjects (all but subject 5). The results averaged over all subjects for the distances from the mean path and the distance from the teacher are shown in Table 3. The mean and standard deviation for the distance from the mean path were 5.27±2.49 cm in the first VE training and 3.21±1.16 cm in the final VE training. The mean and standard deviation of the distances from the teacher at the time the teacher entered the doughnut were 14.22±6.17 cm in the first VE training and 9.87±4.85 cm in the final VE training. At the time the teacher emerged from the doughnut, the distance was 20.84±6.88 cm in the first VE training and 13.00±8.16 cm in the final training. Significant differences were found for the distance from the mean path and the distance from the teacher beween the initial and final VE (p<0.05).

Table 3.

The difference between each subject and the teacher in the trajectory of the upper extremity

Subjects	trials	Mean distance between each trajectory and average trajectory when the teacher entered the doughnut in each subject.		Mean distance between each subject and the teacher for the average trajectory of upper extremeties when the teacher entered the doughnut.		Mean distance between each subject and the teacher for the average trajectory of upper extremeties when the teacher emerged the doughnut.
		First VE mean±SD	Final VE mean±SD	First VE mean±SD	Final VE mean±SD	First VE mean±SD	Final VE mean±SD
1	18	3.57±2.26	1.48±0.66	12.55±3.68	5.43±1.04	22.08±2.55	6.98±1.33
2	18	2.43±0.98	2.42±1.01	8.30±1.00	7.92±1.76	14.47±1.81	10.92±1.42
3	11	4.85±1.99	3.39±1.08	24.66±3.47	14.69±2.31	33.45±3.51	25.81±2.95
4	13	4.17±2.91	3.62±1.61	9.04±3.31	3.98±2.00	21.65±4.97	8.05±2.58
5	11	7.88±4.60	4.91±1.41	18.00±8.81	15.61±1.64	16.91±7.35	20.32±2.54
6	10	8.71±4.16	3.46±1.71	12.74±4.11	11.58±1.29	16.46±5.41	5.92±2.31

unit: cm

DISCUSSION

FMA is a test of sensorimotor recovery, quality of movement and intralimb joint coordination, while WMFT is more of a functional evaluation and a measure of task execution time. That is, for FMA, the quality of the movement performance influences the score the most, whereas for the WMFT, whether a task is accomplished and how fast it is accomplished are the major factors influencing the score. The significant differences in FMA scores, WMFT scores and task times between the initial and final VE suggest that motor quality and performance had improved. This result reflects a relationship with the reaching movement that is an element of VE training. Normally, reaching movements of an upper extremity require movement of the shoulder joint, including the scapula, and the elbow joint, and grasping and opening movements of the fingers. In reaching movements of an upper extremity, the joint movement needed differs depending on the distance and direction of reach. Moreover, reaching movements of the upper extremities in virtual reality are reported to be performed with a similar movement strategy to that of reaching movements in real environments12). The doughnut task in VE training is a reaching task in the anterosuperior direction. Accordingly, the reaching movement of the upper extremity in the doughnut task is thought to be done with abduction and anterosuperior rotation of the scapula; flexion, abduction, and inner rotation of the shoulder joint; and extension of the elbow toward the reaching target. Therefore we conclude that elements of the reaching movement of the upper extremity in VE training influenced the improvements in FMA and WMFT. Elements of the reaching movement of the upper extremity in VE training are related to various actions in daily living, including grasping a cup, turning faucets on and off, washing one’s hands, unrolling toilet paper, and washing the non-paralyzed side. In the present study, a significant difference was seen in FIM. However, the magnitude of the improvement in the FIM score was not as large as for the other functional measure, WMFT (7% increase for FIM vs. 42% improvement for WMFT time score). This discrepancy may be explained by the fact that in the FIM test, subjects may use either the impaired or non-impaired upper extremity to accomplish a task (e.g., dressing, feeding). Thus subjects may score as ‘independent’ in a task even if the affected limb is not used at all during execution. This would create a ‘ceiling’ effect for scores of subjects who may now be able to perform a task with the impaired limb post training, but would still receive the same score as when they did the task with the unimpaired limb. However, for the WMFT, subjects must use the impaired limb for all tasks, and thus improvements in the impaired limb would be better captured by the WMFT score.

The reduction in distance from the mean path for a particular movement shows a decrease in variable error and is a sign of motor learning and more skilled movement performance.

The results of the accuracy of the trajectory movement and the differences between trajectory movements of each subject and the teacher showed the high precision of upper extremity movements.

In the present study, the distance from the mean path decreased in 5 subjects. A significant difference was also seen in the distance from the mean path averaged over all subjects in the comparison between the initial and final VE.

In a study on motor control of the upper extremities of patients with stroke in ipsilateral, contralateral, proximal, and distal reaching tasks, the trajectory of movement of the paralyzed upper extremity differed from that of the upper extremity on the non-paralyzed side and from those of healthy people. The large variance of movement path, the reduction of the reaching distance and lengthening of movement time were observed13). Thus, a significant difference was found for the change of distance from the mean path. This means that there are temporal and spatial improvements in the subjects’ movements in VE training sessions.

The reduced distance from the teacher shows that the movement of the subjects was approaching the movement of the teacher. In the present study, the distance reduced for all subjects at the time the teacher entered the doughnut. In addition, the distance at the time the teacher emerged from the doughnut decreased in 5 subjects. Moreover, a significant difference was seen in the distance from the teacher averaged over all subjects in the comparison between the initial and final VE.

The finding that the movement of the subjects approached the movement of the teacher suggests that motor learning by the subjects was promoted by the VE training. The subjects started moving at the same time as the teacher and moved while receiving visual feedback on errors with respect to the teacher. This required highly accurate reproduction of the path by a dynamic visual presentation of the teacher, in contrast with the task of reaching toward the presented target. Stroke patients show a tendency to turn their eyes to the teacher during VE training14). Thus, patients seem to be visually monitoring the teacher as a target path. Therefore, a dynamic visual presentation with a teacher should promote motor learning of a patient. The theoretical background for motor learning in VE training suggests that path planning is promoted in association areas by the movement of the teacher. To show that this is true, it is necessary to measure the localization of brain functions using brain function imaging during VE training and to further investigate motor learning using static and dynamic visual presentations.

In doughnut tasks, the cube reaches the doughnut, and is then hidden behind it, causing the target path to disappear. In a study on visual feedback and hand-eye coordination15), a certain level of maintenance or improvement in the precision of movement was seen with proprioceptive feedback, such as from a muscle, even without accompanying visual feedback. In VE training, the speed of the teacher is set by the therapist so that the teacher can be followed by the subject. We think this is why a significant difference was seen in the distance from the teacher even at the time the teacher emerged from the doughnut.

Although a reduction in the distance from the teacher was seen in the present study, it is not clear what kind of joint movement occurred in the upper extremity. In a mathematical investigation of the relationship between the position of the fingertips and the arm position range16), the possibility was raised that shoulder and elbow movement could be controlled by the tip position of the fingertips. In addition, the joint degrees of freedom of the distal forearm are less susceptible to control by the position of the fingertips. Moreover, it has been reported that in reaching movements of the upper extremities of stroke patients, movement of the trunk occurs before movement of the upper extremities17). Thus it is believed that during VE training the upper extremities adopt various positions with the participation of the trunk. A motion analysis of the upper extremities that includes trunk movement is therefore necessary. In VE training, because movements corresponding to various tasks are performed, it is also important to investigate whether it is possible to control the distal forearm through motion analysis of the upper extremities.

VE is a kind of mixed reality. Mixed reality is a general term for technology that supplements or augments sensory information obtained from the real world with digital information generated by computer and sensors. It is also called augmented reality (AR), as it merges real and virtual worlds. Subjects performed their arm movement grasping a real object (small board or wine glass) with position sensors while watching the display of VE. Therefore, VE is more related to the real world than to VR game. Moreover, none of the patients experienced virtual sickness during VE. The results indicate that VE did not disturb recovery of the affected upper extremity. In addition, all patients who participated in our study understood their recovery process by VE. Also, they saw that their movement paths gradually similar to the ideal normal movement of the upper extremities. However, our study was limited in that we did not analyze or discuss the effect of natural recovery.

In conclusion, we clarified the effects on motor relearning of paralyzed upper extremities in VE training, and investigated the relationship between improvements in paralyzed upper extremity movement and the clinical condition. The method used was analysis of 3-dimensional coordinate data of the trajectory of movement recorded in VE training, and examination of the relationship between VE training and clinical evaluations.

By converting 3-dimensional coordinate data to distance from the mean path and distance from the teacher, it was possible to quantitatively evaluate the trajectory of movement in VE training. It was also found that during VE training tasks, the movement of the paralyzed upper extremity became less variable and more accurate and approached the movement of the teacher. Significant improvements were also seen in all the clinical evaluations (FMA, WMFT and FIM tests), suggesting the clinical application of VE training.

The results of the 3-dimensional coordinate data analysis performed in the present study demonstrated the motor learning effect with respect to the movement of the teacher at the time the teacher entered the doughnut and at the time the teacher emerged from the doughnut. That is, only the ‘endpoint’ trajectory of the moving limb was analyzed. What is not clear is how these endpoint trajectory movements of the paralyzed upper extremities were achieved by combinations of shoulder, elbow and hand motions. Therefore, in future studies we would also like to perform more extensive motion analyses of paralyzed upper extremities during VE training as well as investigate the extent to which these motions can also be performed during ‘real world’ versions of similar tasks of daily living.