Immediate effects of head-mounted display adaptation in cases of unilateral spatial neglect: study of straight-ahead pointing.

[Purpose] This study aimed to clarify the effect of an adaptation of a deviation of the visual field in three axes on spatial cognition in patients with unilateral spatial neglect and distorted spatial perception in three dimensions. [Participants and Methods] Fifteen patients with cerebrovascular disease and symptoms of unilateral spatial neglect were included. Forty-eight pointing movements with a camera attached to a head-mounted display changed in three axes were compared with the control condition in which the camera was deflected only in the horizontal plane as with the prism adaptation. The main outcome measures were subjective straight-ahead pointing, line bisection, line cancellation, and star cancellation.
[Results] The head-mounted display adaptive therapy was performed under conditions that varied in all three axes. The results indicated that it was possible to deflect the subjective straight-ahead pointing position to the lower left direction.
[Conclusion] In contrast to the prism adaptation, which deflects the visual field in a single axis in the horizontal plane, the tri-axial adaptation corrected the median cognition in the left-right direction as well as the cognition of the body center, including the vertical direction. 2022©by the Society of Physical Therapy Science. Published by IPEC Inc.

INTRODUCTION

Unilateral spatial neglect (USN) is defined as “a consistent, exaggerated spatial asymmetry in processing information in bodily and/or extrabodily space due to an acquired cerebral lesion”1). Cerebrovascular disorders with USN are generally known to have poor functional outcomes2, 3). Therefore, effective treatment is desired. Various treatment methods have been attempted. Nontheless, the prism adaptation (PA) devised by Rossetti et al. is the most effective method to date4). This method uses prism glasses with a visual field 10° biased to the right. Additionally, the upper limbs are extended, and the target is repeatedly touched. In Rossetti et al.’s first report on the effectiveness of PA4), participants performed a subjective straight-ahead pointing (SAP) task, wherein they pointed to a position in the horizontal plane that they perceived to be mid-front of the trunk. Their results revealed that the subjective median localization, which is usually right-biased, shifted to the left in USN cases and also indicated that the sub-items of the behavioral inattention test (BIT) (such as picture copying, line cancelation test, and line bisection test) showed improvement in USN immediately after PA, with the effect lasting for at least 2 hours. Since Rossetti et al.’s report4), improvements were evidenced in the performance of desk tests using paper and pencil5,6,7), activities of daily living8,9,10), and wheelchair driving ability11, 12). The basis for this improvement is the improvement in SAP13,14,15), which also correlates with the line bisection test16, 17) and is used as an indicator of PA effectiveness17,18,19,20,21).

USN patients generally have problems not only with cognition in the left-right direction (i.e., in the horizontal plane) but also with cognition in the frontal and sagittal planes. The visual vertical perception is tilted counterclockwise in the frontal plane22, 23) and backward in the sagittal plane23). As a result, attention is biased to the upper right and less attention is paid to the lower left, thereby delaying the awareness of obstacles near the left leg and often leads to falls, which requires improvement. Cognition changes in three dimensions (3D). Thus, it is considered that conventional PA, which causes visual field deviation only in the horizontal plane, is not sufficient for USN cases with deviated spatial cognition in 3D. This is because PA uses light refraction by a prism, which can shift the field of view in one dimension but cannot rotate it in the frontal plane.

When a head-mounted display (HMD) is used in combination with a web camera, the camera position can be freely changed so that the field of view can be not only shifted in the horizontal plane (yaw angle), but also twisted in the frontal plane (roll angle) and tilted in the sagittal plane (pitch angle), enabling 3D displacement. In a study24) an HMD with a camera mounted in a twisted position in the frontal plane was attached to a healthy adult, and the adult was made to assume a standing posture. It was found that the head and trunk angle was tilted to the left when the camera was rotated 20° to the right (the visual field was rotated 20° to the left), and the floor reaction force vertical component was also increased in the left foot. Although there are some reports on new evaluation methods using HMDs25,26,27,28), only a few intervention studies have been conducted. In one of those, a virtual space was created in a HMD, and a treatment that made the patient turn to the left using a curtain effect was effective29). In previous studies, the same after effect as in PA was achieved by trying the same conditions as in PA using a video camera and HMD30, 31). However, to the best of our knowledge, no study has reported on intervention that includes real world rotation in three axes using HMD. For the treatment of USN, which is considered to be changing in 3D, it is important to have the view deviated not only in the horizontal plane (yaw angle) but also in the frontal plane (roll angle).

This study aimed to clarify how adaptation in the field of view deviated by three axes (yaw angle, roll angle, pitch angle) affects spatial cognition in USN cases. This is the first study to examine the effect of adaptation on USN patients using an HMD and a web camera in a 3D deflected field of view.

PARTICIPANTS AND METHODS

This study was conducted at two regional rehabilitation hospitals and one acute hospital, and it adopted a one-session crossover design of experimental condition and control condition. The participants were 15 patients hospitalized during the acute to convalescent period at the Niiza Hospital, the Higashiyamato Hospital, and the Tokorozawa Central Hospital between October 2018 and January 2021, who met the following criteria: (1) patients with right hemispheric cerebrovascular disease; (2) patients hospitalized within 6 months of stroke onset; (3) patients without severe dementia (Mini-Mental State Examination score ≥15); (4) patients with USN (with reduced score items in the neuropsychological findings described below); (5) patients for whom wheelchair or chair sitting was possible; and (6) patients in whom consent from the attending physician was obtained.

The exclusion criteria were as follows: (1) inability to hold a wheelchair in a seated position; (2) inability to understand tasks due to aphasia or other cognitive impairments; (3) inability to understand Japanese; (4) visual impairment; (5) severe hearing loss; (6) inability to reach due to limited range of motion of the right upper extremity; (7) amputation of the right upper extremity proximal to half of the forearm; (8) severe positional dysesthesia of the fingers due to peripheral neuropathy; and (9) patient refusal to participate.

The neuropsychological findings before the intervention are presented in Table 1.

Table 1.

Individual demographic data

Patient	Age	Gender (M/F)	Etiology	Lesion site	Months from onset	FIM-m	FIM-c	CBS
A	66	M	Infarction	Frontal lobe	3	43	18	4
B	61	M	Hemorrhage	Corona radiata	1	42	22	5
C	64	M	Hemorrhage	Putamen	3	55	19	16
D	73	M	Infarction	corona radiata	6	57	31	6
E	78	M	Infarction	corona radiata−insular cortex	1	46	16	9
F	75	F	Infarction	MCA area	5	73	32	4
G	51	M	Hemorrhage	Putamen	3	41	17	22
H	62	M	Hemorrhage	Putamen	6	25	18	21
I	68	M	Hemorrhage	MCA area	3	25	25	20
J	80	M	Hemorrhage	Putamen	5	66	27	10
K	69	F	Infarction	Temporal lobe	0	46	26	8
L	50	M	Infarction	Occipital lobe	0	14	16	2
M	74	M	Infarction	MCA area	0	39	26	10
N	77	F	Infarction	Pons-Diencephalon	0	37	22	2
O	52	M	Hemorrhage	Putamen	0	38	27	1
Average	67.8				3.6	47.3	22.5	11.7
SD	9.6				2.2	15.0	5.2	6.9

M: male; F: female; MCA: middle cerebral artery; FIM: functional independence measure; CBS: catherine bergego scale.

This study was conducted in accordance with the Declaration of Helsinki and was approved by the Research Ethics Committee of Tokyo Metropolitan University (Approval code: 19031). Written informed consent was obtained from the participants after explaining that participation in the research was voluntary and that it was possible to withdraw at no disadvantage at any time after consent.

The conventional SAP measures the left-right deviation from the midline by pointing to its position on a transversely placed scale on the horizontal plane. In the frontal plane-SAP (FSAP), all participants wore an HMD (Oculus rift, Facebook Technologies, LLC, Menlo Park, CA, USA) with a webcam (Ovrvision, Shinobiya.com, Osaka, Japan) facing directly toward the front and were in a wheelchair (or chair) sitting position with their feet on the ground and trunk in contact with the backrest. The nosepiece was used as an index to fit the participant’s face so that the HMD screen was exactly in front of both eyes. Since the purpose was to measure the cognition of the front of the trunk, there were no restrictions on the movement of the head. To identify the center point, instead of the midline, a 32-inch (1,375 × 767 pixels) touch panel (Retrofitting touch panel, NEWCOM Inc., Saitama, Japan) was placed at 50 cm in front of the body to make the plane parallel to the front of the trunk. The examiner used a cross-line laser leveler (GLL40-20G, Robert Bosch GmbH, Stuttgart, Germany) to locate and record the anterior projection point of the height of the midline and xiphoid process of the trunk on the panel. Subsequently, the participants were asked to touch the location of the touch panel, which they thought was directly in front of the height of the midline and xiphoid process, with their index fingers 10 times each, and the touch position was recorded each time. To measure the touch position, the sensor response position of the touch panel was recorded using the software Grid (NEWCOM Inc.), which can acquire the pixel position, and the x and y positions were calculated as the ratio of the actual size of the panel. This allowed us to measure the touch position by the task and the deviation from the sternal front center identified by the laser pointer and to convert the distance from the trunk to an angle of 50 cm. The participants were instructed to look at the landmarks above the touch panel, referring to the method reported by Farnè et al.32), and their field of vision was adjusted so that they could not see their upper limbs; the image in the HMD was displayed on the screen of the HMD image processing PC (mouse computer NEXTGEAR-NOTE i5702BA1, MouseComputer Co., Ltd., Tokyo, Japan) to confirm the participant’s field of vision.

Before and after the interventions, in addition to FSAP, the behavioral inattention tests (BITs) for the following three sub-items were conducted.

Line bisection: An A4 test sheet was presented on a desk, so that the true center point of a 200-mm long line drawn horizontally across the center of the sheet coincided with the sagittal plane of the participant’s body. Participants were instructed to mark the center of the line with a pencil held in their right hand.

Line cancellation: An A4 test sheet with 40 one-inch-long line segments printed seemingly randomly on the paper was presented on the participant’s front midline desk. After showing all the line segments, the examiner marked the two lines in the middle to illustrate how to make the marking and then instructed the participants to mark all the line segments.

Star cancellation: An A4 paper with 52 large stars and 56 small stars scattered among 12 randomly placed letters and 10 words was presented on the midline desk in front of the participant. The examiner marked two small stars in the center of the form to teach how to make the marking and then instructed the examinee to mark all the small stars.

The HMD was worn in the same posture as in the initial evaluation. As the purpose of PA was to adapt only in the left-right direction (one dimension), there were two targets to reach. However, in this study the purpose was to adapt in a plane (two dimensions); thus, four targets were displayed in front of the chest and randomly pointed to 48 times while visually observing.

Intervention was conducted under two conditions: experimental (E) and control (C) conditions. The order of these two conditions of intervention was randomly determined. In addition, the two interventions (E,C) were performed at 24 hour intervals or more. Among the 15 participants, 8 underwent the condition E to C intervention order and 7 underwent the condition C to E intervention order.

Condition C: Control (yaw angle deviation): The camera attached to the HMD was tilted 10° to the left on the horizontal plane and the participants pointed to the targets (instructed characters) 48 times with the right hand.

Condition E: 3-axis deviation (yaw + roll + pitch angle deviation): With the camera tilted to the left on the horizontal plane, clockwise on the front face, and downward by 10° on the sagittal plane, participants pointed at them 48 times. We created a dedicated tool and set it to 10 degrees in each dimension.

The FSAP assessment and BIT sub-item evaluation were conducted before and after intervention. The angle was calculated from the distance between the trunk and the touched position, and the difference between pre- and post-intervention was taken in the X and Y directions, respectively, and tested using Wilcoxon’s signed-rank test under the E and C conditions.

Changes in the line bisection test and line bisection peripheral test were similarly tested using paired Wilcoxon’s signed-rank test using IBM SPSS Statistics 25 (International Business Machines Corporation, Armonk, NY, USA).

RESULTS

The results for FSAP under each condition are presented in Table 2 and those for the BIT sub-item evaluation are presented in Table 3.

Table 2.

FSAP results

							(°)
Condition		X			Y
		pre	post	post-pre	pre	post	post-pre
E	Average	3.6	−0.7	−4.3	2.3	2.8	0.5
	SD	5.3	6.5	3.4	8.7	7.5	7.0
C	Average	2.3	−1.1	−3.4	4.9	−3.8	−8.7
	SD	3.8	3.8	3.2	4.9	6.3	4.7

The X-axis values are positive on the right and negative on the left, whereas the Y-axis values are positive on the upper side and negative on the lower side.

FSAP: frontal plane-straight-ahead pointing; SD: standard deviation.

Table 3.

BIT sub-item results

Condition		Line bisection test			Line cancellation test			Star cancellation test
		(cm)			(number of errors)			(number of errors)
		pre	post	pre-post	pre	post	pre-post	pre	post	pre-post
E	Average	−0.3	0.2	0.4	3.5	2.7	−0.8	8.6	7.6	−1.0
	SD	1.2	1.3	0.9	6.3	7.2	4.8	13.4	12.6	7.1
C	Average	0.3	0.1	−0.1	2.5	3.0	0.5	8.3	6.0	−2.3
	SD	1.0	1.6	0.9	7.9	7.9	1.6	13.2	13.4	5.6

Line bisection test is positive to the right of the center.

BIT: behavioral inattention test.

On comparing the FSAP position before and after the intervention under the E condition, the FSAP position before the intervention was 2.3 ± 3.8° (mean ± standard deviation) to the right in the X-axis, which tilted 1.1 ± 3.8° to the left after the intervention (p=0.01), and it was 4.9 ± 4.9° to the top, tilting 3.8 ± 6.3° to the bottom in the Y-axis (p=0.005). Thus, there was a significant deviation in the left downward direction.

Comparing the FSAP position before and after the intervention under the C condition, the FSAP position before the intervention was 3.6 ± 5.3° to the right in the X-axis, which was significantly biased to 0.7 ± 6.5° to the left after the intervention (p=0.008). In the Y-axis direction, it was 2.3 ± 8.7° to the top, which was tilted 2.8 ± 7.5° downwards after the intervention (n.s.).

Comparing E and C conditions, the X-direction was shifted to the left by 3.4 ± 3.2°, and the Y-direction was shifted downward by 8.7 ± 4.5° before and after the E condition intervention. The X-direction was shifted to the left by 4.3 ± 3.3°, and the Y-direction was shifted upward by 0.5 ± 7.0° before and after the C condition intervention. Verifying the differences in the means of X and Y-direction shifts under E and C conditions showed that both FSAPs were shifted to the left after the intervention in the X-direction; however, the differences were not significant (p=0.48). There was a significantly larger degree of downward shift under the E condition in the Y-direction (p=0.008).

In the BIT, there was no change in the line bisection test under the E and C conditions, and the average number of missed cases decreased, although there was no significant difference in the line cancellation and star cancellation tests (Table 3).

DISCUSSION

This study demonstrated that the FSAP was significantly deflected downward to the left under the E condition, in which the yaw angle, roll angle, and pitch angle were deflected. Both E and C conditions showed leftward displacement, but only the E condition showed downward deflection. From the result, the same effect as PA was obtained under the C condition. Under the E condition, by deflecting the field of view in the three axes, the leftward adaptation effect of FSAP was obtained in the horizontal axis (left-right direction) along with the deflection of the yaw angle as in PA, and a downward effect was obtained in the vertical axis due to the addition of changes in pitch and roll angles. The rate of change in the left-right direction is reportedly 40% of the prismatic deviation in PA14), and the present results using a webcam and HMD showed almost the same effect: 34% (3.4°) in the E condition and 43% (4.3°) in the C condition against a 10° viewing deviation. Under the E condition, the rate of change in the vertical direction was 8.7°, which was 87% of the 10° viewing deflection. As a result of deflecting the camera by 10° in each axis, the position of the target in front of the field of view during adaptation (shown in thin) and its position in real space (shown in bold) were misaligned, as shown in Fig. 1, suggesting that the FSAP (median judgment) was deflected to the lower left (arrow direction) when the camera was returned to the center after the hand-eye coordination was relearned by adaptation6, 33, 34).

Fig. 1.

Character positions on the head-mounted display (HMD) and actual positions.

In patients with left USN, attention to the left is reduced, resulting in a vicious cycle in which median cognition is biased to the right and attention to the left becomes more difficult35). By performing reach movements for tens of times while wearing a right polarized prism as a treatment, the adaptation between the hand that appears to be shifted to the right on the HMD screen and the actual hand position, median cognition is returned to the left, and a subsequent improvement of left USN is observed14). However, no intervention has been made to deflect the visual field in the vertical axis (up and down directions) in patients with USN to date. It was found that not only can the effect on the horizontal plane be obtained using the camera with the deviation of the yaw angle, the adaptation effect in 3D, including the front face value, can also be obtained by changing the roll and pitch angles. HMD adaptation can modify not only the left-right direction but also the up-down direction. Moreover, in our E condition, it could modify the left-down direction. Prismatic adaptation has been shown to improve attention in the left direction by returning median cognition to the left in patients with USN who have decreased attention to the left. The HMD adaptation can also be expected to improve attention to the lower left by returning the median cognition to the lower left. If we consider that the median cognition in the anterior forehead plane was biased upward to the right because the median cognition was shifted to the right when the vertical visual cognition was tilted counterclockwise22, 23), adaptation with biased visual fields in three axes is likely to contribute to the correction of vertical cognition.

The effect of the desk tests was not significant, despite the change in FSAP. Because there were many patients with mild disease and a small number of patients with severe USN, the possibility of selection bias cannot be ruled out, and the effect of the desk test may have been difficult to demonstrate. Another possibility is the effect of the camera angle. Turton et al.9) reported small changes in BIT in response to changes in SAP in PA, but they used a 6° prism. Some studies36) have recommended that participants be exposed to a 15° (or higher) prism for more than 10 minutes with more than 250 pointing movements. Therefore, it is necessary to compare conditions, including higher-intensity interventions. In the future, this may assist in establishing a more effective treatment in patients whose attention to the lower left is reduced due to occurrence of three-dimensionally shifted visual field adaptation.

This study has some limitations. First, because it was an immediate effect of a single session, it was impossible to measure the cumulative or long-term effects of repeated sessions. To reduce the neglected symptoms in vertical cognitive modification and Activities of Daily Living, which involve various modalities, it is necessary to evaluate the long-term intervention. Second, the limited sample size did not allow for adequate subgroup analysis of response to PA. As USN can be classified into several subtypes based on the patient’s symptoms, the differences in response to PA among various subtypes need to be clarified. Since the HMD used in this experiment was equipped with a heavy object in front of the head, the extensor muscles of the head and neck were loaded, affecting proprioception of the head and neck in the sagittal plane. As a result, adaptation ratio in the left-right direction was about 40%. However, this adaptation ratio in the vertical direction increased to 87%, which may indicate probable effects of that load on adaptation ratios.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of interest

The authors have no conflicts of interest directly relevant to the content of this article.