Haptic stylus and empirical studies on braille, button, and texture display.

This paper presents a haptic stylus interface with a built-in compact tactile display module and an impact module as well as empirical studies on Braille, button, and texture display. We describe preliminary evaluations verifying the tactile display's performance indicating that it can satisfactorily represent Braille numbers for both the normal and the blind. In order to prove haptic feedback capability of the stylus, an experiment providing impact feedback mimicking the click of a button has been conducted. Since the developed device is small enough to be attached to a force feedback device, its applicability to combined force and tactile feedback display in a pen-held haptic device is also investigated. The handle of pen-held haptic interface was replaced by the pen-like interface to add tactile feedback capability to the device. Since the system provides combination of force, tactile and impact feedback, three haptic representation methods for texture display have been compared on surface with 3 texture groups which differ in direction, groove width, and shape. In addition, we evaluate its capacity to support touch screen operations by providing tactile sensations when a user rubs against an image displayed on a monitor.

1. INTRODUCTION

Researchers have proposed a diverse range of haptic interfaces for more realistic communication methods with computers. Force feedback devices, which have attracted the most attention with their capacity to physically push and pull a user's body, have been applied to game interfaces, medical simulators, training simulators, and interactive design software, among other domains [1]. However, compared to force feedback interfaces, tactile displays have not been deeply studied. It is clear that haptic applications for mobile devices, such as PDAs, mobile computers, and mobile phones, will have to rely on tactile devices. Such a handheld haptic system will only be achieved through the development of a fast, strong, small, silent, safe tactile display module, with low heat dissipation and power consumption. Furthermore, stimulation methods reflecting human tactile perception characteristics should be suggested together with a device.

A number of researchers have proposed tactile display systems. In order to provide tactile sensation to the skin, work has looked at mechanical, electrical, and thermal stimulation. Most mechanical methods involve an array of pins driven by linear actuation mechanisms such as solenoids, piezoelectric actuators, or pneumatic actuators. An example is the “Texture Explorer,” developed by Ikei and Shiratori [2]. This flat pin array is composed of piezoelectric actuators and operates at a fixed frequency (250 Hz) with maximum amplitude of 22 μm. Summers and Chanter [3] developed a broadband tactile array using piezoelectric bimorphs and reported empirical results for stimulation frequencies of 40 Hz and 320 Hz with the maximum displacement of 50 μm. Since the aforementioned tactile displays may not result in sufficiently deep skin indentation, Kyung et al. [4] developed a pin-array tactile display which has a small size, long travel, and high bandwidth. However, this system requires a high input voltage and a high power controller. As an alternative to providing normal indentation, Hayward and Cruz-Hernandez [5] and Luk et al. [6] have focused on the tactile sensation of lateral skin stretch and designed a tactile display device which operates by displaying distributed lateral skin stretch at frequencies of up to several kilohertz. However, it is arguable that the device remains too large (and high voltage) to be realistically integrated into a mobile device. Furthermore, despite work investigating user performance on cues delivered by lateral skin stretch, it remains unclear whether this method is capable of displaying the full range of stimuli achievable by presenting an array of normal forces.

Konyo et al. [7] used an electroactive polymer as an actuator for mechanical stimulation. Poletto and Van Doren developed a high-voltage electrocutaneous stimulator with small electrodes [8]. Kajimoto et al. [9] developed a nerve axon model based on the properties of human skin and proposed an electrocutaneous display using anodic and cathodic current stimulation. Unfortunately, these tactile display devices sometimes involve user discomfort and even pain.

We can imagine a haptic device providing both force and tactile feedback simultaneously. Since Kontarinis and Howe applied vibration feedback to a teleoperation in 1995 [10], some research works have had interests in combination of force and tactile feedback. Akamatsu and MacKenzie [11] suggested a computer mouse with tactile- and force feedback-increased usability. However, the work dealt with haptic effects rather than precisely controlled force and tactile stimuli. In 2004, Kammermeier et al. combined a tactile actuator array providing spatially distributed tactile shape display on a single fingertip with a single-fingered kinesthetic display and verified its usability [12]. However, the size of the tactile display was not small enough to practically use the suggested mechanism. As more practical design, Okamura et al. designed a 2D tactile slip display and installed it into the handle of a force feedback device [13]. Recently, in order to provide texture sensation with precisely controlled force feedback, a mouse fixed on 2DOF mechanism was suggested [14]. A small pin-array tactile display was embedded into a mouse body and it realized texture display with force feedback. More recently, Allerkamp et al. developed a compact pin-array and they tried to realize the combination of force feedback and tactile display based on the display and vibrations [15]. However, in previous works, the tactile display itself is quite small but its power controller is too big to be used practically. Our work in this paper deals with this issue as one of applications of our system.

In the area of human tactile perception, Johansson and Vallbo [16] and Johnson and Phillips [17] have studied human mechanoreceptors and their function in connection with tactile perception and the anatomical structure of glabrous skin such as the palm or finger pad. Verrillo et al. have suggested a four-channel model of vibrotaction which shows the variation of the displacement (indentation depth) threshold to frequency [18]. Also, studies have measured the sensation magnitude of thresholds as a function of frequency of vibration [18, 19]. The previous physiological research shows that humans have four types of mechanoreceptors for tactile sense and that each type responds in a specific band of frequency. Therefore, frequency characteristics should be given careful consideration in the design of a tactile display device and stimulation method.

In this paper, we propose a compact tactile display module which can be embedded into small devices and a pen-type haptic interface providing impact and distributed pressure. In Section 2, the design parameters and structure of the proposed tactile display module are described in detail. In Section 3, the implementation of a pen-like haptic interface including the tactile display module and impact generator is presented. In Section 4, we evaluate performance of this system, which we term the “Ubi-Pen II.” In Section 5, performance of a force and tactile feedback interface adopting the suggested pen-like interface is described. Finally, in Section 6, we discuss possible applications of the proposed system including image display on a touch screen.

2. COMPACT TACTILE DISPLAY MODULE

2.1. Design of a tactile display module

In order to make a tactile display module, actuator selection is the first and dominant step. The actuator should be small, light, safe, silent, fast, powerful consume modest amounts of power and emits little heat. Recently, we developed a small tactile display using a small ultrasonic linear motor [20]. We here briefly describe its operation principle and mechanism.

The basic structure and driving principle of the actuator are described in Figure 1. The actuator is composed of a transducer, a shaft, and a moving element. The transducer is composed of two piezoelectric ceramic disks and elastic material membranes. The convex motion of the membranes causes lift in the shaft of the motor. The fast restoring concave motion overcomes the static frictional force between the moving element and the shaft, and it makes the moving element maintain its position. The displacement “A” of one cycle is submicrometer scale, and the rapid vibration of the membrane at a frequency of 45 kHz (ultrasonic range) causes rapid movement of the moving element. The diameter of the transducer is 4 mm and its thickness is 0.5 mm. The thrusting force of the actuator is greater than 0.2 N and the maximum speed of the moving element is around 30 mm/sec. In order to minimize the size of the tactile display module, the actuators were arranged as shown in Figure 2. Essentially, this figure shows the arrangement of two variations on the actuators—each with different shaft lengths. This design minimizes the gap between actuators. Another feature is that the elements previously described as “moving” are now stationary and fixed together, causing the shafts to become the elements which move when the actuators are turned on. This minimizes the size of the contact point with a user's skin (to the 1 mm diameter of the shaft), while maintaining the mechanical simplicity of the system.

Figure 1

Operation principle of an actuator.

Figure 2

The implemented tactile display module.

2.2. Implementation

From the design specification described in Section 2.1, the prototype of the tactile display module has been implemented as shown in Figure 2. In order to embed the module in a pen, we constructed only a pin array. However, it should be noted that the basic design concept is fully extensible; additional columns and rows can be added without electrical interference or changes in pin density. The shaft itself plays the role of tactor and has a travel of 1 mm. The distance between two tactors is 3.0 mm. Since the actuators operate in the ultrasonic range, they produce little audible noise. The average thrusting force of each actuator exceeds 0.2 N, sufficient to deform the skin with an indentation of 1 mm [21]. The total size of the module is  mm and its weight is 2.5 g. Since the maximum speed of a pin is around 30 mm/sec, the bandwidth of the tactile display is approximately 20 Hz when used with a maximum normal displacement of 1 mm. If the normal displacement is lower than 1 mm, the bandwidth could be increased.

3. IMPLEMENTATION OF HAPTIC STYLUS

The styli have become common tools for interacting with mobile communication devices. In the area of haptics, Lee et al. [22] suggested a haptic pen which could provide a sense of contact based around a touch sensor and a solenoid. It could generate a feeling corresponding to clicking a button.

In order to support richer stylus-based tactile cues, we embedded our tactile display module into a pen-like prototype. We termed these kinds of devices the Ubi-Pen and intend it for use as an interface to VR, for the blind, to represent textures, and as a symbolic secure communication device [20]. In our previous version, a small vibrator was installed at the tip of the pen. However, since the vibrator's temporal response is slow, it causes time delay between signal and activation. Although it was effective, it was not realistic.

In this research, instead of a typical vibrator, we installed an impact generator in the head of the pen to provide a sense of contact (see Figure 3). We named this version the Ubi-Pen II. We suggest that it could be used generally as the stylus of a mobile communication device, which provides realistic and interactive haptic cues such as buttons during operation of OS.

Figure 3

The prototype of the Ubi-Pen II.

Figure 4 shows an operation principle of the impact generator. There is a mass inside the generator and electromagnetic force induced by electric signal that makes the mass move along a longitudinal axis of the case. This generator is generally used as a kind of linear vibrator and we otherwise use it as an impact generator. The generator is arranged along a longitudinal axis of the stylus housing. When a rising signal is applied to the generator, the mass moves up fast and it collides with the upper side. When a falling signal is applied to the generator, the mass moves down fast and it collides with the bottom side. The response time of the mass movement is within milliseconds scale.

Figure 4

Operation principle of an impact generator.

4. EVALUATION OF PERFORMANCE

4.1. Braille display of the tactile display module

A common method to evaluate the performance of tactile displays is to test user's performance at recognizing specific patterns [2, 4]. We use Braille as a stimulus set to conduct such a test. Specifically, we conducted a study involving the presentation of the Braille numbers 09 on the Ubi-Pen.

Figure 5 shows the experimental Braille patterns. Subjects were required to hold the pen such that the tip of their index finger rested over the pin-array part of tactile display module. In our previous work, the test was conducted for the normal people and there was small observations for the blind [20]. In this paper, the Braille display test bas been conducted for the normal and the blind.

Figure 5

Braille patterns for the experiment.

After setup stage, we conducted a study on recognition rate of the 10 numeric digits in the Braille character set. As these can be displayed on only four pins, we mapped them to the corner pins on our tactile display module. We chose to do this as our user-base was composed of sighted Braille novices. We used three different stimulation frequencies: 0, 2, and 5 Hz. (Pins move up and maintain static position at the 0 Hz.) Pins movement was synchronized. We presented 60 trials in total, each number at each frequency, twice. All presentations were in a random order, and subjects were not advised about the correctness of their responses. 10 subjects participated in the experiment. The Braille stimuli were generated continuously and changed as soon as the subject respond using the graphic user interface. There were 2-minute breaks after every 20 trials.

Two blind people have participated in the same experiment and the visual guidance in the experiment has been replaced by the speech guidance of experimenter. For all stimuli, they responded exactly and quickly. The Braille expert usually read more than 100 characters [23], and the blind subjects responded that they do not feel any difficulties to read the Braille numbers. Since the duration of each trial was shorter than 12 seconds and they answer in the form of speech, we could not measure the duration exactly.

Moreover, 4 neighborhood pins have been presented again with identical procedure for the blind people; and they responded more quickly since the gap of pins was more familiar with them. Duration of each trial was always shorter than 1 second.

Table 1 shows the summary of experimental results. Although normal subjects were novice in using the tactile display, the average percentage of correct answers exceeded 80 percent. The confusions come from the relatively low tactile sensitivity of the novices compared with the sensitivity of the blind. Since the various analysis of the tactile display for the blind is another interesting topic, this will be investigated in our future work.

Table 1

Experimental results.

	Normal subjects	Blind subjects
Average percentage of correct answers	80.83	100
Average duration of each trial (sec)	5.24	1∼2

Craig's research shows the blind people have extraordinary capability to recognize the vibrotactile patterns at very high frequencies [23]. It might be true that specialized people recognize vibrotactile patterns without respect to frequencies. However, spatial acuity of human tactile perception is a function of the vibration frequency; and we need to determine the best frequency for the tactile pattern display using the developed device. Our previous work shows spatial acuities are better at the range of the Merkel's disk and Meisner's corpuscle [4]. From the comparisons at the frequency range of 0560 Hz, the sensitive range of the Merkel's disk, 13 Hz, was the best frequency for the pattern perception since the mechanoreceptor is mainly related to the sense of surface pattern and distributed pressure [18]. Before conducting the experiment, we needed to look at the frequency bands of peripheral tactile neural responses. There are four mechanoreceptors in the glabrous skin of the palm and fingertip regions. Meissner's corpuscles and Merkel's discs are located in the upper layers, and Ruffini endings and Pacinian corpuscles are located more deeply. These receptors are divided into the following two classes according to their rate of adaptation: the slowly adapting afferent receptors and the rapidly adapting afferent receptors. The slowly adapting afferent receptors comprise Merkel's discs (SA I) and the Ruffini endings (SA II), while the rapidly adapting afferent receptors comprise Meissner's corpuscles (RA I) and the Pacinian corpuscles (RA II). The four mechanoreceptors each have different functions [16, 18]. The SA I afferents respond to quasistatic deformations of the skin, such as force or displacement in the frequency range of 0.4–3 Hz. These receptors play an important role in detecting spatial structures in static contact, such as an edge or a bar. The size of Merkel's receptor is small and shows very high innervation density at the tip of index finger. The SA II afferent receptors provide a neural image related to the direction of the skin being stretched. SA Type II fibers produce a buzz-like sensation in the frequency range of 100–500 Hz. The RA I afferent receptors, which have a frequency range of 2–40 Hz, detect dynamic deformations of the skin such as the sensation of flutter. The RA I afferent receptors are about four times more sensitive than the SA I afferent receptors; in addition, RA I shows best sensitivity in the frequency range of 25–40 Hz. The RA II afferent receptors, which have a frequency response in the range of 40–500 Hz, are the most sensitive to vibration amplitude and are particularly known to serve as detectors of acceleration or vibration. Previous anatomic study shows the size of Pacinian corpuscles to be bigger than the other mechanoreceptors located deeper within the skins, and their innervation density is low [25]. Therefore, it is to be expected that their spatial acuity would be poor. (However, in some cases [23], good spatial resolution may be observed at frequencies expected to activate Pacinian corpuscles.) Based on these findings, we found that humans were more sensitive at a frequency band of 13 Hz in tactile pattern discrimination that they are at surrounding frequencies [4]. This is due to the structure of our neural mechanism for sensing tactile pattern. One part is easily activated by this frequency band. Therefore, we hypothesized that stimuli delivered in that frequency range would outperform those outside it. This was brought out by asking subjects about their impressions of the cues, and 8 of the 10 subjects suggested that some frequencies were easier to detect than others.

However, as shown in Table 2, there is no difference among the percentage of correct answers according to frequencies. Investigating in more detail, we turned to task completion time. Average duration of a trial was 5.98 seconds at the 0 Hz, 4.42 seconds at the 2 Hz, and 5.24 seconds at 5 Hz. Thus, the average duration of a trial is decreased at the 2-Hz frequency. Although, inconclusive, we suggest this indicates that subjects found the sensations delivered at this frequency to be easier to detect. In this section, the performance of the tactile display module has been verified. Especially, its capability of displaying Braille for the blind was proved. In addition, an appropriate stimulating frequency has been investigated.

Table 2

Percentage of correct answers according to frequencies.

	0 Hz	2 Hz	5 Hz
Average percentage of correct answers	79.9	81.9	82.7
Standard deviation	18.6	12.3	9.2

Here, we have some issues to be discussed. As mentioned previously, since the blind people are familiar with rubbing surface to read the Braille, we are not sure that stimulation of 2 Hz is effective for the blind. In fact, after they participated in the experiments, they commented that static display was easier to discriminate than vibrational stimuli. We have to consider user's familiarity when we design tactile stimuli.

4.2. Simulation of button pressing sense

One of the most frequent complaints when using a touch screen is ambiguity about whether a screen tap has resulted in a successful button press. Researchers have proposed that there is a touch screen providing active touch feedback to address this issue [25]. In a previous version of the Ubi-Pen, there is a short-term vibration feedback for notifying button clicking [20]. In a different manner, the Ubi-Pen II also possesses the ability to produce a click-like sensation with an impact generator.

As shown in Figure 6, button pressing is composed of 3 steps. The first step is increasing pressing force. The second step is button pressed state after sudden falling down when the pressing force is greater than a threshold. The third step is releasing the button with an abrupt rising up. We do not have to consider the first step since it naturally occurs on a touch screen. The touch screen itself provides a function of button pressing with a threshold pressure; and the keys of the second and the third steps are sudden change of movement. Because the sudden change is a kind of impact, we can simulate the second and the third steps with our haptic stylus including an impact generator. As shown in Figure 6, the falling down collision of the mass inside the generator gives effect of the button pressing. The rising up collision of the mass provides sense of the button releasing to users.

Figure 6

Procedure of button pressing sense.

Here we test the effectiveness of this feature. We presented subjects with a simple calculator interface, shown in Figure 7. They had to enter each of the 6 equations shown on the right of the screen. Each equation was randomly presented and haptic feedback was also randomly provided in half the trials. Subjects had to calculate every equation twice until they obtained the correct answer to each. This calculator displayed only the results of calculations, not the figures entered. In this study, we measured task completion time

Figure 7

Calculator and presented equations.

The experimental results in Table 3 show that the clicking sense feedback of the Ubi-Pen II decreased the length of time to enter the calculations. The major influence of the click sensation was to add self confidence to users, and this contributed to the production of fewer errors and the reduced duration of the calculations. We asked each participant about the effectiveness of clicking sense feedback and they all agreed that clicking sense feedback gives self confidence and reality. Additionally, we had a chance demonstrating the Ubi-Pen II at an IT exhibition show and 145 of 160 visitors agreed that proposed scheme provide users with reality of a button. From this test, the effectiveness of the Ubi-Pen's button pressing feedback has been verified.

Table 3

Effectiveness button pressing sense feedback.

	Average duration of calculation	Standard deviation
Without haptic feedback	14.04 (sec)	2.62
With haptic feedback	10.66 (sec)	2.15

5. COMBINATION OF FORCE FEEDBACK AND TEXTURE FEEDBACK

5.1. System and experimental design

Currently, the PHANToM is the most widely used haptic interface. It has force feedback capabilities and it provides a stylus-like handle interface [26]. Here we replace its handle with the Ubi-Pen II to add tactile feedback capability to the device. Since the Ubi-Pen provides both impact and texture stimuli, this allows us to compare the effectiveness of various haptic stimulation methods.

In our previous experiment, the previous version of the Ubi-Pen provided texture feedback and vibration feedback [20]. However, we reported that vibration potentially had problems in aspect of control. The stylus is replaced by the Ubi-Pen II in this experiment. We conduct similar experiment here, but we observe the effectiveness of impact feedback on texture display. As shown in Figure 8, the proposed pen-like interface was attached to the handle of a force feedback device (model: PHANToM Omni). In order to test performance of the system, we designed a virtual tangible object. The virtual object is a box and its stiffness is 2 kN/m. (The task in this experiment does not require high interaction force.) The widths are 75 mm (300 pixels) and 67.5 mm (270 pixels). The upper surface of the box has a texture derived from texture mapping an image and a user explores only upper surface. In order to use the image as a texture, this test provides a symbolic pointer in the shape of a square, with a size of pixels. A user can load any gray-scale image. As shown in Figure 9, when the user touches an image on the box with the integrated interface, the area of the cursor is divided into 9(= ) subcells and the average gray value of each cell is calculated. Then, this averaged gray value is converted to the intensity of the stimuli displayed on each pin of the tactile display.

Figure 8

Force and tactile feedback interface.

Figure 9

Methodology of texture display according to the stimulation method.

In this interaction, the stiffness of the box is represented by the PHANToM force feedback device. However, the texture on the surface can be represented in 3 ways. The first is through force feedback presented by the PHANToM since we can feel texture by probe scanning. The second is texture feedback by the Ubi-Pen since the pin's movement can display surface roughness. The third is the Ubi-Pen's impact feedback since such stimuli could facilitate the recognition of obstacles when rubbing a surface. We compared all the 3 possible stimulation methods in this experiment as shown in Figure 9. As mentioned above, the area of virtual cursor is divided into 9 cells each with an individual gray value. However, while the tactile display inside the pen interface has 9 spatially distributed stimulators, the impact and force feedback interface both have only one interaction point. Therefore, force feedback and impact feedback use only the center value.

In case of force feedback, the gray value is converted into the height of pattern and its highest value is 1 mm. In case of tactile feedback, the gray value is converted into the normal displacement of each pin and the maximum displacement is 1 mm. When we use a pin-array-based tactile display, representing resolution of the tactile display is determined by the resolution of the pin-array. Thus, only tactile display with high density pin-array is the solution of the high-resolution display. In order to make up this limitation, we derived an idea that the tactile display plays a role of a texture magnifier. As shown in Figure 10, size of the tactile display is 2.4 times bigger than the symbolic pointer. This kind of skill may decrease reality in aspect of size, but it is a useful tip to convey texture information to a user precisely when we use a low-density pin-array.

Figure 10

Methodology of pattern display.

In case of impact feedback, haptic cues indicate change of region while the pointer across over the texture pattern. When the pointer moves inside texture area, the mass rises up and a user recognizes a ridge of the pattern. When the pointer escapes texture area and the gray value decreases under a threshold value, the mass falls down and the user experiences sudden drop-like feeling. This kind of stimulation may not precisely represent projected shapes of textures that could be effective to display surface patterns.

In order to compare the performance of all stimulation methods, we prepared 3 groups of tactile patterns. Figure 11(a) shows 5 image samples from group I which differ in the direction of the gratings they feature. The size of each image was pixels. Figure 11(b) shows image samples from group II which contains grooves of varying widths. A user feels horizontal gratings while rubbing the surfaces. In order to discriminate these patterns, the tactile stimuli must be integrated with movements on the plane. Figure 11(c) shows 5 image samples from group III, each of which shows different shapes. Discriminating among these patterns will require precise and accurate integration of the tactile cues with the movements on the surface. Feeling distributed pressure (as with the pin array display) may help users to discern the surfaces.

Figure 11

Texture samples.

Ten subjects participated in the experiment. In each trial, one of the five images from one of the groups was texture mapped on the upper surface of a virtual box. However, the graphical representation was hidden, and only a blank surface displayed. When the user touched and rubbed the surface of the object, the gray values of the image were conveyed to the haptic interface. They were then required to state which texture was present. The subjects have shown all images patterns through another screen in order to make their choice. All texture images in a group were presented 4 times at random and the order of test group was also randomly selected. The user felt the stiffness of the box by force feedback, but there were three conditions for representing texture: force feedback, tactile feedback, and impact feedback. In order to prevent practice effects, the order of the stimulation method was also randomized. Finally, sounds produced during the interaction may affect recognition performance, so participants were required to wear noise cancelling headphones (Bose, QuietComfort2).

5.2. Performance and discussion

Table 4 shows experimental results for the force feedback case in the form of a confusion matrix. Likewise, Tables 5 and 6, respectively, show the experimental results for tactile and impact feedback. In case of force feedback, average percentages of correct answers are 86.5% for group I, 73.5% for group II, and 60.5% for group III. In case of tactile feedback, average percentages of correct answers are 97.5% for group I, 91.5% for group II, and 80.5% for group III. In case of impact feedback, average percentages of correct answers are 83.5% for group I, 81.5% for group II, and 61.0% for group III. Figure 12 shows the mean durations of trials in each condition. The experimental results for force feedback and tactile feedback are similar to the previous paper's results [20]. This confirms that both previous and new experimental results are reliable. In case of impact feedback, since impact plays a role of cue to notifying change of texture, experimental results are a bit similar to the case of vibration feedback previously observed.

Table 4

Experimental results for force feedback (%).

Force feedback		1	2	3	4	5
Group I	1	95.0	2.5	—	2.5	—
	2	—	75.0	5.0	12.5	7.5
	3	7.5	5.0	85.0	2.5	—
	4	—	—	5.0	95.0	—
	5	15.0	2.5	—	—	82.5
Group II	1	82.5	2.5	7.5	7.5	—
	2	2.5	67.5	—	12.5	17.5
	3	12.5	10.0	75.0	—	2.5
	4	—	12.5	—	82.5	5.0
	5	2.5	20.0	5.0	12.5	60.0
Group III	1	55.0	15.0	12.5	17.5	—
	2	22.5	60.0	15.0	—	2.5
	3	25.0	7.5	55.0	12.5	—
	4	7.5	5.0	10.0	67.5	10.0
	5	7.5	15.0	—	12.5	65.0

Table 5

Experimental results for tactile feedback (%).

Tactile feedback		1	2	3	4	5
Group I	1	100.0	—	—	—	—
	2	—	100.0	—	—	—
	3	—	—	97.5	2.5	—
	4	—	—	7.5	92.5	—
	5	—	—	—	2.5	97.5
Group II	1	95.0	—	2.5	2.5	—
	2	—	100.0	—	—	—
	3	—	7.5	92.5	—	—
	4	—	—	—	97.5	2.5
	5	—	22.5	—	5.0	72.5
Group III	1	60.0	17.5	12.5	10.0	—
	2	10.0	90.0	—	—	—
	3	5.0	—	95.0	—	—
	4	—	—	7.5	82.5	10.0
	5	10.0	—	—	15.0	75.0

Table 6

Experimental results for impact feedback (%).

Impact feedback		1.0	2.0	3.0	4.0	5.0
Group I	1	85.0	—	5.0	—	10.0
	2	5.0	90.0	5.0	—	—
	3	—	—	85.0	10.0	5.0
	4	—	10.0	15.0	75.0	—
	5	7.5	—	10.0	—	82.5
Group II	1	95.0	5.0	—	—	—
	2	5.0	85.0	—	5.0	5.0
	3	2.5	10.0	82.5	5.0	—
	4	—	—	5.0	85.0	10.0
	5	—	25.0	10.0	5.0	60.0
Group III	1	55.0	25.0	—	15.0	5.0
	2	10.0	60.0	10.0	15.0	5.0
	3	10.0	—	70.0	10.0	10.0
	4	15.0	5.0	15.0	55.0	10.0
	5	5.0	5.0	15.0	10.0	65.0

Figure 12

Duration of each trial.

The texture samples assigned in group I can be discriminated by detecting the direction of the gratings. Users can recognize the direction from the position of the interaction point and the direction in which they rub. In this case, there is no substantial difference between force feedback and impact feedback. However, tactile display provides line load to the finger along the gratings. As shown in Tables 4, 5, and 6 as well as Figure 12, this makes human recognize direction of the gratings more correctly and quickly.

For group II, the images can be discriminated by the variations in the spacing between the ridges. However, the spatial resolution of the human arm is not sufficient to reliably detect variations on the scale of millimeters whereas the skin sense allows discrimination of submillimeter gaps [17]. In addition, pattern display by force feedback inherently results in movement of the arm and even stick slip vibration, factors which may disturb discrimination of gap variation. Therefore, as shown in Table 4, the percentage of correct answers for force feedback is lower than in the other conditions. A good example is that users experienced difficulty discriminating between sample 2 and sample 5. In the case of the tactile feedback, the narrow gaps are discriminated though the skin. This shows the best performance. In the case of the impact feedback, the participants typically rubbed the surface at a constant speed and felt the frequency of the stimulation. This technique was also effective.

As mentioned in Section 5.1, in order to recognize shape of a pattern, the tactile stimuli must be accurately integrated with movements on the plane. However, arm movements do not guarantee the high spatial resolution required for this. For example, when sample 3 of group III was presented, users found it hard to discern it from the other samples; but, in case of the tactile feedback, the distributed pressure cues enabled them to make more accurate choices.

If the tactile display had more pins, it might show better performance. However, over all the tests, the haptic device combined with the built-in compact tactile display showed satisfactory results. Impact feedback was also reasonably effective in texture display with force feedback.

6. APPLICATION OF THE Ubi-Pen II

6.1. Image display on touch screen

As shown in Figure 13, the Ubi-pen mouse enables tactile pattern display when the scheme described in Section 5.1 is applied to the image on a touch screen. In order to verify texture display performance of the Ubi-Pen, the image samples from Section 5 were reused. One of five images from one of the groups was displayed on the screen, but hidden from the participant. Instead, the visual representation was of a blank square the same size as the image. When a user rubs against this square, the gray values from the image are presented to the tactile display on the Ubi-Pen. The experimental results are shown in Table 7 and these data verify that the Ubi-Pen and image display scheme are effective. This scheme can be applied to educational programs for children or interactive drawing software. In the future, this kind of technology could be the basis of a virtual interactive shopping mall.

Figure 13

Tactile image display on a touchscreen.

Table 7

Experimental results.

	Percentage of correct answers					Duration of a trial (second)
	S1	S2	S3	S4	S5	Ave./Std.
Group1	97.5	92.5	85.0	95.0	92.5	10.7/2.9
Group2	92.5	100	77.5	97.5	75.0	13.4/4.0
Group3	62.5	77.5	80.0	72.5	95.0	20.6/10.7

6.2. Medical applications

One possible application of the combination of force and tactile feedback is a palpation medical simulator. Palpation is a kind of diagnosis based on pressure and pressure distribution. Therefore, when we develop a haptic palpation simulator, both force and tactile display interface are required. Kim et al. [27] proposed a palpation simulator based on this structure. However, their tactile display was somewhat cumbersome. The use of our tactile display or the Ubi-Pen might enhance the usability of this system; and there have been many other studies for haptic medical simulators which required a compact tactile display for more realistic and effective skin sense feedback.

6.3. Additional applications

As tested in Section 4.1, one of the most practical uses of our compact tactile display is Braille display. In particular, it can realize a highly portable Braille display. However, we need to conduct more precise evaluations before construction such a system.

Finally, the tactile display module could be installed in new mobile communication devices as well as PDAs and mobile computers.

7. CONCLUSION

This paper presents the Ubi-Pen II, a pen-like haptic interface with a built-in compact tactile display and an impact module, as well as empirical studies on Braille, button, and texture display. Its performance is verified in a series of preliminary evaluations which indicate that it can satisfactorily represent tactile patterns and provide impact feedback. The compact tactile display can represent Braille patterns and the impact feedback provides an effective button pressing sense which can increase user confidence. Furthermore, we investigated its applicability to combined force and tactile feedback interfaces in a haptic device with a pen-like end effecter. Force feedback, tactile feedback, and impact feedback have been compared for texture display. Of these three, combining tactile feedback with force feedback showed enhanced performance. Finally, we evaluated the Ubi-Pen II's capacity to support touch screen operations by providing tactile cues when a user rubs an image displayed on a monitor.

Future work involves improving the performance and usability of the Ubi-Pen II. To make the interface a stand-alone system, a processor and power controller should be embedded into the pen. The future version will be an interactive wireless interface; and more psychophysical and physiological studies will be involved in the next experiment for the Braille and texture display.