A methodology to evaluate contact areas and indentations of human fingertips based on 3D techniques for haptic purposes.

This paper presents a methodology to study the contact of human fingers with surfaces based on 3D techniques. This method helps to investigate the fingertip mechanical properties which are crucial for designing haptic interfaces. The dependence of the fingertip deformation on the applied forces is obtained both with theoretical and experimental approaches. The experimental procedure is based on digital measurements by 3D optical scanners to reconstruct the geometry of the fingertip impression and on force measurements by an instrumented plate. Results highlight the force-displacement trend and can be validated with a Finite Element Model (FEM), with data from literature or with measurements at a force-strain gauge. Gross contact areas, radii and work of adhesion are also detected, and results are compared with contact models available in literature. • A sensorized plate with a thin force sensitive resistor and a dough material layer is used to measure the contact force corresponding to a specific digital imprint. • 3D indentation maps are obtained and evaluated by comparing the 3D scan model of fingertips during imprinting with the digital model of the undeformed fingers and of the imprints. • Force-displacement results can be validated by comparison with a developed FEM, a force-displacement gauge or literature outcomes.

Specifications table

Method details

Recently, in the robotics field, considerable progresses are carried on providing tools more and more aimed to the comparison with the human abilities. Many different grippers [1,2] or supernumerary robotic limbs [3], [4], [5] are studied and developed to grasp different objects. The knowledge of the human hands and fingers in terms of characteristics and features is fundamental for obtaining adequate functionalities of artificial limbs, as grippers and manipulators. Considering the possible application of the proposed method to artificial hands and fingers, artificial skin, and wearable haptic devices such as rings gloves and thimbles [6,7], it is very important to investigate the tactile abilities of the human sense of touch.

The presented method can be defined as part of “haptic tribology”, as defined by authors [8], since it has important implications in haptic technologies having the purpose to reproduce the human touching ability and sensing through vibrations, movements, and forces applied on the user. This is particularly important in wearable haptic devices as, for example, in the field of surgery where many devices allow to work inside the human body, but the surgeon does not have the real sense of touch.

One can observe that, for the sake of wearability, often underactuated and undersensed mechanical solutions are implemented as shown in Fig. 1.

Fig. 1

Example of a wearable haptic device for index fingertip.

The method implies the use of 3D optical scanners [9,10], and software for mesh editing and control [11,12]. The analysis of scan data and the indentation maps generation from 3D optical measurements require the knowledge of reverse engineering techniques [13,14] and mesh postprocessing skills [15]. To apply the method it is also necessary to setup and use sensorized plates mounting a thin force sensitive resistor (FSR) controlled by Arduino, as in [8]. Furthermore, a FEM software is used to generate force-displacement data for validation. Alternatively, results from literature or from measurements with a strain-stress gauge can be used for validation. All the results must be examined in consideration of tribological contact models and mechanical and biological characteristics of human fingers and skin. Thus, since the proposed method allows gathering a complete scenario of the finger deformation under certain loads, it provides a useful procedure to design and implement the human touch in haptic technologies [16,17].

Method materials and equipment

The experimental setup for the method includes the following materials:

Volunteers for the experiments

Software for FEM analysis

Sensorized plate

Dough material and flattening equipment

3D optical scanners with a high accuracy grade

Mesh editing and control software for the elaboration of 3D measurements

Digital force-displacement gauge

(a) Volunteers for experiments

Volunteers with normal conditions of the finger skin must be recruited before starting the experimental procedure. All the volunteers must be approximately the same age to guarantee a similar elasticity of the skin and subcutaneous tissues and have the index fingertip with the same width and height. They are invited to rest for half an hour before tests. Afterwards, the tests are performed by inviting the volunteers to press a finger on the sensorized plate to leave an imprint on the dough material. During this process the applied force is measured by a FSR sensor, and the finger is 3D scanned to get a reference for the subsequent generation of 3D indentation maps. Volunteers can also be invited to press the fingertip against a flat tip of a force-displacement transducer to gather results to use as validation data or as input for the 3D indentation maps.

These experiments must be conducted in compliance with the declaration of Helsinki, assuring no risks of harmful effects on health. Furthermore, no sensible data are necessary to apply the method.

(b) Software for FEM analysis

Any software to create a finite element model for the prediction of the trend force-displacement of a fingertip in contact with a rigid plate can be used: for instance, the authors used COMSOL Multiphysics. The model of the fingertip must be created considering the same dimensions and characteristics of the volunteers’ fingertips participating to the experimental campaign. Some characteristics can be found in biotribology or biology literature [18], [19], [20].

(c) Sensorized plate

The sensorized plate can be manufactured using a flat surface as for example a plexiglass sheet on top of which a thin force sensitive resistor must be attached. The advantages of these kind of sensors are simplicity and affordability. For haptic applications this sensor must have a sensing diameter compatible with the width of a human finger to allow the contact of the entire surface of the fingertip. For adult volunteers a diameter of 12 mm is sufficient, but authors used a sensor with a sensing diameter of 15 mm, as represented in Fig. 2.

Fig. 2

Force sensitive resistor - FSR.

The resistance of the FSR resistor depends on the applied pressure on the sensing area which is made of a robust polymer thick film (PTF) that exhibits a decrease in resistance with increase in applied force.

Usually, this force sensitivity resistor can be used in human touch control of electronic devices such as automotive electronics, medical systems, and in industrial or robotic applications [21].

The used FSR has an actuation force as low as 0.1 N, and can sense applied forces in a range between 100g-10kg. The FSR force curve and schematic diagram are represented in Fig. 3.

Fig. 3

FSR datasheet: (a) Force curve; (b) Schematic diagram.

The evaluation of applied force to the sensor is measured during the squeezing of the fingertip on the plate considered as unreformable and with the FRS working in the linear zone. The FSR can be connected to the open-source electronics platform “Arduino” and a program must be compiled to extract measures when the force is applied. Fig. 4 shows Arduino hardware and software.

Fig. 4

Arduino: (a) board; (b) software.

(d) Dough material and flattening equipment

A dough material (e.g. a common play dough) is needed to gather the fingertips imprints Fig. 5a. Before the tests, this material must be prepared in thin layers of about 3 mm thickness: this dimension is sufficient to accommodate the maximum indentation δ due to the deformation of a human finger-pad pressed against a flat surface, which is about 2 mm [22].To do this, a flattening device can be used, as for instance a manual rolling mill for goldsmiths (Fig. 5b).

Fig. 5

(a) Dough material; (b) a rolling mill for goldsmiths.

(e) 3D optical scanners with a high accuracy grade

The method implies the use of two different 3D optical digitizing instruments with a high accuracy grade, to allow reliable measurements.

A first 3D scanner must be used to acquire the 3D model of the fingertip when it is pressed against the sensorized plate at the specific value of analyzed force (deformed finger) and the 3D model of the undeformed finger. The model of the finger during the imprinting must be used during the method procedure to align the deformed finger on the dough material imprint and the model of the undeformed finger on the deformed one. The 3D scanner most suitable to digitize the deformed and undeformed finger is a portable digitizer which allows fast and real-time acquisitions based on feature and marker alignment. In fact, during the scanning process the 3D scanner must be moved around the finger and all the single scanning frames must be automatically aligned on the basis of the finger geometry or based on the position of physical high reflective markers. For this method, the 3D scanning of the undeformed finger must be performed by using a feature-based alignment, while the deformed finger acquisition can be done by feature and marker alignment. For this reason, portable structured light 3D scanner is the best choice. Authors commonly use a high-resolution portable 3D scanner (EinScan Pro HD, Shining 3D Tech. Co., Ltd., Hangzhou, China). This scanner can also be used as a desktop digitizer to perform the next scanning procedures envisaged by the method (Fig. 6).

Fig. 6

Structured light 3D scanner: (a) portable configuration; (b) desktop configuration.

A second 3D scanner must be used to reconstruct the digital model of the imprints on the dough material after hardening. For this procedure the most suitable instrument is a desktop 3D scanner with an automatic turntable. A single DoF (degree of freedom) turntable is sufficient to acquire the surface of the imprints but 3S scanners with multipe DoFs rotary tables are also better. Authors commonly use the portable 3D scanner represented in Fig. 6b in the desktop configuration, or desktop 3D scanners with a 1 or 2 DoFs turntable (e.g. AutoScan Inspec or EinScan-SE, Shining 3D Tech. Co., Ltd., Hangzhou, China).

(f) Mesh editing and control software for the elaboration of 3D measurements

The 3D digital models obtained by 3D scanners must be postprocessed by using proper mesh editing software to align eventual different scanning frames, fill holes, remove errors. For this purpose software like Geomagic Wrap (3D Systems, Inc., Rock Hill, South Carolina, USA), Meshmixer (Autodesk, Inc., San Rafael, California, USA) or Meshlab (ISTI - CNR, Pisa, Italy) can be used. Then a mesh to cad or mesh to mesh compare software is needed to perform the deformation analysis and measure indentations, contact areas and radii. For this purpose, software like Geomagic Wrap or Geomagic Control X (3D Systems, Inc., Rock Hill, South Carolina, USA), PointShape Inspector (DREAMTNS Co., Ltd., Republic of Korea) can be employed.

(g) Digital force-displacement gauge

A force-displacement gauge can be used to gather results to compare with 3D experimental results or FEM simulations or to have input datasets for creating the 3D indentation maps. The gauge should have a force-displacement transducer with a flat tip to be put in contact with the volunteers’ fingertips.

Method procedure and validation

The method procedure is summarized according to the following steps that are displayed in the graphical abstract.

FEM contact model set up and simulations

Fingertip impression and force evaluation

Fingertip 3D scan during imprinting

Digital impression 3D scanning

Postprocessing and alignment

Evaluation of the dough material deformation and final alignment

3D indentation maps

Method validation

Contact area experimental evaluation

Contact area theoretical evaluation

Contact area comparison

FEM contact model set up and simulations

In this step a relationship between the applied force and the fingertip deformation must be evaluated. Two methods can be used for this purpose: FEM simulations or force-displacement measurements with a digital transducer.

For setting a contact model of the fingertip, a fingertip model of an adult is represented with a multi-layered structural 3D Finite Element (FE) in contact with a flat surface (Fig. 7). Skin is a non-homogeneous, anisotropic, and non-linear viscoelastic material whose characteristics can be found in literature [23]. Skin and subcutaneous tissues are hypothesized as hyperelastic and linear viscoelastic materials; nails and bones are hypothesized as linear elastic materials.

Fig. 7

Front view of the 3D fingertip model (a) in contact with a flat surface (b).

Skin is considered as stratified in three layers (Fig. 8): the epidermis, the dermis (a dense fibroelastic connective tissue) and the hypodermis (loose fatty connective tissue). The first layer has a thickness between 75 and 150 μm and the second one is about 1-4 mm thick; all the layers are very sensitive to the tactile stimuli.

Fig. 8

Schematic cross section of human skin.

To perform the simulations all the Young moduli and the Poisson ratio of dermis, epidermidis, subcutaneous tissues, bones and nails must be known, as well as the deformability of the plexiglass plate [24]. The simulations must be carried on by statically pre-compressing the fingertip with different defined displacements imposed to the plate toward the finger. The constraints should be defined by fixing the center of the fingernail and the back cross-section of the finger in both the horizontal and vertical direction.

The use of FEM analyses allows to create simulated indentation maps to be compare with experimental ones.

Another method to get a relationship between the applied force and the fingertip displacement is to use a digital force-displacement transducer with a flat tip. In this case, volunteers must be invited to press their fingertip against a flat tip of a force-displacement transducer (Fig. 9) until the specified values of the forces established for the experiment are reached. Corresponding displacements must be recorded. Results can be used both for validation purposes and to have a force-displacement data set to use as input for the 3D indentation maps.

Fig. 9

Force-displacement tests with digital transducer.

Fingertip impression and force evaluation

The volunteers should be invited to rest for some minutes before starting the experimental campaign.

In the meantime, some thin layers of dough material can be prepared, and the first one can be attached to the FSR sensor on the sensorized plate. Then, the volunteers should be invited to press a finger on the dough material on the plate until the desired value of force is read by Arduino (Fig. 10).

Fig. 10

Force evaluation test and fingertip impression.

Different layers of dough material must be used for different force evaluations and different volunteers. Tests must be carried out in temperature and humidity standard conditions.

Fingertip 3D scan during imprinting

When the desired value of force is read by Arduino, the volunteer must be asked to keep his finger firm and the finger on the dough material must be 3D scanned capturing the entire visible surface of the dough material and the visible part of the finger. It is very important to scan the entire nail which is a rigid part to use as reference area to align the deformed and undeformed finger models.

The scanning procedure can be performed by using only a feature-based frame alignment of the scanner or also a marker alignment, depending also on the used 3D scanning instrument.

In the 3D scanner is provided with a marker alignment, authors suggest to use both the alignments by attaching on the sensorized plate some physical markers before starting the tests.

After this step, the impression on the dough material can be removed by the plate without stretching or deforming the layer and it can be put in a proper tray where it can harden.

Then another layer can be put on the plate and another test can be carried out with the same procedure.

The finger used by the volunteers during the tests must be also 3D scanned in the undeformed configuration. For this procedure the volunteers must be asked to keep the hand form while the portable 3D scanner is moved around it (Fig. 11). The feature-based alignment in this case can be sufficient as the application of markers on the skin is not always possible.

Fig. 11

Undeformed finger 3D scan.

Digital impression 3D scanning

After 24 hours of hardening the digital impressions can be digitized on the desktop 3D scanner (Fig. 12).

Fig. 12

Impression 3D scanning.

Sometimes the dough material must be sprayed with a anti-glare powder for 3D scanning, if the deformed thickness is too small. In fact, in these cases the material appears as semitransparent.

Postprocessing and preliminary alignment

All the 3D digital models of deformed and undeformed fingers and of the fingerprints must be processed by using a mesh editing software and restoring all the possible errors such as holes, spikes, self-intersections, etc. The undeformed finger model can be trimmed by the 3D scanning of the volunteer's hand and then the deformed finger model must be used as a reference to align the fingerprint model and the undeformed finger model. The fingerprint model can be aligned using the visible regions of the dough material as reference and the undeformed finger model can be superimposed on the deformed one by using the nail as reference. After the alignment is complete, the reference model of the deformed finger can be removed obtaining the preliminary alignment between the finger imprint and the undeformed finger model, as represented in Fig. 13.

Fig. 13

Preliminary alignment.

Evaluation of the dough material deformation and final alignment

Based on the Winkler bed of springs model [25], [26], [27], the value of the maximum indentation due to the dough material deformation can be calculated. On the basis of the resulting value the preliminary alignment obtained in the previous step can be corrected by subtracting the indentation due to the dough layer. To do this, the model of the undeformed finger can be moved farther from the imprint on the vertical axis, obtaining the final alignment. The vertical axis is the axis perpendicular to the plexiglass plate, which is known from the 3D scanning of the undeformed finger.

As a variant of the method, the corrected displacement representing the deformation of the fingertip can be gathered form the measurement obtained with the digital force-displacement transducer. The final alignment can be done according to these results.

3D indentation maps

From the final alignment obtained in the previous step, the 3D indentation map can be built, highlighting the deformation of the finger when it is pressed against the plexiglass plate. Fig. 14 represents a typical 3D indentation map created with the presented method.

Fig. 14

3D indentation map.

Method validation

The method can be validated by comparing the curves force vs maximum indentations resulting from the 3D indentation maps with corresponding data from the FEM analysis, from the measurements obtained by the force-displacement gauge or from literature [24,28]. Another comparison can be done by superimposing the 3D indentation maps from the FEM analysis and from the experimental campaign. An example of this comparison is reported in Fig. 15.

Fig. 15

3D indentation map comparison: (a) experimental; (b) simulated.

Contact area experimental evaluation

The final alignment used to create the 3D indentation maps can be used to measure the gross contact area. This area can be found considering the intersection between the undeformed finger and the imprint models. The resulting curve can be a 3D polyline or spline which can be projected on the plane normal to z. Then a mesh with this planar curve as border can be created and used to evaluate the contact area extent. Being the contact area elliptical, it is possible to measure the maximum and minimum axes and the equivalent contact radius corresponding to the radius of a circumference with the same surface as the contact area.

Contact area theoretical evaluation

Theoretical gross contact areas and radii corresponding to the same maximum indentations found in the previous steps, can be calculated applying different contact mechanics models, such as the Hertz theory, the Johnson, Kendall and Roberts (JKR) contact model or others, using the experimental indentations δ as inputs [19,[29], [30], [31], [32], [33], [34].

Contact area comparison

The fitting between experimental and theoretical gross contact areas and contact radii can be studied and differences or similarities can be found and analysed to find similarities or discrepancies and discussing the results [[35], [36]] (Fig. 16).

Fig. 16

Experimental contact area.

Conclusion

The methodology proposed in this paper is aimed at identifying fingertip deformation characteristics when interacting with a compliant material. From the results of this study, fingertip impedance properties can be investigated. Such information is important for understanding hand grasp properties, that can be brought in the design of effective robotic hands, prostheses, and grippers. Another important application for this study is the design of wearable haptic interfaces. Future developments of this research will include the inclusion of more users with different hand sizes.

CRediT author statement

All the authors contributed equally to all the phases of the work.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Subject Area;	Engineering
More specific subject area;	Tribology for haptics
Method name;	3D study of human touch
Name and reference of original method;	S. Logozzo, M.C. Valigi, M. Malvezzi, Modelling the human touch: A basic study for haptic technology, Tribology International 166 (2022) 107352. https://doi.org/10.1016/j.triboint.2021.107352
Resource availability;	Details are provided in the manuscript [8]S. Logozzo, M.C. Valigi, M. Malvezzi, Modelling the human touch: A basic study for haptic technology, Tribology International 166 (2022) 107352. https://doi.org/10.1016/j.triboint.2021.107352.