3D-3D facial registration method applied to personal identification: Does it work with limited portions of faces? An experiment in ideal conditions.

Personal identification of faces represents a challenging issue, especially for what concerns the quantification of the comparison. The 3D-3D superimposition approach proved to distinguish between matches and mismatches. However, the potential of this procedure applied to cases where only parts of faces are visible still has to be verified. This study aimed at verifying the applicability of a 3D-3D procedure to faces divided into three thirds. 3D models of fifty male subjects acquired through stereophotogrammetry were used. The 3D facial models were divided into upper, middle, and lower thirds and registered onto other models belonging to the same and different individuals according to the least point-to-point distance. In total, 50 matches and 50 mismatches were analyzed. RMS value (root mean square) of point-to-point distance between the two facial surfaces was calculated through VAM® software. Statistically significant differences between matches and mismatches in each facial third were assessed through Mann-Whitney test (p < 0.05). On average, RMS value in matches was 0.32 ± 0.12 mm in upper third, 0.36 ± 0.15 mm in middle third, and 0.40 ± 0.20 mm in lower third, respectively; in mismatches, RMS value was 1.40 ± 0.32 mm in upper third, 1.96 ± 0.58 mm in middle third, and 2.39 ± 0.90 mm in lower third, respectively. Differences in RMS values between matches and mismatches were significantly different for all facial thirds, without superimpositions (p < 0.01). This study shows that the existing 3D-3D superimposition methods may be useful also when only a limited portion of face is visible in ideal conditions. Their application to forensic cases of identification still needs to be verified.

This study examines 3D to 3D face comparisons using root mean square (RMS) distance.

RMS distances of three facial thirds were calculated in matches and mismatches.

RMS values between matches and mismatches were different for all facial thirds (p < 0.01).

3D‐3D registration and face comparison were possible under controlled conditioned with 50 subjects.

INTRODUCTION

The comparison of facial morphology represents a crucial procedure in several forensic contexts, including identification of culprits recorded by video surveillance systems [1]. Usually in forensic practice, several methods are used to compare different facial traits, being morphological analysis the most commonly used [2, 3]. Other methods involve 2D‐3D superimposition procedure, where 3D models acquired from the suspect are superimposed onto the 2D image of the culprit extracted from video surveillance records: The advantages of using 3D models derive from the chance of comparing images of the culprit and the possible suspects at the same orientation and size [4, 5]. However, this approach is affected by important limitations, including the intra‐ and inter‐observer variability in collocation of facial landmarks (especially on 2D images) [6] and the partial superimposition of linear distances recorded in matches and mismatches [4].

A potentially decisive innovation may derive in the future from the development and diffusion of a 3D‐3D approach. In literature, experiments concerning 3D‐3D superimposition have been performed so far only in ideal conditions, using 3D models acquired through stereophotogrammetric procedures. A procedure of identification based on the comparison of 3D facial acquisitions was proposed by literature, showing important advantages such as the chance of calculating differences between models on the entire surface and not according to specific facial landmarks, and the lack of superimposition between the distributions of distances obtained in matches and mismatches [7, 8]. So far, this procedure has been applied only to complete faces, where the entire facial silhouette is clearly visible; however, often forensic practice has to address cases where culprits with surface obstructions such as sunglasses, hats, masks, and helmets [9]. In these conditions, only a portion of the face is visible and may be used for 3D‐3D comparison.

The present study aimed at verifying the potential of a recently published 3D‐3D superimposition procedure in correctly distinguishing between matches and mismatches in cases where only limited portions of faces are analyzed. Results will provide novel data about the reliability of modern procedure 3D‐3D procedure for personal identification also in difficult conditions where only parts of faces are available for comparison.

MATERIALS AND METHODS

Fifty male subjects, aged between 18 and 45 years (mean age: 25 ± 7 years), were selected from a database of 3D facial models. The database includes three‐dimensional models collected for several anatomical and medical research purposes (among the others: analysis of normal facial anatomy, comparison between patients affected by congenital pathologies and control healthy groups, analysis of facial mimicry, assessment of mimicry impairment in case of facial palsy, etc.). Subjects affected by congenital or acquired pathologies influencing facial morphology, obesity, with medical history of surgery as well as subjects with beard, mustache, piercings, or jewelry potentially influencing the stereophotogrammetric acquisition were excluded from the study. Only male subjects were included in order to exclude possible RMS values fluctuations in matches and mismatches due to sexual dimorphism (especially for what concerns the possible different size of 3D facial models in males and females).

For each individual, two 3D facial models were performed in neutral position through stereophotogrammetric devices (VECTRA‐3D M3: Canfield Scientific, Inc., Fairfield, NJ). Technical characteristics of VECTRA‐3D® M3 are as follows: sample density: 1.2 mm geometric resolution; capture volume: 400 × 300 × 250 mm; and speed of acquisition: 3.5 ms [10]. The chosen system is one of the gold standard devices for 3D surface acquisition [10]. Time span ranging between the two acquisitions was between 1 and 2 min and 50 months. The earliest acquisitions from all the individuals were included in group 1, whereas the latest ones were included in group 2.

The stereophotogrammetric acquisition is entirely safe and is not affected by any biological risk. The study was approved by the University ethical committee (26.03.14; n° 92/14) and follows the guidelines provided by Helsinki Declaration. All the subjects signed an informed consent.

All 3D models were analyzed through VAM® software (Canfield Scientific, Inc., Fairfield, NJ) in .stl format. From each 3D model, a FAI (facial area of interest) was obtained, including the upper (UT), middle (MT), and lower (LT) thirds of the face. The UT area was defined as included among the trichion and right and left frontotemporale, exocanthion and endocanthion landmarks, the MT among right and left frontotemporale, exocanthion, endocanthion, zygion, and cheilion landmarks, LT among gnathion and right and left zygion, gonion, and cheilion landmarks (Figure 1) [11]. FAIs from each facial third from 3D models belonging to groups 1 and 2 were then registered one on each other according to the protocol by Gibelli et al. [7]. In detail, the FAIs from group 2 were automatically moved onto the FAIs from group 1 according to the least point‐to‐point distance on the entire surface; finally, RMS (root mean square) point‐to‐point distance between the two FAIs was automatically calculated by VAM® software. The calculation of RMS point‐to‐point distance was performed using FAI of group 1 as the reference one, separately for UT, MT, and LT (Figure 2).

FIGURE 1

Division of a facial 3D model according to facial thirds: ch, Cheilion; en, Endocanthion; ex, Exocanthion; ft, Frontotemporale; gn, Gnathion; go, Gonion; LT, Lower third. Tr, Trichion; MT, Middle third; t, Tragion; UT, Upper third; zy, Zygion [8]

FIGURE 2

3D‐3D registration of 3D models belonging to the same individual (match) and different individual (mismatch) in upper (UT), middle (MT), and lower (LT) third: (A) 3D model from group 1; (B) 3D model from group 2; (C) registration of the two 3D models according to the least point‐to‐point distance on the entire surface: Superimposed areas in different colors between the two models are visible (D) chromatic visualization of differences between the two models: In green, coincident points between the two models: In light and dark blue, recessing areas of model from group 2 according to model from group 1: In red and yellow, protruding areas of model from group 2 according to model from group 1

The procedure was repeated registering FAIs from the same individual in order to obtain 50 matches; moreover, FAIs from different individuals were registered one on each other, always registering models from group 2 onto model from group 1. Among the possible 2,450 combinations, other 50 registrations were randomly performed to obtain the mismatches group for each facial third. The number of matches and mismatches was equal in order to avoid possible discrepancies in sample size potentially influencing the results of the statistical analyses.

The entire procedure from FAI selection to calculation of RMS point‐to‐point distance was repeated for ten matches and ten mismatches by the same observer and another one to test intra‐ and inter‐observer error, expressed as absolute and relative technical error of measurement (TEM and rTEM). The reported parameters give the amount in mm and percentage, respectively, of RMS value determined by the intra‐ and inter‐observer errors, separately [12].

To verify possible influence of time span on facial differences found between 3D models from the same individual, Pearson's correlation index was calculated between the time passing between the two scans and the RMS values in the matches group (p < 0.05).

Statistically significant differences between matches and mismatches were assessed through Mann–Whitney test (p < 0.05), separately for each facial third. All statistical analyses were performed through SPSS® software (IBM, New York, USA).

RESULTS

Intra‐observer error was 1.7% both in matches and in mismatches; inter‐observer error was respectively 1.8% and 3.0% in matches and mismatches (Table 1). In all the cases, repeatability could be classified as “very good” according to Camison et al. [13].

TABLE 1

TEM (absolute technical error of measurement) and rTEM (relative technical error of measurement) of intra‐ and inter‐observer error, divided between matches and mismatches

	Intra‐observer error		Inter‐observer error
	TEM (mm)	rTEM (%)	TEM (mm)	rTEM (%)
Matches	0.01	1.7	0.01	1.8
Mismatches	0.03	1.7	0.05	3.0

No significant correlation was found between time span and RMS values in matches group (correlation index: 0.27; p > 0.05).

On average, RMS values in matches were 0.32 ± 0.12 mm in the upper third, 0.36 ± 0.15 mm in the middle third, and 0.40 ± 0.20 mm in the lower third, respectively; on the contrary, in mismatches, RMS values were on average 1.40 ± 0.32 mm in the upper third, 1.96 ± 0.58 mm in the middle third, and 2.39 ± 0.90 mm in the lower third, respectively (Table 2). Differences in RMS values between matches and mismatches were statistically significant for all facial thirds (p < 0.01).

TABLE 2

Minimum, maximum, and average RMS values recorded in matches and mismatches, for each facial third: All the values are expressed in mm

	Min	Max	Mean	SD
Upper third
Matches	0.14	0.66	0.32	0.12
Mismatches	0.81	2.07	1.40	0.32
Middle third
Matches	0.11	0.68	0.36	0.15
Mismatches	1.07	3.44	1.96	0.58
Lower third
Matches	0.11	0.92	0.40	0.20
Mismatches	1.04	4.19	2.39	0.90

Analyzing the distribution of data, no overlaps were found between the group of matches and mismatches for any facial third (Figure 3). For all the facial thirds, false match rate (FMR) and false non‐match rate (FNMR) amounted up to 0%.

FIGURE 3

Distribution of RMS (root mean squared) values recorded from all the comparisons in different facial thirds: Within the red circle the group of matches. LT, Lower third; MT, Middle third; UT, Upper third. No overlapping of RMS values between matches and mismatches can be found in any facial third

DISCUSSION

Facial identification still represents a challenge in forensic practice, mainly for the difficulties in reaching a quantification of the comparison between facial images [1, 14]. Moreover, the presence of surface obstructions may reduce the visibility of the entire facial morphology, with clear limits in assessing a reliable analysis [15].

The development of 3D acquisition systems has enabled the operators to devise novel methods of facial identification, based on manual or automatic procedures [16]. Modern 3D‐3D superimposition methods, in ideal conditions, proved to reliably distinguish among matches and mismatches of facial models acquired through stereophotogrammetric devices [7]. In this case, 3D analysis is performed through a registration between two 3D facial models and the difference is expressed as RMS point‐to‐point distance between them on the entire surface. A recent study verified that the 3D‐3D superimposition procedure was able to distinguish matches from mismatches, without overlapping of RMS values [7]. However, the reliability of the proposed method has still to be verified in cases where only a limited facial portion is visible.

The present study showed that also when only portions of the entire face expressed as facial thirds (upper, middle, and lower) are visible, the current 3D‐3D superimposition procedure can clearly distinguish between matches and mismatches, without overlaps in values of RMS point‐to‐point distances. In detail, in the upper third, matches yielded the maximum RMS value of 0.66 mm, whereas the minimum RMS value in mismatches was 0.81 mm. For the middle third, the maximum RMS value in matches was 0.68 mm, whereas the minimum RMS value of mismatches was 1.07 mm. Finally, in the lower third, RMS values of all the matches were below 1.00 mm, whereas in mismatches, they were always higher.

These results suggest that 3D‐3D superimposition techniques are highly promising for personal identification, as already demonstrated in other fields of personal identification [17, 18]. In addition, the proposed method may represent an improvement in comparison with existing 2D‐3D registration procedures (where a 3D model is superimposed onto a 2D image), which are based on calculation of distances between corresponding facial landmarks identified on the two compared images; in fact, the proposed protocol is not based on facial landmarks, but for the initial definition of FAI, as the calculation of RMS value is performed point‐to‐point on the entire surface of the model. In fact, literature has widely demonstrated that the use of facial landmarks is not sufficient for personal identification [19]; moreover, their collocation is affected by important intra‐ and inter‐observer variability for some of them [6].

However, some limitations to the present study need to be adequately discussed. First, we have to observe that the proposed procedure was applied to ideal 3D images acquired through high‐quality stereophotogrammetric devices; from this point of view, the used models are far from being representative of common images used for personal identification in forensic contexts, for example, from video surveillance systems. In fact, in spite of the extensive advancements recorded in the field of 3D technologies, the resolution of existing cameras is still inadequate to produce a 3D facial model of sufficient quality for being analyzed through the proposed 3D‐3D registration procedures [20]. Moreover, the comparability between 3D facial models obtained through 3D surface acquisition devices and from video surveillance cameras has still to be verified, especially considering that the superimposition of 3D models acquired through different devices usually leads to an increase in RMS values recorded between the two surfaces [1]. Therefore, the next studies will focus on the application of the proposed protocol to 3D models acquired through conventional video surveillance cameras.

Secondarily, another possible obstacle to personal identification through not only modern 3D‐3D but also traditional 2D‐3D procedures is represented by voluntary facial mimicry: The proposed 3D‐3D method has been applied only to faces in rest position where only involuntary facial movements were unavoidable. This variability was assessed also by the present study, as average RMS values seemed to increase passing from the upper to the middle and lower third, both in matches and mismatches, where facial involuntary mimicry is more pronounced. However, the possible influence of more important facial modification due to voluntary facial mimicry has to be explored, especially for what concerns modern 3D‐3D methods. Moreover, none of the recruited subjects for the present study had facial hair, as it reduces the quality of the obtained 3D model; from this point of view, this limit represents an additional limit in applying the present results to the forensic context.

Finally, the present article took into consideration a hypothetical scenario where subjects show only a part of the entire face; therefore, cases of surface obstructions were not considered and still represent a challenge for any morphological comparison of faces [21].

In conclusion, the present study shows that modern 3D‐3D superimposition methods can reliably distinguish between matches and mismatches also in cases when only a part of the entire face is visible. Once again, results suggest the high advantages which may derive from the proposed procedure; however, the present results cannot still be applied to the real case scenarios, although data obtained in ideal conditions are promising. The next studies will focus on verifying the reliability of 3D‐3D superimposition procedure also in cases where only 2D images of low quality are available.