Sensor for high speed, high precision measurement of 2-d positions.

A sensor system to measure the 2-D position of an object that intercepts a plane in space is presented in this paper. This sensor system was developed with the aim of measuring the height and lateral position of contact wires supplying power to electric locomotives. The sensor comprises two line-scans focused on the zone to be measured and positioned in such a way that their viewing planes are on the same plane. The report includes a mathematical model of the sensor system, and details the method used for calibrating the sensor system. The procedure used for high speed measurement of object position in space is also described, where measurement acquisition time was less than 0.7 ms. Finally, position measurement results verifying system performance in real time are given.

Introduction

The use of visual information to detect the position of objects in relation to other objects is a fundamental function of computer vision systems. Many methods and applications have been developed in order to perform this task, all with their respective advantages and disadvantages with regards to computational efficiency, complexity, robustness, accuracy and performance. In the majority of cases, more than one camera [1-4], or a camera and a structured light source [5-7] have been used in order to establish the position of an object. In some applications, the use of line-scans has contributed to an overall improvement of the system, as measurement acquisition is faster than with matrix cameras, less information needs to be processed, and sensors with greater spatial resolution (higher number of pixels) can be used [8,9]. However, a disadvantage of linear sensors is that neither traditional calibration methods nor the object detection algorithms developed for matrix camera based systems can be applied [10-12]. The calibration patterns used in calibration of the matrix cameras, used in 3D measurements, can not be used for calibration of line-scan sensors, because it is virtually impossible to match the captured line-image with the interest points of the patterns (circles centers, interception lines).

In the calibration of line-scan sensors, we cannot get 3D measurements without a priori assumption of one coordinate, so it is more accurate in this case to optimize the parameters for a 2D calibration. A line-scan calibration method using calibration pattern line-images in different positions is presented in [13]. Although this requires extreme precision when positioning the pattern, and could thus represent a disadvantage, [14] describes the resolution of this potential problem through the use of calibration patterns at different depths. Nevertheless, when more than one line scan is used, these methods are only capable of obtaining individual intrinsic parameters for each line-scan. However, it is not possible to obtain accurate extrinsic parameters, as one of the limitations of these methods is that the line-scan sensor array must be approximately parallel to the calibration pattern planes. If two line-scans are used to measure position with triangulation techniques, both sensors must be separated and thus the pattern cannot be situated in such as way that the planes are parallel to the sensor arrays.

In this study, we used a 2-D sensor based on two line-scans. Section 2 describes the sensor employed and the sensor modeling. Section 3 presents the calibration method used. Section 4 gives the experimental results. Finally, Section 5 summarizes the main conclusions.

Sensor System

The overall function of the sensor system is based on the capture of contact wire images with two line-scans. Following line-image processing and triangulation, it is possible to calculate 2-D coordinates for the objects in relation to a specific reference system.

The system comprises a computer with an image acquisition and processing board (IAB) for each line-scan, as shown in Figure 1. This board is responsible for information transfer (images and control) between the cameras and the PC. The PC performs line-image processing to determine the 2-D coordinates for objects. Image acquisition, control, processing and data presentation is carried out using software in C language.

Figure 1.

Block diagram of 2-D sensor system.

Sensor Modeling

The reference coordinate system does not usually coincide with camera or line-scan coordinates (Figure 2). To resolve this issue, and thus obtain coordinates for the line-image of a 2-D point in space with respect to a reference system, a projective transformation in 2-D is performed. This enabled us to obtain camera system coordinates which corresponded to a scene point.

Figure 2.

Relation between line-scan coordinate system and world coordinate system.

Using Figure 2 and a 2-D projective transformation, it was possible to change from one coordinate system to the other (Equation 1): where R is the rotation matrix defined by: and T is the translation vector which defines the relative position between the optical centre of the line-scan camera and the world coordinates centre (Equation 3):

If Equation (1) is expressed by homogeneous coordinates, we obtain Equation (4): the values represented in the projection model were calculated through camera calibration. These extrinsic parameters (t) link the relative position between the world coordinate system and the camera coordinate system.

Using the pin-hole camera model for a 1-D sensor, as is the case of line-scans, the projection of a point P(x) from the scene onto a line-image will bear the coordinate x:

If (5) is shown in matrix form and with homogeneous coordinates, we obtain:

Substituting (4) in (6), a general expression is obtained for relating a point in the world coordinate system with its corresponding projection onto the line-image:

A diagram explaining the pin-hole model for a line-scan is shown In Figure 3. If x is represented by pixel coordinates x, and we take into account that the optical axis may coincide with a pixel c different to the centre of the sensor, it is then possible to formulate the Equation (8).

Figure 3.

Diagram explaining the pin-hole model for a line-scan.

The scale factor s (mm/pixel) is the parameter which relates the line-image system of metric coordinates to the pixel array coordinate system provided by the line-scan. This value corresponds to pixel size. In this case, the theoretical value given by the manufacturer is 12 μm.

Substituting (5) in (8) and doing fx = f/s, the Equation (9) is obtained, which models the line-scans according to the parameters of the pin-hole model:

Using (9), Equation (7) can be rewritten in the following form:

When the intrinsic parameter matrix M and the extrinsic parameter matrix M are multiplied, a general expression for the projection matrix M = M · M is obtained, which represents the relation between the scene points and their projection onto a line-image.

If the matrix coefficients are represented by m…m, (10) they can be rewritten as (11):

Calculation of Calibration Parameters

An alternative method for obtaining projection matrix M coefficients is to assign a value to one of the coefficients (in this case, the value m = 1 is chosen), and to express the other projection matrix coefficients according to this value (12). Thus, the Direct Linear Transformation (DLT) coefficient vector L = [L] is obtained:

If the DLT coefficient vector is substituted, and the matrices in (11) are multiplied, the unknown quantity producing scene point projection onto the line-image is found:

As can be seen, Equation (13) has five DLT coefficients (L … L), so at least five 2-D point correspondences, visible to both line-scans, are necessary. Therefore, the pattern must have at least five known points to establish their correspondence with the captured line-image.

The number of points of correspondence between the real world and line-images is represented by h. The more points used, the greater calibration accuracy becomes. A matrix of h rows is formed, where each row corresponds to a point in the pattern:

To find L, the least squares estimate is used:

The use of m = 1 is justified because the solution is subject to a scale factor, given that the projection matrix is homogeneous. The parameter m is the t component of the translation vector which locates the line-scan in the world 2-D reference system. Thus, if t were null, it would not be valid for the proposed solution. The parameter m is obtained from the L and L parameters of the DLT coefficient matrix:

To obtain the projection matrix M from the DLT coefficients, an inverse scale change is carried out. This is achieved by multiplying each of the elements calculated by m:

Intrinsic parameters

Once the projection matrix has been calculated, calculation of the intrinsic parameters is a simple operation:

Extrinsic parameters

The extrinsic parameters are obtained as follows:

Calibration Pattern

A fundamental steep in the calibration process is the selection of an adequate calibration pattern. As for the calibration of matrix cameras, 3-D patterns offer the best results for line-scans calibration. In this case, calibration was carried out using a 3-D calibration pattern, comprising a series of parallel threads in different positions (Figure 4). The pattern is located in such a way that the threads cross the vision plane of the line-scans perpendicularly, as shown in Figure 5.

Figure 4.

Calibration pattern comprising threads.

Figure 5.

Diagram that show the viewing planes crossed by calibration pattern threads.

When many reference threads are used in the proposed pattern, overlapping of the different threads projected may occur. This can be detected by the lack of concordance between the number of reference points in our pattern and the number of points seen in the line-images.

Calibration Results

In our case, calibration was carried out using a total of 16 threads in the pattern. Table 1 gives the calibration parameters obtained, and calibration error, ε. This error quantifies the difference between the coordinates for each point on the real line-images x with respect to those calculated by means of its projection x. Applying the projection matrix x for each of the calibration pattern h points:

Table 1.

Calibration results.

Parameter	Left hand Line-scan	Right hand Line-scan
tx, cm	53.34	−53.35
ty, cm	106.35	106.12
α, degrees	−21.8	21.7
fx	2,577.0	2,565.9
cx, pixels	1,033.9	1,019.7
ε, pixels	0.63	0.49

With the value of s = 12 μm and the scale factors obtained, focal length for each line-scan can be calculated. Focal length of the left hand line-scan is f = 30.7487 mm, and focal length of the right hand line-scan is f = 30.4995 mm.

Calculation of 2-D Position with Two Calibrated Line-scan

Once both line-scans have been calibrated, it is possible to obtain a correspondence between a 2-D point and its projection on both line-images. The model for a single calibrated line-scan, based on a pin-hole model, can be expressed by (11). If the same calibration pattern, located in a particular position, is used to calibrate both line-scans, the Equation (23) can be obtained, which establishes the relation between the two previous models, and yields the parameters [x] according to the line-images captured for each line-scan, and the corresponding projection matrices: the system of linear equations (23) represents two straight lines which are cut at the points [x]. In this system of equations, the other unknown parameters are m and m. To obtain the geometric location of the points [x], an inverse operation to that carried out for Equation (23) is performed, giving (24): with the projection matrices for each line-scan and the Equation (24), it is possible to calculate the 2-D position of a point in the measurement zone, whose projection in each line-image is and . Using the n (n = 2048) values of and , we calculated two matrices of nxn (2048 × 2048), where each value corresponded to the position in x (lateral decentring) and the position in y (height). We called these matrices the “Sensor Matrices”.

To summarise, the system for measuring 2-D position is defined by two matrices, called “sensor matrices”, which contain the coordinates (x,y) for each scene point projected onto the line-scans at the coordinate and . By reading these matrices, the geometric location of a scene point can be identified. For this, it is only necessary to know pixel position in the projection of the object to be measured on each line-scan. These pixel values are then used to read the sensor matrices, stored during the calibration process. In this way, the calculation time for measurements carried out in real time is reduced.

Experimental Results

In this section, we describe some practical experiments which were carried out with the aim of verifying the performance of the measuring system proposed. The experiments were aimed at establishing the accuracy of measurements under real operating conditions. In order to achieve this, measurements were taken of a moving contact wire, to verify efficiency of the tracking algorithms. In addition, static measurements were taken of the threads in different positions, in order to calculate magnitude of error in measurements.

Measuring the 2-D Position of Static Objects

The aim of this experiment was to verify the accuracy of lateral decentring (X) and height (Y) measurements taken with the system when the system sensor and the measured objects were static. The calibration pattern structure, with 16 white 0.5 mm diameter threads, was placed in a known position with the coordinates (X). The base reference system was situated at a central point between the two line-scans. Once the position measurements had been taken with the system sensor (X), the values obtained were compared with the real values. Figure 6 shows the calibration pattern with the 16 reference threads placed in the sensor measurement zone. The various threads are numbered and marked by a yellow dot.

Figure 6.

Diagram showing the planes of vision to be crossed by the calibration pattern threads. The distance between the line-scans is 106.5 cm, and the angles α = 67.68 degrees and α = 67.28 degrees.

As an example, Table 2 gives both real and measured coordinate values for the threads in one particular calibration pattern position. The same experiment was carried out for different thread and pattern positions, but always ensuring that the position of the threads within the measurement zone coincided. From a total of 524 measurements, maximum error of x was 2.1 mm, and standard error was 0.82 mm. For height measurements (y), maximum error was 3.2 mm and standard error was 0.94 mm. This error was due to incorrect thread placement, calibration error and/or sensor system resolution.

Table 2.

Real and sensor measured coordinates for different points (threads).

Thread	XS1 realmm	XS1 sensormm	Errormm	YS1 realmm	YS1 sensormm	Errormm
1	−490	−489.6322	0.3678	1,070	1,069.5631	0.4369
2	−220	−219.7668	0.2332	1,070	1,071.817	1.817
3	0	0.1267	0.1267	1,070	1,069.8788	0.1212
4	220	221.2802	1.2802	1,070	1,069.5836	0.4164
5	490	489.8626	0.1374	1,070	1,071.0224	1.0224
6	−490	−490.0087	0.0087	1,370	1,368.6644	1.3356
7	−220	−220.5975	0.5975	1,370	1,369.9805	0.0195
8	0	−0.5668	0.5668	1,370	1,369.8184	0.1816
9	220	219.1745	0.8255	1,370	1,370.5471	0.5471
10	490	490.9952	0.9952	1,370	1,369.9767	0.0233
11	−220	−220.2104	0.2104	1,670	1,671.0066	1.0066
12	0	−0.4616	0.4616	1,670	1,669.8844	0.1156
13	220	219.5798	0.4202	1,670	1,668.9608	1.0392
14	−220	−219.9084	0.0916	2,030	2,031.5914	1.5914
15	0	1.7755	1.7755	2,030	2,029.9706	0.0294
16	220	218.9772	1.0228	2,030	2,030.5424	0.5424

Measuring the 2-D Position of a Moving Object

This second experiment aimed to verify the validity of the monitoring algorithm and the system's capacity for measuring the position of an object (contact wire) moving at high speed.

To move the contact wire one end was attached to a bearing placed in a constant position so that only the wire could rotate. The other end was attached to another support located on an aluminium bar which in turn was attached to an engine rotor. The engine rotates the aluminium bar parallel to the plane of view of the line-scans. The rotor axis was positioned so that whatever the position of the rotating bar, the contact wire was always located within the field of vision of the cameras. A photograph of the experimental structure assembled in order to generate contact wire movement is shown in Figure 7.

Figure 7.

Structure assembled in order to generate contact wire movement.

The graphs in Figure 8 give height and decentring measurements at a sample speed of 100 frames per second (fps). The small jumps in the curves are due to the experimental structure used.

Figure 8.

Contact wire measurements at a sample speed of 100 fps: Height and lateral decentring.

To verify real time sensor system operation, line-scan tests were carried out for various acquisition times. Maximum acquisition and processing speed without loss of samples was 1,430 frames per second, using a PC with P4 2.0 GHz processor and 512 MB of RAM. This high speed was achieved because it was only necessary to find centroids of the line-images captured. Once the centroids had been obtained, a matrix reading is sufficient to obtain the values of x and y.

Conclusions

This paper has presented a 2-D sensor system based on two line-scans. Among other applications, it can be used to verify the geometry of contact wires supplying power to electric locomotives. A mathematical model has been reported.

In addition, has been proposed and described a method for calibrating the sensor system. This method is based on the calibration of both line-scans using the calibration pattern position. The calibration pattern is a 3-D structure with various parallel threads attached. Calibration provides the matrices containing the coordinates (x,y) for each scene point, corresponding to the projection of these points onto each line-scan. To obtain the coordinates (x,y) is only necessary to know pixel position in the projection of the object in each line-scan. These pixel values are then used to read the sensor matrices which contain the coordinates (x,y). In this way, processing time may be less than 0.7 ms.

Experiments were carried out in order to verify system operation. The first experiment examined static measurement error, and from a total of 524 measurements, the maximum error of x was found to be 2.1 mm, with a standard error of 0.82 mm. In the case of height measurement (y), maximum error was 3.2 mm and standard error was 0.94 mm. The second experiment measured contact wire position when moving, in order to verify monitoring algorithms. Maximum acquisition and processing speed without sample loss was 1430 frames per second, using a PC with P4 2.0 GHz processor and 512 MB of RAM.