GB2202104A

GB2202104A - Ranging by correlation

Info

Publication number: GB2202104A
Application number: GB08803289A
Authority: GB
Inventors: Walter Prendin; Fabio Rocca; Danilo Maddalena; Ciro Cafforio
Original assignee: Tecnomare SpA
Current assignee: Tecnomare SpA
Priority date: 1987-02-13
Filing date: 1988-02-12
Publication date: 1988-09-14
Anticipated expiration: 2008-02-12
Also published as: FR2611053B1; IT1202534B; FR2611053A1; GB2202104B; GB8803289D0; IT8719364A0

Abstract

Equipment for continuously detecting the distance from said equipment of a fixed or moving point comprises four stores 9,10,11,12 in which successive images from two synchronized video cameras 1,2 are stored alternately. A cursor is displayed 22 on one image. A tracking computer 16 searches for the new position of the designated point in the successive image from the same camera by comparing an area, or window, in the neighbourhood of the cursor and analogous windows in the successive image. A distance-measuring computer 17 detects, by means of comparisons of windows, the point corresponding to the new position of the point in the corresponding image from the other camera. The difference in position between these points gives the desired distance value. <IMAGE>

Description

"EQUIPMENT FOR CONTINUOUSLY DETECTING THE DISTANCE FROM IT OF A PREFIXED, ALSO MOVING, POINT" The object of the present invention is a special equipment which, by analysing images of television type, makes it possible the distance between a framed point and the shooting system to be continuously measured, also in the presence of a relative motion between the framed point and the shooting system.

The detection of such distance is very important, in particular for the following application: - the automatic handling of things in the robotics field; in such case, in fact, it is necessary to supply the distance of the things to be grasped too, in order that a whatever automatic system may correctly address the mechanical arm; - the automatic positioning of submarine vehicles in the nearby of offshore structures; in this case, it is important to know the distances between the vehicle and a whatever point of the offshore platform, in order to be able to stabilize the position of the vehicle relatively to the platform: a stable position of such vehicles makes it possible, in its turn, such operations as inspections and maintenance operations to be carried out successfully; - submarine re-entries; during these operations, inserting a riser (in practice, a tube) hanging from a boat, or from a pontoon, on a structure laying at sea-bottom level (a well head, or BOP = Blow Out Preventer") is required; knowing the distance between the end of the riser, and the structure at sea-bottom level makes it possible this operation to be made easier, and the relevant times and costs to be reduced.

The presently available techniques for measuring the distances in the above described fields are substantially of acoustical, mechanical or optical type.

The acoustical measurements can be carried out by using an equipment which operates in a similar way to a sonar: the device sends a sound impulse with suitable frequency and length towards the point whose distance has to be measured, and measures the reflected echo.

From the time elapsed between the emission of the impulse and the reception of the echo, the distance of the point can be computed. This system of measurement can be used both in air and in a submarine environment.

The main drawback shown by this method appears in case of relative motion between the measurement system, and the point to be measured.

In this case, it is necessary to continuously modify, by a manual procedure, the direction of the sound beam, in order to keep it always correctly aiming at the interesting point.

Another technique for measuring the distances, used in a submarine environment, provides for a certain number of "transponders" - i.e., of devices which emit a sound impulse responding to a received impulse - to be used. In this case too, the times elapsed between the inquiry impulses and the responses of the various transponders are used in order to measure the distances. For example, in case of submarine re-entries, the transponders are positioned on the sea-bed, in the nearby of the interesting structure (i.e., a well head) and/or on the same structure, and these transponders are then calibrated in order to learn the actual mutual position.

The acoustical transducer is located on the bottom. end of the riser, and is connected with the inquiring equipment, located at sea surface, by means of an electrical cable.

The main drawbacks shown by this latter technique relate both to the need of calibration, with a consequent time waste, and to the need of using at least four transponders, which require servicing (for example, the periodical replacement of the batteries, which, obviously, become exhausted with use). In order to perform the above said servicing, the transponders have to be lifted to the sea surface, and, after the servicing, they have to be positioned again on the sea bed, and recalibrated.

The mechanical methods provide for the measurement of the distance to be carries out by means of, e.g., a calibrated rope placed between the point to be measured, and the measuring equipment: the length of the unwound rope directly supplies the desired distance value, also in case the point is moving relatively to the measuring device.

In such case, the practical drawbacks are anyway evident, which are due to the need of fastening the rope to the interested point, and to the encumbrance caused by the presence of the same rope.

Finally, the optical methods typically use two mutually parallel objectives and a system of adjustable mirrors and/or prisms, in order to superimpose the two so-obtained images to each other: by measuring the angles of the mirrors/prisms, and the distance between the objectives, the distance of the desired point is computed (telemeter).

Other optical methods provide, on the contrary, for a modulated laser beam to be used. In both cases, the considerable drawback has to be coped with, that resort has to be made to manual operations in order to keep the measurement equipment correctly aiming at the selected point in case of a relative motion.

The purpose of the present invention is to obviate the above-said drawbacks, and therefore provide an equipment which continuously supplies the measure of the distance between a point and the same measuring equipment with no need for auxiliary systems (e.g., transponders, or calibrated ropes), and which, furthermore, features the continuous tracking of the selected point, such as to supply correct values also in case of a relative motion between the selected point, and the measurement equipment.

This purpose is substantially achieved by means of two telecameras having mutually parallel optical axes, and synchronised with each other, which scan, by successive frames, the same shot scene, inside which the selected point, whose distance has to be kept measured, may move. The video signals relevant to successive frames shot by each of both telecameras, after sampling and conversion into digital form, are stored in succession into either of two storages per each telecamera, from one of which a tracking computer copies to its working storage a small reference area, or window, in the neighbourhood of said selected point now displayed by means of a cursor positioned by means of a joystick, and compares it to analogous comparative areas or windows drawn from the following frame, or image, contained in the other storage and selected within neighbours close to the reference window. When the window which contains the new position of the selected point is found, said window is copied to the working storage of a distance-measuring computer, which searches, by comparison, for the most similar window which is present in the corresponding image stored inside one of the two storages relevant to the other telecamera, and then determines the difference between the positions of these two latter windows, from which the value of the sought distance is then easily computed.

In the following, the invention is better clarified by referring to the hereto attached drawings, which depict a preferred form of practical embodiment supplied for purely exemplifying and non-limitative purposes, in that technical, technological or structural variants may be supplied at any time, without departing from the scope of the present invention.

In such drawings: Figure 1 shows, by a block diagram, the equipment for continuously detecting the distance from it of a prefixed point, accomplished according to the present invention; Figure 2 shows the reference area, or reference window, of a frame, or image, of a telecamera used by the equipment of Figure 1; Figure 3 shows the comparative areas, or comparative windows, close to the reference window, analysed on the following frame or image shot by the same telecamera by the equipment of Figure 1, in order to determine the new position of the point; Figures 4 and 5 respectively show the reference area or window of the new found position of the point, as well as the comparative areas or windows analysed on the corresponding image shot by a second telecamera, used by the equipment of Figure 1 in order to determine the sought distance.

Referring to the Figures, two similar telecameras 1 and 2 are shown mounted by means of a bracket 3, in such a way that their optical axes are mutually parallel, and their line scannings and frame scannings are maintained synchronized, by means of the connection 4, by the synchronism signals generator 5, which also supplies the timing signals for the remainder of the equipment, as disclosed in the following.

The electrical video signals coming from the two telecameras (the right-hand telecamera 1, and the lefthand telecamera 2) are simultaneously sampled and converted into digital form by two analog-digital converters, respectively indicated by the reference numerals 6 and 7, controlled by the synchronism generator by means of the connection 8.

The two video signals sampled and converted into digital form by said analog-digital converters 6 and 7 are then stored into four' storages of RAM (Random Access Memory) type, of which: the storages 9 and 10 are destined to store the images coming from the right-hand telecamera 1, and the storages 11 and 12 are destined to store the images coming from the left-hand telecamera 2.

Using a double storage for each image makes it possible, as is better clarified in the following, the image stored in a storage to be processed simultaneously to the storing of the successive image into the other storage The sequences of storing of the successive images into the four storages 9, 10, 11, 12 are supplied by an address generator 13 on the basis of the timing signals coming from said synchronism generator 5.

More specifically, the address generator 13 enables to weriting the storages 9 and 11 by means of the control impulses transmitted through the connection 14, and the storages 10 and 12 by means of the control impulses transmitted through the connection 15. When the two storages 9, 11 or 10, 12 are completely loaded with the relevant video signals, they turn into free for reading by the tracking computer 16 and by the distance-computing computer 17, respectively connected to them by means of the connections 19 and 18, and enabled to access said storages in succession by the synchronism signals generator 5 through the connections 20 and 21.

The purpose of the tracking computer 16 is of -supplying, for each video image, the position (in terms of coordinates inside the frame) of the point selected as the reference point by the operator, whose distance has to be measured, as well as of tracking it, if it is a moving point, until it remains inside the frame shot by the telecameras.

The selection of the reference point is carried out by means of a video monitor 22 shunted from one of the two outputs from the telecameras 1 and 2, e.g., shunted from the output 23 of the left-hand telecamera 2 (see Figure 1), which displays the output signal from said telecamera 2, on which a cursor (e.g., a small cross) is overlapped, which must be positioned on the prefixed point of the shot image. This overlapping is carried out by means of an electronic character entering means 24 interposed along the path of the TV signal between the telecamera 2 and the video monitor 22, and connected to the tracking computer 16 by means of the connection 25.

On the other hand, the above-said positioning of the cursor on the point is carried out by means of a joystick 26 through the tracking computer 16, with which it is connected by means of the connection 27.

When the step of selection of the reference point is ended, the tracking computer 16 copies into its own working storage 28 a small area or window 29 (a square), around the point selected by the operator, and shown by a small cross (see Figure 2), of the video image supplied by the left-hand telecamera 2 and stored inside the storage 11, and then performs the step of tracking of the point, during which it will supply the new position of the reference point both to the distance-measuring computer 17, through the connection 30, and to the character entering means 24, through said connection 25, in order to keep the cursor overlapped to the reference point.

The reference point, in fact, could either move by its own motion, or move by apparent motion, due to the relative movement of the shooting system (telecameras) relatively to the same point.

The tracking of the reference point is carried out by the computer by comparing the above said reference window 29 stored inside the working storage 28 to comparative areas or windows 31, 32, 33,..., having the same dimensions as of the reference window 29, contained inside the corresponding left-hand storage 12 ,and ~relevant to a successive image or frame (see Figure 3).

This comparison is accomplished by means of a verisimilitude function, for example, the mutual correlation, whose value is computed by the computer in correspondence of the above-said comparative windows 31, 32, 33,...

The computer selects these windows inside neighbourhoods close to the reference window 29, and large enough as to comprise all of the possible positions due to the (either true or apparent) shift of the reference point during the period of two successive frames or images.

In order to make this search step as short as possible, the tracking computer performs it, according to the present invention, by successive approximations. In other terms, during a first pass, the computer 16 does not perform the search on all of the possible new windows 31, 32, 33,... existing in the nearby of the reference window 29, but it only takes into consideration a portion thereof, a It e rnati,ve ly excluding some of them (e.g., every fourth new window). Subsequently, the accuracy is enhanced by using a larger and larger number of windows in the neighbourhood of the new point determined by the preceding processing.

This process enables the new true position to be found very quickly, which the movable reference point has come to during the period of two successive frames or images.

At the end of the above-said process, the information of the new position of the reference point is transmitted to the distance-measuring computer 17 by means of the connection 30, and furthermore the tracking computer 16 sends the command through the connection 25, to the character entering means 24, for this latter to delete the cursor from the old position of the point, and to print it again in the new position, after which the above disclosed cycle is repeated again, as applied to the successive images or frames shot by the telecameras, and alternatively stored into said storages.

Once that the tracking computer 16 ends the search for the new position of the reference point relevant to the image stored inside the left-hand storage 12, the distance-measuring computer 17 starts computing the distance from the telecameras 1 and 2 of said reference point, by using the stereoscopic technique which enables the measure of said distance to be automatically obtained (i.e., without any actions by the operator), with a considerable accuracy.

Those skilled in the art know that, according to the above-said stereoscopic technique, the sought distance is defined by the values of the difference in position of the reference point on the images or frames simultaneously shot by the right-hand telecamera 1 and the left-hand telecamera 2.

In order to achieve the above said end result within minimum computation times, according to a feature of the present invention, the distance-measuring computer 17 measures such difference by operating according to two distinct steps following each other.

In the first step, it supplies a rough information as to the above-said distance by copying from the lefthand storage 12 to its own working storage 34 a small -reference area or window 35 (see Figure 4) around the new reference point found (always represented by the crossshaped cursor), searching, by comparison, the most similar window 36, 37,... (see Figure 5), of the same dimensions as of the reference window 35 contained inside the corresponding right-hand storage 10, and computing the difference in position between these two windows 35 and (36 or 37 or .

In case the telecameras 1 and 2 are used positioned at a same level, the computer 17 reduces the area of this search inside said right-hand memory 10 to the only possible windows, i.e., to those windows which are located to the left relatively to the reference window 35, and aligned to one another along a horizontal line (see Figure 5) and performs the comparison by means of the same technique as used by the tracking computer 16, looking for the maximum of the verisimilitude function.

It is however clear that, instead of a positioning side-by-side along a horizontal line of the telecameras 1 and 2, any other assemblage arrangement is possible as well, e.g., the telecameras can be positioned side-byside along a vertical direction, in which case the search for the difference in position by the distance-measuring computer 17 will be carried out along a vertical line only.

However, the so-obtained difference in position has a measurement accuracy which obviously depends on the resolution of the sampling of the signals, performed by the elements 6 and 7, and is therefore limited.

In order to considerably improve this accuracy, the distance-measuring computer 17 continues operating by -performing the second step, which consists in determining, for each point inside the concerned window (35), a refining of the so computed position difference value, in order to obtain a more accurate value of the sought distance.

Said refinement is accomplished by the distance meauring computer 17 for each individual point x. of the above-said window, by means of the following iterative computation, derived from the Taylor series development truncated to the linear term: f2(xi) - f1(xi + ai-1) ai = ai-1 + -------------------- f'1(xi + ai-1) wherein:: i is an index ranging from 1 up to the total number of point contained in the above said window, a. is - the already determined rough value of the difference in position, f2(xi) and f1(xi + ai-1) are the respective values of luminance in the point x. of the left-hand window 35 and in the point (xi+ai-1) of the right-hand window 36 or 37 or ... (see Figure 5), and the derivative f'1(xi+ai-1) is computed by means of the approximation of the finite differences.

Once that the values of the above said differences in position are computed, the same distance-measuring computer 17 converts them into distance values, whose arithmetic average is then sent, by means of the connection 38, to the character entering device 24, for it to be displayed by means of the monitor 22.

It is finally evident that the invention is not limited to the disclosed form; so, instead of two computers, as used in the block diagram shown in Figure 1, it is possible to use one computer only, which sequentially performs both the tasks performed by the two computers, as well as, by using a suitable system of mirrors and/or of prisms, using only one telecamera is possible, to which both the images are conveyed, each of which occupies half screen. Furthermore, in place of the mutual correlation, in the operation of comparison in order to determine corresponding windows, other verisimilitude functions, such as, e.g., the sum of the absolute values of the deviations between corresponding values of the two compared windows, can be used.

Claims

1. Equipment for continuously detecting the distance therefrom of a fixed or moving point, characterised in that it comprises two telecameras whose optical axes are parallel to each other, and which are'synchronized with each other by a synchronism signal generator, the output from each of said telecameras being fed, through an analog-digital converter, to two stores in which, on command by an address generator, driven by said synchronism signal generator, successive video images or frames shot by the telecamera are alternately stored, the outputs from said stores being passed to (a) a tracking computer capable of comparing a reference area or window positioned around said point, which was displayed on one of the two telecameras by means of a cursor entered by the same computer using a character entering device and positioned using a joystick, and then stored in one of the said stores, with analogous comparative windows detected in the subsequent image or frame contained inside the other of the said stores in order to determine the new position of the point in this latter subsequent image, and (b) a distance-measuring computer to which the output from said tracking computer is also passed, said distance-measuring computer being capable of comparing a new reference area or window positioned around said new determined position of the point with analogous comparative areas or windows detected in the image or frame stored inside the corresponding store of the other two stores relevant to the other telecamera in order to find the corresponding position of said point in this latter image, and of measuring the difference in position between these two points, from which the required distance value is computed, said computers being furthermore synchronized, said computers being furthermore synchronized with each other by the above said synchronism signal generator, and said distance measuring computer being connected with said character entering device, in order to move the said cursor to the new determined position of said point, and preferably means being finally provided for improving the accuracy of the measurement of the above said difference in position.

2. Equipment according to claim 1, wherein, in order to improve the accuracy of the measurement of the above said difference in position of the points, the distance-measuring computer is adapted to determine more accurate values of the difference in position for the various points Xi of the window which by means of the formula: f2(xi) - f1(xi + ai-1) ai = ai-1 + - f'1(xi + ai-1) wherein:: i is an index ranging from 1 up to the total number of points contained in the above said window, ai is the already determined rough value of the difference in position, f2(xi) and F1(xi + ai-1) are the respective values of luminance in the point x. of the reference window and in the point (x +a > of the comparative window, and 1 i-I the derivative f'1 is computed by means of the approximation of the finite differences, in converting said values of the difference in position into distance values, and in computing the arithmetic average of these latter values.

3. Equipment for continuously detecting the distance from it of a prefixed point, also in the presence of a relative motion between said point and said equipment, characterized in that it is constituted by two telecameras with optical axes parallel to each other, and synchronized with each other by a synchronism signal generator, the output from each of said telecameras being respectively connected, through an analog-digital converter, to two storages into which, on command by an address generator, driven by said synchronism signal generator, two successive video images or frames shot by the telecamera are alternatively stored, the outputs from said storages being respectively connected with a tracking computer capable of comparing a

reference area or window positioned around said prefixed point which was displayed on one of the two telecameras by means of a cursor entered by the same computer by means of a character entering device and positioned by means of a joystick, and then stored into one of the above-said two storages, to analogous comparative windows detected in the subsequent image or frame contained inside the other of the above-said two storages in order to find out the new position of the point in this latter subsequent image, as well as, with a distance-measuring computer to which too the output from said tracking computer is sent, said distance-measuring computer being capable of comparing a new

reference area or window positioned around said new found position of the point to analogous comparative windows detected in the image or frame stored inside the corresponding storage of the other two storages relevant to the other telecamera in order to find the corresponding position of said point in this latter image, as well as of measuring the difference in position between these two points, from which the sough distance, value is computed, said computers being furthermore synchronized with each other by the above said synchronism signal generator, and said distance measuring computer being connected with said character entering device, in order to move the above said cursor

to the new position of said prefixed point, and X means being finally provided for, in order to improve the accuracy of the measurement of the above said difference in position.

4. Equipment for continuously detecting the distance from it of a prefixed point according to claim 3 characterized in that said means for improving the accuracy of the measurement of the above said difference in position of the concerned point consist in determining, by means of the said distance-measuring computer, more accurate values of the difference in position for the various points x. of the concerned window by means of the formula: f2(xi) - f1(xi + ai-1) ai = ai-1 + -------------------- f'1(xi + ai-1) wherein: i is an index ranging from 1 up to the total number of points contained in the above said window, a. is the already determined rough value of the difference in position, f2(xi) and f1(xi + ai-1) are the respective values of luminance in the point x of the reference window and in the point (xi-a; 1) of the comparative window, and the derivative f' 1 is computed by means of the approximation of the finite differences, in converting said values of the difference in position into distance values, and in computing the arithmetic average of these latter values.

5. Equipment for continuously detecting the distance

from it of a

point according tok claims 12, characterized in that said computers compare said reference window to the said analogous comparative windows alternatively excluding some of them.

Equipment for continuously detecting the distance

from it of a

point according tok claims characterized in that the above said two telecameras with optical axes parallel to each other are mounted side-byside to each other along a horizontal line, or along a vertical line.

Equipment for continuously detecting the distance from it of a prefixed, also moving, point as substantially herein disclosed and illustrated.