RU2363018C1 - Method of selecting objects on remote background - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: method of selecting objects on a remote background involves reception and formation of reference and compared images using two identical video systems, installed at a small distance from each other about a remote background, with parallel principal optical axes, and simultaneous recording of both images. The images are recorded using two identical video systems based on multiple-element high-speed photodetectors, for example, CMOS matrices. Two differential images are formed, the first by subtracting the compared image from the reference image, and the second through subtraction of the reference image from the compared image. Negative values in the differential images are zeroed, after which coordinates of each non-zero fragment on the first differential image are associated with a shift value between the given fragment and the corresponding fragment on the second differential image, until their maximum matching in the direction of background parallactic shift, if there is such a fragment on the second differential image in the given direction. Similarly, coordinates of each non-zero fragment on the second differential image are associated with a shift value between the given fragment and the corresponding fragment on the first differential image. Distance to the detected objects is calculated using the formula R=L·A/Δ_i, where L is the distance between centres of the objectives of the video systems, and A is the distance from the video matrix to the principal optical axis of the objective of the video system. Δ_i the value of parallactic shift of the i^th image of the object.

EFFECT: increased accuracy of detecting (selecting) one or more objects with unknown brightness in conditions of high-contrast background objects located relatively near each other at various distances.

4 cl, 8 dwg

Description

The invention relates to passive optical methods for selecting objects on a complex contrast dynamically changing background in the observed scene.

A known method of detecting objects on complex backgrounds, including the operation of signal preprocessing, represented by a time sequence of images of a scene on which an object may appear, memorizing reference signals, and subtracting the corresponding reference signals from the corresponding current signals, the processing algorithm provides for the separation of each image into fragments , measuring the magnitude of the characteristics of each fragment, for example, optical density, storing the corresponding signals, and after subtracting signals, comparing the differences obtained with a predetermined threshold value, generating control signals performing fragmentary filtering of the time sequence of current images to transmit only those difference signals whose absolute values exceed the threshold value (US Pat. No. 2250478 of 04/11/2003, IPC G01S 17/06 )

The disadvantage of this method is the complexity of the differential image processing algorithm and the limitation of its application if it is necessary to detect a rapidly moving object, because there will not be enough data for this processing algorithm to provide a reliable result.

As a prototype, the method of selecting an object on a remote background was selected according to RF patent No. 2081435 dated 04/02/84, IPC G01S 17/00, which consists in receiving and forming two images at two spatially separated points, simultaneously registering the generated images, determining the parallactic background displacement by forming the inter-correlation function of two registered images and determining its maximum, the displacement of the first of the registered images by the amount of background parallactic displacement in the direction opolozhnom this offset, obtaining a differential image by subtracting the offset and second registered images, dividing the differential image into areas having opposite signs, and analysis fragments areas.

The known method has the following disadvantages.

1. Limited functionality, since the known method requires complete certainty with the brightness of the desired object compared to the background. The initial uncertainty in the brightness of the object in the event a group of objects appears in the controlled zone will inevitably lead to false positives.

2. Limited applicability, since the algorithm of the known method is not designed for background objects located at different distances from the observation point, and not at infinite distance.

The challenge facing the present invention is to increase the accuracy of detection (selection) of one or more objects of unknown brightness in conditions of high contrast background objects relatively close at different distances.

The problem is solved as follows.

In the method of selecting objects on a remote background, which consists in receiving and forming a reference and compared images with two identical optical systems installed at a small distance from each other relative to the remote background, with parallel main optical axes and simultaneously recording both images, according to the invention, the images are recorded by two identical video systems based on multi-element high-speed photodetectors, for example CMOS-matrices, in this case two difference images are formed phenomena, the first of which is formed by subtracting the compared image from the reference image, and the second difference image by subtracting the reference image from the compared, zero the negative values in the difference images, after which the coordinates of each nonzero fragment in the first difference image are associated with the shift value between the specified fragment and the corresponding fragment in the second difference image to their maximum coincidence in the direction of the parallactic displacement of the background, if such a fragment in the second difference image in the indicated direction is present, similarly to the coordinates of each nonzero fragment in the second difference image, the shift value between the specified fragment and the corresponding fragment in the first difference image is associated, the distances to the detected objects are calculated by the formula R = L · A / Δ _i , where L is the distance between the centers of the lenses of the video systems, A is the distance from the video matrix to the main optical axis of the lens of the video system, Δ _i is the parallactic shift of the ith image of the object.

The technical essence of the invention consists in suppressing a remote, high-contrast, dynamically changing background while simultaneously highlighting the coordinates of objects of different distances (relative to the background) in the processed registered images (first and second difference image). The distances to the detected objects are determined by the values of the parallactic displacements of the corresponding fragments in the processed images. As a result, information is obtained about the detected objects located at different distances to the remote high-contrast background, in the form of the coordinates of the objects in the reference image and the distances to the objects set by it.

относительно опорного изображения до максимального совпадения удаленного фона.In addition, to eliminate the error associated with the parallelism of the main optical axes of identical optical systems, before receiving the difference images, the compared image is parallactically shifted by

relative to the reference image until the remote background matches as much as possible.

In addition, to eliminate the sensitivity spread of the photodetector devices, the reference and compared images are aligned in brightness before the formation of differential images.

In addition, to eliminate noise caused, for example, by optical aberration of the lenses of optical systems, in difference images, the values are zeroed from a certain level corresponding to the sensitivity spread of the photodetectors of video systems.

Figure 1 shows a diagram of an optical stereoscopic system, consisting of two identical recording channels (A - reference, B - compared), installed at a small distance between each other relative to the distant background, with parallel main optical axes (like a human visual apparatus). Each recording channel consists of an optical lens 1 and an array multi-element photodetector 2 (CMOS matrix). The system also includes a block 3 of mathematical processing (for example, a programmable logic integrated circuit - FPGA).

On figa shows a diagram of the image registered by the reference channel A, and figb - compared channel B.

Figure 3 (a, b) presents the registered images - reference A1 (a) and compared A2 (b). On the recorded images (figure 3) on a remote high-contrast background there are objects lighter than the background surrounding them (C), and objects are darker than the background (T).

и

горизонтальная и вертикальная проекции параллактического смещения удаленного фона соответственно. Величина параллактического смещения удаленного фона принимается постоянной для оптических систем и вычисляется по формуле

.Figure 4 shows a diagram of the displacement (cropping) of the image matrices A1 and A2 relative to each other in the presence of a deviation from parallelism of the main optical axes of the recording systems, where

and

horizontal and vertical projections of the parallactic displacement of the distant background, respectively. The magnitude of the parallactic displacement of the remote background is taken constant for optical systems and is calculated by the formula

.

Figure 5 (a, b) shows the obtained first and second difference images Raznosti and Raznost2, respectively. On the received differential images (figure 5) the following images of objects are highlighted:

C1 — fragments of “bright” objects in the reference image;

C2 - fragments of “bright” objects in the compared image;

T1 - fragments of “dark” objects in the reference image;

T2 - fragments of “dark” objects in the compared image;

Fig.6 shows a diagram for determining the parallactic displacement of an object lighter (white square) than the background surrounding it (Δ ₁ , Fig.6a), and an object darker than the background - a black square (Δ ₂ , Fig.6b).

Figure 7 presents the result of determining the parallactic displacement of objects, a is lighter than the surrounding background, b is darker than the background; where the images of the objects are shown each in its own color corresponding to the offset (the color decoding of the correspondence is shown in the middle).

On Fig shows one of the options for presenting the result of the algorithm, images of objects are highlighted in color, and in this case, the color corresponds to the distance to the object (in the right corner is the color decoding).

The method is implemented as follows. Optical stereoscopic system (figure 1) simultaneously record two images (reference - figa, 3A and compared - fig.2b, 3b).

In the reference (Fig. 2a) and compared (Fig. 2b) images, the removed background is not offset, and objects closer to the background in the compared image are shifted to the right relative to the same objects in the reference image, and, as can be seen from Fig. 1, the value displacement depends on the distance of the object. Based on the largest geometric size of objects (D _max ) and their maximum distance R _max (for example, meteorological visibility range), the distance L is selected so that the displacement (Δ _i ) of the most distant object with the largest geometric dimensions is not less than its maximum size in the image in the direction of parallactic displacement (figure 1).

The reference and compared images (Fig. 3) are presented in the form of matrices A1 and A2 with a size of IxJ, respectively.

If the main optical axis of the recording systems have a slight deviation of parallelism between them, the matrices are “cut off” in the direction opposite to the parallactic background displacement (Fig. 4). We take the size of the resulting matrices equal to IxJ and keep their notation (A1 and A2).

In the event of a brightness deviation in the recorded images, caused, for example, by errors in the production of photodetectors of optical recording systems, the reference and compared images are aligned in brightness to their maximum coincidence.

The following sequence of actions carried out in block 3 of mathematical processing suppresses the remote high-contrast background and simultaneously select objects with previously unknown brightness (light objects on a dark background or dark objects on a light background).

- Form two differential images.

1. The first difference image corresponds to a matrix equal to: Raznoct1 = A1-A2.

2. The second difference image corresponds to a matrix equal to: Raznoct2 = A2-A1.

- In the received differential images, negative values are zeroed. If noise is present on the difference images, due, for example, to the optical aberrations of the lenses 1, then the zeroing of the values in the difference matrices is carried out starting from a certain positive value, i.e.:

Raznoct1 (i, J) = 0 if Raznoct1 (i, j) <b,

Raznoct2 (i, j) = 0 if Raznoct2 (i, j) <b.

The value (b) is constant and determined in advance based on the difference of the removed background in the reference and compared images.

On the obtained first difference image (Fig. 5a), fragments corresponding to lighter objects in the reference image, as well as fragments corresponding to darker objects in the compared image, are highlighted (not zeroed), and all other values are reset. In the obtained second difference image (Fig. 5b), fragments corresponding to darker objects in the reference image, as well as fragments corresponding to lighter objects in the compared image, are highlighted (not zeroed), and all other values are reset.

To calculate the distances to the selected objects in block 3 of the mathematical processing determine the parallactic displacement Δ of the fragments corresponding to the objects in the difference images. The determination of bias values is divided into two stages:

1) determination of displacements for objects brighter than the background surrounding them;

2) determining the offset for objects darker than the background surrounding them.

- The first stage (determining the displacement of the object lighter than the background) is to determine Δ as an argument to the function F ₁ (Δ), at which it is minimal.

The function F ₁ (Δ) is the module of the arithmetic mean difference superimposed on the second difference image with a coordinate offset (from 0 to dX) in the direction of the background parallax shift, a nonzero fragment of the first difference image and the corresponding section of the second difference image:

Where

Raznoct1 (X, Y) - non-zero fragment of the first difference image corresponding to the object;

Raznoct2 (X + Δ, Y) - the portion of the second differential image, the size and geometry of which corresponds to Raznoct1 (X, Y) and shifted by Δ in the direction of the background parallax;

N - fragment size (in square units of area or pixels);

dX is the maximum parallactic displacement determined by the detection range of objects and their sizes.

Moreover, the values of the function F ₁ (Δ) for fragments corresponding to darker objects are constant due to the absence on the second difference image of the corresponding fragments shifted in the direction of the parallactic shift (Fig. 6a). As a result, Δ is taken equal to zero, i.e. the fragment is equated to the area of the removed background (figa).

- The second stage (determining the displacement of the object darker than the background) is to determine Δ as an argument to the function F ₂ (Δ) at which it is minimal. The function F ₂ (Δ) is the module of the arithmetic mean difference superimposed on the first difference image with a coordinate offset (from 0 to dX) in the direction of the background parallax shift, a nonzero fragment of the second difference image and the corresponding section of the first difference image:

Where

Raznoct2 (X, Y) - non-zero fragment of the second difference image corresponding to the object;

Raznoct1 (X + Δ, Y) is the portion of the first differential image corresponding in size and geometry to Raznoct1 (X, Y) and shifted by Δ in the direction of background parallax shift.

Moreover, the values of the function F ₂ (Δ) for fragments corresponding to lighter objects are constant due to the absence on the first difference image of the corresponding fragments shifted in the direction of the parallactic shift (Fig.6b). As a result, Δ is taken equal to zero, i.e. the fragment is equated to the remote background area (Fig.7b).

The distance to the detected objects is calculated by the formula R = L · A / Δ _i , where L is the distance between the centers of the lenses 1 of the video system, A is the distance from the video matrix 2 to the main optical axis of the lens 1 of the video system, Δ _i is the magnitude of the parallactic shift of the i-th object .

As a result, at the signal output of the mathematical processing unit 3, information is obtained about the detected objects located at different distances to a remote high-contrast, dynamically changing background, in the form of the coordinates of the objects in the reference image and the distances to the objects assigned to them (Fig. 8).

The implementation of the proposed method is quite possible, because Today, the industry has mastered all the main components of the optical system, the recorder and the mathematical processing unit.

Claims (4)

1. The method of selecting objects on a remote background, which consists in receiving and forming the reference and compared images with two identical video systems installed at a small distance between themselves relative to the remote background, with parallel main optical axes, and simultaneously registering both images, characterized in that the images are recorded two identical video systems based on multi-element high-speed photodetectors, for example CMOS matrices, while two difference images are formed, the two of which are formed by subtracting the compared image from the reference image, and the second difference image by subtracting the reference image from the compared, zero the negative values in the difference images, after which the coordinates of each nonzero fragment in the first difference image are associated with the shift value between the specified fragment and the corresponding fragment in the second difference image, to the maximum coincidence in the direction of the parallactic background displacement, if A new fragment in the second difference image in the indicated direction is present, similarly to the coordinates of each nonzero fragment in the second difference image, the shift value between the indicated fragment and the corresponding fragment in the first difference image is associated, the distances to the detected objects are calculated by the formula R = L · A / Δ _i, where L - distance between centers of the lenses of video systems, video matrix a is the distance from the lens to the main optical axis of the video system, Δ _i - value with parallactic Whig i-th object image. 2. The method of selecting objects on a remote background according to claim 1, characterized in that before receiving the difference images, the compared image is parallactically shifted by

relative to the reference image until the remote background matches as much as possible. 3. The method of selecting objects on a remote background according to claim 1, characterized in that the reference and compared image prior to the formation of difference images are aligned in brightness. 4. The method of selecting objects on a remote background according to claim 1, characterized in that in the difference images, the values are zeroed from a certain level corresponding to the sensitivity spread of the photodetectors of video systems.

Images