JP2023007867A - Detection device and detection method - Google Patents

Abstract

To efficiently detect a curve mirror from a photographed image.SOLUTION: A detection device comprises: a calculation unit; a generation unit; and a detection unit. The calculation unit calculates a circular object passing through an edge with Hough transformation for obtaining the perfect circular shape on the basis of the edge of a photographed image obtained by imaging the travel direction of a vehicle. The generation unit generates an integrated circular object formed by integrating portions of the plurality of circular objects whose distance between center coordinates is equal to or less than a threshold among the circular objects calculated by the calculation unit. The detection unit detects a curve mirror imaged in the photographed image on the basis of the integrated circular object generated by the generation unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a detection device and detection method.

Conventionally, a technology has been disclosed in which a curved mirror is extracted from an image captured around the vehicle, and a moving object approaching or moving away from the vehicle is extracted based on the movement vector of the feature point reflected in the curved mirror. (See, for example, Patent Document 1).

JP-A-2005-234999

However, in the conventional technology, there is room for improvement in efficiently detecting curved mirrors from captured images.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a detection apparatus and a detection method capable of efficiently detecting a curved mirror from a captured image.

In order to solve the above-described problems and achieve the object, the detection device according to the present invention includes a calculator, a generator, and a detector. The calculator calculates a circular object passing through the edge by a Hough transform for obtaining a perfect circular shape, based on the edge of the image captured in the traveling direction of the vehicle. The generation unit generates an integrated circle object by integrating some of the circle objects calculated by the calculation unit, the distance between the center coordinates of which is equal to or less than a threshold. The detection unit detects a curved mirror appearing in the captured image based on the integrated circle object generated by the generation unit.

According to the present invention, it is possible to efficiently detect a curved mirror from a captured image.

FIG. 1 is a diagram showing an overview of the detection method. FIG. 2 is a block diagram of the detection device. FIG. 3 is a schematic diagram showing time-series changes of integrated circle objects. FIG. 4 is a flow chart showing a processing procedure executed by the detection device.

Hereinafter, embodiments of the detection device and detection method disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

First, an outline of a detection device and a detection method according to an embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an overview of the detection method. Note that the detection method according to the embodiment is executed by the detection device 1 shown in FIG.

As shown in FIG. 1, the detection device 1 is mounted, for example, on the own vehicle MC. For example, the detection device 1 is a drive recorder equipped with a camera that captures an image of the traveling direction (for example, forward) of the own vehicle MC, and has an obstacle detection function of detecting an obstacle from an image captured by the camera.

For example, as shown in FIG. 1, assuming a case where the vehicle MC enters a T-junction, the driver of the vehicle MC normally looks at the curve mirror M to see the T-junction from the other direction. The presence of another vehicle C entering is recognized. On the other hand, the detection device 1 performs image processing on the captured image to detect an obstacle such as another vehicle C reflected in the curved mirror M and, for example, notify the driver of its presence.

By the way, in detecting the curved mirror M from the captured image, a method of obtaining an elliptical shape that indicates the characteristic of the shape of the curved mirror M by Hough transform is conceivable. However, in general, the Hough transform dramatically increases the processing load as the number of variables increases. Therefore, compared with the Hough transform for obtaining a perfect circular shape, the Hough transform for obtaining an elliptical shape requires an enormous amount of calculation, which is not preferable from the viewpoint of the processing load.

Therefore, in the detecting device 1 according to the embodiment, the curved mirror M is detected using the Hough transform that obtains a perfect circular shape. Specifically, the detection device 1 first performs a Hough transform for obtaining a perfect circular shape based on the edges of the captured image I (step S1). Note that the edges of the captured image I are extracted by a known method based on the luminance gradient of each pixel of the captured image I or the like.

Here, in the Hough transform for obtaining a perfect circular shape in step S1, a perfect circular circular object o passing through each edge is calculated. In addition, in the Hough transform for obtaining a perfect circular shape, out of an infinite number of perfect circles that pass through each edge, the number of circles that pass through a large number of edges, in other words, the number of circles that are likely to be actually captured in the captured image I is Find the center coordinates and radius.

Subsequently, the detecting device 1 generates an integrated circle object oi by integrating a portion of a plurality of circles whose center coordinate distance is equal to or less than a threshold among the circle objects o calculated in step S1 (step S2).

For example, in the process of step S2, the integrated circle object oi is generated by calculating the distance between the center coordinates of each circle object o and integrating the trajectories of the circle objects o for which the calculated distance is equal to or less than a threshold.

In the example shown in FIG. 1, the integrated circle object oi has an elliptical shape. Therefore, the processing in steps S2 and S3 can be said to be processing for indirectly extracting an elliptical shape from the captured image I.

That is, the detection device 1 indirectly obtains the edge shape of the curved mirror M from the plurality of circular objects o. This makes it possible to obtain an elliptical shape while avoiding the Hough transform for obtaining an elliptical shape.

After that, the detection device 1 detects the curved mirror M appearing in the captured image I based on the integrated circle object oi (step S3). For example, the detection device 1 detects one of the integrated circle objects oi as the curved mirror M. FIG.

As described above, the detection device 1 according to the embodiment calculates the circular object o by the Hough transform that obtains a perfect circular shape, and integrates the calculated circular objects o to generate the integrated circular object oi.

Then, in the detection device 1 according to the embodiment, the curved mirror M reflected in the captured image I is detected using the integrated circle object oi. As a result, the detection device 1 according to the embodiment can detect the elliptical curved mirror M while avoiding the Hough transform that obtains an elliptical shape, so that the curved mirror M can be efficiently detected from the captured image I. can be done.

Next, a configuration example of the detection device 1 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram of the detection device 1. As shown in FIG. Note that FIG. 2 also shows a camera 100 and a display 110 in addition to the detection device 1 . The camera 100 captures an image of the traveling direction of the host vehicle MC at predetermined intervals. For example, the camera 100 includes a lens such as a fisheye lens, and an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

The display 110 is an in-vehicle display, and displays various kinds of information (for example, a warning image) regarding an obstacle appearing on the curved mirror M detected by the detection device 1 . Further, the display 110 may have an audio output unit and output warning sounds as various types of information regarding obstacles. Also, as the display 110, for example, a so-called head-up display that projects an image onto the windshield of the vehicle MC may be employed.

As shown in FIG. 2 , the detection device 1 includes a storage section 10 and a control section 20 . The storage unit 10 is configured by a storage device such as a nonvolatile memory, data flash, or hard disk drive, for example. The storage unit 10 stores mirror detection information 11, feature point information 12, various programs, and the like.

The mirror detection information 11 is various information regarding the detection result of the curved mirror M. FIG. For example, information about the relative position, size, direction, etc. of the vehicle MC and the detected curved mirror M is stored in the storage unit 10 as the mirror detection information 11 .

The feature point information 12 is various types of information regarding feature points extracted from the captured image I. FIG. For example, information about time-series changes for each feature point is stored as feature point information 12 in the storage unit 10 .

The control unit 20 includes a calculation unit 21, a generation unit 22, a detection unit 23, a determination unit 24, and an output unit 25, and includes, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM ( Random Access Memory), hard disk drives, input/output ports, etc., including computers and various circuits.

The CPU of the computer functions as the calculation unit 21, the generation unit 22, the detection unit 23, the determination unit 24, and the output unit 25 of the control unit 20 by reading and executing programs stored in the ROM, for example.

In addition, at least one part or all of the calculation unit 21, the generation unit 22, the detection unit 23, the determination unit 24 and the output unit 25 of the control unit 20 are ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc. It can also be configured with hardware of

The calculation unit 21 calculates a circular object o passing through the edge by a Hough transform for obtaining a perfect circular shape, based on the edge of the captured image I that captures the traveling direction of the own vehicle MC. For example, the calculation unit 21 acquires the captured image I from the camera 100 and extracts the edges of the captured image I by edge extraction processing on the captured image I.

Subsequently, based on the edges extracted from the captured image I, the calculation unit 21 calculates a circular object o that passes through each edge by a Hough transform that obtains a perfect circular shape. Here, the Hough transform for obtaining a perfect circular shape is a process of detecting a perfect circle from the captured image I, and a circular object o passing through a plurality of edges is calculated by the Hough transform.

For example, the calculator 21 calculates a myriad of circular objects o from one captured image I by a circular Hough transform. After calculating the circle object o, the calculation unit 21 passes information about the circle object o (eg, center coordinates, radius) to the generation unit 22 .

Here, for example, the calculation unit 21 may set an edge composed of pixels having an orange color as a calculation target of the Hough transform based on the color information of each pixel. This is because curved mirrors generally have an orange appearance.

That is, by focusing on the orange color, it is possible to subject only the edges having the characteristic of the color of the curved mirror to the Hough transform. As a result, the number of edges used for the Hough transform can be reduced, thereby reducing the processing load.

The generation unit 22 generates an integrated circle object oi by integrating some of the circle objects o calculated by the calculation unit 21, the distance between the center coordinates of which is equal to or less than a threshold value. For example, the generating unit 22 generates an integrated circular object oi by integrating circular objects o whose circumferences intersect with each other.

At this time, the generating unit 22 may generate the integrated circular object oi by, for example, integrating a part of the circular object o so as to form an elliptical shape. As a result, the elliptical shape can be derived from the perfect circular shape, so that the processing load can be reduced compared to the case of using the Hough transform for obtaining the elliptical shape.

The detection unit 23 detects the curved mirror M appearing in the captured image I based on the integrated circle object oi generated by the generation unit 22 . For example, the detection unit 23 detects, as the curved mirror M, one of the integrated circle objects oi that is closest to a preset curved mirror shape. Upon detecting the curved mirror M, the detection unit 23 writes the detection result to the storage unit 10 as the mirror detection information 11 .

Further, the detection unit 23 may detect the curved mirror M using information about each integrated circle object oi in each time-series frame and the circle object o used to generate the integrated circle object oi.

FIG. 3 is a schematic diagram showing time-series changes of integrated circle objects. For example, as shown in FIG. 3, the detection unit 23 detects the curved mirror M in the current frame from time-series changes in the curved mirror M detected up to the current frame.

First, the detection unit 23 detects the coordinates of the center of gravity of the first object group G1, which is a set of circle objects o used to generate the integrated circle object oi in the current frame In, and The distance from the barycentric coordinates of the second object group G2, which is a set of circle objects o, is calculated. Note that the second object group G2 here is, for example, a set of circle objects o used to generate the integrated circle object oi detected as the curve mirror M in the past frame.

For example, the detection unit 23 detects the first object group G1 in which the distance between the barycentric coordinates of the first object group G1 and the barycentric coordinates of the second object group G2 (hereinafter referred to as the distance between the barycenters) is equal to or less than a threshold. To be detected. In other words, the first object group G1 for which the distance between the centers of gravity exceeds the threshold is excluded from the detection targets of the curved mirror M.

As a result, the currently detected curved mirror M is estimated as a position relatively close to the position of the curved mirror M detected in the past, so that the processing can be reduced for a part of the image. .

Subsequently, the detection unit 23 superimposes, for example, the first object group G1 and the second object group G2 on the first object group G1 whose distance between the centers of gravity is equal to or less than the threshold, and divides the overlapping area R1 and the non-overlapping area R2. Calculate the percentage. Here, superimposing the first object group G1 and the second object group G2 means, for example, matching the barycentric coordinates of the first object group G1 and the barycentric coordinates of the second object group G2. , the first object group G1 and the second object group G2 can be overlapped by sliding the second object group G2 to the position of the first object group G1.

Subsequently, the detection unit 23 detects an integrated circular object of the first object group G1 in which the overlapping area R1 between the first object group G1 and the second object group G2 is equal to or greater than the threshold and the non-overlapping area R2 is equal to or less than the threshold. oi is detected as a curved mirror M in the current frame.

In other words, the detection unit 23 detects the integrated circle object oi of the first object group G1 whose shape is similar to the second object group G2 as the curved mirror M, while detecting the first object group G1 whose shape is not similar to the second object group G2. The integrated circle object oi of the object group G1 is not detected as the curved mirror M.

In this way, by detecting the integrated circle object oi similar to the shape of the curved mirror M detected in the past as the curved mirror M, the detection accuracy of the shape of the curved mirror M can be improved.

Note that the detection process by the detection unit 23 is not limited to the above example. For example, the curve mirror M may be detected by focusing on the time-series change of the feature points on the outer edge of the curve mirror M and the time-series change of the feature points inside the mirror surface.

Specifically, since the curved mirror M is fixedly installed, the optical flow of the feature points on its outer edge depends on the movement component of the host vehicle MC. On the other hand, for example, when a moving object such as another vehicle is reflected on the curved mirror M, the optical flow of the feature points within the mirror surface contains the movement component of the moving object.

Therefore, the detection unit 23 can determine whether or not the integrated circle object oi is the curved mirror M by comparing the optical flow of the integrated circle object oi with the optical flow inside the integrated circle object oi.

It should be noted that, for example, when the host vehicle MC is traveling toward the curved mirror M, the curved mirror M reflected in the captured image I gradually becomes larger. Alternatively, the sizes of the objects oi may be compared.

In other words, the detection unit 23 may detect, as the curved mirror M, one of the integrated circle objects oi that has a larger area than or the same size as the previously detected curved mirror M.

As a result, the integrated circle object oi having a smaller area can be excluded from candidates for the curved mirror M, so that the detection accuracy of the curved mirror M can be improved.

In addition, since the curved mirror M itself does not move, it is considered that the position and size of the mirror are unlikely to change greatly between frames. Therefore, the position of the curved mirror M in the current frame may be predicted from the past detection results, and only the integrated circle object oi within the predicted range may be targeted for the curved mirror M to be detected. Note that this point can be similarly applied to the calculation processing of the circle object o. That is, the circle object o may be calculated for edges existing within the predicted range.

Based on the curve mirror M detected by the detection unit 23 , the determination unit 24 determines whether or not there is an approaching object reflected in the curve mirror M. Here, the approaching object refers to, for example, a moving body that approaches the curved mirror M from a direction different from that of the own vehicle MC. For example, in the example shown in FIG. 1, another vehicle C running toward a T-junction corresponds to the approaching object. Note that the approaching object may be a person, a bicycle, a motorcycle, or the like.

For example, when determining that an approaching object exists, the determination unit 24 notifies the output unit 25 of an instruction to output warning information. Note that the determination regarding the presence or absence of an approaching object is not particularly limited, and any method may be used, for example. Also, the orientation of the curved mirror M can be detected by an existing method.

The output unit 25 outputs the warning information to the display 110, for example, when receiving the warning information output instruction from the determination unit 24. FIG. Here, the warning information is information for notifying the driver of the own vehicle MC of the presence of an approaching object. For example, warning information includes images, illustrations, texts, voices, and the like.

Next, a processing procedure executed by the detection device 1 according to the embodiment will be described with reference to FIG. FIG. 4 is a flow chart showing a processing procedure executed by the detecting device 1. As shown in FIG. Note that the processing procedure described below is repeatedly executed by the control unit 20 of the detection device 1 .

As shown in FIG. 4, when the detection device 1 acquires the captured image I (step S101), it performs a Hough transform to obtain a perfect circular shape based on the edges of the captured image (step S102). Subsequently, the detection device 1 generates an integrated circle object oi from the circle object o calculated by the Hough transform (step S103).

Subsequently, the detection device 1 detects the curved mirror M appearing in the captured image I based on the integrated circle object oi (step S104), and determines an approaching object approaching the own vehicle MC based on the detected curved mirror M. (Step S105).

Subsequently, the detection device 1 determines whether or not there is an approaching object as a result of step S105 (step S106), and if it determines that there is an approaching object (step S106; Yes), it outputs warning information (step S107), the process ends. If the detecting device 1 determines that there is no approaching object (step S106; No), the process ends by omitting the process of step S106.

As described above, the detection device 1 according to the embodiment includes the calculator 21 , the generator 22 and the detector 23 . The calculation unit 21 calculates a circular object o passing through the edge by a Hough transform for obtaining a perfect circular shape, based on the edge of the captured image I captured in the traveling direction of the vehicle.

The generation unit 22 generates an integrated circle object oi by integrating some of the circle objects o calculated by the calculation unit 21, the distance between the center coordinates of which is equal to or less than a threshold value. The detection unit 23 detects the curved mirror M appearing in the captured image I based on the integrated circle object oi generated by the generation unit 22 .

Therefore, according to the detecting device 1 according to the embodiment, the curved mirror M can be efficiently detected from the captured image I.

By the way, in the above-described embodiment, a case has been described in which a moving object reflected in the curved mirror M is extracted from the captured image I that captures the front of the vehicle MC, but the captured image I captures the sides and rear of the vehicle MC. It may be

Further, in the above-described embodiment, when a moving object reflected in the curved mirror M is detected, the driver of the host vehicle MC is notified of its presence, but the present invention is not limited to this. That is, the present invention may be applied to automatic driving. For example, in this case, when a moving object reflected in the curved mirror M is detected, various collision avoidance measures such as deceleration may be performed.

Further, in the above-described embodiment, the case where the circle object o is calculated using the Hough transform has been described. good too.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

1 detection device 21 calculation unit 22 generation unit 23 detection unit 24 determination unit 25 output unit M curve mirror o circle object oi integrated circle object

Claims (6)

a calculation unit that calculates a circular object that passes through the edge by a Hough transform that obtains a perfect circular shape based on the edge of the captured image that captures the traveling direction of the vehicle;
a generation unit that generates an integrated circle object by integrating a portion of a plurality of the circle objects whose center coordinate distance is equal to or less than a threshold among the circle objects calculated by the calculation unit;
and a detection unit that detects a curved mirror appearing in the captured image based on the integrated circle object generated by the generation unit. The calculation unit
2. The detection device according to claim 1, wherein the circular object is calculated for the edge having an orange color based on the color information of each pixel of the captured image. The detection unit is
The barycentric coordinates of a first circle object group, which is a set of circle objects used for the integrated circle object in the current frame, and the barycenter of a second circle object, which is a set of circle objects used for the integrated circle object in the past frame. 3. The detection device according to claim 1, wherein the integrated circle object whose distance from coordinates is equal to or less than a threshold is detected as the curved mirror. The detection unit is
When the first circle object group and the second circle object are overlapped and an overlapping area is equal to or larger than a threshold and a non-overlapping area is equal to or smaller than the threshold, the integrated circular object is detected as the curved mirror. 4. The detection device according to claim 3. The detection unit is
5. The detecting device according to claim 1, wherein the integrated circular object having a larger area than the previously detected curved mirror is detected as the curved mirror. a calculating step of calculating a circular object passing through the edge by a Hough transform for obtaining a perfect circular shape, based on the edge of a captured image obtained by capturing the traveling direction of the vehicle;
a generation step of generating an integrated circle object by integrating a portion of a plurality of the circle objects whose center coordinate distance is equal to or less than a threshold among the circle objects calculated by the calculation step;
and a detection step of detecting a curved mirror appearing in the captured image based on the integrated circle object generated by the generation step.

Images