JP2019101470A - Vehicle outside environment recognition device - Google Patents

Abstract

To improve discrimination accuracy between a two-wheeler and a pedestrian.SOLUTION: In a vehicle outside environment recognition device, a computer functions as a candidate identifying part identifying a partial image 300, 302 that is a pedestrian or a two-wheeler, in an image, a representative identifying part identifying a representative value of a distance for each of a plurality of vertical regions S arranged in a lateral direction of the image for the partial image 300, 302, and a determination part determining whether the partial image 300, 302 is a pedestrian or a two-wheeler based on variations in representative values in the plurality of vertical regions S.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an external environment recognition device for identifying a three-dimensional object ahead of a host vehicle.

  Conventionally, there is known a technique for specifying a three-dimensional object such as a vehicle, a two-wheeled vehicle, or a pedestrian located in front of the host vehicle based on an image captured by an imaging device (for example, Patent Documents 1 and 2).

JP, 2015-195018, A Unexamined-Japanese-Patent No. 2010-097541

  When the above-described imaging device captures a motorcycle from the front or the rear, it is difficult to distinguish characteristic parts such as the wheels of the motorcycle. Therefore, even with the techniques of Patent Documents 1 and 2, it may be difficult to distinguish a two-wheeled vehicle from a pedestrian. Therefore, development of a technique for improving the accuracy of identification of a two-wheeled vehicle and a pedestrian is desired.

  An object of the present invention is to provide an external environment recognition device capable of improving the identification accuracy of a two-wheeled vehicle and a pedestrian in view of such problems.

  In order to solve the above problems, in the external environment recognition device of the present invention, a computer is arranged in a lateral direction of an image, with respect to a candidate specifying unit for specifying a partial image that is a pedestrian or a motorcycle. It functions as a determination unit that determines whether the partial image is a pedestrian or a two-wheeler based on a representative specifying unit that specifies a representative value of the distance for each of the plurality of vertical regions and a variation in the representative values in the plurality of vertical regions.

  The determination unit may determine that the partial image is a two-wheeled vehicle when the index value of the variation of the representative values exceeds a predetermined threshold.

  The computer may further function as a correction unit that reduces the estimated distance to the two-wheeler than before it was determined to be a two-wheeler.

  In order to solve the above-described problems, in another external environment recognition apparatus according to the present invention, the computer determines that the partial image in the image is a two-wheeled vehicle and the partial image is a two-wheeled vehicle Also, it functions as a correction unit that reduces the estimated distance to the two-wheeler than before it was determined to be a two-wheeler.

  According to the present invention, it is possible to improve the identification accuracy of a two-wheeled vehicle and a pedestrian.

It is a block diagram showing the connection relation of the environment recognition system outside a car. It is an explanatory view for explaining a luminosity picture and a distance picture. It is a functional block diagram showing a schematic function of an outside environment recognition device. It is a flowchart which shows the flow of an environment recognition process outside a vehicle. It is a figure for demonstrating the difference between a pedestrian and a two-wheeled vehicle.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values and the like shown in the embodiment are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the specification and the drawings, elements having substantially the same functions and configurations will be denoted by the same reference numerals to omit repeated description, and elements not directly related to the present invention will not be illustrated. Do.

(Outside environment recognition system 100)
FIG. 1 is a block diagram showing the connection relationship of the outside environment recognition system 100. As shown in FIG. The external environment recognition system 100 includes an imaging device 110, an external environment recognition device 120, and a vehicle control unit (ECU: Engine Control Unit) 130.

  The imaging device 110 is configured to include an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), captures an environment outside the vehicle 1 and includes at least information on luminance. A luminance image (color image or monochrome image) can be generated. Moreover, the imaging devices 110 are spaced apart in the substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates a luminance image obtained by imaging a three-dimensional object present in a detection area in front of the host vehicle 1 every frame (60 fps) of, for example, 1/60 seconds. Here, three-dimensional objects recognized by the imaging device 110 are only two-dimensional objects such as two-wheelers (bicycles, etc.), pedestrians (people), vehicles, traffic lights, roads (traveling roads), road signs, guard rails, buildings Also, it includes things such as the head and shoulders of pedestrians that can be identified as a part thereof.

  In addition, the environment recognition device outside the vehicle 120 acquires a luminance image from each of the two imaging devices 110, and corresponds to a block (for example, an array of 4 horizontal pixels × 4 vertical pixels) arbitrarily extracted from one luminance image. The so-called pattern matching, which is searched from the other luminance image, is used to derive disparity information including disparity and a screen position indicating a position in the screen of an arbitrary block. Here, horizontal indicates the screen horizontal direction of the captured image, and vertical indicates the screen vertical direction of the captured image. As this pattern matching, it is conceivable to compare the luminance (Y) in arbitrary block units between a pair of images. For example, SAD (Sum of Absolute Difference) which takes a difference of luminance, SSD (Sum of Squared intensity Difference) which uses the difference as a square, NCC which takes similarity of the dispersion value which subtracted the average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The environment outside environment recognition device 120 performs such block-based parallax derivation processing for all blocks displayed in the detection area (for example, 600 pixels × 200 pixels). Here, the block is 4 pixels × 4 pixels, but the number of pixels in the block can be set arbitrarily.

  However, in the external environment recognition apparatus 120, although parallax can be derived for each block which is a detection resolution unit, it is not possible to recognize what part of a three-dimensional object the block is. Therefore, disparity information is not independently derived from a three-dimensional object unit, but independently at a detection resolution unit (for example, a block unit) in a detection area. Here, an image associated with the parallax information derived in this manner is referred to as a distance image to distinguish it from the above-described luminance image.

  FIG. 2 is an explanatory diagram for explaining the luminance image 126 and the distance image 128. As shown in FIG. For example, it is assumed that a luminance image 126 as illustrated in FIG. 2A is generated for the detection area 124 through the two imaging devices 110. However, here, only one of the two luminance images 126 is schematically shown in order to facilitate understanding. The external environment recognition device 120 obtains the parallax for each block from such a luminance image 126, and forms a distance image (image) 128 as shown in FIG. 2 (b). Each block in the distance image 128 is associated with the parallax of that block. In FIG. 2, for convenience of explanation, the blocks from which the parallax is derived are represented by black dots.

  In addition, the environment recognition device outside the vehicle three-dimensional in real space including the luminance value (color value) based on the luminance image 126 and the relative distance (distance) to the vehicle 1 calculated based on the distance image 128 Blocks with equal color values and similar three-dimensional position information are grouped as three-dimensional objects to identify what three-dimensional object in the detection area in front of the host vehicle 1 (for example, pedestrian etc.) Do. Further, when the three-dimensional object is specified in this way, the outside environment recognition device 120 avoids the collision with the three-dimensional object (collision avoidance control), or keeps the inter-vehicle distance with the preceding vehicle at a safe distance. Control (cruise control). The relative distance can be obtained by converting disparity information of each block in the distance image 128 into three-dimensional position information using a so-called stereo method. Here, the stereo method is a method of deriving the relative distance of the three-dimensional object to the imaging device 110 from the parallax of the three-dimensional object by using the triangulation method.

  Referring back to FIG. 1, the vehicle control device 130 receives the operation input of the driver through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and transmits it to the steering mechanism 142, the drive mechanism 144, and the braking mechanism 146. Control the vehicle 1 Further, the vehicle control device 130 controls the steering mechanism 142, the drive mechanism 144, and the braking mechanism 146 in accordance with an instruction of the external environment recognition device 120.

  Hereinafter, the configuration of the external environment recognition device 120 will be described in detail. Here, the process of identifying three-dimensional objects (for example, pedestrians and two-wheelers) in the detection area ahead of the host vehicle 1 which is characteristic in the present embodiment will be described in detail, and the configuration irrelevant to the features of the present embodiment will be described. I omit it.

(Exterior environment recognition device 120)
FIG. 3 is a functional block diagram showing a schematic function of the external environment recognition device 120. As shown in FIG. As shown in FIG. 3, the external environment recognition apparatus 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing two-way information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 is configured by a RAM, a flash memory, an HDD, and the like, and holds various information necessary for the processing of each functional unit described below.

  The central control unit 154 is constituted by a semiconductor integrated circuit including a central processing unit (CPU), a ROM in which programs and the like are stored, and a RAM as a work area, and the I / F unit 150 and the data holding unit Control 152 and the like. Further, in the present embodiment, the central control unit 154 also functions as a solid object region specifying unit 160, a candidate specifying unit 162, a representative specifying unit 164, a candidate determining unit 166, and a distance correcting unit 168. Hereinafter, the external environment recognition processing for recognizing a pedestrian and a two-wheeled vehicle, which is characteristic of the present embodiment, will be described in detail based on the operation of each functional unit of the central control unit 154.

(Outside environment recognition processing)
FIG. 4 is a flow chart showing the flow of the outside environment recognition process. In the environment recognition process outside the vehicle, first, a three-dimensional object area specifying process (S200) is performed in which the three-dimensional object area specifying unit 160 specifies a three-dimensional object area including (photographed) a three-dimensional object Ru. Then, a candidate identification process (S202) is performed in which the candidate identification unit 162 identifies a three-dimensional object region (partial image) in which a three-dimensional object of a pedestrian or a two-wheeled vehicle is included. When a three-dimensional object region including a three-dimensional object of a pedestrian or a two-wheeled vehicle is identified, a representative identifying unit 164 performs a representative value identifying process (S204) for identifying a representative value described later. Candidate determination process (S206) to determine whether the three-dimensional object is a two-wheeled vehicle is executed, and finally the correction process (S208) in which the distance correction unit (correction unit) 168 corrects the relative distance To be executed. The process of recognizing the environment outside the vehicle is repeatedly performed for each frame in which the luminance image 126 and the distance image 128 are acquired.

(Three-dimensional object region identification processing S200)
As described above, the three-dimensional object area specifying unit 160 specifies three-dimensional objects by grouping in the detection areas of the luminance image 126 and the distance image 128. The three-dimensional object region specifying unit 160 extracts, from the luminance image 126 and the distance image 128, a three-dimensional object region (partial image) in which the specified three-dimensional object is included (captured).

(Candidate identification processing S202)
Subsequently, the candidate identifying unit 162 identifies what the identified three-dimensional object is based on the extracted three-dimensional object region. For example, as the three-dimensional object, as described above, types such as motorcycles, pedestrians (people), vehicles, traffic lights, roads (traveling roads), road signs, guard rails, buildings, head and shoulders of pedestrians, etc. are included. Be A setting condition is provided in advance for each type, and the specified three-dimensional object is determined as the corresponding type when the three-dimensional object region matches the setting condition.

  Here, it may be difficult to distinguish between pedestrians and motorcycles. For example, when the two-wheeled vehicle on which a person gets on is facing the front or the back with respect to the imaging device 110, the identification is difficult. The wheel, which is a feature of a two-wheeled vehicle, becomes a black rod-like object, for example, and the contour is difficult to identify. Moreover, the upper body of the person located above the two-wheeled vehicle is indistinguishable from that of the pedestrian. Here, for example, the two-wheeled vehicle refers to one in which the front wheel and the rear wheel are single wheels (one pair).

  Therefore, the candidate identifying unit 162 identifies the three-dimensional object as a type “pedestrian or two-wheeled vehicle” without identifying whether it is a pedestrian or a two-wheeled vehicle. The candidate identifying unit 162 identifies “pedestrian or two-wheeled vehicle” by, for example, the outline shape, size, moving speed, etc. of the three-dimensional object.

(Representative value identification process S204)
FIG. 5 is a diagram for explaining the difference between a pedestrian and a two-wheeled vehicle. In FIG. 5, two areas A and B of the distance image 128 are extracted and shown. In the region A, a partial image 300 including a pedestrian is positioned, and in the region B, a partial image 302 including a two-wheeled vehicle is positioned. Further, below the regions A and B, respectively, image diagrams 304 and 306 of representative values of relative distances are shown.

  As shown in FIG. 5, in the partial image 302, the two-wheeled vehicle is facing the back, and as described above, the rear wheel 308 is difficult to identify. Therefore, the candidate identifying unit 162 determines that the three-dimensional object is “a pedestrian or a two-wheeled vehicle” for both of the partial images 300 and 302.

  However, there are cases where a three-dimensional object can identify a pedestrian or a motorcycle. For example, if the moving speed is equal to or higher than the threshold and it can not be a pedestrian, it is identified as a two-wheeled vehicle. In such a case, the candidate identifying unit 162 identifies a pedestrian or a motorcycle without performing the following process of the representative identifying unit 164 and the candidate determining unit 166.

  Subsequently, for each of the partial images 300 and 302, the representative specifying unit 164 sets the representative value of the relative distance for each of a plurality of vertically elongated vertical regions S across the upper and lower ends of the image (screen) arranged in the horizontal direction of the distance image 128. Identify. Specifically, the distance image 128 is divided into a plurality of vertical regions S divided in the horizontal direction (horizontal direction) of the distance image 128.

  For any vertical region S, a frequency distribution of relative distances of all blocks contained in the vertical region S is generated. In the frequency distribution, for example, the number of blocks having relative distances belonging to each of the distance sections divided into predetermined distances is indicated as a frequency. Here, when the same three-dimensional object is compared, the area occupied by the three-dimensional object in the distance image 128 increases as the relative distance decreases. Therefore, for example, the central value of the distance section having the highest frequency is taken as the representative value of the relative distance of the vertical region S. As a result, the relative distance of a three-dimensional object close to the host vehicle 1 is likely to be the representative value. In addition, even if there is noise in a block whose relative distance is close, the influence on the representative value is avoided unless the frequency is high.

  Here, the case where the central value of the distance division with the highest frequency is the representative value has been described. However, for example, among the distance divisions in which the frequency is equal to or more than the predetermined value, the central value of the distance division closest to the host vehicle 1 may be a representative value. Further, the representative value may represent the relative distance of the blocks in the vertical region S, and may be, for example, an average value of the relative distances in the vertical region S.

  For all vertical regions S, representative values are similarly identified. In the image diagrams 304 and 306, representative values of the relative distance of the vertical regions S are indicated by black squares. In the image diagrams 304 and 306, the upper side in the figure indicates that the relative distance is longer.

  The representative value of the relative distance is used for the above grouping. For example, vertical regions S adjacent to each other are compared, and a vertical region group is generated in which vertical regions S having representative values in close proximity are grouped. Within the vertical area group, with blocks at relative distances close to the representative value as a base point, blocks whose relative distance, horizontal distance, height, etc. are within predetermined ranges are grouped as one solid object Be

  Therefore, the representative specifying unit 164 specifies a representative value of the vertical region S before the grouping process of the three-dimensional object region specifying unit 160. The three-dimensional object region identification unit 160 performs grouping processing as described above using the representative value identified by the representative identification unit 164.

  However, the representative value may not be used for the grouping process by the three-dimensional object region identification unit 160. In this case, the representative specifying unit 164 may, for example, classify the partial images 300 and 302 only into the vertical region S and specify the representative value of the vertical region S. At this time, the representative value may be, for example, an average value of relative distances of blocks in a predetermined range below the partial images 300 and 302.

(Candidate determination processing S206)
The candidate determination unit 166 determines whether the partial images 300 and 302 are a pedestrian or a two-wheeled vehicle based on variations in representative values in a plurality of vertical regions. As shown in the image diagram 304, the representative value of the relative distance of the vertical region S including the partial image 300 of the pedestrian has a small variation. On the other hand, as shown in the image diagram 306, the representative value of the relative distance of the longitudinal region S including the partial image 302 of the two-wheeled vehicle has a large variation. This is because the relative distance of the rear wheel 308 of the two-wheeled vehicle is smaller than that of the person riding the two-wheeled vehicle.

  The candidate determination unit 166 uses this characteristic to identify a pedestrian and a two-wheeled vehicle. The candidate determination unit 166 determines that the partial image 302 is a two-wheeled vehicle when the index value of variation of the representative values exceeds a predetermined threshold. Here, the index value of variation may be, for example, a statistical index such as variance, standard deviation, or simply the difference between the maximum value and the minimum value of the representative values. In any case, the degree of variation of the representative value may be indicated.

(Correction process S208)
As described above, the relative distance to a three-dimensional object such as a pedestrian or a two-wheeled vehicle is estimated by the representative value of the relative distance or the like. However, the contours of the wheels of the two-wheeled vehicle (the rear wheel 308 in the rear and the front wheels in the front) are difficult to identify. Also, in general, the wheels of a two-wheeled vehicle have a narrow width, so it is difficult to obtain accurate parallax information. As a result, the relative distance to the two-wheeled vehicle in the partial image 302 is originally the relative distance to the end of the rear wheel 308 (the end at the vehicle 1 side), for example, to the back of the person riding the two-wheeled vehicle It becomes relative distance of. Thus, the relative distance is derived larger than in reality.

  That is, in the image diagram 306, the representative value of the vertical region S including the rear wheel 308 is a value farther from the host vehicle 1 than the actual end portion of the rear wheel 308. In this case, the control of maintaining the proper distance between the host vehicle 1 and the two-wheeled vehicle may be hindered.

  Therefore, when the partial image 302 is determined to be a two-wheeled vehicle, the distance correction unit 168 performs correction to reduce the estimated value of the relative distance to the two-wheeled vehicle corresponding to the partial image 302 (estimated distance). That is, the distance correction unit 168 reduces the estimated distance to the two-wheeler than a value before the candidate determination unit 166 determines that the two-wheeler is a motorcycle (for example, a value derived by the representative identification unit 164). Specifically, the distance correction unit 168 performs, for example, a correction of subtracting a predetermined value (for example, 70 cm or the like) from the estimated distance to the two-wheeled vehicle. The size of the wheel of the two-wheeled vehicle is generally limited, and the predetermined value is set on the safe side to correspond to the larger wheel.

  As described above, in the present embodiment, the candidate determination unit 166 can appropriately distinguish between a pedestrian and a two-wheeled vehicle. Further, as for the two-wheeled vehicle, by reducing the estimated distance, an appropriate distance between the vehicle 1 and the two-wheeled vehicle can be maintained.

  In addition, a program that causes a computer to function as the external environment recognition device 120, and a computer readable storage medium such as a flexible disk, a magneto-optical disk, a ROM, a CD, a DVD, a BD, etc., storing the program are also provided. Here, the program refers to data processing means described in any language or description method.

  Although the preferred embodiments of the present invention have been described above with reference to the accompanying drawings, it goes without saying that the present invention is not limited to such embodiments. It is obvious that those skilled in the art can conceive of various changes or modifications within the scope of the claims, and it is naturally understood that they are also within the technical scope of the present invention. Be done.

  For example, in the embodiment described above, a case has been described in which the candidate determination unit 166 determines that the partial image 302 is a two-wheeled vehicle when the index value of variation of representative values exceeds a predetermined threshold. However, if the determination is made based on the variation of the representative value, the comparison process of the index value and the threshold is not essential.

  Further, in the embodiment described above, the case where the outside environment recognition device 120 also functions as the distance correction unit 168 has been described. However, the distance correction unit 168 is not an essential component.

  Further, in the above-described embodiment, the case where the external environment recognition device 120 also functions as the candidate specifying unit 162 and the representative specifying unit 164 has been described. However, if the outside environment recognition device 120 functions as the candidate determination unit 166 and the distance correction unit 168, it may not function as the candidate specification unit 162 and the representative specification unit 164. In this case, the candidate determination unit 166 may determine whether at least the partial images 300 and 302 are two-wheeled vehicles. The determination logic of the two-wheeled vehicle is not limited to the above, and any logic may be used.

  Note that each process of the outside environment recognition process of the present specification does not necessarily have to be processed in chronological order according to the order described as the flowchart, and may include processes in parallel or by a subroutine.

  The present invention can be used for an outside environment recognition device for identifying a three-dimensional object ahead of the host vehicle.

120 Vehicle outside environment recognition device 128 Distance image (image)
160 three-dimensional object area specifying unit 162 candidate specifying unit 164 representative specifying unit 166 candidate determining unit (determining unit)
168 Distance correction unit (correction unit)
300, 302 partial images

Claims (4)

The computer is
A candidate identification unit that identifies a partial image that is a pedestrian or a motorcycle in the image;
A representative specifying unit that specifies a representative value of the distance for each of the plurality of vertical regions arranged in the horizontal direction of the partial image;
An external environment recognition device that functions as a determination unit that determines whether the partial image is the pedestrian or the two-wheeled vehicle based on variations in the representative values in a plurality of the vertical regions.   The apparatus according to claim 1, wherein the determination unit determines that the partial image is the two-wheeled vehicle when the index value of variation of the representative values exceeds a predetermined threshold. The computer
The vehicle external environment recognition device according to claim 1, further functioning as a correction unit that reduces the estimated distance to the two-wheeled vehicle than before the two-wheeled vehicle is determined. The computer is
A determination unit that determines whether a partial image in the image is a two-wheeled vehicle;
An external environment recognition device that functions as a correction unit that reduces the estimated distance to the two-wheeled vehicle compared to before the two-wheeled vehicle is determined if the partial image is determined to be a two-wheeled vehicle.

Images