JP2009070344A - Image recognition device, image recognition method, and electronic control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a probable position of a recognition object. <P>SOLUTION: Predetermined pattern matching is carried out on a region of an input image extracted through a search window, and the degree of the pattern matching is derived. Pattern matching is carried out multiple times while moving the search window so that each of regions extracted through the search window partially overlaps with one of the regions. The derived degree is included as the degree of the region, and as to a part of the region on which pattern matching has been repeatedly carried out multiple times as a result of carrying out pattern matching multiple times, the degree derived in each pattern matching is integrated multiple times as the degree of a part of the region. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image recognition device, an image recognition method, and an electronic control device.

  It is particularly important to avoid collisions with pedestrians while the vehicle is running. For this reason, conventionally, techniques aimed at recognizing pedestrians around the host vehicle have been developed.

  For example, the image recognition device performs Fourier transform on the input image captured by the camera and selects a frequency region corresponding to the contour of the pedestrian, or corresponds to individual differences such as the background of the road and the pattern of clothes. There is a technique for recognizing a pedestrian by excluding the frequency domain.

  Further, for example, in Patent Document 1, the image recognition device detects a lane marking from an input image photographed by a visible light camera, sets a search area, and then, in the search area from an input image photographed by an infrared camera. A technique for recognizing a pedestrian is disclosed. Further, for example, Patent Document 2 discloses a technique in which an image recognition apparatus calculates a distance between a recognized pedestrian and a vehicle by using an input image captured by a stereo camera.

JP 2002-362302 A JP 2005-62910 A

  By the way, in the above-described conventional technology, there is a problem in that it is impossible to detect a certain position of the recognition target. That is, for example, when the image recognition device performs fast Fourier transform on the input image and recognizes the pedestrian using the result of the fast Fourier transform, the robustness against the displacement increases (see FIG. 13), and the pedestrian It is impossible to obtain a certain position (position where a pedestrian is likely to exist).

  Such a problem becomes particularly prominent when there are a plurality of pedestrians. That is, for example, if the image recognition device recognizes a plurality of pedestrians walking with the result of the fast Fourier transform, it may not be possible to determine the number of pedestrians as well as the pedestrian's possible position. There is.

  Note that the techniques disclosed in Patent Document 1 and Patent Document 2 all use a plurality of cameras, and do not use an image captured by a single camera as an input image. Since the new problem (for example, the structure of an apparatus becomes complicated) which arises arises, in any case, the above-mentioned problem cannot be solved appropriately.

  Accordingly, the present invention has been made to solve the above-described problems of the prior art, and provides an image recognition device, an image recognition method, and an electronic control device capable of detecting a probable position of a recognition target. For the purpose.

  In order to solve the above-described problems and achieve the object, an image recognition device, an image recognition method, and an electronic control device according to the present invention perform predetermined pattern matching on an input image region cut out by a search window, and Deriving the degree of matching. Further, the pattern matching is executed a plurality of times while moving the search window so that the areas cut out by the search window partially overlap each other. Then, the calculated degree is counted as the degree of the area, and pattern matching is executed a plurality of times, and as a result of the pattern matching being executed a plurality of times, the pattern matching is performed once for a part of the area where the pattern matching is executed a plurality of times. The degree derived in (1) is integrated a plurality of times as a part of the area.

  According to the present invention, the image recognition device, the image recognition method, and the electronic control device perform predetermined pattern matching on an input image region cut out by a search window, derive a degree of the pattern matching, and cut out the image by the search window. While the search window is moved so that the areas to be partially overlap each other, pattern matching is executed a plurality of times, the derived degree is counted as the degree of the area, and a plurality of pattern matching is performed. As a result of being executed twice, for a part of the region where pattern matching has been executed a plurality of times, the degree derived by the pattern matching for each time is integrated several times as a part of the region, so It is possible to detect a probable position of the recognition target.

  Exemplary embodiments of an image recognition device, an image recognition method, and an electronic control device according to the present invention will be described below in detail with reference to the accompanying drawings. In the first embodiment, an in-vehicle image recognition apparatus will be described. In the following, the main terms used in the embodiment, the outline and features of the image recognition device according to the first embodiment, the configuration of the image recognition device according to the first embodiment, the processing procedure of the image recognition device according to the first embodiment, And the effect of Example 1 is demonstrated, and another Example is described next.

[Explanation of terms]
First, main terms used in the following examples will be described. “Pattern matching” is a method of comparing an input image with a reference pattern for the purpose of recognizing a recognition target in the input image. Specific methods include pattern matching using a classifier using a neural network and pattern matching using a cross-correlation coefficient between an input image and a reference pattern. Although the present invention is not limited to a specific method of pattern matching, the first embodiment assumes pattern matching using a classifier based on a neural network.

  A classifier based on a neural network is a learning type classifier that learns from a reference pattern. That is, the classifier using a neural network learns in advance various target images such as “child” images and “adult” images as reference patterns, and learns images of various sizes in advance. When the input is received, it is possible to identify whether or not the recognition target in the image is “person”. When such a discriminator receives an input of an image, the discriminator converts the image into a predetermined size handled by the discriminator, and uses the pixel value of the converted image as an input to determine whether the image is “human”. The identification result is output as a value of 0 to 1 as the “degree of pattern matching”.

  Here, the image input to the discriminator is specifically an area of the input image cut out by the “search window”. Generally, in “pattern matching”, when the input image to be processed is an image larger in size than the reference pattern, the area of the input image cut out by the “search window” is compared with the reference pattern. Is done. The “search window” performs “pattern matching” on the entire input image by appropriately moving the input image. Since the shape of the “search window” has already been set to a rectangle having the same size as the recognition target, such a “search window” is assumed in the first embodiment. However, the present invention is not limited to this.

[Outline and Features of Image Recognition Apparatus According to First Embodiment]
Next, the outline and features of the image recognition apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining the outline and features of the image recognition apparatus according to the first embodiment.

  As described above, the image recognition apparatus according to the first embodiment outlines that the recognition target in the input image is recognized by performing pattern matching on the input image region cut out by the search window. The main feature is to detect a probable position.

  This main feature will be briefly described. The image recognition apparatus according to the first embodiment performs pattern matching on the area of the input image cut out by the search window, and derives the degree of pattern matching. For example, as shown in (1) of FIG. 1, the image recognition apparatus performs pattern matching using a classifier based on a neural network on the area of the input image, and as shown in (2) of FIG. The degree of matching “0.6” is derived. In the first embodiment, it is assumed that the classifier identifies whether or not the input image is “person”.

  Here, the image recognition apparatus according to the first embodiment executes the pattern matching as described above a plurality of times. At this time, the regions cut out by the search window partially overlap each other, Move the search window. For example, as shown in (3) of FIG. 1, the image recognition apparatus moves the search window and cuts out the peripheral area of “male” that is the recognition target a plurality of times. The search window is moved little by little so that it partially overlaps with.

  Further, the image recognition apparatus according to the first embodiment counts the derived degree as the degree of the area for each of the areas cut out by the search window, and the pattern matching is executed a plurality of times. As for a part of the area where the pattern matching has been executed, the degree derived by each pattern matching is integrated several times as a part of the area.

  For example, as shown in (4) of FIG. 1, the image recognition apparatus counts “6” for the area from which “output 0.6” is derived in (3) of FIG. For the area from which “8” is derived, “8” is counted. In addition, the image recognition apparatus performs a part of the region from which “output 0.6” and “output 0.9” are derived as a result of performing the pattern matching twice in (3) of FIG. Adds “6” and “9” and counts “15”. Further, the image recognition apparatus derives “output 0.6”, “output 0.9”, and “output 0.8” as a result of executing the pattern matching three times in (3) of FIG. For a part of the area, “6”, “9” and “8” are added up and “23” is counted. In addition, the image recognition apparatus performs “pattern output matching” four times in (3) of FIG. 1, resulting in “output 0.6”, “output 0.9”, “output 0.8”, and “output”. For a part of the area from which “0.8” is derived, “6”, “9”, “8”, and “8” are added together to count “31”.

  Then, the image recognition apparatus according to the first embodiment outputs the integrated result to the output unit. For example, as shown in (5) of FIG. 1, the image recognition apparatus renders the result into an image by expressing the gradation divided into levels according to the accumulated value, and outputs the result. In the example of (5) in FIG. 1, gradation expression (area is filled) is closer to white as the integrated value is larger and closer to black as the integrated value is smaller.

  For this reason, the image recognition apparatus according to the first embodiment can detect a likely position of the recognition target.

  That is, as shown in FIG. 1 (5), the output of the image recognition apparatus according to the first embodiment is displayed with a particularly emphasized portion where the integrated value is large ((5) in FIG. 1). In FIG. 5, the white-filled portion is the highlighted portion). Here, the portion where the accumulated value is large is a portion where the value identified as `` is a person '' is large, and among them, the portion where the value obtained by integrating the degree of multiple times is large is, This means that there is a particularly high possibility that “people” exist. If the part that is most likely to have a “person” is displayed with particular emphasis, for example, the user who is notified by such output recognizes the probable position of the “person” to be recognized. Is possible.

[Configuration of Image Recognition Apparatus According to First Embodiment]
Next, the configuration of the image recognition apparatus according to the first embodiment will be described with reference to FIGS. FIG. 2 is a block diagram illustrating a configuration of the image recognition apparatus (vehicle-mounted image recognition apparatus) according to the first embodiment, FIG. 3 is a diagram for explaining an input image, and FIG. 4 is a search window. 5 is a diagram for explaining the matching execution unit, FIG. 6 is a diagram for explaining the matching degree integrating unit, and FIG. 7 is a result output unit ( FIG. 8 is a diagram for explaining the result output unit (color coding), and FIG. 9 is a diagram for explaining the result output unit (superposition). FIG. 10 is a diagram for explaining the result output unit (afterimage).

  The image recognition device 10 mounted on the vehicle is connected to the navigation device 30, the camera 31, the radar 33, and the pre-crash ECU 40.

  The navigation device 30 is an in-vehicle device that sets and guides a travel route by using a position of the host vehicle specified by communicating with a GPS (Global Positioning System) artificial satellite and map data 30a stored in advance. In addition, the navigation device 30 provides the image recognition device 10 with position information of the host vehicle, surrounding map information, a planned travel route, and the like.

  The camera 31 captures the vicinity of the host vehicle and inputs the captured result to the image recognition apparatus 10. The radar 33 detects an object around the host vehicle and measures a distance to the object, and inputs the detected object to the image recognition apparatus 10.

  The pre-crash ECU 40 receives the control of the image recognition device 10 when the image recognition device 10 predicts a collision of the host vehicle, controls the operation of the vehicle by the brake 41 and the engine control device (EFI) 42, This is an electronic control device that performs notification by the speaker 44.

  The display 43 notifies the user, that is, the vehicle occupant by display, and the speaker 44 notifies by voice. The display 43 and the speaker 44 can be shared by various in-vehicle devices such as the navigation device 30 and an in-vehicle audio device (not shown) in addition to performing output under the control of the pre-crash ECU 40.

  The image recognition apparatus 10 includes a preprocessing unit 11, a vehicle recognition unit 16, a white line recognition unit 17, a pedestrian recognition unit 18, a pattern database 19, and a collision determination unit 20 therein. Here, the vehicle recognition unit 16, the white line recognition unit 17, the pedestrian recognition unit 18, and the collision determination unit 20 are realized by, for example, a single microcomputer 10a (an arithmetic processing unit including a combination of a CPU, a ROM, and a RAM). And so on.

  The preprocessing unit 11 performs processing such as filtering, edge detection, and contour extraction on the image captured by the camera 31, and then outputs the image to the vehicle recognition unit 16, the white line recognition unit 17, and the pedestrian recognition unit 18. .

  The vehicle recognition unit 16 performs pattern matching on the image output from the preprocessing unit 11 to recognize the vehicle, and outputs the recognition result to the collision determination unit 20. Further, the white line recognition unit 17 recognizes a white line by performing pattern matching or the like on the image output from the preprocessing unit 11, and outputs the recognition result to the collision determination unit 20. The specific method of pattern matching is not limited, but the image recognition apparatus 10 may store a reference pattern for pattern matching in the pattern database 19 as necessary.

  As shown in FIG. 2, the pedestrian recognition unit 18 includes a matching degree derivation unit 18a, a matching execution unit 18b, a matching degree integration unit 18c, and a result output unit 18d. The matching degree deriving unit 18a corresponds to the “degree deriving unit” described in the claims, and the matching execution unit 18b corresponds to the “execution unit” described in the claims, and the matching degree integrating unit Reference numeral 18c corresponds to the “accumulation means” recited in the claims, and the result output section 18d corresponds to the “output means” recited in the claims.

  The matching degree deriving unit 18a performs pattern matching and derives the degree of pattern matching. Specifically, the matching degree deriving unit 18a performs pattern matching on the input image region cut out by the search window from among the input images that have been subjected to predetermined processing by the preprocessing unit 11, and determines the degree of pattern matching. The derived degree is used for processing by the matching degree integrating unit 18c.

  Here, the matching degree deriving unit 18a includes a classifier using a neural network that performs pattern matching. As described above, when the classifier using the neural network according to the first embodiment receives an input of an image, the classifier converts the image into a predetermined size handled by the classifier, and receives the pixel value of the converted image as an input. Whether the person is “person” or not is identified, and the identification result is output as a “degree of pattern matching” with a value from 0 to 1. Note that the image recognition apparatus 10 may store a reference pattern for learning of the discriminator in the pattern database 19 as necessary.

  For example, when the input of the image as shown in FIG. 3 is received as the input image that has been subjected to the predetermined processing by the preprocessing unit 11, the matching degree deriving unit 18a displays the input image in the search window as shown in FIG. A region is cut out, the cut out region is converted into a predetermined size handled by the classifier, and the pixel value of the converted region is input to identify whether or not the region is `` person ''. Output as “degree of pattern matching”.

  In the first embodiment, when the derived “pattern matching degree” does not satisfy “output 0.5”, the matching degree deriving unit 18 a identifies that the person is not “person” and matches the matching degree integrating unit 18 c. In order to omit the integration process according to, the derived degree is not transmitted to the matching degree integration unit 18c, and the process proceeds to the next process as it is.

  The matching execution unit 18b executes pattern matching a plurality of times. Specifically, the matching execution unit 18b performs pattern matching by the matching degree deriving unit 18a a plurality of times while moving the search window so that the regions cut out by the search window partially overlap each other. Let it run.

  For example, as shown in FIG. 5 (although FIG. 5 shows a part of the moved search window for convenience of explanation), the matching execution unit 18b is configured so that the regions cut out by the search window are respectively The search window is moved so as to partially overlap any of the areas. That is, as shown in FIG. 5, the matching execution unit 18 b moves the search window and cuts out the peripheral area of “male” to be recognized a plurality of times. The search windows are moved little by little so as to overlap. Along with this movement, the matching degree deriving unit 18a identifies the region cut out by the search window by the discriminator and outputs it as the “pattern matching degree”. Here, how much the matching execution unit 18b should move while shifting the search window may be appropriately selected. The purpose of the image recognition apparatus 10 is a method of shifting every pixel, a method of shifting every several pixels, or a method of moving only the periphery of an object to be recognized precisely and moving it to the next instead of shifting it. It may be appropriately selected depending on the above.

  Note that the matching execution unit 18b moves the search window relative to the input image, but what kind of search window should be used may be appropriately selected in the present invention. In the first embodiment, it is assumed that the image recognition device 10 is mounted on a vehicle and used particularly for the purpose of avoiding a collision with a pedestrian. Assume that images with a sense of perspective as shown are often input. If this is the case, even if the person is the same, if the Y coordinate shown in FIG. 4 has a small value, it can be said that the search window should be large, and the Y coordinate has a large value. In some cases, it can be said that a small search window should be used. However, “people” include “adults” and “children”, “standing people” and “sitting people”, and “people” with different heights and body types. ], There is no simple change in the size of the search window according to the Y coordinate. The image recognition apparatus 10 according to the first embodiment stores information on the size of a search window that can be considered as an appropriate selection according to the Y coordinate in a database (not shown), and the matching execution unit 18b searches the search window from this database. The information about the size of is acquired, and an appropriate search window is acquired.

  The matching degree integrating unit 18c counts the derived degree. Specifically, the matching degree accumulating unit 18c counts the degree derived by the matching degree deriving unit 18a as the area degree, and the matching execution unit 18b executes pattern matching a plurality of times. For a part of the region where the pattern matching is executed by overlapping a plurality of times by 18a, the degree derived by each pattern matching is integrated several times as a part of the region, and the result of integration is It is used for processing by the output unit 18d.

  For example, as shown in FIG. 6, the matching degree integrating unit 18 c is derived with “output 0.6” in FIG. 5 (although FIG. 6 shows a part of integration for convenience of explanation). “6” is counted for the remaining area, and “8” is counted for the area from which “output 0.8” is derived. Further, as a result of the pattern matching being performed twice in FIG. 5, the image recognition apparatus 10 has obtained “output 0.6” and “output 0.9” in a part of the region from which “ Add “6” and “9” to add “15”. In addition, the image recognition apparatus 10 is the region in which “output 0.6”, “output 0.9”, and “output 0.8” are derived as a result of performing pattern matching three times in FIG. As for a part of, “6”, “9” and “8” are added up and “23” is counted. Further, as a result of the pattern matching being performed four times in FIG. 5, the image recognition apparatus 10 performs “output 0.6”, “output 0.9”, “output 0.8”, and “output 0.8”. "6", "9", "8", and "8" are added to a part of the area from which "is derived" and "31" is counted.

  Here, considering FIG. 6 again, first, FIG. 6 suggests that there is a possibility that an object to be recognized (“person” in the first embodiment) exists in two places in the input image. It can be said that. On the other hand, for example, when focusing on the left part, the recognition target “person” covers a wide area from the point where the value “6” is written to the point where the value “8” is written. This means that it can exist anywhere in the entire region. It is hard to say that a probable position of the “person” to be recognized can be detected only by the information that there is a possibility that the “person” exists in any one of the entire areas. For example, assuming an emergency situation in which a vehicle and a pedestrian may collide, there will be a “8” value indicating whether there is a “person” around the value “6” written. It can be said that there is a big difference in whether there is a “person” around.

  In this regard, according to the present invention, the integrated value contributes to the detection of a probable position of the “person” to be recognized. That is, as shown in FIG. 6, a large value of “31” is written in a part of the entire wide area. A large integrated value means that there is a particularly high possibility that “person” exists, and the position where the value “31” is written is the most likely position of “person”.

  The result output unit 18d outputs the integrated result to the output unit. Specifically, the result accumulated by the matching degree accumulation unit 18c is output to an output unit such as the display 43 (by being controlled by the collision determination unit 20 or the pre-crash ECU 40).

  For example, as shown in FIG. 7, the result output unit 18 d converts the level into a gradation according to the accumulated value, thereby imaging the result and outputting it to the display 43. In the example of FIG. 7, a gradation expression (area is filled) is closer to white as the integrated value is larger and closer to black as the integrated value is smaller. That is, as shown in FIG. 7, the output of the image recognition apparatus 10 is displayed with a particularly emphasized portion where the integrated value is large (in FIG. 7, the white-painted portion is highlighted and displayed). Part). Here, the portion where the accumulated value is large is a portion where the value identified as “person” is large, and among them, the portion where the accumulated value is accumulated by integrating the degree of multiple times This means that the possibility that “person” exists is particularly high (corresponding to the position where the value “31” in FIG. 6 is written). If the part that is most likely to have a “person” is displayed with particular emphasis, for example, the user who is notified by such output recognizes the probable position of the “person” to be recognized. Is possible.

  It should be noted that the result output unit 18d according to the first embodiment converts the integrated value into a value that does not satisfy a predetermined threshold (for example, “5” (corresponding to “0.5” in terms of output)). All the results are output so as to be the same level of expression (for example, a blacked-out portion). In addition, the result output unit 18d according to the first exemplary embodiment expresses the imaged result all at the same level (for example, white-filled) for a value that is equal to or greater than a predetermined threshold (for example, “30” or greater) among the integrated values. Output). However, the present invention is not limited to such an output, and can cope with various output variations.

  For example, as shown in FIG. 8, the result output unit 18 d may display the result as an image by expressing the color classification divided into levels according to the integrated values and output the result to the display 43. In the example of FIG. 8, if the integrated value is equal to or greater than a predetermined threshold (for example, “30” or greater), it is “red”, the next value (or range of values) is “yellow”, and the next value ( Alternatively, the value range is expressed in “green”, and in addition, if the integrated value does not satisfy a predetermined threshold (for example, “5”), it is expressed in “blue”. That is, as shown in FIG. 8, the output of the image recognition device 10 is displayed with a particularly emphasized portion where the integrated value is large (in FIG. 8, the portion painted in red (see legend)). Is the highlighted part).

  Further, for example, the result output unit 18d may output the imaged result and the input image in a superimposed manner. That is, as shown in FIG. 9, an input image may be used as a background, and an output as shown in FIG. Further, for example, the result output unit 18d may alternately output the imaged result and the input image in a short time. That is, as shown in FIG. 10, the imaged result may be inserted after the input image and continuously output to the display 43 like a so-called flip book. In this case, the user can grasp the relationship between the input image and the probable position by using the afterimage of the user.

  The collision determination unit 20 uses the recognition results obtained by the vehicle recognition unit 16, the white line recognition unit 17, and the pedestrian recognition unit 18, the detection results obtained by the radar 33, and the position information output from the navigation device 30 to detect pedestrians and other vehicles. The collision risk with the own vehicle is determined.

  Specifically, the collision determination unit 20 determines the probability that a collision with a pedestrian or another vehicle will occur, the collision time, the distance to the collision position, the collision angle, etc., and based on the determination result, An information display instruction for the display 43, a warning sound output instruction for the speaker 44, a brake control instruction, an EFI control instruction, and the like are output to the pre-crash ECU 40.

  When the collision determination unit 20 according to the first embodiment receives a result output instruction from the result output unit 18d of the pedestrian recognition unit 18, for example, the collision determination unit 20 determines that the output result includes a value equal to or greater than a predetermined threshold. Thus, the collision is determined and transmitted to the pre-crash ECU 40, and the result is output to the display 43 via the pre-crash ECU 40.

[Processing Procedure of Image Recognition Device According to First Embodiment]
Next, processing performed by the image recognition apparatus according to the first embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is a flowchart illustrating the flow of processing (whole) of the image recognition apparatus, and FIG. 12 is a flowchart illustrating the flow of processing of the pedestrian recognition unit.

[Processing of image recognition device (overall)]
The flowchart shown in FIG. 11 is started when a power switch (which may be interlocked with an ignition switch or the like) is turned on and the camera 31 captures an image, and for each processing of an image frame (for example, every several milliseconds). This process is repeatedly executed.

  First, the image recognition apparatus 10 performs processing such as filtering, edge detection, and contour extraction by the preprocessing unit 11 on the image captured by the camera 31 (step S101). Next, the image recognition device 10 executes white line recognition processing (step S102) by the white line recognition unit 17 and vehicle recognition processing (step S103) by the vehicle recognition unit 16.

  Thereafter, the pedestrian recognition unit 18 performs pedestrian recognition (step S104), the collision determination unit 20 performs a collision determination (step S105), and outputs the determination result to the pre-crash ECU 40 (step S106). finish.

[Processing of pedestrian recognition unit]
Next, the flowchart shown in FIG. 12 will be described. First, the matching execution unit 18b acquires information on the size of a search window that can be considered as an appropriate selection according to the Y coordinate from a database (not shown), and performs an appropriate search. A window is acquired (step S201).

  Next, when the matching execution unit 18b moves the search window and cuts out an input image area in the search window, the matching degree deriving unit 18a executes pattern matching and derives the degree of pattern matching (step S202).

  Subsequently, the matching degree deriving unit 18a determines whether or not the derived pattern matching degree satisfies a predetermined threshold (for example, “output 0.5”) (step S203). Here, when the degree does not satisfy the threshold (No at Step S203), the matching degree deriving unit 18a identifies that it is not “person”, and does not transmit the derived degree to the matching degree integrating unit 18c, and does not perform the step. The process proceeds to S205.

  On the other hand, when the degree satisfies the threshold (Yes at Step S203), the matching degree deriving unit 18a transmits the derived degree to the matching degree integrating unit 18c, and the matching degree integrating unit 18c integrates the derived degree. (Step S204). At this time, the matching degree accumulating unit 18c counts the degree transmitted from the matching degree deriving unit 18a as the degree of the area, and also accumulates a part of the area where the pattern matching is executed a plurality of times.

  And the matching execution part 18b determines whether the pattern matching by the said search window was complete | finished (step S205), and when not complete | finished (step S205 negative), moves a search window (step S206). ) Again, the process proceeds to the pattern matching execution process (step S202). In step S206, the matching execution unit 18b moves the search window so that the areas cut out by the search window partially overlap each of the areas.

  On the other hand, when the matching execution unit 18b determines in step S205 that the pattern matching by the search window has ended (Yes in step S205), the matching execution unit 18b determines whether or not the search window has ended (step S207). ) If the search window has not ended (No at Step S207), the matching execution unit 18b acquires the search window again (Step S201). When the search window ends (Yes at Step S207), the matching execution unit 18b transmits the integrated result to the result output unit 18d, and the result output unit 18d outputs the imaged result to the display. (Step S208).

[Effect of Example 1]
As described above, according to the first embodiment, predetermined pattern matching is performed on the area of the input image cut out by the search window, the degree of the pattern matching is derived, and each of the areas cut out by the search window is selected. While moving the search window so as to partially overlap that region, pattern matching is performed a plurality of times, the derived degree is counted as the region degree, and the result of the pattern matching being performed a plurality of times, For a part of the region where pattern matching has been performed multiple times, the degree derived by each pattern matching is integrated several times as a part of the region, and the integrated result is determined as a predetermined value. Since the data is output to the output unit, it is possible to detect a likely position of the recognition target.

  In addition, the image recognition apparatus according to the present invention is resistant to misalignment and can detect a likely position of the recognition target. As a result, erroneous detection can be reduced.

  In addition, according to the first embodiment, since the result is imaged and output, for example, it is possible to notify the driver of the vehicle or the like of the likely position of the recognition target.

  In addition, according to the first embodiment, the imaged result and the input image are superimposed and output, or the imaged result and the input image are alternately output in a short time. It becomes possible to notify the position appropriately.

In addition, according to the first embodiment, the result is imaged and output by expressing the color classification or gradation classified according to the integrated value, so that the probable position of the recognition target can be It becomes possible to notify after dividing.
In addition, when expressing color coding, it is possible to reduce oversight of a notified person (such as a driver of a vehicle) by filling an area (for example, a rectangular area) instead of a frame.

  Further, according to the first embodiment, for the values that do not satisfy the predetermined threshold value among the integrated values, the imaged result is output so that all the results are expressed at the same level, and the value is equal to or greater than the predetermined threshold value. With respect to, since all the imaged results are output so as to have the same level of expression, it is possible to efficiently notify the likely position of the recognition target.

[Other embodiments]
Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above.

  In the first embodiment, the example in which the image recognition apparatus according to the present invention is mounted on a vehicle and the object to be recognized is identified as “human” has been described. However, the present invention is not limited thereto. It is not a thing. The present invention can be applied to other cases as an image recognition apparatus that recognizes an image and detects a likely position of a recognition target.

  In the first embodiment, a method of performing pattern matching using a classifier using a neural network is assumed, but the present invention is not limited to this. For example, the image recognition apparatus stores one or a plurality of reference patterns in the pattern database, and the matching degree derivation unit determines whether the input image area extracted by the search window is between the reference pattern stored in the pattern database. The present invention can be similarly applied to a method of performing pattern matching using a cross-correlation coefficient.

  In addition, when this method is used, the image recognition device stores an “adult” pattern or a “children” pattern as a reference pattern, and integrates the results of performing pattern matching with the “adult” pattern. Further, the integration result obtained by integrating the results of performing the pattern matching with the “child” pattern may be managed by individual planes (individual planes correspond to, for example, one output image). That is, for example, the matching degree integration unit manages the integration results based on the “adult” pattern and the integration results based on the “child” pattern on separate planes, and also manages the total integration results on other planes. The result output unit may output the total integration result or output only the integration result based on the “child” pattern as necessary.

  As described above, the method of managing the integration results with a plurality of planes instead of one plane can be applied to the case where a classifier using a neural network is used. That is, for example, if there are a plurality of classifiers classified into neural network classifiers that identify whether or not they are “people” and classifiers that identify whether or not they are “animals”, the degree of matching The integration unit manages the integration results from the “person” classifier and the integration results from the “animal” classifier on separate planes, and also manages the totalized result on other planes, resulting in a result output unit. However, if necessary, the total integration result may be output, or only the integration result based on the “person” pattern may be output. Each plane may manage the integration result for each search window.

  In the first embodiment, the method of imaging the output result and displaying it on the display has been described. However, the present invention is not limited to this. For example, the image processing apparatus may output the X coordinate, the Y coordinate, etc. as data indicating the likely position detected, and input this to the collision determination unit, thereby leaving it to appropriate control by the collision determination unit. .

[System configuration, etc.]
In addition, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated (for example, FIG. 2). In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  The image recognition method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. The program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD and being read from the recording medium by the computer.

  As described above, the image recognition device, the image recognition method, and the electronic control device according to the present invention recognize the recognition target in the input image by performing pattern matching on the region of the input image cut out by the search window. This is useful, and is particularly suitable for detecting a probable position of the recognition target.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining an overview and features of an image recognition apparatus according to a first embodiment. 1 is a block diagram illustrating a configuration of an image recognition apparatus according to a first embodiment. It is a figure for demonstrating an input image. It is a figure for demonstrating a search window. It is a figure for demonstrating a matching execution part. It is a figure for demonstrating a matching degree integrating | accumulating part. It is a figure for demonstrating a result output part (gradation). It is a figure for demonstrating a result output part (color classification). It is a figure for demonstrating a result output part (superposition). It is a figure for demonstrating a result output part (afterimage). It is a flowchart shown about the flow of a process (whole) of an image recognition apparatus. It is a flowchart shown about the flow of a process of a pedestrian recognition part. It is a figure for demonstrating a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image recognition apparatus 10a Microcomputer 11 Pre-processing part 16 Vehicle recognition part 17 White line recognition part 18 Pedestrian recognition part 18a Matching degree deriving part 18b Matching execution part 18c Matching degree accumulation part 18d Result output part 19 Pattern database 20 Collision judgment part 30 Navigation Device 30a Map data 31 Camera 33 Radar 40 Pre-crash ECU
41 Brake 42 EFI
43 Display 44 Speaker

Claims (7)

A degree deriving unit that performs predetermined pattern matching on an input image region cut out in the search window and derives a degree of the pattern matching;
Execution means for performing pattern matching by the degree deriving means a plurality of times while moving the search window so that the areas cut out by the search window partially overlap each other, and
A region where the degree derived by the degree deriving unit is counted as the degree of the region, and the pattern matching is performed a plurality of times by the degree deriving unit as a result of the pattern matching being performed a plurality of times by the executing unit. For a part of, the integration means for integrating the degree derived by pattern matching for each time as a part of the region for a plurality of times,
An image recognition apparatus comprising:   It further comprises output means for imaging the result of integration by the integration means and outputting it to a predetermined output unit by expressing the color classification leveled according to the value integrated by the integration means. The image recognition apparatus according to claim 1.   The image processing apparatus further includes an output unit configured to express gradations divided into levels according to the values integrated by the integration unit, thereby imaging the result integrated by the integration unit and outputting the result to a predetermined output unit. The image recognition apparatus according to claim 1.   The output means outputs a value that does not satisfy a predetermined threshold among the values integrated by the integration means so that all of the imaged results are the expression of the same level. Item 4. The image recognition device according to Item 2 or 3.   The output means outputs, for a value that is equal to or greater than a predetermined threshold value among the values accumulated by the accumulation means, so that all of the imaged results are the expression at the same level. The image recognition apparatus as described in any one of 2-4. A degree derivation step of performing predetermined pattern matching on an input image region cut out in the search window and deriving the degree of pattern matching;
An execution step of performing pattern matching by the degree derivation step a plurality of times while moving the search window so that the regions cut out in the search window partially overlap each other, and
Area where the degree derived by the degree derivation step is counted as the degree of the region, and pattern matching is executed multiple times by the degree derivation step as a result of pattern matching being executed multiple times by the execution step For a part of, the integration step of integrating a plurality of times as the degree of a part of the region, the degree derived by pattern matching for each time,
An image recognition method comprising: A degree deriving unit that performs predetermined pattern matching on an input image region cut out in the search window and derives a degree of the pattern matching;
Execution means for performing pattern matching by the degree deriving means a plurality of times while moving the search window so that the areas cut out by the search window partially overlap each other, and
A region where the degree derived by the degree deriving unit is counted as the degree of the region, and the pattern matching is performed a plurality of times by the degree deriving unit as a result of the pattern matching being performed a plurality of times by the executing unit. For a part of, the integration means for integrating the degree derived by pattern matching for each time as a part of the region for a plurality of times,
Control means for performing vehicle operation control and / or notification control for the driver when a collision with an object is predicted based on the result accumulated by the accumulation means;
An electronic control device comprising: