JP6348758B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that uses an edge extracted from a captured image to recognize an image to be detected such as a lane mark.

  Conventionally, it has been generally performed to extract an edge from a captured image of an imaging apparatus and to recognize various detection target images using the edge.

  For example, in Patent Document 1, a binary image is generated by extracting edges that are candidates for a white line on a road surface from a captured image of an imaging device mounted on a vehicle, and the white line on the road surface is recognized based on the binary image. What to do is described.

  In this case, the edge that is a candidate for the white line is extracted by comparing the pixel value of the edge image indicating the luminance value change of the captured image with a predetermined threshold value.

  Patent Document 1 describes a technique for changing the threshold values in both the left and right regions in accordance with the edge count value in the right region of the captured image or the edge count value in the left region.

JP-A-9-288737

  The captured image obtained from the imaging device may have a relatively large difference in image brightness (average luminance) due to the surrounding environment of the imaging device, the influence of the weather, and the like. For example, when an image of the front of the host vehicle is captured by an imaging device, a part of the captured image is relatively dark due to a shadow of a structure such as a bag, while the other image part is relatively bright. Situations may arise.

  In such a situation, when an edge is to be extracted from a captured image, if the threshold for edge extraction is set to be the same in the entire image area to be extracted, depending on a part of the image area, There is a possibility that the same type of edge (for example, the edge of the lane mark on the road surface) may or may not be extracted.

  However, in the conventional technique as seen in Patent Document 1, the threshold for edge extraction is set to the same value in the entire image region from which the edge is to be extracted.

  For this reason, such an edge is not extracted in an image area where the edge to be extracted actually exists, or many unnecessary edges are extracted in an area where the edge to be extracted does not exist. It is easy to produce.

  The present invention has been made in view of such a background, and an object thereof is to provide an image processing apparatus capable of extracting a required edge from a captured image with high reliability.

In order to achieve such an object, the image processing apparatus of the present invention compares the pixel value change in the captured image of the imaging apparatus that captures the monitoring area with a predetermined threshold value, and extracts an edge from the captured image And an edge extraction target area that is an area in the captured image in which the edge extraction means executes the process of extracting the edge from the captured image. The edge extraction unit executes processing for extracting an edge in each partial area of the edge extraction target area using the threshold value set corresponding to the partial area. The basic configuration is configured as described above.

According to this , since each different threshold value is set for each partial region of the edge extraction target region, the edge extraction unit determines the state (distribution state, average pixel value) of the entire pixel value of each partial region. Etc.) can be extracted using the threshold suitable for the partial region.

  For example, in the edge extraction target region, it is possible to extract edges in these partial regions by using different threshold values for the relatively dark partial region and the bright partial region.

Therefore, according to the basic configuration , it is possible to extract a required edge from the captured image with high reliability.

Note that the edge extraction target area in the basic configuration may be either the entire area of the captured image or a partial area of the captured image. Furthermore, the edge extraction target region and each partial region may be either one continuous region or a region composed of a plurality of regions separated from each other.

In one aspect of the basic configuration, the threshold value is a threshold value in a partial region where an image of a portion having a relatively large distance from the imaging device in real space is projected among the plurality of partial regions. It is preferable that the distance is set to be smaller than a threshold value in a partial area where an image of a relatively small portion is projected ( reference invention).

That is, an image at a location having a relatively large distance from the imaging device tends to be unclear compared to an image at a location having a relatively small distance. In the reference invention, in consideration of this, the threshold value in each partial region is set as described above.

  Thereby, the extraction of the edge in each partial area can be appropriately performed regardless of the distance from the imaging device of the location shown in the partial area.

Further, in the basic configuration or the reference invention , the threshold value in each of the one or more partial areas of the plurality of partial areas is extracted by the edge extraction unit in the local area of the partial areas. Threshold value updating means for updating according to the number of data may be further provided ( first invention).

  In addition, the local area | region of arbitrary one partial area means the area | region of a part of this partial area. The same applies to other inventions described later.

According to the first invention, the threshold value in each of the one or more partial areas, are updated according to the number of the edges extracted in the local region of the partial region. For example, when an edge is extracted in the local region using a certain threshold value, it is easy to extract an edge when the number of extracted edges is smaller than a predetermined number (when the number of extracted edges is too small). Reducing the threshold value in the partial region having the local region for, or when the number of extracted edges is larger than a predetermined number (when the number of extracted is too large), It is possible to increase the threshold value in the partial area having the local area.

  For this reason, the threshold value in each partial area can be appropriately updated with respect to a change in the image state in each of the one or more partial areas.

  In addition, since the threshold value in each of the one or more partial areas is updated according to the number of extracted edges in a partial local area of the partial area, this update process can be efficiently performed with a small calculation load. It can be carried out.

In the basic configuration or the reference invention , the following modes can be adopted as more specific modes.

That is, in the basic configuration or the reference invention described above, for example, when the monitoring region captured by the imaging device is a region including a traveling road surface of the vehicle, the edge extraction target region including the image of the traveling road surface of the vehicle Lane mark recognizing means for recognizing an image of the lane mark on the road surface using the edge extracted by the edge extracting means, and one having a possibility that an image of at least the lane mark of the plurality of partial areas is captured. The local region in which each of the one or more partial regions is set such that the threshold value in each of the partial regions is a region in which an image of the road surface between the right and left lane marks on the road surface is captured. And a threshold updating unit that updates the number of edges according to the number of edges extracted by the edge extracting unit. (The second invention).

According to this , the threshold value in each of the one or more partial areas having the possibility of capturing the image of the lane mark is set in the local area set in each of the one or more partial areas as in the first invention. It is updated according to the number of edges extracted by the edge extraction means.

  In this case, each local area of the one or more partial areas is an area that is set to be an area in which an image of the road surface between the right and left lane marks of the traveling road surface is captured. Therefore, the local region is a region having a stable pixel value with less noise components and the like.

  For this reason, the threshold value in each of the one or more partial regions can be appropriately updated with high stability (so as not to be easily influenced by noise components).

Alternatively, in the basic configuration or the reference invention , for example, when the monitoring area captured by the imaging device is an area including a traveling road surface of the vehicle, the edge extraction target area including an image of the traveling road surface of the vehicle Lane mark recognizing means for recognizing an image of the lane mark on the road surface using the edge extracted by the edge extracting means, and one having a possibility that an image of at least the lane mark of the plurality of partial areas is captured. The number of the edges extracted by the edge extraction means in the local area where the threshold value in each of the partial areas is set in each of the one or more partial areas so as to be an area where the image of the lane mark appears. It is possible to employ a configuration further comprising threshold updating means for updating in accordance with ( third invention).

According to this , the threshold value in each of the one or more partial areas having the possibility of the image of the lane mark being set in each of the local areas set in the one or more partial areas as in the third aspect of the invention. It is updated according to the number of edges extracted by the edge extraction means.

  In this case, each local region of the one or more partial regions is a region set so as to be a region where the image of the lane mark is reflected. For this reason, the threshold value in each of the one or more partial regions can be updated to a value suitable for extracting an edge corresponding to the lane mark.

In the second or third aspect of the invention, the threshold update unit is configured to update the new captured image based on the position of the lane mark already recognized by the lane mark recognition unit before the threshold is updated. It is preferable that the location of the lane mark is estimated and the local region is set based on the estimation result ( fourth invention).

According to this , since the location of the lane mark is estimated and the local region is set based on the estimation result, the local region can be set at a required position with high reliability. Therefore, it is possible to set a suitable threshold value as a threshold value for extracting an edge corresponding to the lane mark with high reliability .
Furthermore, the present invention may employ an aspect in which, in the above basic configuration, the plurality of partial areas include partial areas divided so as to be divided into left and right on the captured image (fifth invention). .

1 is a diagram illustrating a system configuration of an image processing apparatus according to an embodiment of the present invention. The figure which shows an example of the captured image of the imaging device 2 shown in FIG. The figure which illustrates the arrangement pattern of the partial field and local field in a 1st embodiment. The figure which illustrates the arrangement pattern of the partial field and local field in a 2nd embodiment.

[First Embodiment]
A first embodiment of the present invention will be described below with reference to FIGS.

  The image processing apparatus 1 according to the present embodiment is mounted on a vehicle 100 traveling on a road. The image processing apparatus 1 receives an image captured by the imaging apparatus 2 from the imaging apparatus 2. Then, the image processing apparatus 1 recognizes a predetermined recognition target image from the captured image acquired from the imaging apparatus 2. In the present embodiment, the recognition target is a lane mark attached to a road surface (traveling road surface) on which the vehicle 100 travels.

  The imaging device 2 is configured by a monocular camera or the like. The imaging device 2 is mounted on the host vehicle 100 so as to capture an image of a monitoring area around the vehicle 100 (hereinafter may be referred to as the host vehicle 100), for example, a front area. The captured image of the imaging device 2 may be a color image or a monotone image. Further, the imaging device 2 may be a stereo camera configured by two or more cameras.

  The image processing apparatus 1 includes an electronic circuit unit including a CPU, RAM, ROM, interface circuit, and the like.

  The image processing apparatus 1 executes a process of extracting an edge that is a candidate for a lane mark from a captured image of the imaging apparatus 2 as a function realized by a mounted program or a function realized by a hardware configuration. An edge extracting unit 5 that performs processing for updating (variably setting) a threshold value used in processing of the edge extracting unit 5, and an edge extracted by the edge extracting unit 5. And a lane mark recognition unit 7 that executes processing for recognizing the lane mark image.

  Note that the edge extraction unit 5, the threshold update unit 6, and the lane mark recognition unit 7 correspond to an edge extraction unit, a threshold update unit, and a lane mark recognition unit in the present invention, respectively.

  Next, the processing of the image processing apparatus 1 will be described in detail.

  The image processing apparatus 1 acquires a captured image of the imaging apparatus 2 at a predetermined calculation processing cycle, and stores and holds captured images from the current time to a predetermined time before in an image memory (not shown).

  The image processing apparatus 1 performs processing of the edge extraction unit 5 and the lane mark recognition unit 7 on the captured image acquired at each calculation processing cycle.

  FIG. 2 schematically shows an example of a captured image acquired from the imaging device 2. The captured image in the illustrated example illustrates a captured image acquired when the host vehicle 100 is traveling on the traveling road surface 111 between the right lane mark 110R and the left lane mark 110L.

  In the illustrated example, the right lane mark 110R is a solid white line, and the left lane mark 110L is a dashed white line. Further, in the captured image of the illustrated example, an area with stippling (an area on the traveling road surface 111) represents a shaded area where light is blocked by the eaves 112 erected on the right side of the traveling road surface 111. Therefore, the image of the traveling road surface 111 in the illustrated example includes a bright part (sunlight part) having a relatively high luminance and a dark part (shade part) having a relatively low luminance.

  The edge extraction unit 5 executes processing for extracting an edge in an edge extraction target region set as shown in FIG. 2 in the captured image. The edge extraction target region is set as a region where a lane mark image as a recognition target in the present embodiment is highly likely to exist (in the illustrated example, a rectangular region excluding a portion near the periphery of the captured image). The

  The edge extraction target area may be a predetermined fixed area, but the size or the arrangement position may be changed as appropriate depending on the situation such as the imaging environment. Further, the entire captured image may be set as the edge extraction target region. In addition, the edge extraction target region may be a region having a shape other than a square shape (for example, a trapezoidal shape).

  The edge extraction unit 5 uses a differential filter or the like from the captured image of the edge extraction target region to indicate the change (gradient) of the pixel value (for example, luminance value) in the horizontal direction (or vertical direction) of the captured image. Create The edge image is an image in which the size of each pixel value increases as the gradient of the change in the pixel value of the original captured image increases.

  When the captured image is a color image, it is also possible to use a hue or saturation as the pixel value of the captured image and create an edge image based on the change in the hue or saturation.

  The edge extraction unit 5 further extracts, as a component of the edge, a pixel whose pixel value is equal to or greater than a predetermined threshold in the created edge image (that is, a pixel whose pixel value change in the original captured image is equal to or greater than the predetermined threshold). To do.

  In this case, in the present embodiment, the threshold value is set for each of a plurality of partial regions obtained by dividing (segmenting) the edge extraction target region by the processing of the threshold update unit 6 described later.

  As an example, as shown in FIG. 3, the edge extraction target region is divided into seven partial regions 11 (1) to 11 (7) (each of which is surrounded by a bold frame). Different thresholds are set for each of 1) to 11 (7). In the present embodiment, each of the partial regions 11 (1) to 11 (7) is a region (unit region) including one or more unit regions (small regions surrounded by a broken line frame in FIG. 3) having a predetermined area. Is set as an area of an integer multiple of.

  Of the seven partial areas 11 (1) to 11 (7), the partial areas 11 (1), 11 (2), and 11 (3) are arranged in order from the lower side in the right half of the edge extraction target area. The regions and partial regions 11 (4), 11 (5), and 11 (6) arranged in the direction are the regions arranged in the vertical direction in order from the bottom in the left half of the edge extraction target region, and the partial region 11 (7). These are regions located above the partial regions 11 (3) and 11 (6) and at the upper end of the edge extraction target region.

  In other words, the right partial areas 11 (1), 11 (2), and 11 (3) of the edge extraction target area are the areas where the image of the right lane mark 110R is shown, and the left partial areas 11 (4), 11 ( In other words, 5) and 11 (6) are areas where the image of the left lane mark 110L is shown. Also, the partial area 11 (7) at the upper end of the edge extraction target area is an area where an empty image is mainly captured.

  Although details will be described later, the threshold update unit 6 sets different thresholds for edge extraction for each of the partial regions 11 (1) to 11 (7).

  Then, the edge extraction unit 5 performs edge extraction in each of the partial areas 11 (1) to 11 (7) using a threshold value for each of the partial areas (1) to 11 (7). More specifically, the edge extraction unit 5 causes the pixel value of the edge image to be equal to or greater than the threshold value corresponding to the partial areas 11 (1) to 11 (7) in each of the partial areas 11 (1) to 11 (7). Pixels are extracted as constituent elements of edges (edges that are candidates for edges of lane marks) in the partial areas 11 (1) to 11 (7).

  Supplementally, in FIG. 3, the partial areas 11 (1), 11 (2) and 11 (3) on the right side of the edge extraction target area and the partial areas 11 (4), 11 (5) and 11 (6) on the left side. The position of the boundary line is the center position in the left-right direction of the edge extraction target area, but the position of the boundary line may be a position shifted from the center position to the left side or the right side.

  Next, the image processing apparatus 1 executes the process of the lane mark recognition unit 7 using the edge extracted by the edge extraction unit 5. In this case, the lane mark recognition unit 7 recognizes the presence position and direction of the lane mark image in the captured image (existence position and direction on the captured image) by the edges continuously arranged on the right side and the left side of the host vehicle 100. To do.

  For example, the existence positions and directions of the lane marks 110R and 110L in the captured image shown in FIG. 2 are recognized. Then, the lane mark recognition unit 7 estimates the positional relationship between the host vehicle 100 and the lane mark in real space based on the presence position and direction of the lane mark recognized on the captured image.

  The estimation result is used, for example, for travel control of the host vehicle 100 (travel area control, etc.) or alarm control for the driver.

  Next, details of the processing of the threshold update unit 6 will be described. In the following description, a reference sign th (i) is attached to the threshold for edge extraction corresponding to an arbitrary partial region 11 (i) (i = 1, 2,...).

  In the present embodiment, the threshold update unit 6 updates the threshold for edge extraction in each partial region 11 (i) (i = 1, 2,...) At a predetermined calculation processing cycle. In this case, the calculation processing cycle of the processing of the threshold update unit 6 may be the same as the calculation processing cycle of the processing of the edge extraction unit 5, but even if the cycle is longer than the calculation processing cycle of the processing of the edge extraction unit 5. Good.

  Furthermore, the processing of the threshold update unit 6 may be executed using a change in the imaging environment of the imaging device 2 as a trigger.

  In the present embodiment, the threshold update unit 6 applies the partial areas 11 (i) (i = 1, i) to the partial areas 11 (1), 11 (2), and 11 (3) on the right side of the edge extraction target area. 2 and 3), the threshold value th (i) for edge extraction is such that the position of the partial area 11 (i) (position on the captured image) is higher (in other words, the partial area 11 (i), the threshold value for edge extraction th (i) is set to be relatively small as the distance from the host vehicle 100 becomes larger. That is, these threshold values th (1), th (2), and th (3) are set so that th (1)> th (2)> th (3).

  Similarly, the threshold update unit 6 applies the partial areas 11 (i) (i = 4, 5, 5) to the left partial areas 11 (4), 11 (5), 11 (6) of the edge extraction target area. The threshold value th (i) for edge extraction corresponding to 6) is set so that th (4)> th (5)> th (6).

  Here, the image shown in the upper partial area is a projected image of a location where the distance from the own vehicle 100 (or from the imaging device 2) is larger, and therefore, compared to the image shown in the lower partial area. It tends to be unclear. On the other hand, the image shown in the lower partial area is a projected image of a portion having a smaller distance from the own vehicle 100 (or from the imaging device 2), and therefore, compared with the image shown in the upper partial area. It tends to be clear.

  Therefore, as described above, by setting the thresholds th (1), th (2), and th (3) so that th (1)> th (2)> th (3), the partial region 11 ( 1), 11 (2), and 11 (3), the edges of the lane marks (for example, the lane mark 110R shown in FIG. 2), regardless of the distance from the vehicle 100 (the distance from the imaging device 2), Extraction can be performed with high reliability.

  Similarly, by setting the thresholds th (4), th (5), and th (6) so that th (4)> th (5)> th (6), the partial regions 11 (4), The edges of the lane marks (for example, the lane mark 110L shown in FIG. 2) shown in 11 (5) and 11 (6) are extracted with high reliability regardless of the distance from the host vehicle 100 (distance from the imaging device 2). It becomes possible to do.

  In the present embodiment, the partial area 11 (7) at the upper end of the edge extraction target area basically corresponds to the partial area 11 (7) because the lane mark image is not captured. The threshold th (7) for edge extraction is set to a value sufficiently larger than the other thresholds th (1) to th (6), for example.

  More specifically, the thresholds th (1) to th (6) of the partial areas 11 (1) to 11 (6) are set as follows.

  That is, the threshold value th (i) corresponding to each 11 (i) (i = 1, 2,..., 6) of each of the partial areas 11 (1) to (6) is the local area included in the partial area 11 (i). When an edge is extracted by the processing of the edge extraction unit 5 using the threshold th (i) in the region 12 (i), the number of extracted edges (the number of pixels extracted as an edge component) is calculated. The threshold value th (i) is set so that the number matches or substantially matches the target value determined corresponding to the partial area 11 (i).

  More specifically, the threshold update unit 6 firstly calculates a local region 12 (i) for counting the number of extracted edges for each of the partial regions 11 (i) (i = 1, 2,..., 6). Set.

  In this case, in the present embodiment, the local region 12 (i) indicates that the image shown in the local region 12 (i) is the lane mark on the right side and the left side of the host vehicle 100 on the traveling road surface in front of the host vehicle 100. It is set to be an image of a road surface portion (a road surface portion to which no lane mark is attached).

  More specifically, the threshold update unit 6 detects the positions of the right and left lane marks (on the captured image) that have already been recognized by the processing of the lane mark recognition unit 7 executed before the current time (from the current time to a predetermined time before). In the captured image in which a threshold for edge extraction is newly set (the threshold is to be updated) based on the position of the vehicle 100, the lane marks on the right side and the left side of the host vehicle 100 are reflected. Estimate the expected approximate position.

  Note that the approximate position where the lane mark is expected to be captured may be estimated from only the current captured image according to a rule determined experimentally in advance.

  Then, based on the estimated position of the lane mark, the partial area 11 (i) (i = 1, 2,..., 6) is an area in which an image of the road surface portion between the right and left lane marks is captured. A local region 12 (i) corresponding to is set.

  As an example, in a situation where the host vehicle 100 is traveling in a substantially central position between the lane marks 110R and 110L shown in FIG. 2, for example, regions 12 (1) to 12 (6) hatched in FIG. Are set as local regions corresponding to the partial regions 11 (1) to 11 (6), respectively.

  The arrangement positions and sizes (areas) of the regions 12 (1) to 12 (6) shown in FIG. 3 are examples, and the patterns of these arrangement positions and sizes are limited to the patterns shown in FIG. is not.

  For example, the area (or horizontal or vertical width) of the local regions 12 (1) and 12 (4) corresponding to the lowermost partial regions 11 (1) and 11 (4) is the upper partial region 11 (2), It may be the same as the local regions 12 (2), 12 (3), 12 (5), and 12 (6) corresponding to 11 (3), 11 (5), and 11 (6), respectively.

  Also, each of the local regions 12 (1), 12 (2), 12 (3) of the right partial regions 11 (1), 11 (2), 11 (3) and the left partial region 11 (4), Each of the local regions 12 (4), 12 (5), and 12 (6) of 11 (5) and 11 (6) may be a region having a different size from each other. Alternatively, the arrangement positions of the local regions 12 (1), 12 (2), and 12 (3) in the right partial regions 11 (1), 11 (2), and 11 (3) and the left partial region 11 ( The arrangement positions of the local regions 12 (4), 12 (5), and 12 (6) of 4), 11 (5), and 11 (6) may be different from the arrangement positions that are symmetric.

  Further, any of the local regions 12 (1) to 12 (6) may be a region smaller than the unit region described above.

  Then, the threshold update unit 6 sets a threshold th (i) for edge extraction within a predetermined range in the local region 12 (i) of each partial region 11 (i) (i = 1, 2,..., 6). A plurality of candidate values (for example, th, th + α, th + 2 · α, th + 3 · α,..., Th + n · α, which are different from each other by a predetermined value α) are set in order, and threshold values of these candidate values are set. Using th (i), the same processing as that performed by the edge extracting unit 5 is performed, so that the number N () of edges extracted in the local region 12 (i) corresponding to the threshold value th (i) of each candidate value. count th (i)).

  The threshold update unit 6 sets a target value for the number of extracted edges for each local region 12 (i) of each partial region 11 (i) (i = 1, 2,..., 6). In this case, for each partial region 11 (i) (i = 1, 2,..., 6), a target value for the number of extracted edges per unit area is set in advance based on experiments or the like. Then, the threshold update unit 6 sets the target value of the number of extracted edges in each local region 12 (i) to a value obtained by multiplying the target value per unit area by the area of the local region 12 (i). .

  Then, the threshold updating unit 6 determines that the number N (th (i)) of edge extractions among the candidate values of the threshold th (i) in each local region 12 (i) (i = 1, 2,..., 6). A candidate value that is closest to the target value set corresponding to the local region 12 (i) is selected as the threshold value th (i) in the entire partial region 11 (i).

  In this case, it is necessary that the number N (th (i)) of edge extraction is equal to or less than the target value, and the number N (th (i)) of edge extraction among candidate values that satisfy the necessary condition. A candidate value that maximizes the threshold value may be selected as the threshold value th (i) in the partial region 11 (i).

  Alternatively, on the condition that the number of extracted edges N (th (i)) is equal to or greater than the target value, the number of extracted edges N (th (i)) among the candidate values satisfying the necessary condition is minimized. Such a candidate value may be selected as the threshold value th (i) in the partial region 11 (i).

  Supplementally, the target value of the number of extracted edges per unit area in each partial region 11 (i) (i = 1, 2,..., 6) is a threshold value that is selectively determined as described above in the present embodiment. th (1) to th (6) are set such that th (1)> th (2)> th (3) and th (4)> th (5)> th (6).

  For the partial region 11 (7) at the upper end of the edge extraction target region, the threshold value th (7) is set to a predetermined value in this embodiment. However, the threshold th (7) may be variably set so that the number of extracted edges in the partial region 11 (7) matches or substantially matches a predetermined target value.

  As described above, the thresholds th (1) to th (7) set corresponding to the partial areas 11 (1) to 11 (7) are processed in the current calculation processing cycle of the edge extraction unit 5, or Used in the processing after the next arithmetic processing cycle.

  According to the present embodiment described above, a different threshold th (i) is set for each partial region 11 (i) (i = 1, 2,...) Of the edge extraction target region of the captured image. Therefore, for each partial region 11 (i) (i = 1, 2,...), The edge can be extracted using an appropriate threshold th (i) for extracting the edge. As a result, it is possible to prevent a situation in which a required edge (the edge of a lane mark in this embodiment) cannot be extracted from each partial region 11 (i) or a large number of extra edges are extracted. .

  For example, as shown in FIG. 2, even if a relatively dark shaded portion and a relatively bright sunny portion are mixed on the road surface 111, each partial region 11 (i) (i = 1) , 2,...), The edge of the lane mark 110R or 110L can be appropriately extracted.

  Thus, a required edge can be extracted with high reliability.

  Further, in the present embodiment, the threshold th (i) in each of 11 (i) (i = 1, 2,..., 6) of each of the partial areas 11 (1) to 11 (6) where the lane mark is highly likely to be captured is Since it is set according to the number of extracted edges in the local region 12 (i) set so that a portion between the left and right lane marks is captured, it is difficult to be affected by noise components in the captured image. A highly reliable threshold th (i) can be set appropriately.

  Further, the threshold value th (i) in each of the partial areas 11 (1) to 11 (6) 11 (i) (i = 1, 2,..., 6) is a partial area of the partial area 11 (i). Since it is set according to the number of extracted edges in a certain local region 12 (i), the setting process can be performed efficiently with a small calculation load.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG. Note that this embodiment is different from the first embodiment only in the processing of the threshold update unit 6. For this reason, the description of the present embodiment will be focused on matters that are different from the first embodiment, and the description of the same matters as the first embodiment will be omitted.

  In the present embodiment, the threshold update unit 6 includes 11 (i) (i = i = 11) of each of the partial areas 11 (1) to 11 (6) among the partial areas 11 (1) to 11 (7) of the edge extraction target area. The local region 12 (i) for counting the number of extracted edges in 1, 2,..., 6) is set in a different form from the first embodiment.

  In the present embodiment, the local region 12 (i) is an image of one of the right lane mark and the left lane mark of the host vehicle 100 on the road surface ahead of the host vehicle 100. It is set to be reflected in.

  More specifically, the threshold update unit 6 detects the positions of the right and left lane marks (on the captured image) that have already been recognized by the processing of the lane mark recognition unit 7 executed before the current time (from the current time to a predetermined time before). The right and left lane marks of the host vehicle 100 are expected to be captured in the captured image in which a threshold for edge extraction is to be newly set (to be updated). Estimate the position.

  Then, the threshold update unit 6 determines the local regions 12 (1), 12 (2) in the right partial regions 11 (1), 11 (2), 11 (3) based on the estimated position of the right lane mark. ), 12 (3), the local regions 12 (1), 12 (2), 12 (3) are set so that the right lane mark is shown.

  Similarly, the threshold update unit 6 determines the local regions 12 (4), 12 (12) in the left partial regions 11 (4), 11 (5), 11 (6) based on the estimated position of the left lane mark. The local regions 12 (4), 12 (5), and 12 (6) are set so that the left lane mark appears in 5) and 12 (6).

  As an example, in the situation where the host vehicle 100 is traveling substantially at the center position between the lane marks 110R and 110L shown in FIG. 2, for example, regions 12 (1) to 12 (6) hatched in FIG. Are set as local regions corresponding to the partial regions 11 (1) to 11 (6), respectively.

  Note that the arrangement positions and sizes (areas) of the regions 12 (1) to 12 (6) shown in FIG. 4 are examples, and the patterns of these arrangement positions and sizes are limited to the patterns shown in FIG. is not. As these arrangement positions and sizes, various patterns can be adopted as in the case described with respect to the first embodiment.

  The present embodiment is the same as the first embodiment except for the matters described above.

  In this embodiment, as in the first embodiment, an appropriate threshold th (i) is used for extracting an edge for each partial region 11 (i) (i = 1, 2,...). Thus, the edge can be extracted. As a result, a required edge (the edge of the lane mark in this embodiment) can be extracted with high reliability in each partial region 11 (i).

  Further, in the present embodiment, the threshold th (i) in each of 11 (i) (i = 1, 2,..., 6) of each of the partial areas 11 (1) to 11 (6) where the lane mark is highly likely to be captured is The threshold value th (i) suitable for extracting the edge of the lane mark is set according to the number of extracted edges in the local region 12 (i) set so that the right or left lane mark is captured. Can be set.

  Furthermore, the threshold value th (i) in each of the partial areas 11 (1) to 11 (6) 11 (i) (i = 1, 2,..., 6) is the same as in the first embodiment. Since it is set according to the number of extracted edges in the local area 12 (i), which is a partial area of (i), the setting process can be efficiently performed with a small calculation load.

[Modification]
Next, some modifications related to the above-described embodiment will be described.

  In each of the above embodiments, the number of partial areas 11 (i) (i = 1, 2,...) In the edge extraction target area and the arrangement position and size of each partial area 11 (i) are fixed. The number of partial areas, the arrangement pattern of each partial area, and the size of each partial area may be changed as appropriate in accordance with 100 driving environments (and thus an imaging environment).

  Further, although cases have been described with the above embodiments as examples where the edges of the lane marks on the road surface are extracted, the present invention can also be applied to the case where the edges of an object such as another vehicle are extracted from the captured image.

  In addition, the imaging device 2 is not limited to the imaging device 2 that is mounted in advance on the vehicle 100, for example, an imaging device that can be carried by the driver, or an imaging device 2 that is mounted on an arbitrary moving body other than the vehicle 100. There may be.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 2 ... Imaging device, 5 ... Edge extraction part (edge extraction means), 6 ... Threshold update part (threshold update means), 7 ... Lane mark recognition part (lane mark recognition means), 100 ... Vehicle, 111 ... traveling road surface, 110R, 110L ... lane mark, 11 (1) to 11 (7) ... partial area, 12 (1) to 12 (7) ... local area.

Claims (4)

An image processing apparatus comprising edge extraction means for extracting an edge from a captured image by comparing a change in a pixel value in a captured image of an image capturing apparatus that captures a monitoring region with a predetermined threshold,
The edge extraction target area, which is an area in the captured image where the edge extraction unit executes the process of extracting the edge from the captured image, is divided into a plurality of partial areas each having a different threshold value. ,
The edge extraction means is configured to execute processing for extracting an edge in each partial region of the edge extraction target region using the threshold value set corresponding to the partial region,
Threshold update means for updating the threshold in each of one or more partial areas of the plurality of partial areas according to the number of edges extracted by the edge extraction means in a local area of the partial areas An image processing apparatus further comprising: An image processing apparatus comprising edge extraction means for extracting an edge from a captured image by comparing a change in a pixel value in a captured image of an image capturing apparatus that captures a monitoring region with a predetermined threshold,
The edge extraction target area, which is an area in the captured image where the edge extraction unit executes the process of extracting the edge from the captured image, is divided into a plurality of partial areas each having a different threshold value. ,
The edge extraction unit is configured to execute using the threshold value set in correspondence with processing of extracting an edge in the partial region in the partial regions of the edge extraction target area,
The monitoring area imaged by the imaging device is an area including the traveling road surface of the vehicle,
Lane mark recognition means for recognizing an image of the lane mark on the road surface using the edge extracted by the edge extraction means in the edge extraction target area including the image of the road surface of the vehicle;
The threshold value in each of one or more partial areas having the possibility of capturing an image of at least the lane mark in the plurality of partial areas is set to the road surface between the right and left lane marks of the traveling road surface. Threshold update means for updating according to the number of edges extracted by the edge extraction means in a local area set in each of the one or more partial areas so as to be an area in which an image is shown. An image processing apparatus. An image processing apparatus comprising edge extraction means for extracting an edge from a captured image by comparing a change in a pixel value in a captured image of an image capturing apparatus that captures a monitoring region with a predetermined threshold,
The edge extraction target area, which is an area in the captured image where the edge extraction unit executes the process of extracting the edge from the captured image, is divided into a plurality of partial areas each having a different threshold value. ,
The edge extraction means is configured to execute processing for extracting an edge in each partial region of the edge extraction target region using the threshold value set corresponding to the partial region,
The monitoring area imaged by the imaging device is an area including the traveling road surface of the vehicle,
Lane mark recognition means for recognizing an image of the lane mark on the road surface using the edge extracted by the edge extraction means in the edge extraction target area including the image of the road surface of the vehicle;
Among the plurality of partial areas, at least one of the one or more partial areas having the possibility of capturing the lane mark image is set to the one or more partial areas so that the lane mark image is captured. An image processing apparatus, further comprising: a threshold update unit that updates the local region set in each of the partial regions according to the number of edges extracted by the edge extraction unit. The image processing apparatus according to claim 2 or 3,
The threshold update means estimates the location of the lane mark in a new captured image based on the position in the captured image of the lane mark that has already been recognized by the lane mark recognition means before the threshold is updated, An image processing apparatus configured to set the local region based on an estimation result.