JP2020106484A - Adhering matter detection device and adhering matter detection method - Google Patents

Abstract

To prevent erroneous detection of a cloud for a water droplet and increase detection accuracy.SOLUTION: An adhering matter detection device according to an embodiment comprises an extraction unit and a detection unit. The extraction unit extracts edge information included in an image imaged by an imaging device for each pixel. The detection unit detects a candidate region where a water droplet is estimated to be present in the image using a predetermined detection algorithm based on the edge information extracted by the extraction unit. The detection unit also excludes a candidate region corresponding to a predetermined exclusion condition based on the position, luminance, and size of a candidate region from the detected candidate region.SELECTED DRAWING: Figure 2

Description

The disclosed embodiments relate to an adhering matter detecting device and an adhering matter detecting method.

BACKGROUND ART Conventionally, a vehicle-mounted camera that is mounted on a vehicle and captures an image around the vehicle is known. The image captured by the vehicle-mounted camera is displayed on a monitor, for example, to assist the driver's visual field, or used for sensing to detect a white line on the road, an object approaching the vehicle, or the like.

Note that, for example, raindrops, snowflakes, dust, mud, and other attached matter may adhere to the lens of the vehicle-mounted camera, which may interfere with the above-described visibility assistance and sensing. Therefore, for example, there is a technique of extracting a so-called edge from an image captured by a vehicle-mounted camera, detecting an adhered matter to the lens based on the extracted edge, and ejecting cleaning water or compressed air toward the lens to remove the adhered matter. Proposed. (For example, refer to Patent Document 1).

JP 2001-141838 A

However, the above-described conventional technique has room for further improvement in that, when a cloud is reflected in an outdoor scene, the cloud is prevented from being erroneously detected as a water drop and the detection accuracy is improved.

One aspect of the embodiment is made in view of the above, and provides an adhering matter detection device and an adhering matter detection method that can suppress erroneous detection of a cloud as a water droplet and improve detection accuracy. The purpose is to do.

The adhering matter detection device according to one aspect of the embodiment includes an extraction unit and a detection unit. The extraction unit extracts edge information of each pixel included in a captured image captured by the image capturing device. The detection unit detects a candidate region in which the water droplet is estimated to exist in the captured image, using a predetermined detection algorithm based on the edge information extracted by the extraction unit. Further, the detection unit excludes, from the detected candidate regions, the candidate regions that meet a predetermined exclusion condition based on the position, brightness, and size of the candidate region.

According to one aspect of the embodiment, it is possible to suppress erroneous detection of a cloud as a water drop and improve detection accuracy.

FIG. 1A is a schematic explanatory diagram (part 1) of an adhering matter detection method according to the embodiment. FIG. 1B is a schematic explanatory diagram (part 2) of the attached matter detection method according to the embodiment. FIG. 1C is a schematic explanatory diagram (part 3) of the attached matter detection method according to the embodiment. FIG. 2 is a block diagram of the adhering matter detection device according to the embodiment. FIG. 3 is a diagram showing a vector calculation method. FIG. 4A is an explanatory diagram (1) of the method of calculating the representative value. FIG. 4B is an explanatory diagram (part 2) of the method of calculating the representative value. FIG. 5A is a diagram showing an example of a template. FIG. 5B is a diagram showing an example of the matching process. FIG. 6 is a diagram illustrating an example of the detection process. FIG. 7A is an explanatory diagram of candidate areas. FIG. 7B is an explanatory diagram of xy coordinates of the candidate area. FIG. 7C is a diagram showing an example of data content regarding the candidate areas included in the detection information. FIG. 8 is an explanatory diagram of exclusion conditions included in the exclusion information. FIG. 9 is a flowchart showing a processing procedure executed by the adhering matter detection device according to the embodiment.

Embodiments of an adhering matter detecting device and an adhering matter detecting method disclosed in the present application will be described in detail below with reference to the accompanying drawings. The present invention is not limited to the embodiments described below.

Further, in the following, a case will be described as an example in which a water droplet attached to an on-vehicle camera mounted on a vehicle to image the surroundings of the vehicle is detected as an attached matter.

In addition, hereinafter, an outline of the attached matter detection method according to the embodiment will be described with reference to FIGS. 1A to 1C, and then an attached matter detection device 10 to which the attached matter detection method according to the embodiment is applied will be described with reference to FIGS. This will be described using 9.

First, the outline of the attached matter detection method according to the embodiment will be described with reference to FIGS. 1A to 1C. 1A to 1C are schematic explanatory diagrams (No. 1) to (No. 3) of the attached matter detection method according to the embodiment.

Vehicles are equipped with vehicle-mounted cameras such as a front camera, a back camera, a right side camera, and a left side camera in order to capture an image around the vehicle. In addition, below, this vehicle-mounted camera is described as "camera 2."

The camera 2 includes an image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and the image pickup device picks up an image of the vehicle periphery. A wide-angle lens such as a fish-eye lens is adopted as the lens of the camera 2, and each camera 2 has an angle of view of 180 degrees or more, and by using these, it is possible to image the entire circumference of the vehicle.

Then, the image captured by the camera 2 is output to an adhering matter detection device (not shown) mounted on the vehicle.

The attached matter detection device analyzes the image acquired from the camera 2 for each frame, and uses a technique such as template matching based on the edge information of each pixel (such as the gradient of brightness) to detect the attached matter in the image. The existing area is detected.

For example, as shown in FIG. 1A, the adhering matter detection device detects a candidate area Da that is an area in which a water droplet is estimated to be present in the captured image I. 1A to 1C, reference numerals are given only to the two candidate areas Da, but the white line frames in the drawings are all candidate areas Da.

However, in detecting such water droplets, when a cloud is reflected in the captured image I, for example, as shown in FIG. 1B, the attached matter detection device according to the related art has water droplets in an area where the cloud is present. There was a case where the candidate area Da was erroneously detected. This is because the cloud may have an irregular shape, but may have a rounded contour like a water drop.

Therefore, in the attached matter detection method according to the embodiment, with respect to the candidate area Da in the case where such a cloud is erroneously detected as a water drop, its characteristics are extracted in advance from a plurality of sample data and the like is conditioned ( Step S1).

In addition, in the deposit detection method according to the embodiment, the condition is used as an exclusion condition, and the candidate area Da that corresponds to the exclusion condition is excluded from the area where water droplets are estimated to exist (step S2). This can prevent the cloud from being erroneously detected as a water drop. Therefore, according to the deposit detection method according to the embodiment, the detection accuracy can be improved.

Note that details of the characteristics in the case of erroneously detecting a cloud as a water drop and the exclusion conditions based on this will be described later with reference to FIG.

Hereinafter, an embodiment of the adhering matter detecting device 10 to which the adhering matter detecting method according to the above-described embodiment is applied will be described more specifically.

FIG. 2 is a block diagram of the adhering matter detection device 10 according to the embodiment. Note that, in FIG. 2, only the components necessary for explaining the features of the present embodiment are represented by functional blocks, and the description of general components is omitted.

In other words, each component shown in FIG. 2 is functionally conceptual, and does not necessarily have to be physically configured as shown. For example, the specific form of distribution/integration of each functional block is not limited to that shown in the figure, and all or part of the functional block may be functionally or physically distributed in arbitrary units according to various loads and usage conditions. -It can be integrated and configured.

As shown in FIG. 2, the adhering matter detection device 10 according to the embodiment includes a storage unit 11 and a control unit 12. Further, the adhered matter detection device 10 is connected to the camera 2 and various devices 50.

Note that FIG. 2 shows the case where the adhering matter detection device 10 is configured separately from the camera 2 and the various devices 50, but the present invention is not limited to this, and is integrated with at least one of the camera 2 and the various devices 50. It may be configured.

Since the camera 2 has already been described, its description is omitted here. The various devices 50 are devices that obtain the detection result of the adhering matter detection device 10 and perform various controls of the vehicle. The various devices 50 may display, for example, a display device for notifying the user of an adhered substance on the lens of the camera 2 or a user's instruction to wipe off the adhered substance, ejecting fluid or gas toward the lens to remove the adhered substance. It includes a removal device for removal, a vehicle control device for controlling automatic driving, and the like.

The storage unit 11 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. In the example of FIG. 2, the template information 11a and The detection information 11b and the exclusion information 11c are stored.

The template information 11a is information about a template used in a matching process executed by the matching unit 12d described later. The detection information 11b is information including the detection conditions of the candidate area Da and data regarding the detected candidate area Da. The exclusion information 11c is information including the exclusion conditions described above.

The control unit 12 is a controller, and various programs stored in a storage device inside the adhering matter detection device 10 work on the RAM by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). It is realized by being executed as a region. Further, the control unit 12 can be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 12 includes an acquisition unit 12a, an extraction unit 12b, a conversion unit 12c, a matching unit 12d, and a detection unit 11e, and realizes or executes the functions and actions of information processing described below.

The acquisition unit 12a acquires a camera image for each frame from the camera 2 and executes a pre-process required for image analysis.

The acquisition unit 12a converts the camera image into grayscale, for example, as preprocessing. Note that the grayscale conversion is a process of converting each pixel in the camera image so as to be expressed by each gradation from white to black according to the brightness. Such gray scale conversion may be omitted.

The acquisition unit 12a also changes the camera image to a predetermined size, for example, as another pre-processing. The acquisition unit 12a also outputs the preprocessed camera image to the extraction unit 12b.

The extraction unit 12b extracts edge information of each pixel in the camera image by using, for example, a Sobel filter in the camera image input from the acquisition unit 12a. Here, the edge information refers to the edge strength of each pixel in the X-axis direction and the Y-axis direction.

Further, the extraction unit 12b outputs the extracted edge information in association with the grayscale image to the conversion unit 12c. Note that the extraction unit 12b may use another edge extraction method such as a Laplacian filter instead of the Sobel filter.

The conversion unit 12c calculates an edge vector of each pixel based on the edge strength in the X-axis direction and the Y-axis direction of each pixel input from the extraction unit 12b, and encodes the edge direction. In addition, in the adhering matter detection device 10 according to the embodiment, for example, a representative value of edges in a plurality of pixels is obtained, and the representative value is encoded. Details of this point will be described later with reference to FIGS. 4A and 4B.

Further, the conversion unit 12c outputs the grayscale image in which the edge direction is encoded to the matching unit 12d. The matching unit 12d performs a matching process using a regular expression between the coded grayscale image input from the conversion unit 12c and the code pattern indicating the characteristics of the water droplet. The regular expression mentioned here is a set of code strings represented by one code.

The code pattern indicating the characteristics of the water droplet is stored in the template information 11a. A specific example of such a code pattern will be described later with reference to FIG. 5A. Details of the matching processing by the matching unit 12d will be described later with reference to FIG. 5B.

The detection unit 12e detects a candidate area Da in which the water droplet is estimated to be present in the captured image I by using a predetermined detection algorithm based on the edge information extracted by the extraction unit 12b.

Specifically, the detection unit 12e detects a candidate area Da in which the presence of water droplets is estimated based on the code pattern extracted by the matching unit 12d. Further, the detection unit 12e determines, for each of the detected candidate areas Da, whether or not the cloud is erroneously detected as a water droplet.

Further, regarding the candidate area Da that is determined to be a water drop based on the determination result, the detection unit 12e notifies the various devices 50 of the candidate area Da. On the other hand, the detection unit 12e does not notify the various devices 50 of the candidate area Da determined to correspond to the erroneous detection based on the determination result, and excludes it from the processing target in the subsequent stage.

That is, the detection unit 12e excludes the candidate area Da having low reliability. By excluding unnecessary image areas in this way, it is possible to improve the accuracy of detecting the adhering matter and reduce the processing load in the subsequent stage. The details of the detection processing by the detection unit 12e will be described later with reference to FIGS.

Here, an example of each algorithm executed by the conversion unit 12c, the matching unit 12d, and the detection unit 12e will be specifically described with reference to FIGS. 3 to 8. FIG. 3 is a diagram showing a vector calculation method. 4A and 4B are explanatory views (No. 1) and (No. 2) of the method of calculating the representative value.

Further, FIG. 5A is a diagram showing an example of the template. Further, FIG. 5B is a diagram showing an example of the matching process. FIG. 6 is a diagram showing an example of the detection process.

Further, FIG. 7A is an explanatory diagram of the candidate area Da. Further, FIG. 7B is an explanatory diagram of xy coordinates of the candidate area Da. Further, FIG. 7C is a diagram showing an example of data content regarding the candidate area Da included in the detection information 11b. Further, FIG. 8 is an explanatory diagram of exclusion conditions included in the exclusion information 11c.

First, specifically, the conversion unit 12c calculates a vector of each pixel by using a trigonometric function based on the edge strength in the X-axis direction and the Y-axis direction, as shown in FIG. In the following, the angle θ formed by the calculated vector shown in FIG. 3 and the X axis on the positive side is referred to as “edge direction”, and the length L of the vector is referred to as “edge strength” of each pixel.

Note that the conversion unit 12c does not have to calculate the edge orientations for all the pixels, and for regions of low priority, the processing may be simplified, such as calculating the edge orientations for each pixel at a predetermined interval. ..

The conversion unit 12c also encodes the calculated edge direction. For example, the conversion unit 12c obtains a representative value of edges in a plurality of pixels and encodes the representative value. Specifically, as shown in FIG. 4A, the conversion unit 12c handles pixel units of 8×8 pixels as one “cell” and pixel units of 3×3 cells as one “block”, for example. In addition, the cell in the center of one block is treated as a "cell of interest".

The conversion unit 12c creates a histogram indicating the edge direction and edge strength of each pixel for each block. Such a histogram will be described with reference to FIG. 4B. Here, the conversion unit 12c derives the edge direction of the center coordinate of the cell of interest from the histogram of the block.

Then, when the conversion unit 12c derives the representative value of the target cell in one block, the block is shifted by one cell to create a histogram, and the representative value of the target cell in the block is calculated.

That is, the conversion unit 12c can reduce the data amount by calculating the representative value for each of the plurality of pixels. In the example shown in FIG. 4A, the cell size is 8×8 pixels, so the data amount is reduced to 1/64.

Note that the numbers of pixels in the blocks and cells shown in FIG. 4A are examples, and the numbers of pixels in the cells and blocks can be set arbitrarily. At this time, the number of pixels in each cell can be changed according to the size of the water droplet to be detected.

For example, when detecting a small water drop, the number of pixels of the cell is set to be small, and when detecting a large water drop, the number of pixels of the cell is set to be large. As a result, it is possible to efficiently detect a water droplet having a desired size.

Alternatively, the conversion unit 12c may simply create a histogram for each cell and calculate the representative value of each cell based on the histogram. The conversion unit 12c may encode all pixels without calculating the representative value.

Next, the histogram will be described with reference to FIG. 4B. In FIG. 4B, the vertical axis represents the edge strength and the horizontal axis represents the edge-oriented class. As shown in FIG. 4B, the conversion unit 12c creates a histogram by allocating the edge direction to each of 18 grades every 20°, for example.

Specifically, the conversion unit 12c creates the histogram in the block by adding the edge strength of each pixel in the block to the class corresponding to the edge direction. Subsequently, the conversion unit 12c obtains the class having the maximum sum of edge strengths from the created histogram.

In the example shown in FIG. 4B, the case where the class of 80 to 100° takes the maximum value is shown. At this time, the conversion unit 12c sets the class as a representative value when the sum of the edge strengths in the class is equal to or more than the threshold value.

In the example shown in FIG. 4B, the sum of the edge strengths exceeds the threshold in the class of 80 to 100°, so the above condition is satisfied. Therefore, the class of the cell of interest in this block is 80 to 100°.

Subsequently, the conversion unit 12c converts the cell of interest into a code assigned according to the class. Here, 18 kinds of codes, for example, 0 to 9 and A to H are assigned to each class, respectively. It should be noted that 0 to 9 and A to H are codes assigned to each class in increments of 20 degrees from 0° to 360°. Further, when the representative value does not exceed the threshold value, that is, a cell having a low edge strength is assigned a code of Z, for example.

In this way, the conversion unit 12c encodes all cells. As a result, in the coded camera image, the codes are arranged in a grid pattern. Note that the conversion unit 12c may calculate the representative value using another statistical calculation method, instead of calculating the representative value described above.

Further, in FIG. 4B, the case where the edge direction is classified into 18 steps has been described, but the present invention is not limited to this, and the number may be less than or more than 18 steps. Further, in FIG. 4B, the case where the reference numerals are A to H and Z is shown, but other characters such as hiragana or numbers, or figures may be used as the reference numerals.

Subsequently, the matching unit 12d performs the matching process between the encoded camera image and the template stored as the template information 11a, as described above. An example of such a template is shown in FIG. 5A. Note that in FIG. 5A, in order to make it easy to understand visually, the template is not shown by the above-mentioned reference symbols, but pin-shaped symbols are used to schematically show the actual edge direction.

As shown in FIG. 5A, the template information 11a has a code pattern that is a code string indicating the characteristics of water droplets as a template. Specifically, for example, it has an upper side pattern, a lower side pattern, a left side pattern, and a right side pattern.

Here, the pattern of each side shown in FIG. 5A corresponds to each side of a rectangle that contains a water droplet inside or is contained inside a water droplet. Here, “including a water drop inside” includes a case where the water drop is inscribed in a rectangle. Further, "included inside the water droplet" includes the case where the rectangle is inscribed in the water droplet. In the present embodiment, “rectangle” includes a square. Further, FIG. 5A shows a case where the edge direction of the pattern on each side is toward the center. In this case, the brightness of the water drop increases from the end to the center, that is, the feature of the water drop is bright in the center and dark in the end.

Note that, conversely, the template information 11a may have a pattern of each side showing the feature of a water drop in which the brightness of the water drop increases from the center toward the end, that is, the center is dark and the end is bright. Good. By doing so, various water drops can be detected.

It should be noted that although FIG. 4 exemplifies four patterns of up, down, left, and right, a pattern including an oblique direction can also be used. By doing so, the detection accuracy of water droplets can be improved.

Next, FIG. 5B shows an example of matching processing using the pattern of each side. Here, for convenience of explanation, the upper side patterns shown in FIG. 5A are shown by using the symbols A to F, respectively. Further, a and b in the same figure schematically show a part of the encoded camera image.

As illustrated in a of FIG. 5B, the matching unit 12d determines that the code patterns match the upper side pattern if the code patterns are arranged in order from A to F.

Specifically, the matching unit 12d is repeated, for example, A three times, B, C, D, and E two times, and F three times, as shown in a of FIG. 5B. If the arrangement satisfies the arrangement order of the respective codes of the upper side pattern, it will be extracted as the upper side pattern.

This is because the number of times the code is repeated differs depending on the size of the water droplet. That is, the larger the water droplet, the longer the length of each code string. In this way, by allowing the repetition of such codes, it is possible to extract a code string indicating a plurality of water droplets having different sizes by one matching process.

Therefore, the water drop can be detected while reducing the processing load. Note that the matching unit 12d may prepare a pattern on each of a plurality of sides in which the length of the code string differs depending on the size of the water droplet, and extract the code string using all such patterns.

Further, since the water droplets are generally spherical, the number of times each code is repeated should be line-symmetrical from the center. Therefore, the matching unit 12d decides to exclude the unbalanced code strings from the extracted code strings.

Specifically, as shown in b of FIG. 5B, the matching unit 12d closely examines the balance between A and F located at both ends, for example. Here, in the figure, the case where A is repeated 3 times and F is repeated 10 times is shown.

At this time, the matching unit 12d excludes the code string pattern even when the arrangement order is satisfied when the numbers of A and F differ by two or more. By doing so, it is possible to prevent erroneous extraction of unnecessary code patterns other than water drops, and suppress erroneous detection of water drops.

Further, in the matching unit 12d, for example, when the extracted code string is longer than the threshold value, the code string can be excluded from the matching. This is because if the code string is long, it is unlikely to be a water drop. This can prevent erroneous detection of water droplets. It should be noted that an optimum value of the threshold is derived in advance by statistics or the like.

Next, FIG. 6 shows an example of the detection processing of the candidate area Da by the detection unit 12e after the matching processing. Note that, in FIG. 6, similar to FIG. 5A, the actual edge direction is schematically illustrated instead of the reference numerals.

Further, a case will be described here where the upper side pattern is first extracted by the matching process. The detection unit 12e first sets a substantially square-shaped candidate area Da1 based on the width of the extracted upper side pattern.

Next, it is assumed that the right side pattern is extracted at a position deviating from the candidate area Da1. At this time, if the center coordinates of the right area pattern candidate area Da2 are within the candidate area Da1, the detection unit 12e performs a process of integrating both candidate areas Da1 and Da2.

After that, for example, when the lower side pattern or the left side pattern is extracted in the integrated candidate area Da3, the detection unit 12e extracts the candidate area Da3 as an area in which it is estimated that water droplets are present. In other words, the detection unit 12e detects the candidate area Da in which water droplets are estimated to exist, with the detection condition (hereinafter, referred to as “direction condition”) that patterns indicating the respective sides in three or more different directions are extracted. To do.

In addition to the direction condition, the detection unit 12e detects that the pattern indicating each side is extracted a predetermined number of times (for example, four times including up, down, left, and right) in the integrated candidate area Da3 (for example, a detection condition ( Hereinafter, it may be referred to as a “count condition”).

As described above, by setting the direction condition of three or more directions or the number of times condition as the detection condition, the candidate area Da in which the water droplet is estimated to exist is extracted even if the patterns of all sides of the upper, lower, left, and right sides are not extracted. You can That is, it is possible to detect, for example, a semicircular water drop that can be cut off from the camera image. Note that these detection conditions are stored, for example, included in the detection information 11b.

Although FIG. 6 shows the case where the candidate area Da is integrated when the center coordinates of the candidate area Da2 are within the candidate area Da1 of the upper side pattern, the present invention is not limited to this. That is, if at least a part of the candidate area Da1 and the candidate area Da2 overlap, they may be integrated.

Further, the integrated candidate area Da3 may be a logical product of the candidate areas Da1 and Da2 or may be a logical sum. Further, although FIG. 6 shows the case where the candidate area Da1 and the candidate area Da2 are rectangular, the present invention is not limited to this and may be another shape such as a circular shape.

Then, the detection unit 12e stores data regarding the detected candidate area Da in the detection information 11b. Specifically, as illustrated in FIG. 7A, the detection unit 12e defines, for example, a rectangular candidate area Da with upper left coordinates (x, y), a width w, and a height h.

Here, for the sake of convenience of the following description, in the present embodiment, the xy coordinate system has an origin in the upper left corner of the captured image I and a horizontal direction to the right in the x-axis positive direction as shown in FIG. 7B, for example. , The downward direction in the vertical direction is defined as the positive y-axis direction. The resolution of the captured image I is, for example, 640×480 as shown in FIG.

Then, as shown in FIG. 7C, the detection information 11b includes, for example, an "area ID" item, an "area information" item, and a "brightness average" item. The "area ID" item stores the identification information of the candidate area Da, and the detection information 11b is managed for each area ID.

The “area information” item stores the upper left coordinates (x, y) of the candidate area Da shown in FIG. 7A, the width w, the height h, and the like. The “average brightness” item stores the average brightness value for each of the detected candidate areas Da. That is, the detection unit 12e calculates a brightness average value for each of the detected candidate areas Da and stores it in the "brightness average" item.

Then, the detection unit 12e determines, based on the detection information 11b and the exclusion information 11c, whether each of the candidate areas Da does not correspond to a case where a cloud is erroneously detected as a water drop.

Specifically, exclusion conditions are set in advance in the exclusion information 11c. More specifically, as shown in FIG. 8, the exclusion information 11c includes, in the exclusion information 11c, “features of the candidate area Da at the time of normal detection” in which a water droplet is detected as a water drop and “candidates at the time of false detection in which a cloud is detected as a water droplet. The "corresponding condition of the cloud" is set in advance based on "the characteristics of the area Da".

The “characteristics of the candidate area Da at the time of correct detection” and the “characteristics of the candidate area Da at the time of erroneous detection” are extracted based on a plurality of sample data collected, for example, at the time of an experiment during development of the adhered matter detection device 10. To be done.

For example, in the present embodiment, from a plurality of sample data, the “feature of the candidate area Da at the time of correct detection” has a distribution over the entire surface, and the size is relatively large (93% is a side length≧40) and the brightness is high. Is relatively dark (76% luminance average <150). The length of one side may be the larger of the width w and the height h described above.

Similarly, the distribution of the “characteristics of the candidate area Da at the time of erroneous detection” is biased toward the upper part of the screen (100% y<200), and the size is relatively small (81% is one side length<60. ), the brightness is relatively bright (85% is luminance average ≧150). Note that y here is, for example, the y coordinate of the lower left corner or the y coordinate of the lower right corner of the candidate area Da.

As a result, in the present embodiment, as shown in FIG. 8, the exclusion condition that is the “corresponding condition of the cloud” is, for example, “(y<200) AND (length of one side<60) AND (luminance average≧150). ". The detection unit 12e determines, for each of the candidate areas Da stored in the detection information 11b, whether or not the exclusion condition is satisfied, and if so, false detection is performed, and if not, correct detection is performed. Each will be judged.

It should be noted that, of the exclusion conditions, "y<200" corresponds to the above-described resolution of 640×480, and, of course, if the vertical resolution "480" is different, it will be adapted accordingly. Can be changed to For example, in the case of vertical “960”, “y<400” may be set. Alternatively, regardless of the resolution in the vertical direction, the condition may be specified such that the percentage is in the upper part of the image (about 42% according to the present embodiment).

Next, a processing procedure executed by the adhering matter detection device 10 according to the embodiment will be described with reference to FIG. 9. FIG. 9 is a flowchart showing a processing procedure executed by the adhering matter detection device 10 according to the embodiment. Note that FIG. 9 shows a processing procedure for a camera image for one frame.

First, the acquisition unit 12a acquires a camera image (step S101). Then, the extraction unit 12b extracts the edge information of each pixel (step S102).

Subsequently, the conversion unit 12c, the matching unit 12d, and the detection unit 12e execute a predetermined detection algorithm based on the edge information extracted by the extraction unit 12b (step S103). Then, the detection unit 12e extracts the candidate area Da in which the presence of water droplets is estimated based on the execution result of the algorithm (step S104).

That is, the conversion unit 12c and the matching unit 12d execute each process described with reference to FIGS. 3 to 5B as an example, and the detection unit 12e detects the candidate area Da by the detection process described with reference to FIG. ..

The detection algorithm for detecting the candidate area Da is not limited to the example described above. For example, the conversion unit 12c calculates the edge strength of each pixel based on the edge information extracted by the extraction unit 12b, and the periphery of the camera 2 is calculated. Each pixel may be binarized by comparing the edge strength with a binarization threshold that takes a different value depending on the environment.

In addition, the conversion unit 12c calculates the orientation of the edge of each pixel based on the edge information extracted by the extraction unit 12b, and uses the parameters in which the orientations of the opposite edges have a one's complement relationship after conversion. Each pixel may be converted.

In such a case, the matching unit 12d may perform matching processing with a template of the data format indicating a water drop according to the data format converted by the conversion unit 12c.

Following step S104, the detection unit 12e determines whether or not there is an area corresponding to a cloud among the detected candidate areas Da based on the detection information 11b and the exclusion information 11c (step S105).

If there is an area corresponding to the cloud (step S105, Yes), the detection unit 12e excludes the area (step S106). When there is no area corresponding to the cloud (step S105, No), the control is moved to step S107.

Then, in step S107, the detection unit 12e notifies the various devices 50 of the candidate area Da and ends the process.

As described above, the adhering matter detection device 10 according to the embodiment includes the extraction unit 12b and the detection unit 12e. The extraction unit 12b extracts edge information of each pixel included in the captured image I captured by the camera 2 (corresponding to an example of “imaging device”). The detection unit 12e uses a predetermined detection algorithm based on the edge information extracted by the extraction unit 12b to determine a candidate area Da (corresponding to an example of a “candidate region”) in which a water droplet is estimated to be present in the captured image I. To detect. Further, the detection unit 12e excludes, from the detected candidate areas Da, the candidate areas Da that meet a predetermined exclusion condition based on the position, brightness, and size of the candidate area Da.

Therefore, according to the adhering matter detection device 10 according to the embodiment, it is possible to prevent erroneous detection of a cloud as a water drop, for example, and improve detection accuracy.

Further, when the position of the candidate area Da is higher than the predetermined position, the average brightness of the candidate area Da is higher than the predetermined threshold, and the size of the candidate area Da is smaller than the predetermined threshold, the detection unit 12e concerned. The candidate area Da is excluded.

Therefore, according to the adhering matter detection device 10 of the embodiment, it is possible to prevent the cloud from being erroneously detected as a water drop.

In addition, in the above-described embodiment, various thresholds in the exclusion conditions are described as examples of fixed values for the traveling state of the vehicle, but these are dynamically changed according to the traveling state of the vehicle. It may be one that does.

For example, when the vehicle is traveling on a slope or the like, the sky region in the captured image I is narrowed or widened. In such a case, an acceleration sensor, a gyro sensor, or the like among various sensors mounted on the vehicle is used. The threshold value regarding the y-coordinate of the candidate area Da in the exclusion condition may be changed based on the sensor value.

Further, in the above-described embodiment, the example in which the exclusion condition is used for the candidate area Da to be excluded has been described, but it may be used for the candidate area Da to be detected. That is, it may be used when it is desired to detect the candidate area Da corresponding to the cloud.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

2 camera 10 adhering matter detection device 11 storage unit 11a template information 11b detection information 11c exclusion information 12 control unit 12a acquisition unit 12b extraction unit 12c conversion unit 12d matching unit 12e detection unit 50 various devices Da candidate area I captured image

Claims (3)

An extraction unit that extracts edge information of each pixel included in a captured image captured by the imaging device,
Using a predetermined detection algorithm based on the edge information extracted by the extraction unit, a detection unit that detects a candidate region in which the water droplet is estimated to be present in the captured image,
The detection unit,
An adhering matter detection device, characterized in that, from among the detected candidate areas, the candidate areas that meet predetermined exclusion conditions based on the position, brightness, and size of the candidate area are excluded. The detection unit,
Excluding the candidate area when the position of the candidate area is higher than a predetermined position, the average brightness of the candidate area is larger than a predetermined threshold, and the size of the candidate area is smaller than a predetermined threshold. The adhering matter detection device according to claim 1, which is characterized in that. An extraction step of extracting edge information of each pixel included in the captured image captured by the imaging device;
Using a predetermined detection algorithm based on the edge information extracted in the extraction step, a detection step of detecting a candidate region in which the water droplet is estimated to be present in the captured image,
The detection step,
An adhering matter detection method, characterized in that, from among the detected candidate areas, the candidate areas that meet predetermined exclusion conditions based on the position, brightness, and size of the candidate area are excluded.