JP2002042131A - Marker detecting device, marker detecting method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To satisfactorily recognize a marker even when the marker to be detected is changed in scale or inclined. SOLUTION: A specific color discriminating section 11 discriminates an area having a specific color from an input color image. An area splitting section 12 splits the input color image into lattice-like areas by splitting the input color image in the vertical direction and the horizontal direction so as to surround the discriminated area. A marker area judging section 13 judges whether each split lattice-like area is a marker area forming the portion of the marker or not. A marker information detecting section 14 detects the significant information in the marker area judged by the marker area judging section 13 as marker information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sign detecting apparatus and a sign detecting method suitable for detecting, for example, a road sign and alerting a driver or automatically controlling a vehicle.

[0002]

2. Description of the Related Art A video camera mounted on a vehicle photographs a road in the traveling direction of the vehicle and its surroundings, recognizes road signs and the like from the photographed image, and alerts the driver or automatically controls the vehicle. It has been proposed to make it available (for example, Japanese Patent Publication No. 6-52).
554, JP-A-9-185703, JP-A-11-73598, etc.).

[0003] The road sign detection method proposed in the above publications and the like basically uses pattern matching. For example, in the invention described in Japanese Patent Application Laid-Open No. 9-185703, a recognition method based on pattern matching that focuses on a road sign having a specific color combination and the shape of the sign is used.

[0004]

However, in the case of the recognition method using pattern matching, it is necessary to prepare a template of the sign itself in advance, which is very troublesome. Moreover, every time a road sign is newly added, it is necessary to add the sign to the template each time, which is troublesome.

[0005] Further, measures against the problem of pattern matching caused by the moving vehicle are not taken into account in the conventional sign detection methods described in the above publications. That is, the scale of the road sign to be taken changes with the progress of the vehicle, or the sign is taken in at an angle, and the recognition rate deteriorates when only one template is prepared for one road sign. I do. In order to improve this problem and recognize well, it is necessary to prepare templates at every scale and every slope for each road sign, but it is virtually impossible to prepare such a large number of templates. It is almost impossible, and even if it can be prepared, it increases the size and cost of the recognition device and lowers the recognition speed.

In view of the above points, the present invention makes it possible to satisfactorily recognize a sign to be detected even if the scale of the sign to be detected changes or the sign is inclined, for example, as the vehicle moves. It is an object to provide a method and an apparatus.

[0007]

In order to solve the above-mentioned problems, a sign detecting apparatus according to the present invention comprises a specific color area extracting means for extracting an area having a specific color from an input color image, and the specific color area extracting means. By dividing the input color image in a vertical direction and a horizontal direction so as to surround the region cut out by the above, the input color image is divided by a region dividing unit into a lattice-like region, and divided by the region dividing unit. For each of the divided areas, a sign area determining means for judging whether or not the area is a sign area which is a sign, and a sign area determined by the sign area judging means signifies significant information therein. And a sign information detecting means for detecting as information.

Further, in the marker detection method according to the present invention, a specific color region extracting step of extracting an area having a specific color from an input color image, and the input area is surrounded by the specific color area extracting step so as to surround the input area. By dividing the color image in the vertical direction and the horizontal direction, an area dividing step of dividing the input color image into a grid-like area, and for each of the divided areas divided in the area dividing step, the area is marked with a sign. A marker area determining step of determining whether or not the area is a marker area, and a marker information detecting step of detecting significant information inside the marker area as the marker information in the marker area determined in the marker area determining step. It is characterized by having.

In the sign detecting apparatus and method according to the present invention having the above-described structure, first, an area having a specific color used as a sign is cut out from an input color image. In cutting out this specific color region, it is not necessary to exactly extract only the marker portion.

Next, the input color image is divided vertically and horizontally so as to surround the cut-out area, and the input color image is divided into grid-like areas. As described above, since the cutout of the specific color region includes not only the sign portion but also other portions, the divided lattice-shaped region includes a region including the sign portion and a region not including the sign portion. And

Then, for each of the grid-like divided regions, it is determined whether the region is a sign region that is a sign portion or another region. Then, as a result of this determination, in the marker area determined to be the marker part, information inside the marker area, which can be significant as a marker, is detected as marker information.

As described above, according to the present invention, a sign can be recognized irrespective of the scale or inclination of the sign taken.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a sign detecting device according to the present invention will be described below with reference to the drawings, in the case of a road sign detecting device mounted on a vehicle.

FIG. 2 shows an example of the overall system configuration of the sign detection apparatus according to this embodiment. This embodiment is an example in which the system is mounted on a vehicle.

In this embodiment, the video camera 1
It is mounted on the vehicle so as to photograph the road in front of the vehicle and its surroundings. The color image information obtained by shooting with the video camera 1 is converted into a digital signal by the A / D converter 2 and supplied to the arithmetic processing unit 3.

The arithmetic processing unit 3 is provided with a microcomputer.
PU (Central Processing Uni)
t) 31 and a ROM (Read Only Memory) in which information such as programs and characters is stored.
32 and a RAM for work area (Random Ac)
access memory 33, an image memory 34M for capturing image data, and an image memory 34B for calculation.
, Display controller 35, hard disk interface 36, CD-ROM (Compac
t Disc Read Only Memory) drive interface 38 is connected.

A hard disk drive 37 is connected to the hard disk interface 36 and a CD-ROM
A CD-ROM drive device 39 is connected to the drive interface 38. The display 4 is connected to the display controller 35.

A program for a road sign detection process to be described later can be stored in the hard disk device 37 in advance. A CD-ROM storing the program;
Is loaded into the CD-ROM drive device 38 and the CD
-The program can be read from the ROM and used, or the program can be read from the CD-ROM and written to the hard disk of the hard disk device 37 for use.

Further, although not shown, a communication interface connected to, for example, a mobile phone is connected to the system bus 30, the program is downloaded through the mobile phone, and the downloaded program data is written to a hard disk of a hard disk device. Can also be used.

The color image data converted into a digital signal by the A / D converter 2 is taken into the image memory 34M of the arithmetic processing unit 3. The capture of the color image data into the image memory 34M is controlled by the CPU 31. The image memory 34M captures color image data as still image data at time intervals in consideration of the time required for a road sign detection process described later. Then, the arithmetic processing section 3 performs a road sign detection process on the still image fetched into the image memory 34M, and displays the detection result on the display 4 to notify the driver.

As shown in FIG. 1, the arithmetic processing section 3 has a specific color area cutout section 11, an area division section 12,
It is composed of a sign area determination unit 13 and a sign information detection unit 14. The area dividing section 12 includes a horizontal / vertical frequency distribution calculating section 15, a change point detecting section 16, and a grid-shaped area dividing section 17. These units correspond to the respective steps in the case of the road sign detection method. Hereinafter, the configuration and processing operation of each unit will be described.

[Specific Color Area Extraction Unit 11] The data of the still image color image read from the image memory 34M is supplied to the specific color area extraction unit 11. The specific color area cutout unit 11 performs processing using the image memory 34B for calculation.

Road signs use only certain colors, such as blue, red, yellow, black, and white.
In view of the special color that these colors do not have a fine level change, the specific color area cutout unit 12 extracts these specific color areas from the color still image fetched into the image memory 34M, and extracts the extraction result. Write to the image memory 34B.

The extraction of the specific color area is performed, for example, when the color image signal is composed of three primary color signals of R (red), G (green), and B (blue). The thresholds of the respective values of R, G, and B are set in advance, and using these thresholds, whether each pixel unit or a plurality of pixel units of the color still image signal of the image memory 34 belongs to the specific color area is used. This is performed by performing a process of determining whether or not to perform the determination.

In this case, the extracted pixel data of the specific color area is extracted as it is, but the pixel data of the area other than the specific color area is extracted from the R, G, B pixels.
Are all set to 0.

As described above, since the colors used as road signs are limited, and the colors do not change in small levels, the number of thresholds prepared in advance may be limited. it can.

In order to accurately extract only the sign portion, the threshold of each of the R, G, and B values for detecting a specific color region is set according to a change in the surrounding brightness. However, this embodiment is a two-stage process including a process of determining whether or not each region is a marker portion and removing a region other than the marker as described later. The setting of the threshold value of each of the R, G, and B values is not strict, and there is no problem even if an area other than the sign is cut out in this specific color area cutout processing.
That is, it is not necessary to finely set a threshold value for cutting out a specific color region according to a change in brightness or the like.

For example, if the input color still image fetched into the image memory 34M is as shown in FIG. 3, the processing in this specific color area cutout unit 11 causes the image memory 34B for calculation to In FIG. 4, the marker region MS as shown in FIG. 4 is almost cut out and stored.

[Region Division Unit 12] The region division unit 12 divides an input color image into a plurality of grid-like regions using the data of the specific color region cut out by the specific color region cutout unit 11.

The data of the specific color area extracted by the specific color area extraction section 11 is supplied to the horizontal / vertical frequency distribution calculation section 15 of the area division section 12. In the horizontal / vertical frequency distribution calculation unit 15, in the region of the specific color cut out,
A histogram of the number of pixels having the specific color in the horizontal and vertical directions is obtained.

That is, as shown in FIG. 5, at each position X in the horizontal direction (x direction) of the area cut out by the specific color area cutout section 11, the image data of the area is scanned in the vertical direction, and The number of pixels of a specific color at the horizontal position X is counted, and the count value (frequency) Hh (x) is stored in, for example, the RAM 33 in correspondence with each position coordinate X in the horizontal direction.

Similarly, at each position Y in the vertical direction (y-direction) of the area cut out by the specific color area cut-out section 11, the image data of that area is scanned in the horizontal direction, and at each vertical position Y The number of pixels of a specific color is counted, and the count value Hh (y) is stored, for example, in the RAM 33 in correspondence with each position coordinate Y in the vertical direction. As described above, a histogram as represented by the curves 21 and 22 in FIG. 5 is obtained.

As described above, the data of the horizontal and vertical histograms calculated by the horizontal / vertical frequency distribution calculator 15 are supplied to the change point detector 16. In the change point detection unit 16, in the histograms in the horizontal direction and the vertical direction, a significant change point (horizontal direction) that can distinguish between a region including a marker (hereinafter, this region is referred to as a marker region) and the other region. (Direction position coordinates and vertical position coordinates).

In this embodiment, the change point detecting section 16 obtains a position coordinate which becomes a peak value of the histogram as a significant change point. The method of detecting a peak point in this embodiment is as follows.

First, when detecting the peak point in the horizontal direction, assuming that the frequency at a certain horizontal coordinate X is Hh (x), the peak point is determined by the threshold determination represented by the following equations (1) and (2). Is detected.

Hh (x)> Hth (1) {2 × Hh (x) −Hh (x−1) −Hh (x + 1)}> ΔHth (2) Here, Hth and ΔHth are set in advance. Hh (x−1) is a frequency at a horizontal coordinate (X−1) immediately before the horizontal coordinate X, and Hh (x + 1) is a horizontal threshold. This is the frequency at the horizontal coordinate (X + 1) one after the coordinate X.

Similarly, in detecting a peak point in the vertical direction, assuming that the frequency at a certain vertical coordinate Y is Hv (y), the peak value is determined by the threshold value determination represented by the following equations (3) and (4). Detect points.

Hv (y)> Vth (3) {2 × Hv (y) −Hv (y−1) −Hv (y + 1)}> ΔVth (4) Here, Vth and ΔVth are set in advance. Hv (y-1) is the frequency at the vertical coordinate (Y-1) immediately before the vertical coordinate Y, and Hv (y + 1) is the vertical threshold. This is the frequency at the vertical coordinate (Y + 1) immediately after the coordinate Y.

The threshold values Hth, ΔHth, Vth, ΔV
For each value of th, an appropriate value is determined in advance by a preliminary experiment or the like.

In the case of the example of FIG. 5, peak points Pxa and Pxb are detected from horizontal histograms at horizontal position coordinates Xa and Xb.
At the position coordinates Ya and Yb in the vertical direction, the peak points Pya and Pyb
Is detected. The position coordinate information of these detected peak points is supplied from the change point detecting section 16 to the grid-like area dividing section 17.

When there are a plurality of specific colors to be cut out as a marker area in the specific color area cutout section 11, the specific color area cutout section 11 adds information on the specific color of that area to each cutout area. In addition, the horizontal / vertical frequency distribution calculation unit 15 calculates the histogram for each specific color using the information of the added specific color, and uses the histogram to calculate the peak point from the histogram for each specific color. The change point is detected by the change point detection unit 16 and the detection result is supplied to the lattice-like area division unit 17.

The specific color area cutout unit 11, the horizontal / vertical frequency distribution calculation unit 15, and the change point detection unit 16
The processing may be performed for each specific color to be cut out as a marker area, and the change point detection result for each specific color may be supplied to the grid-like area dividing unit 17.

The grid area dividing section 17 uses the positional coordinate information of all the changing points supplied from the changing point detecting section 16 to convert the input color still image stored in the image memory 34M into a grid. Divide into regions. That is, in the case of FIG. 5 described above, the position coordinates Xa, Xb, Ya, and Yb of the detected peak point in the horizontal and vertical directions are supplied to the lattice-like region division unit 17, so that the lattice-like region division unit In FIG. 17, horizontal position coordinates Xa, Xb are set as shown in FIG.
The input color still image is horizontally divided into three by the dividing lines 23 and 24 passing through. The vertical position coordinates Ya,
The input color still image is vertically divided into three by the dividing lines 25 and 26 passing through Yb. As a result, the input color image is divided into nine grid regions as shown in FIG. When the division of the lattice area is performed in this manner, the storage contents of the image memory 34M for calculation are cleared (all are deleted).

[Signature Area Determining Unit 13] As described above, the image data for each lattice area divided by the lattice area dividing unit 17 is supplied to the sign area determination unit 13 as an output of the area dividing unit 12. Is done. In the sign area determination unit 13,
From the characteristics of the image for each lattice area from the area division unit 12, it is determined whether each area is a marker area.

As described above, a road sign has a frame of a specific color, and a specific sign such as a sign character or an arrow exists inside the frame of the specific color. ing. Therefore, the peak point of the histogram of the specific color pixel of the specific color area cut out by the specific color area cutout unit 11 corresponds to the frame portion, and the inside of the area surrounded by this peak point is the inside of the sign, It can be said that the characters and symbols of the sign exist in it.

However, as described above, when the threshold value is not strict in the extraction processing of the specific color area, the color area other than the sign is also extracted, thereby detecting the peak of the histogram of the area other than the sign. In some cases.
Further, some input color images may include a plurality of markers.

Therefore, in this embodiment, in each of the grid-like regions divided by the region dividing unit 12 as described above, whether or not the region is inside the sign is determined by the sign region judging unit 13. To do.

The marker area determination section 13 of this embodiment determines whether or not the area is a marker area by using the characteristics of the pixel value of the area inside the marker. That is, although characters and symbols exist inside the marker area, there is no small change in color pixels, and only two or three values exist as pixel values. For example, blue characters and symbols are described on a white background, and white characters and symbols are described on a blue background.

Therefore, the marker area determination unit 13 of this embodiment pays attention to, for example, the G (green) signal of each pixel and determines whether the pixel value in each lattice area is close to binary in this example. Thereby, it is determined whether or not the lattice area is inside the sign.

FIG. 8 is a block diagram showing the function of the marker area determining unit 13 of this embodiment, which corresponds to a case where the marker area determining unit 13 has a hardware configuration. As shown in FIG. 8, the marker area determination unit 13 includes an in-area maximum value detection unit 131, a difference calculation unit 132, a maximum value neighborhood pixel determination unit 133, and a maximum value neighborhood pixel counting unit 134.
, An area minimum value detection unit 135, and a difference calculation unit 136
, A minimum value neighboring pixel determination unit 137, a minimum value neighboring pixel counting unit 138, a total number of pixels in area calculation unit 139, and a marker area determination extraction unit 130.

The image data of each lattice area unit from the area division unit 12 first detects the level of the maximum G signal in the area in the area maximum value detection unit 131. Then, the difference calculation unit 132 calculates a difference between the G value of each pixel in the area and the detected maximum G signal. The difference value is supplied to the maximum value neighboring pixel determination unit 133. The maximum value neighboring pixel determination unit 133 compares the difference value with the threshold value, and when the difference value is smaller than the threshold value, determines that the pixel is a pixel having a value near the maximum G value, The count value of the pixel counting unit 134 is incremented. Therefore, the maximum value neighboring pixel counting unit 134 counts the number Nb of pixels near the maximum G value among all the pixels in each lattice area.

The image data of each lattice area unit from the area dividing unit 12 detects the level of the minimum G signal in the area in the area minimum value detecting unit 135. Then, the difference calculation unit 136 calculates a difference between the G value of each pixel in the area and the detected minimum G signal. The difference value is supplied to the minimum value neighboring pixel determination unit 133. The minimum value neighboring pixel determination unit 137 compares the difference value with the threshold value, and when the difference value is smaller than the threshold value, determines that the pixel is a pixel having a value near the minimum G value, and The count value of the pixel counting unit 138 is incremented. Therefore, the minimum value neighboring pixel counting unit 138 counts the number Nc of the pixels near the minimum G value among all the pixels in each lattice area.

In parallel with the above processing, the intra-region total pixel number calculation section 139 calculates the total pixel number Na included in the lattice area. The calculated total number of pixels Na is supplied to the marker area determination extraction unit 130. Also, the number Nb of pixels near the maximum G value counted by the maximum value neighboring pixel counting unit 134
And the number Nc of pixels near the minimum G value counted by the minimum value pixel counting unit 138 are also supplied to the marker area determination extraction unit 130.

In this example, the marker area determination / extraction unit 130 calculates (Nb + Nc) / Na, and the calculation result is
When the value is larger than a predetermined threshold value, that is, the pixel value in the area is close to binary, the grid-like area is determined as a marker area, and the data of the area is stored in, for example, an image memory 34B for calculation. This is set as a target of the next sign information detection processing in the sign information detecting unit 14.

On the other hand, when the calculation result is equal to or smaller than the threshold value, the lattice area is determined to be an area other than the marker area, and
The data in that area is discarded without being stored in the image memory 34B for calculation, and is excluded from the target of the next sign information detection processing by the sign information detecting unit 14.

As described above, the marker area determination unit 13 of this embodiment dynamically calculates the maximum value and minimum value of predetermined information such as the brightness of a pixel for each grid area, and Based on the calculated maximum value and minimum value, it is determined whether or not the pixel value in each lattice area is close to a binary value. Therefore, it is possible to determine the marker area without depending on the surrounding brightness such as weather. Can be.

FIG. 9 is a flowchart in the case where the marker area determination section 13 of this embodiment is executed by software (program) of the arithmetic processing section 3.

First, the data of the first grid-like area to be determined is fetched (step S1), and the number of pixels Na in the area is detected (step S2). Next, the maximum value of the G signal is detected from the G signals of all the pixels in the lattice area (step S3). Then, the difference between the G signal of each pixel in the lattice area and the maximum value of the detected G signal is obtained, and the number Nb of pixels whose difference is smaller than a predetermined threshold is calculated (step S4).

Next, the minimum value of the G signal is detected from the G signals of all the pixels in the lattice area (step S).
5). Then, the G signal of each pixel in the lattice area,
The difference from the minimum value of the detected G signal is obtained, and the number Nc of pixels whose difference is smaller than a predetermined threshold is calculated (step S6).

Next, using the pixel numbers Na, Nb, and Nc obtained in steps S2, S4, and S6,
(Nb + Nc) / Na is calculated, and it is determined whether or not the calculation result is larger than a predetermined threshold value Nth (step S7). As a result of the determination, the calculation result becomes the threshold N
When it is larger than th, the grid-like area is determined to be a marker area, the data of the area is held in the image memory 34B, and the next marker information detecting unit 14 performs the next marker information detection processing (step S12). S8). Also, the calculation result is
If the value is equal to or smaller than the threshold value Nth, the data of the lattice-shaped area is discarded as an area other than the marker area, and is excluded from the target of the marker information detection processing by the next marker information detector 14 (step S9).

Next, it is determined whether or not the determination has been completed for all the lattice regions (step S10). If not, the data of the next lattice region is fetched (step S11), and step S2 is performed. And the above processing is repeated. When the determination for all the grid-like regions is completed, the processing of the marker region determination is completed.

In the case where the input color image is as shown in FIG. 6, the sign area judging section 13 judges whether or not each of the nine grid areas is a sign area as described above. The marker area as shown in FIG. 10 is stored in the calculation image memory 34B as the determination result.

In the above example, the marker area determination unit 13
Has focused on the G signal to determine whether the pixel value in each lattice area is close to binary. However, other R (red) and B (blue) signals can be used. Since there is no detailed texture in the lattice area and the property that the pixel value is close to binary or ternary holds, the present invention is not particularly limited to the G signal. Also, instead of the G signal, R,
A luminance signal Y is generated from each of the G and B signals, and the luminance signal Y is subjected to the same processing as described above, whereby
It can also be determined whether each grid area is inside a sign.

[Sign Information Detection Unit 14] Sign Information Detection Unit 1
Reference numeral 4 is determined as a marker area by the marker area determination unit 13 and detects significant information as a marker such as a character or a symbol portion such as an arrow from the image data of the lattice area stored in the image memory 34B. Three examples of a method of detecting significant information in the sign information detecting unit 14 will be described.

[First Example of Sign Information Detection] In this first example, the coloration of information that is significant as a sign is, as described above,
Since there are a plurality of determined colors, the color information of each of the plurality of colors is determined as a threshold value. For this reason, in the first example, a threshold value for each color signal component of R, G, and B is determined in this example for each color arrangement, and the threshold value and each color signal of R, G, and B of each pixel are determined. Are compared, it is determined whether or not each pixel is a significant information portion. The processing for detecting the sign information in the first example will be described with reference to the flowchart of FIG. 11 and the subsequent flowchart of FIG.

That is, first, a threshold value is set for each of the R, G, and B color signals in the first color scheme to be detected among a plurality of color schemes (step S21). Then, the first pixel data in the marker area is read (step S22), and the R signal of the pixel data and
The threshold value for the R signal set in Step 1 is compared with the threshold value (Step S23), and it is determined whether or not the condition for significance of the R signal is satisfied (Step S24). If the significance condition for the R signal is not satisfied, for example, a flag indicating that pixel is set as a pixel other than the significant information (step S30).

When the significance condition for the R signal is satisfied, the G signal of the pixel data is compared with the threshold value for the G signal set in step S21 (step S25). It is determined whether or not the condition of significance for is satisfied (step S26). If the significance condition for the G signal is not satisfied, for example, a flag indicating that pixel is set as a pixel other than the significant information (step S30).

If the significance condition for the G signal is satisfied, the B signal of the pixel data is compared with the threshold value for the B signal set in step S21 (step S27). It is determined whether or not the condition of significance for is satisfied (step S28). If the significance condition for the B signal is not satisfied, for example, a flag indicating that pixel is set as a pixel other than the significant information (step S30).

When the significance condition for the B signal is satisfied, that is, when the significance condition is satisfied for all of R, G, and B, the pixel data is regarded as significant information and displayed on the display 4. The image data is stored in the image memory 34B for storing the display image data (step S29).

Next, it is determined whether or not the significance detection has been completed for all the pixels in the marker area (step S31). If not, the next pixel data in the marker area is read. (Step S32), the process returns to step S23, and the above-described processes after step S23 are repeated.

If it is determined in step S31 that the detection of significance for all pixels in the marker area has been completed, all the color schemes prepared for detecting significant information on the marker are determined. It is determined whether or not the detection has been completed (step S33). If not, a threshold value is set for each of the R, G, and B color signals in the next color scheme (step S34). And step S
Returning to 22, the detection of the significant information on the new color scheme is performed for the pixels in the area as described above.

At this time, in the detection of the significant information on the color scheme before that, when it is determined that the pixel is a pixel other than the significant information, and a flag to which the flag is added is newly regarded as significant information, In step S29, the flag is erased, and the pixel data that is regarded as significant information is retained.

As described above, when the detection of the significant information for all the color arrangements is completed, the process proceeds to step S33.
Then, the process proceeds to step S35, where it is determined whether the detection processing of the significant information has been completed for all the marker areas. When there is another sign area for which the detection processing of the significant information has not been completed, the processing is shifted to the sign area (step S3).
6), repeat the processing from step S21.

When it is determined in step S35 that the detection processing of the significant information has been completed for all the marker areas, the pixel data flagged as pixels other than the significant information is deleted (step S37). The routine of the significant information detection processing ends.

As a result of the above processing, the image memory 34
10B, only the sign portion as shown in FIG. 10 remains. By reading this and displaying it on the display 4, an image similar to that of FIG. Will be notified.

[Second Example of Sign Information Detection] In the second example, the number of pixels serving as significant information such as characters and symbols in a sign area is smaller than the number of other pixels. It is based on features. The processing for detecting the sign information in the case of the second example will be described with reference to the flowchart of FIG.

First, the first marker area in which significant information is to be detected is specified (step S41), and each pixel in the marker area is quantized to a binary or ternary value in the R, G, B color space. (Step S42). Next, the number of pixels for each quantization level is counted (step S43), and the quantization level Qmin having the smallest number of pixels is detected (step S44).

Then, for each pixel, it is determined whether or not the level after quantization corresponds to the detected quantization level Qmin (step S45), and the data of the corresponding pixel is stored in the image memory 34B as significant information. (Step S46), the data of the non-corresponding pixel is stored in the image memory 34.
Erase from B (step S47).

Next, it is determined whether or not the detection processing of the significant information has been completed for all the marker areas (step S4).
8) If there is another sign area for which the significant information detection processing has not been completed, the processing is shifted to that sign area (step S49), and the processing from step S42 is repeated.

Also in the case of the second example, significant information such as characters and symbols in the marker area can be satisfactorily detected in the same manner as in the first example.

[Third Example of Sign Information Detection] In this third example, significant information inside the sign is gathered near the center of the sign. It is a method that utilizes the fact that there are many. The processing for detecting the sign information in the third example will be described with reference to the flowchart of FIG.

First, the first marker area in which significant information is to be detected is specified (step S51), and the center coordinates in the area are calculated (step S52). Is set (step S53). Here, the small area is sufficiently smaller than the label area.

Next, binarization processing is performed on the pixel data inside the minute grid-like area (step S5).
4). Next, the number of pixels in the minute area is counted for each binarization level (step S55). Then, the level having the larger number of pixels is specified (step S56).

Then, it is determined whether or not the binarization level of each pixel in the minute area corresponds to the level having the larger number of pixels (step S57). The image data is retained in the memory 34B (step S58), and the data of the non-corresponding pixels is deleted from the image memory 34B (step S59).

Next, it is determined whether or not the process of detecting significant information has been completed for all the marker areas (step S6).
0) When there is another sign area for which the detection processing of the significant information has not been completed, the processing is shifted to the sign area (step S61), and the processing from step S52 is repeated.

Also in the case of the third example, significant information such as characters and symbols in the marker area can be detected satisfactorily in the same manner as in the first and second examples described above.

[Other Modifications] In the above-described embodiment, all of the plurality of color arrangements assumed as the road sign are processed by each unit, and the processing is shifted to the next stage. For each one of the colors, the processing from the extraction of the specific color area to the processing of detecting the sign information may be performed.

In the above-described embodiment, the change point detecting section 16 of the area dividing section 12 determines the peak points of the horizontal and vertical histograms calculated by the horizontal and vertical frequency distribution calculating section 15 as significant points. Although a change point is detected, a significant change point is not limited to a peak point. For example, in the histogram, a point at which the frequency gradually increases, a point at which the frequency changes to a flat change, or a point at which the frequency changes so as to gradually process from the flat change may be detected as a significant change point. Good.

Although the above embodiment has been described with reference to the case where the detection target is a road sign, it goes without saying that the detection target sign of the present invention is not limited to a road sign.

[0090]

As described above, according to the present invention, not only a simple pattern matching as in the conventional art,
Since the label is detected based on the characteristics of the label, the detection can be performed well without depending on the inclination of the label or the scale of the label.

Further, after extracting a specific color area, it is a two-step process in which the extraction result is used to divide the area into a lattice area and to determine a marker portion for each lattice area. Therefore, even if there is a change in brightness around the sign, the sign can be reliably detected.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a main part of an embodiment of a sign detection device according to the present invention.

FIG. 2 is an overall system configuration diagram of an embodiment of a marker detection device according to the present invention.

FIG. 3 is a diagram provided for describing a label detection processing operation according to the present invention;

FIG. 4 is a diagram provided for describing a marker detection processing operation according to the present invention;

FIG. 5 is a diagram provided for explanation of a marker detection processing operation according to the present invention.

FIG. 6 is a diagram provided for explanation of a marker detection processing operation according to the present invention.

FIG. 7 is a diagram provided for explanation of a marker detection processing operation according to the present invention.

FIG. 8 is a functional block diagram of an example of a marker area determination unit in the embodiment of the marker detection device according to the present invention.

FIG. 9 is a flowchart for explaining an example of a marker area determination unit in the embodiment of the marker detection device according to the present invention.

FIG. 10 is a diagram provided for explanation of a marker detection processing operation according to the present invention.

FIG. 11 is a part of a flowchart illustrating an example of a sign information detecting unit in the sign detecting apparatus according to the embodiment of the present invention;

FIG. 12 is a part of a flowchart for explaining an example of a sign information detecting unit in the sign detecting apparatus according to the embodiment of the present invention;

FIG. 13 is a flowchart for explaining another example of the sign information detecting unit in the embodiment of the sign detecting apparatus according to the present invention.

FIG. 14 is a flowchart for explaining still another example of the sign information detecting unit in the embodiment of the sign detecting apparatus according to the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Video camera, 2 ... A / D converter, 3 ... Operation processing part, 4 ... Display, 11 ... Specific color area cut-out part,
12 area dividing section, 13 sign area determining section, 14 sign information detecting section, 15 horizontal / vertical frequency distribution calculating section, 16 ...
Change point detecting section, 17: grid area dividing section, 130: marker area determining / extracting section, 131: area maximum value detecting section, 133
... A maximum value neighboring pixel determining unit, 134 a maximum value neighboring pixel counting unit, 135 a region minimum value detecting unit, 137 a minimum value neighboring pixel determining unit, 138 a minimum value neighboring pixel counting unit, 139.
Total pixel count calculation area

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5H180 AA01 BB13 CC04 CC05 FF14 FF32 5L096 AA02 AA06 BA18 DA02 EA35 EA43 FA04 FA36 GA28 GA38 GA51

Claims (27)

[Claims] 1. A specific color area cutout means for cutting out an area having a specific color from an input color image, and the input color image is vertically and horizontally surrounded by the area cut out by the specific color area cutout means. Area dividing means for dividing the input color image into grid-like areas by dividing the input color image into grid areas, and for each of the divided areas divided by the area dividing means, whether the area is a sign area which is a sign part Sign region determining means for determining whether or not, in the sign region determined by the sign region determining means,
And a sign information detecting means for detecting significant information inside the sign information as sign information. 2. The marker detection device according to claim 1, wherein the area dividing means divides the pixels of the specific color in a horizontal direction and a vertical direction in the area cut by the specific color area cutting means. Counting, frequency distribution calculating means for obtaining a frequency distribution in each direction, and a change point detecting means for detecting a significant change point in the horizontal and vertical frequency distributions calculated by the frequency distribution calculating means, Dividing means for dividing the input color image into a grid-like area by dividing the input color image in a vertical direction and a horizontal direction at the significant change point position detected by the change point detecting means. A marker detection device characterized by the above-mentioned. 3. The marker detection device according to claim 1, wherein the specific color region cutout unit extracts a specific color region by performing processing using a predetermined threshold value for a color included in the marker. A sign detection device wherein all pixel values are set to zero except for a specific color area. 4. The sign detecting device according to claim 2, wherein the change point detecting means detects peak points of the frequency distribution in the horizontal direction and the vertical direction as the significant change points. Sign detector. 5. The marker detecting device according to claim 1, wherein the marker region determining means determines whether the value of a pixel included in the region to be determined is close to binary or ternary. A marker detection device that determines a region. 6. The marker detection device according to claim 1, wherein the marker region determination unit includes a detection unit configured to detect all pixel numbers Na included in the region to be determined, and a marker region included in the region to be determined. A maximum value detection unit that detects a maximum value of predetermined information of a pixel; a minimum value detection unit that detects a minimum value of the predetermined information of a pixel included in the region to be determined; Means for counting the number of pixels near the maximum value, which detects the number of pixels Nb whose difference from the maximum value is equal to or less than a predetermined value; Determining a region to be determined as a region inside the marker or a region outside the marker based on a result of comparing the value of (Nb + Nc) / Na with a predetermined threshold value; Judgment means A sign detection device, comprising: 7. The sign detection device according to claim 1, wherein the sign information detecting means determines a color of each pixel in the sign area determined by the sign area determining means and a color used for the sign. A marker detection device that compares and detects the pixel as a pixel of a significant information portion when the difference between the two is smaller than a predetermined value. 8. The sign detection device according to claim 1, wherein the sign information detecting means quantizes a pixel in the sign area determined by the sign area determining means, and Means for calculating the number of pixels for each quantization level for the pixels in the marker area; means for calculating the quantization level of the smallest number of pixels among the quantization levels; and Means for judging whether the quantization level corresponds to the minimum number of pixels and, if so, detecting the pixel as a pixel of a significant information portion. 9. The sign detection device according to claim 1, wherein the sign information detecting means obtains a most frequent pixel level for a pixel included in a minute area near the center of the sign area; Means for determining whether each pixel in the marker area is at a level close to the obtained pixel level, and if so, means for detecting the pixel as a pixel of a significant information portion. Sign detector. 10. A specific color area extracting step of extracting an area having a specific color from an input color image, and the input color image is vertically and horizontally surrounded by surrounding the area extracted in the specific color area extracting step. Dividing the input color image into a grid-like area by dividing the input color image into a grid-like area; and for each of the divided areas divided in the area dividing step, whether the area is a sign area that is a sign part In a sign region determining step of determining whether or not, in the sign region determined in the sign region determining step,
A sign information detecting step of detecting significant information inside the sign information as sign information. 11. The marker detection method according to claim 10, wherein, in the area segmentation step, the pixels of the specific color are horizontally and vertically separated in the area cut out in the specific color area cutout step. Counting, a frequency distribution calculating step of obtaining a frequency distribution in each direction, a change point detecting step of detecting a significant change point of the horizontal and vertical frequency distributions calculated in the frequency distribution calculating step, Dividing the input color image into a grid-like area by dividing the input color image in the vertical and horizontal directions at the significant change point position detected in the change point detection step. A method for detecting a label, characterized in that: 12. The method for detecting a sign according to claim 10, wherein, in the step of cutting out the specific color area, a specific color area is extracted by performing processing using a predetermined threshold value for a color included in the mark. A marker detection method characterized in that all pixel values are set to zero except for a specific color region. 13. The method for detecting a sign according to claim 12, wherein in the change point detecting step, peak points of the frequency distribution in the horizontal direction and the vertical direction are detected as the significant change points. Label detection method. 14. The marker detection method according to claim 10, wherein, in the marker area determination step, the marker included in the area to be determined is determined based on whether a value that can be taken by a pixel is close to binary or ternary. A marker detection method characterized by determining a region. 15. The marker detection method according to claim 10, wherein the marker area determination step includes a detection step of detecting all pixel numbers Na included in the area to be determined, and a label area determination step. A maximum value detection step of detecting a maximum value of predetermined information of a pixel, a minimum value detection step of detecting a minimum value of the predetermined information of a pixel included in the area to be determined, and, in the area to be determined, A maximum value neighboring pixel counting step for detecting the number of pixels Nb whose difference from the maximum value is equal to or less than a predetermined value; and a number of pixels whose difference from the minimum value is equal to or less than a predetermined value in the area to be determined. A step of counting the number of pixels in the vicinity of the minimum value for detecting Nc; and a result of comparison between the value of (Nb + Nc) / Na and a predetermined threshold value. Judgment A method for detecting a label, comprising the steps of: 16. The marker detection method according to claim 10, wherein in the marker information detecting step, the color of each pixel in the marker area determined in the marker area determining step and the color used for the marker are determined. A sign detection method comprising comparing and detecting the pixel as a pixel of a significant information portion when the difference between the two is smaller than a predetermined value. 17. The marker detection method according to claim 10, wherein the marker information detecting step includes a step of quantizing pixels in the marker area determined in the marker area determining step; For the pixels in the marker region, a step of calculating the number of pixels for each quantization level; a step of calculating a quantization level of the smallest number of pixels among the quantization levels; Determining whether the quantization level corresponds to the minimum number of pixels, and if so, detecting the pixel as a pixel of a significant information portion. 18. The marker detection method according to claim 10, wherein the marker information detecting step includes a step of obtaining a most frequent pixel level for a pixel included in a minute area near the center of the marker area. Determining whether each pixel in the marker area is at a level close to the determined pixel level, and if so, detecting the pixel as a pixel of a significant information portion. Label detection method. 19. A specific color area extracting step of extracting an area having a specific color from an input color image, and the input color image is vertically and horizontally surrounded by surrounding the area extracted by the specific color area extracting step. By dividing the input color image into grid-like areas, and for each of the divided areas divided by the area dividing procedure, whether the area is a sign area that is a sign part In the sign area determination procedure to determine whether or not, in the sign area determined in the sign area determination procedure,
A label information detection procedure for detecting significant information inside the label information as a label information; 20. The recording medium according to claim 19, wherein in the area dividing step, the pixels of the specific color are counted in a horizontal direction and a vertical direction in the area cut out by the specific color area cutting out step. A frequency distribution calculation procedure for obtaining a frequency distribution in each direction; a change point detection procedure for detecting a significant change point in the horizontal and vertical frequency distributions calculated in the frequency distribution calculation procedure; Dividing the input color image into a grid-like area by dividing the input color image in the vertical and horizontal directions at the significant change point position detected in the point detection procedure. Recording medium characterized by the above-mentioned. 21. The recording medium according to claim 19, wherein, in the specific color area cutout procedure, a specific color area is extracted by performing processing using a predetermined threshold value for a color included in the marker. A recording medium characterized in that all pixel values are set to zero except for the color region of (1). 22. The recording medium according to claim 19, wherein the change point detecting step detects the peak points of the frequency distribution in the horizontal direction and the vertical direction as the significant change points. Medium. 23. The recording medium according to claim 19, wherein the step of determining the marker area includes determining whether a value of a pixel included in the area to be determined is close to binary or ternary. The recording medium characterized by comprising: judging. 24. The recording medium according to claim 19, wherein the marker area determining step includes: detecting a number Na of all pixels included in the area to be determined; A maximum value detection procedure for detecting a maximum value of the predetermined information, a minimum value detection procedure for detecting a minimum value of the predetermined information of a pixel included in the area to be determined, and A procedure for counting the number of pixels near the maximum value, which detects the number of pixels Nb whose difference from the maximum value is equal to or less than a predetermined value; The area to be determined is determined to be an area inside the sign or an area outside the sign based on a procedure for counting the number of pixels in the vicinity of the minimum value for detecting the value of (Nb + Nc) / Na and a predetermined threshold value. Judgment procedure And a recording medium comprising: 25. The recording medium according to claim 19, wherein the marker information detecting step compares the color of each pixel in the marker area determined in the marker area determining step with the color used for the marker. When the difference between the two is smaller than a predetermined value, the pixel is detected as a pixel of a significant information portion. 26. The recording medium according to claim 19, wherein the marker information detecting step includes a step of quantizing pixels in the marker area determined in the marker area determining step, and the quantized marker. For the pixels in the region, a procedure for calculating the number of pixels for each quantization level, a procedure for calculating the quantization level of the smallest number of pixels among the quantization levels, and Determining whether the quantization level corresponds to the quantization level of the smallest number of pixels, and if so, detecting the pixel as a pixel of a significant information portion. 27. The recording medium according to claim 19, wherein the marker information detecting step includes a step of obtaining a most frequent pixel level for a pixel included in a minute area near a center in the marker area. Determining whether each pixel in the marker area is at a level close to the determined pixel level, and if so, detecting the pixel as a pixel of a significant information portion. Medium.