JP2003016386A - Method and device for character recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and a device for character recognition, which can widely be applied to character recognition on the site, e.g. outdoors or in a factory. SOLUTION: A plurality of character patterns are obtained by binarizing a photographed image of an object character to be recognized by using a plurality of thresholds and those character patterns are projected and added in X and Y directions to cut a character area out. The obtained characters are inputted to a neural network and the character having the maximum output value exceeding a set reference value is outputted as a final recognition result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a character recognition method and apparatus, and more particularly, to characters on a container, marking characters on parts, printed characters with a high degree of dirt and damage, and one having a plurality of fonts for each character. The present invention relates to a character recognition method and apparatus suitable for automatically recognizing characters.

[0002]

2. Description of the Related Art Generally, a container to be loaded and unloaded at a container yard is provided with an identification code from the viewpoint of inventory management and inventory management. Also, in various manufacturing processes with a large number of parts, an identification code may be added to represent the kind of parts. It is very preferable for automation of work if the characters such as codes can be automatically recognized and discriminated.

A conventional character recognition device generally illuminates a printing surface with an illuminating device, and a CCD camera.
The characters printed on the surface are imaged to obtain a grayscale image. Then, a process of increasing the contrast difference between the character and the background is performed, and thereafter, the grayscale image is converted into a binary image by using a threshold value for separating the character and the background. Next, after removing noise included in the binary image by a filter,
By comparing the feature amount of the character extracted from the value image with the corresponding reference feature amount, the character printed on the surface is identified.

However, when the printed characters are dirty or defective, normal character recognition cannot be performed. As a character recognition device for this kind of problem, a system for recognizing an identification code written on a container has been proposed (JP-A-9-50484). This is because when an identification code is taken by an image pickup device and normal image recognition processing is performed for recognition, if there is a character that cannot be recognized and identified due to dirt or the like, the relationship between the unrecognizable character and other characters. The identification code including the estimated value is displayed at the same time by performing the estimation process based on the identification code creation rule, so that even if one of the characters is not recognized, it can be dealt with.

[0005]

However, the above method does not improve the degree of recognition, but attempts to perform an estimation process by using a rule for creating a written character string when a character cannot be recognized. It cannot be applied to the recognition of characters without. Further, for example, in the case of container characters, the identification code has different fonts and character intervals depending on the shipping company, and in many cases, the identification code becomes dirty during transportation. Further, since there is no standard for different container character fonts to be unified by shipping companies, the recognition rate of characters cannot be improved even if the above conventional method is applied.

Further, in the case of characters printed on a member, there are stains, bleeding, blurring, etc., and there is a distribution of brightness in the image. Further, depending on the operating condition of the printing device, the thickness and interval of the dots forming the characters may change, and the printing may differ from the normal printing shown in FIG. 3. In this case, the recognition becomes impossible.

Further, even if the neural network is simply applied to character recognition, if the character has damage such as stains, bleeding, and blurring, the character recognition rate is as low as 90% or less, and the commercially available O
Even if CR software is applied, since it is adjusted for printing characters, if there is dirt, blurring, or blurring of characters such as container characters or component characters, the character recognition rate will also be low.

That is, although the conventional apparatus targets alphanumeric characters for recognition, the characters are often accompanied by defects or deformation such as dirt, faintness, blurring, and blurring, and the conditions for recognition are set for each application target. There is a need to. An object of the present invention is to provide a character recognition method and apparatus that can be generally applied to character recognition in the field such as outdoors or in factories, focusing on the above conventional problems. Another object of the present invention is to provide a character recognizing method and device having a performance equivalent to that of visual judgment.

[0009]

In order to achieve the above object, a character recognition method according to the present invention obtains a plurality of character patterns obtained by binarizing a photographed image of a character to be recognized with a plurality of thresholds. A character area is cut out by projectively adding a plurality of character patterns in the X direction and the Y direction, the obtained characters are input to the neural network, and the output value exceeding the set reference value takes the maximum value. It is characterized by outputting as. If the character area cut out by the projective addition is adjusted to a certain aspect ratio, once enlarged and then reduced and converted, the cutout character size is unified to the learning character size and input to the neural network. Good. As the threshold value, a reference threshold value determined by an arbitrary method and a plurality of threshold values obtained by adding or subtracting a constant value before and after based on the reference threshold value are used. After such a reference threshold value is determined by the discriminant analysis method, the reference threshold value is converted into a percentage display value of the P tile method, and a reference value obtained by adding and subtracting a fixed upper and lower value to and from the reference P tile percentage display value. It suffices to use the calculated front and rear threshold values.

Further, the character recognition apparatus according to the present invention includes preprocessing means for obtaining a plurality of character patterns created with a plurality of thresholds when binarizing a photographed image of a character to be recognized, and a plurality of obtained characters. A neural network having an input layer of patterns and an output layer of at least recognized character types; and a discriminating means for outputting, as a final recognition result, a neural network having a maximum certainty factor among a plurality of determination results by the neural network. It has a feature.

Further, particularly in the case of a framed character such as a check digit of a container character, after obtaining a plurality of character patterns created by a plurality of thresholds when binarizing a photographed image of a character to be recognized, Pre-processing means for cutting out the periphery of the character pattern to obtain a frame-exposed character pattern, a neural network having an input layer of the obtained plurality of character patterns and an output layer of at least the recognized character type, and a plurality of determinations by the neural network Out of the results
It may be configured to have a discriminating means for outputting the one with the maximum certainty value as the final recognition result.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Specific embodiments of a character recognition method and apparatus according to the present invention will be described below with reference to the drawings.
The details will be described. In the present embodiment, a character recognition method and apparatus applying image processing and neural network technology will be described, targeting 36 alphanumeric characters collected outdoors or at a site such as a factory.

FIG. 1 shows a processing flowchart of the character recognition method according to the embodiment. Basically, character recognition is performed by the following procedure. (1) A plurality of character patterns are created by binarizing the input character image with a plurality of threshold values based on the threshold value by the discriminant analysis method. (2) The plurality of created patterns are input to a three-layer hierarchical neural network, and the recognition result with the highest certainty factor among the recognition results of the neural network for each pattern is output as the final recognition result.

In order to recognize a character included in an input image, it is necessary to first extract a character area in the input image and finally cut out the character in character units. As the basic processing, as shown in the flow chart of character cutting shown in FIG. 2, an image of a character string to be recognized is captured (step 200), binarized (step 202), A character string region is cut out by removing noise (step 204) and projective addition (step 206) (step 208). Then, after the character string inclination angle is calculated and corrected (step 210), the projective addition is performed (step 212) to determine the character area (step 2).
14) The character image is cut out (step 21).
6).

The character cutting process will be described by taking a marking character image of a shipbuilding component as an example. The target is the marking character, but the character can be cut out in the same way for the container number. First, the input image is binarized. As a result, a bright portion including the character area is extracted. Next, noise processing for removing spike-like noise is performed on the binarized image, and then projection addition in the X and Y directions is performed. By this processing, the projection addition value of the portion where the character string exists increases. Therefore, a horizontally long character string area surrounded by a white line is cut out from a portion where the projection addition value in the X direction is high, the inclination angle is corrected for each character string area, and then the projection addition is performed again. Determine the character area for each character. Finally, a region corresponding to the character region determined from the input image can be cut out to obtain a character image.

Here, in the present embodiment, the individual characters cut out as described above are to be recognized, and a character pattern for recognition processing is created from the cut out character image. Since the cut-out character image may include the background area, binarization is performed, the character is cut out with the circumscribing frame, and then a character pattern having a size of vertical M pixels × horizontal N pixels is created. . Since the sizes of the characters cut out by the circumscribing frame are not constant, they are arranged to have the size of M × N by using affine transformation.

By the way, in the present embodiment, particularly, the threshold value for binarization is determined by the discriminant analysis method (see "Image Processing Engineering" Taniguchi et al., Kyoritsu Shuppan) as a reference value, and the reference threshold value is sandwiched. A plurality of thresholds are created before and after, and the thresholds are binarized. As a result, a plurality of character patterns are created. The reason for creating a plurality of threshold values is that the threshold values determined by the discriminant analysis method may not be binarized well when the density distribution of the characters and the background are close to each other or the characters are small. This is because there are cases in which it is erroneously recognized. As a plurality of threshold values, the ratio of pixels having a pixel value larger than the threshold value calculated by the discriminant analysis method is obtained (this is defined as X%), and the p tile method (X-10)%, X% (by the discriminant analysis method). (Corresponding to the calculated threshold value), and three threshold values determined by (X + 10)% are used.

The processing flow for creating such a character pattern will be described in detail with reference to FIG. The cut-out character image data is input to the array (step 100), and the threshold value (T _hr DA) for binarization by the discriminant analysis method is determined (step 102). Then, the% of the P-tile method that gives this threshold value T _hr DA: P _ct DA is calculated (step 10).
4). Next, the threshold value T _hr when the P tile method% is X is calculated (step 106). A character image cut out in advance as a recognition target is binarized with this threshold value T _hr (step 108). Then, the obtained binary image is X
By adding projections in the direction and Y direction (step 11
0), the net character area should be cut out (step 11).
2).

By the way, when a character has a large aspect ratio such as "1" or "I", most of the character patterns cut out by projective addition in the X and Y directions are character regions, and distinction can be made. Disappear. Therefore, it is determined whether or not the aspect ratio of the character size is equal to or larger than the set value (3.3 or larger in the embodiment) (step 114), and if larger than the set value, the learning pattern size in the neural network is matched. The margins are added to the left and right of the display (step 116). In the embodiment, since the size is adjusted to 15 × 30, the margin is added so that the aspect ratio becomes 2.

Then, after the cut-out character size is Affine converted to a size of 45 × 90 which is three times as large as the learning pattern (step 118), it is reduced to 1/3 (step 120). This is because the character pattern becomes clear by expanding the character pattern once and then reducing it to a predetermined comparison size.

After the size adjustment as described above, the judgment is made by the neural network (step 12).
2) In the present embodiment, a plurality of thresholds for binarizing the character image in step 108 described above are prepared, a plurality of character patterns are acquired for one recognition target character, and then the neural network is used. The determination process is performed according to. That is, the character image cut out beforehand as a recognition target by the reference threshold T _hr was binarized first, but this time, a plurality of thresholds corresponding to the reference threshold T _hr corresponding to a value of ± 10% of the P-tile method is used. X = (P _ct DA-1
0)%, and the (updated to P _ct DA + 10)%, as determined threshold T _hr1, T _hr2 corresponding to the update X, binarizes these thresholds (step 124).

As a result, a plurality of character patterns binarized with a plurality of threshold values are obtained for one character to be recognized, and these are input to the neural network for judgment processing (step 126). The recognition process by the neural network is as follows. FIG. 3 shows a three-layer hierarchical neural network 10 used for character recognition.
Shows the configuration of. Each unit of the input layer 12 and the intermediate layer 14,
The respective units of the intermediate layer 14 and the output layer 16 are coupled by the coupling coefficient and the bias value obtained by learning. The unit of the input layer 12 outputs the input value as it is. Each unit of the intermediate layer 14 and the output layer 16 is an upstream layer (if each unit of the intermediate layer 14 is the input layer 1
2. If it is each unit of the output layer 16, the output of each unit of the intermediate layer 14) is multiplied by the coupling coefficient and added, and the result input to the nonlinear sigmoid function f (z) shown in Expression (1) is output. To do.

[Equation 1] Here, n: the number of units in the upstream layer, w _j : coupling coefficient,
x _j : input from a unit in the upper layer, w ₀ : bias value.

In order to perform character recognition by the neural network 10, alphanumeric characters to be recognized are assigned to each unit of the output layer 16, and only the unit of the output layer 16 corresponding to the character pattern input to the input layer is assigned. The coupling coefficient and the bias value are adjusted so that the value close to 1 is output and the output of the other units is 0. Although this process is called learning, the error backpropagation method is used as a learning algorithm in this embodiment.

When a character pattern is input to the constructed neural network 10, each unit of the output layer 16 has an output between 0 and 1, and the character assigned to the unit that outputs the value closest to 1 is the recognition result. . Further, the output value at this time is referred to as the certainty factor, and when 1 is set to 100%.

In this embodiment, a plurality of character patterns are input to the constructed character recognition neural network 10 and the best value of the recognition result is used as the final recognition result. As an example, when it was applied to the part number recognition for the distribution / temporary assembly process automation of the shipbuilding small assembly line for the purpose of automating the production process, and the container number recognition of the container yard gate system for the purpose of automating the logistics process, respectively. A high recognition rate was obtained and it was confirmed that the performance was equivalent to the visual judgment.

<Embodiment 1> An example in which the above-described character recognition device is applied to automation of shipbuilding small set process will be described. First,
The concept of the automatic distribution temporary assembly system is as follows.
At the shipbuilding site, the introduction of three-dimensional CAD,
The automation of aggregate cutting and welding processes and welding processes is progressing.
However, with regard to material distribution and temporary assembly (welding plate attachment) between both processes, most of them still rely on manpower, and automation is desired.

The small assembly process is composed of processes such as temporary distribution of materials, welding, and finish strain relief. Of these, in the provisional roughing process, the worker selects parts based on the numbers printed on the parts from the subassembly parts stock, arranges them along the attachment lines drawn on the base panel, and performs temporary welding. Is done in a way that

Therefore, in order to automate this process, the introduction of an automatic material distribution temporary assembly system as shown in FIG. 4 for automating the material distribution and temporary assembly of small assembly parts was examined. This system consists of parts 20.
An image recognition part for recognizing the above marking characters and the position of the mounting line, a handling part 22 for handling the small set parts, and a plate welding machine 2 for temporarily welding the small set parts.
4, and the CAD information is collated with the information obtained by the visual sensor 26, the base panel 28 as shown in FIG.
The preliminary assembly work is performed by disposing the small set component 20 on the upper attachment line. The technology required to realize this system is a recognition technology for the marking characters 30 indicating the part number printed on the subassembly 20 and a mounting line recognition technology on the base panel. In this embodiment, the result of applying the character recognition technique to the marking characters 30 on the component among these will be described.

The characteristics of the marking characters in the shipbuilding subassembly process are as follows. The marking character 30 printed on the member is the ship number, the block name to attach the part,
Information such as the assembly number and material of the parts is symbolized by alphanumeric characters, and is printed on the center of the member with white paint.
There are some characters that are shorter than four characters and some characters that exceed ten characters, and the length of the character string also varies depending on the description content.

A standard pattern of marking characters 30 printed on the component 20 is a 7 × 9 dot pattern.
However, in reality, there are various low-quality characters such as dot column shift, dot expansion, dot omission in a specific column, and member stain. Therefore, the marking character 30
Let's learn. In order to construct the marking character recognition neural network 10, characters were cut out from an image taken in an actual factory to create a character pattern. The size of the character pattern is 18 ×, which is twice the size of the marking character 30 because the size of the marking character 30 is 9 × 7.
A size of 14 and a character pattern of 5 data per alphanumeric character was created as learning data.

These learning data were learned by a three-layer hierarchical neural network consisting of a human power layer 252 units, an intermediate layer 40 units, and an output layer 36 units. As a result of recognizing the marking characters 30 after the pattern learning, the best value of the recognition results of the plural character patterns for each character image was taken as the final recognition result, and for 1218 characters cut out from 80 images, 1205 characters were correctly recognized, and a recognition rate of 99% was obtained for each character. When this was set as the correct answer only when all the characters included in the number were recognized, the recognition rate per part number was 77 out of 80, which was 96%.

If the answer is not correct, the character is classified as erroneous recognition for recognizing the character as another character and unrecognizable with low certainty of the recognition result, but any erroneous character corresponds to unrecognizable character. Finally, by collating the recognition result with the CAD data, the recognition result is verified, the unrecognizable character is estimated, and the part number is specified, whereby the recognition accuracy can be improved.

<Embodiment 2> An embodiment in which the above-mentioned character recognition device is applied to a container yard gate cisnum is as follows. First, the container yard gate system is a system that recognizes the number of a container loaded on a trailer and enters a port facility, confirms the container number, and instructs the storage location. At present, the workers are visually checking, but in order to automate and improve the efficiency of the entire container yard,
It is necessary to automatically recognize the container number in the image captured by the camera.

The characteristics of the container character are as follows.
The container number is drawn on the upper right side of the container door surface, and consists of a total of 11 letters, which are the four alphabets indicating the owner code, a serial number consisting of six numbers, and one check digit. The check digit indicates a numerical value obtained as a result of performing a predetermined operation on the sequence number. From this, alphanumeric characters 36
And 10 check digits must be recognized.

The container image is taken at the timing when the trailer entering the gate is judged by the beam switch. For this reason, the brightness and size of the character differ depending on the shooting situation, and the character has dirt, damage, blurring, and the like. Moreover, since there is no standard for characters, the thickness of the characters and the font are not constant.

In order to recognize the character having such a characteristic by the recognition device according to the embodiment, the container character is first learned. In order to construct a neural network for container character recognition, a character pattern created from a container image was created in the neural network. Since the actual container aspect ratio is 2: 1, the size of the character pattern is 30x15, and 4 to 6 for each alphanumeric character.
Individual character patterns were created as learning patterns. However, with respect to the check digit, there is no difference from the numbers in the serial number except for the frame, so the method of removing the frame was adopted.

The constructed neural network was a three-layer hierarchical type, with 450 units for the input layer, 80 units for the intermediate layer, and 36 units for the output layer. After such learning, the container character was recognized. For alphanumeric characters other than check characters, a binarization threshold value was changed to create a multi-character pattern, and the best value of the recognition results by the neural network was used as the final recognition result. As a result, 5
567 data out of 79 data were correctly recognized, and the recognition rate was 98%.

In the case of a check digit, if the density of the frame is small, the frame part disappears in the binarization process.
As with other characters, the recognition method using multiple character patterns may give the correct answer. However, if the frame is dark, it is misrecognized. Therefore, when the certainty factor of the recognition result is lower than 80%, after removing the assumed frame width from the periphery of the character, the character pattern is created and is recognized by the neural network. FIG. 6 shows a flow of the recognition method. First, input image data (step 300), p
A small threshold value determined by tile 80% is calculated (step 30
2) A binarized character image is obtained by binarizing with this threshold value (step 304). The projection addition and the character region are not cut out (steps 306 and 308), then the region inside the set frame width is cut out from the character image (step 310), and character recognition is performed using a plurality of character patterns (step 312). . When the frame width was set within the range of 1 pixel to 4 pixels (step 314), it was possible to obtain numbers without a frame for all check digit character images. FIG. 7 shows a check digit recognition result in which the check digit "4" is taken as an example.

FIG. 7A shows a character image, and FIG. 7B shows a framed binary image. Further, FIG. 7C shows 12 character patterns created by changing the binarization threshold and the frame width to be removed and the recognition result thereof. The upper numerical values shown below each pattern represent the% value and the frame width of the p tile for which the threshold is determined, and the lower numerical values represent the recognition result and the certainty factor.

For example, the pattern on the upper left is 55% for the P tile.
It is shown that the recognition result is "T" in the pattern created under the condition that the threshold value at that time is 1 and the frame width to be removed is 1 pixel, and the confidence factor at that time is 33%. In this case, it is correctly accepted as “4” as the 12 best values. When 101 check digits were recognized, 98 were correctly recognized, and a recognition rate of 97% was obtained for each character.

Therefore, when this embodiment was applied to 61 container images, all 59 container numbers were correctly recognized, and a recognition rate of 96% was obtained. The recognition result of the neural network was verified by comparing the alphabet with the owner code registered in the database, and the numerical value by comparing the operation result of a series of 6 numbers and the check digit. .

As described above, the character recognizing method and apparatus according to the present invention can be input even when the target is an alphanumeric character that is dirty, bleeding, or blurred for the purpose of automating the production process or the physical distribution process. The neural network recognizes a plurality of character patterns created with a plurality of thresholds created based on the threshold determined by the discriminant analysis method from the character image, and the best value among them is output as the final recognition result, which gives a high recognition rate. Can be realized. In the example applied to marking characters of parts for the purpose of automating the process of shipbuilding assembly, recognition rate of 99% per character and 96% per part can be obtained, and it is also applied to container character recognition of container yard gate system. In the example described above, a recognition rate of 98% for each character and 96% for each container image could be obtained. In any case, the recognition rate is high, and it can be said that the character recognition device has performance equivalent to that of visual judgment.

The threshold value may be determined by using another method, mode method or the like. The range to be changed may be changed for each application target. Regarding the check digit, the frame width was further changed within the range of 1 to 4 pix, but it is not limited to 1 to 4 pix. The range to be changed may be changed for each application target.

In the embodiment, the input layer is 450 units (character size: vertical 30 pix × horizontal 15 pix), the intermediate layer is 80 units, and the output layer is 36 units (alphabet 26 characters + numeral 10 characters). The sizes of the input layer and the intermediate layer are not limited to the number of units in this embodiment. In the output layer, the recognition target is 26 uppercase letters and 10 numbers, so 36 units are used, but it can be changed according to the number of categories to be recognized.

[0045]

As described above, according to the present invention,
Created with multiple threshold values created based on the threshold value determined by a predetermined method from the input character image when recognizing characters that are dirty, blurred, or blurred alphanumeric for the purpose of automating the production process or logistics process This is a method of recognizing a plurality of character patterns by a neural network and using the best value among them as the final recognition result, and it is possible to obtain a high recognition rate for alphanumeric characters as the recognition rate for each character.
As a recognition rate for each character, it is possible to give the check digit a high recognition rate as well.

[Brief description of drawings]

FIG. 1 is a processing flowchart of a character recognition method according to the present embodiment.

FIG. 2 is a flowchart of cutting out a character string described in a member or the like.

FIG. 3 is a configuration example of a neural network.

FIG. 4 is a conceptual configuration diagram of an automatic material distribution temporary assembly system.

FIG. 5 is an explanatory diagram of the arrangement of the small set components on the attachment line.

FIG. 6 is a flowchart showing a check digit recognition processing method.

FIG. 7 is an example of a check digit character pattern.

[Explanation of symbols]

10 ... Neural network, 12 ... Input layer, 14 ... Intermediate layer, 16 ... Output layer, 20 ...
Assembly parts, 22 ......... Handling part, 24 ......... Welding machine, 26 ... Visual sensor, 28 ... Base panel,
30 ......... Marking characters.

Claims (6)

[Claims] 1. A plurality of character patterns obtained by binarizing a captured image of a recognition target character with a plurality of threshold values are obtained, and the plurality of character patterns are projectively added in the X and Y directions to cut out a character region, A character recognition method, characterized in that the obtained characters are input to a neural network, and the one having the maximum output value exceeding the set reference value is output as the final recognition result. 2. The character area cut out by the projective addition is adjusted to a certain aspect ratio, once enlarged and then reduced and converted, and the extracted character size is unified to the learning character size and input to the neural network. The character recognition method according to claim 1, wherein: 3. The character recognition method according to claim 1, wherein the threshold uses a reference threshold defined by an arbitrary method and a plurality of thresholds obtained by adding or subtracting a constant value before and after based on the reference threshold. 4. The reference threshold value is determined by a discriminant analysis method, and then the reference threshold value is converted into a percentage display value of the P tile method, and a predetermined upper and lower value is added to or subtracted from the reference P tile threshold value to obtain a front and rear P value. The character recognition method according to claim 3, wherein a tile threshold is used. 5. Preprocessing means for obtaining a plurality of character patterns created with a plurality of thresholds when binarizing a photographed image of a recognition target character, an input layer of the obtained plurality of character patterns, and at least a recognized character type. A character recognition device, comprising: a neural network having an output layer; and a discrimination means for outputting, as a final recognition result, one having a maximum certainty value among a plurality of determination results by the neural network. 6. A pre-process for obtaining a frame-extracted character pattern by cutting out the periphery of the character pattern after obtaining a plurality of character patterns created with a plurality of thresholds when binarizing a captured image of a recognition target character. Means, a neural network having an input layer of a plurality of obtained character patterns and an output layer of at least a recognized character type, and among a plurality of determination results by the neural network, the one having the maximum confidence is the final recognition result. A character recognition device having a discrimination means for outputting as.