JP4431335B2 - String reader - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for extracting a character line and a character string in a photographed image in an image input means, specifically, a portable information terminal or a mobile phone having a camera.
[0002]
[Prior art]
Conventionally, a device that reads characters printed or handwritten on paper is known as an OCR (Optical Character Reader). The main application fields are form processing, mail classification, and text conversion of documents. In typical OCR, characters are read in the following procedure. First, the paper is photoelectrically converted into a computer using a scanner (image input), the area of the character line to be read is extracted (character line extraction), and individual characters are extracted from the character line (character extraction). Is identified (character identification), and a group of characters read using language information is interpreted as a character string (post-processing).
[0003]
As an example of post-processing in character recognition, there is a function of correcting an error in the character recognition result by collating the individual characters in the recognition result with words registered in the language information dictionary. Taking an English word as an example, the character recognition result “oharacter”, whose leading character is misrecognized, is checked against the word “character” in the dictionary to correct the incorrect character and obtain the correct recognition result as a word. Obtainable. As a conventional technique for such post-processing, the method of Non-Patent Document 1 and the method of Non-Patent Document 2 in which character extraction, recognition, and collation are integrated have been proposed.
[0004]
As a post-processing target, a method of Patent Document 1 is proposed in which not only the word but also the notation format of the address address part of the address is to be collated. In the method of Patent Document 1, a notation format such as N-NN-NN (where-is a delimiter and N is a number from 0 to 9) is stored in the dictionary, and the character recognition result is compared with the notation format. Correct the error in the chome address. When this technology is applied to telephone number recognition, it is possible to extract only the telephone number part from a line in which characters other than the telephone number such as “TEL” are written, and obtain a recognition result. Specifically, the phone number notation format such as 0N-NNNN-NNNN (where 0 is the number 0, N is a number from 0 to 9, and-is the delimiter) is stored in the dictionary, and the character recognition result By comparing with, characters other than the telephone number can be deleted from the recognition result.
[0005]
On the other hand, in recent years, attempts have been made to read characters such as documents, signs, signs, etc. using a camera mounted on a portable information terminal such as a mobile phone or a PDA (personal digital assistant) as an image input means. Patent Documents 2 and 3 have been proposed as conventional examples in which a camera is provided in a mobile phone. The recognition targets of these devices are telephone numbers, e-mail addresses, URLs, words, and the like. In particular, Patent Document 3 proposes a method in which a user selects a recognition target in advance to improve recognition accuracy, and a character type limitation or post-processing method is switched according to the recognition target.
[0006]
In general, one of the problems in character recognition is to extract a character area to be read. In the form OCR, a method is generally known in which an area in which characters to be read are written is specified in advance, and an image in the area is cut out and recognized from the entire image of the form. In addition, in document OCR or the like, there is also a method of designating a recognition target area with a mouse or the like for a document displayed on the screen. On the other hand, in the portable information terminal of Patent Document 2, a method of automatically extracting and recognizing a character line area close to a marker displayed on a screen has been proposed.
[0007]
[Patent Document 1]
Japanese Patent Laid-Open No. 11-207266 [Patent Document 2]
JP 2003-78640 A [Patent Document 3]
JP 2002-152696 A [Patent Document 4]
JP 2002-366463 A [Patent Document 5]
Japanese Patent Laid-Open No. 11-203404 [Non-Patent Document 1]
Katsumi Marukawa et al., “Error Correction Algorithm for Handwritten Kanji Address Recognition”, Transactions of Information Processing Society of Japan, Vol.35, No.6, 1994.
[Non-Patent Document 2]
OR Agazzi, et al., "Connected And Degraded Text Recognition using Planar Hidden Markov Models," Proceedings of International Conference on Acoustics, Speech, and Signal Processing, pp. V-113-V-116, 1993
[0008]
[Problems to be solved by the invention]
There may be a plurality of character strings or words in the same line on paper, and a part to be read may be a part of them. In such a case, it is necessary to automatically discriminate the character string to be read by the device in accordance with a predetermined rule according to the application field.
[0009]
However, in conventional examples of character recognition using portable information terminals such as Patent Document 2, Patent Document 3, and Patent Document 4, there is only one character string to be read in the same line, and unnecessary character strings are It was based on the premise that it does not exist in the image.
[0010]
On the other hand, there are actually character strings other than the recognition target in the same line. There is no problem if only the character string portion to be recognized can be input as an image. However, in general, since an image input from a camera cannot adjust a focus to obtain a user's desired angle of view, character strings other than recognition targets are often included in the image. In such a case, since an unnecessary character string is added to the recognition result, there is a problem that a character recognition result different from the user's intention is obtained.
[0011]
Taking URL recognition as an example, as shown in Fig. 5 (a), if there is an unrelated character string in addition to the URL, the recognition result is the URL and unnecessary character string combined together. .xxxxx.co.jptel: 012-3456-7890 (dial-in) ", so the target Web page cannot be connected. The same is true for e-mail address recognition.
[0012]
For example, when it is desired to recognize an English word and obtain a translation result, it is difficult to extract and translate only one word if the specification can recognize characters only in line units. As an example, a case where a sentence “This is a pen.” Is described in a line and only the pen is to be recognized will be described. The simple recognition result is "Thisisapen." Although it is possible to extract only pen by word matching, if the delimiter between words is not clear, the number of word matching increases (matching with apen, sapen, isapen, etc.), so the mobile phone with low CPU performance It is not suitable for implementation on information terminals. In addition, if the character recognition is incorrect, there is a problem that correct results cannot be obtained even by word matching. For example, if a is misrecognized as o and “a pen” is recognized as “open”, open may be recognized as a result of word matching.
[0013]
In order to solve the above problem, there is a method for detecting a character string or a word using a space between character strings. However, since the width of the space between the character space and the character space is not always clearly distinguishable, an error may occur when extracting the character string using the space. If a character string shorter than the actual one is selected by mistake in the reading target area at this stage, a correct recognition result can never be obtained.
[0014]
Therefore, the present invention has been made in view of the above points, and solves some or all of the above problems, and particularly in character recognition using a portable information terminal or a mobile phone, etc. To provide a character string selection method by which a character string image to be recognized can be easily selected by arbitrarily selecting a character string image to be recognized by an operation, and a character recognition method for recognizing the selected character string image. It is an object.
[0015]
[Means for Solving the Problems]
In order to achieve the above object, the image data includes an image input unit that captures, acquires, or inputs an image, a display unit that displays image data from the image input unit, an operation unit that can perform an input operation, and image data. In a portable information terminal having a calculation unit for recognizing characters and a control unit for controlling the units, a character string to be read or an image data from an image input unit depending on a recognition mode input from the operation unit. Automatically switching the extraction method of character lines or words, automatically extracting character strings or character lines or words, and displaying the image content extracted together with the image data on the display unit as character strings, character lines or words, Provided is a portable information terminal or a character recognition method suitable for recognizing a specific character string, character line, or word displayed on the display unit by a selection operation from the operation unit.
[0016]
Specifically, the line extraction method is switched depending on whether the character recognition target is a phone number, e-mail or URL, or an English word. This process is based on the policy that if the reading target can be limited by the above-described post-processing, the entire character line is the recognition target, and if the reading target cannot be limited, the line is divided using a blank or the like to limit the recognition target. More specifically, if it is a phone number, the entire text line is extracted, if it is an e-mail or URL, it extracts from the left end of the line to the end of the e-mail or URL. To extract. The grounds for the above processing are as follows. In recognition of a telephone number, since only the characters of the telephone number can be determined from the line by the method using the above-mentioned Patent Document 1, the entire line can be recognized. In e-mail and URL recognition, character strings such as “http: //” and “e-mail:” can be detected by word matching, so the left side of the recognition target need not be limited. However, if there is an unnecessary character string on the right side as described above, the address recognition is incorrect, so the unnecessary character string on the right side is deleted and made a recognition target. In recognition of English words, left and right blanks are detected and only words are recognized.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 9.
FIG. 1 is a configuration diagram showing an outline of a portable information terminal or a cellular phone 101 (or simply referred to as a portable terminal or a portable device) having an image input unit according to the present embodiment.
The portable information terminal 101 includes an image input unit 102 such as a camera, a calculation unit 103, a display unit 104, an operation unit 105, a communication unit 106, and a storage unit 107. The arithmetic unit 103 controls each unit according to the procedure specified by the program stored in the storage unit 107 and executes processing such as character line extraction and image encoding. The display unit 104 is configured by a device such as a liquid crystal panel, for example, and visually displays information such as images and characters. The input unit 105 is composed of a device such as a button, for example, and receives an input from a person operating the device. The communication unit 106 is configured by a device such as a wireless LAN, for example, and performs communication with the outside. The storage unit 107 is, for example, a static RAM, and stores a variable value necessary for a program for storing processing procedures and image encoding processing such as character line extraction processing and character recognition. Note that the communication unit 106 is not essential for a portable terminal such as a PDA.
[0018]
Each unit described above and below can be expressed as a means, a mechanism, and a unit, and is basically a function that is processed and controlled by software or hardware, or a combination of software and hardware. It should be noted that it is desirable that an image captured, acquired, input, etc. is stored in a memory such as a control unit or a memory card provided in a portable information terminal so as to be used for character recognition described later.
[0019]
FIG. 2 is a diagram for explaining a character recognition processing procedure according to the first embodiment using the portable information terminal of FIG.
[0020]
In step 200, the user selects an attribute (recognition mode) of the character string to be recognized from the menu displayed on the display unit 104 using the operation unit 105. Examples of the recognition mode include “telephone number”, “e-mail”, “URL”, and “English word” as shown in FIG. In addition, a recognition mode such as “address” and “name” may be added as appropriate.
[0021]
Next, in step 202, an image of a business card, magazine, signboard or the like that is a character recognition target is photographed and stored using the image input unit 102 such as a CCD or an image sensor provided in the portable information terminal or the cellular phone 100. The image is taken into the unit 107 as a digital image. At this time, as shown in Patent Document 2 and FIG. 5 (b), an image and a marker are simultaneously displayed on the display unit 104, and the user collects an image so that the position of the marker is on the character line to be recognized. For example, the user can instruct the portable device of the position of the character line to be read. Note that the shape of the marker in FIG. 5B is a cross shape, but the shape of the marker is not limited. The step 200 for determining the attribute of the character string to be recognized may be performed after the image input step 202.
[0022]
Next, in step 204, any one of character line extraction, character string extraction, and word extraction processing is switched and executed on the captured digital image according to the recognition mode. Details of the processing in step 204 will be described later with reference to FIG. This extraction result is displayed on the display unit 104 in step 206 so that the user can understand it.
[0023]
Next, after the user has confirmed the character line to be character-recognized, it is extracted as a line, a character string, or a word in step 208 by inputting a confirmation or recognition execution instruction using the operation unit 105. Character segmentation is performed for each character in the segmented area, and character identification character identification is performed on each segmented character pattern. Further, in step 210, collation with a notation dictionary previously provided as a result of character identification is performed and output as a recognition result. Finally, in step 212, the character recognition result is displayed on the display unit 104.
[0024]
FIG. 4 is a diagram for explaining the processing procedure of line / character string / word extraction in step 204 of FIG. In FIG. 4, only the phone number, e-mail, URL, and English word recognition modes are shown, but other recognition modes may be added. You may add suitably the process of the back | latter stage of the added recognition mode.
[0025]
First, in step 400, the process branches depending on the recognition mode. If the recognition mode is a telephone number, character line extraction is performed in step 402, and the process ends. As a technique for extracting character lines, the methods described in Patent Document 5 and Patent Document 2 can be used. In the method of Patent Document 5, after binarizing an input image, a region of rows is extracted by extracting a range in which many black pixels are distributed from a peripheral distribution generated by projecting black pixels in the direction of writing characters. Is identified. In the method of Patent Document 2, after binarizing an input image, a connected component that is a block of black pixels is generated, and a region of a line is specified by integrating neighboring connected components in the character writing direction.
[0026]
FIG. 5 is a display example of the display unit 104 in the telephone number recognition mode. FIG. 5A is a recognition target. FIG. 5B shows a display example when inputting an image in step 202 of FIG. Here, since the telephone number “012-3456-7890” is selected as the character string to be recognized, the image is input after adjusting the marker to be on the character string. FIG. 5C shows an example in which the character line area extracted in step 204 in FIG. In the telephone number mode, since only the telephone number can be extracted by post-processing as described above, row extraction is selected in step 204 (step 402).
[0027]
Next, the case where the recognition mode is URL will be described. First, line extraction is performed in step 402. Next, in step 404, a blank on the right side in the line is detected, and in step 408, a region obtained by dividing the character line by this blank is taken as a character string extraction result. As a method for detecting a blank, a method of determining a difference between a character interval and a word interval in a word from a distribution of intervals of connected components in a line can be considered.
[0028]
FIG. 6 is a display example of the display unit 104 in the URL recognition mode. FIG. 6A is a recognition target. FIG. 6B shows a display example when inputting an image in step 202 of FIG. Here, since the URL “http://www.xxxxx.co.jp” is selected as the character string to be recognized, an image is input after adjusting the marker to be on this character string. FIG. 6C shows an example in which the character string area extracted in step 204 in FIG. In the URL mode, since “http: //” can be extracted by post-processing as described above, the processing flow of steps 402-404-408 in FIG. The e-mail recognition mode is the same as in FIG.
[0029]
Next, the case where the recognition mode is English words will be described. First, line extraction is performed in step 402. Next, in step 406, blanks on both sides of the word to be read in the line are detected, and in step 410, an area obtained by dividing the character line by the blanks is set as a word extraction result.
[0030]
FIG. 7 is a display example of the display unit 104 in the English word recognition mode. FIG. 7A is a recognition target. FIG. 7B is a display example when inputting an image in step 202 of FIG. Here, since the English word “thousands” is selected as the character string to be recognized, an image is input after adjusting the marker to be on this word. FIG. 7C shows an example in which the word region extracted in step 204 in FIG. In the English word mode, the processing flow of steps 402-406-410 in FIG.
[0031]
The feature of the first embodiment described above is that when a user selects a recognition target and shoots an image, a method for extracting a recognition target region is automatically selected according to the recognition mode, which is shown in FIG. By displaying on the display unit as shown in c), FIG. 6C, and FIG. 7C, the user can confirm the recognition target area.
[0032]
FIG. 8 is a diagram for explaining the processing procedure of the second embodiment according to the present invention. 8 that have the same reference numerals as those in FIG. 2 have the same functions, description thereof is omitted here. In FIG. 8, after inputting an image in step 202, a recognition mode is selected in step 200. The subsequent processing is the same as in FIG.
[0033]
In the first and second embodiments, character recognition (step 208) and post-processing (step 210) need not be performed in the portable device. For example, it is possible to transfer the image of the recognition target area to the server using the communication function and perform the above-described processing on the server.
[0034]
Next, the marker display formats in FIGS. 5B, 6B, and 7B will be supplemented. The display format of the first marker is displayed at a fixed position on a display device such as a display. In this case, the character line to be recognized is selected by moving the form device. In the second display format, the marker position is not fixed, but can be moved by an instruction from the operation unit, or by cutting out a character string from the input image and displaying a cursor at the character string position.
[0035]
FIG. 9 is a display example of the display unit 104 when an idiom is recognized in the English word recognition mode according to the third embodiment of the present invention. FIG. 9A is a recognition target. FIG. 9B shows a display example when inputting an image in step 202 of FIG. Here, the English phrase “thousands of” is selected as the character string to be recognized. Here, the image is input after adjusting the marker position in the space between “thousands” and “of”. FIG. 7C is an example in which the simple phrase area extracted in step 204 of FIG. In this case, in step 204, the processing flow of steps 402-406-410 in FIG. 4 is selected. However, since a marker is placed in the space between words, this space is detected as a left and right space as a break position, and as a result, a two-character idiom can be selected.
[0036]
【The invention's effect】
As described above, only the character string to be read can be automatically extracted and recognized from the image acquired from the input unit of the portable information terminal.
[Brief description of the drawings]
FIG. 1 shows a configuration diagram of a portable information terminal.
FIG. 2 is a flowchart of a character recognition method according to the first embodiment.
FIG. 3 is a diagram illustrating a menu for selecting a recognition mode according to the first embodiment.
FIG. 4 is a flowchart of a line / character string / word extraction method according to the first embodiment.
FIG. 5 is a diagram for explaining display of row extraction in the telephone number mode according to the first embodiment.
FIG. 6 is a diagram for explaining display of character string extraction in the URL mode according to the first embodiment.
FIG. 7 is a diagram for explaining display of word extraction in the English word mode according to the first embodiment.
FIG. 8 is a flowchart of a character recognition method according to a second embodiment.
FIG. 9 is a diagram for explaining display of idiom extraction in English word mode according to the third embodiment.

Claims (1)

Portable information having an image input unit for inputting an image, a display unit for displaying the image, an operation unit capable of input operation, and a calculation unit for controlling each unit and extracting a character line image from the image In the calculation part of the terminal,
At least the word recognition mode and e-mail, U by the selection input received in the operation unit
A first step of switching between two types of RL recognition modes;
A second step of extracting a range for character recognition from the character line image by an algorithm according to the recognition mode;
A third step of displaying a range for character recognition on the display unit;
And execute
The algorithms for the two types of recognition modes are algorithms that use the positional relationship of the character line image, the marker shown in the image, and the blank in the character line image, and the character line image A character string reading program characterized in that a part of the character string is extracted as a range for character recognition,
When the recognition mode is the word recognition mode, in the algorithm of the second step, when the character line image extracted from the image is horizontal writing, white space existing on both sides of the marker is detected in the character line image. , Execute the process of extracting the area between the blanks,
When the recognition mode is e-mail or URL recognition mode, when the character line image extracted from the image is horizontal writing in the algorithm of the second step, the character line image exists immediately to the right of the marker. A character string reading program, comprising: detecting a blank to be detected, dividing a character line by the blank, and extracting a region from a left end of the character line image to immediately before the blank.

Images