JP2005216102A - Personal digital assistant and image processing method thereby - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform data processing, specialized for individual operation, of an image read by using a cell phone with a camera generally sold on the market. <P>SOLUTION: The cell phone 10 is provided with: a user area 12b of a memory 12 which disables rewriting of information by a user, and firmware for operating basic functions such as an image capturing function, a character recognition function, an e-mail function and a display function of a personal digital assistant with a camera is stored; a system area 12a of the memory 12 which enables rewriting of the information by the user, and a format control descriptive expression (format control file 20) including each function processing and hardware control of the cell phone is stored; and a CPU 18 which if the character recognition function is designated during the operation of the basic function by the firmware, interprets the format control file 20 stored in the user area 12b, and controls, according to the description contents of the file, the image capturing function, the character recognition function, the e-mail function and the display function. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a portable information terminal used for business, for example, for processing a form, and an image processing method using the portable information terminal.

  In general, in the delivery industry and the like, a mobile phone is used for communication, delivery management, etc. between a driver (delivery person) of a vehicle delivering goods and a telephone operator of a distribution center.

At the distribution center, a delivery person collects a customer's voucher for delivering an article from the delivery person and applies the OCR (optical character reader) to convert it into electronic data, which is stored in a database. In business management and inquiry.
Information to be converted into electronic data is image data of a slip and character data (slip number, delivery amount, etc.) read by partially recognizing the image data.
In the distribution center, due to the extreme concentration of slips, sometimes millions of slips are processed in one day. For this reason, hundreds of clerk are engaged in processing work.

  In recent years, functions of mobile phones have been remarkably improved, and high-resolution camera functions have been added as standard functions.

  Therefore, it is possible to eliminate the voucher processing at the distribution center, reduce the number of personnel, and improve the efficiency of work by letting the delivery person carrying the mobile phone use the camera function and directly convert the voucher into electronic data. .

As a technology for character recognition using the camera function of a conventional mobile phone, for example, a still image is extracted by an image extraction unit to perform character recognition from a moving image obtained by imaging using a camera, and the extracted still image A technique for performing character recognition is disclosed (see, for example, Patent Document 1).
This technology incorporates a camera control function and a character recognition function in a mobile phone.

However, since mobile phones are usually manufactured by phone manufacturers and sold by mobile carriers, even if a camera function specializing in reading a letter on a delivery slip is developed, the purchaser is limited, so It is difficult to make it.
JP 2003-216893 A

  As described above, although a technology for partially recognizing characters from an image photographed by a camera has been disclosed as a conventional technology for mobile phones, this type of technology involves hardware control and is downloaded and used. It cannot be handled by additional software (applet), it must be implemented as an embedded program (firmware), and it is difficult to commercialize and put it to practical use, especially for use only for data processing of delivery business forms. was there.

  The present invention has been made to solve such a problem, and uses a commercially available portable information terminal to perform processing of an image captured by the imaging means of the portable information terminal specialized for each job. It is an object of the present invention to provide a portable information terminal capable of performing image processing and an image processing method using the portable information terminal.

  In order to solve the above-described problems, a portable information terminal according to the present invention includes an imaging unit for capturing an image, and a first area for character recognition in the image captured by the imaging unit. A storage means for storing a format control description expression; a character recognition function for recognizing characters from the first area in the image captured by the imaging means based on the format control description expression; and the captured image. And a transmission means for transmitting the character recognition result to a predetermined transmission destination.

  The format control description formula includes a description for displaying a guide for image alignment.

  An image processing method using a portable information terminal according to the present invention is an image processing method using a portable information terminal including a photographing unit, a character recognition function, a transmission unit, and a storage unit. From the first area in the image captured by the imaging step based on the format control description formula for setting the first area of the character recognition target in the image captured by the imaging means. The character recognition function includes a step of recognizing a character, and a step of transmitting the image captured by the imaging step and the character recognition result to a predetermined transmission destination.

  In the present invention, when the character recognition function is operated in the portable information terminal, an image is picked up by the photographing means, and characters are taken from the first region in the picked-up image based on the format control description formula stored in the storage means. Recognition is performed, and the character recognition result and the captured image are transmitted to a predetermined destination, so that it is possible to respond to various tasks without incorporating a character recognition function specialized for each task in advance into the portable information terminal. Can be processed.

  As described above, according to the present invention, it is possible to perform data processing of an image captured by an image capturing unit of a mobile information terminal by using a commercially available mobile information terminal specialized for each job.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing the configuration of a physical distribution management system according to one embodiment of the present invention.
As shown in the figure, this physical distribution management system includes a server computer 1 (hereinafter referred to as “server 1”), a portable computer that can communicate with this server 1 via a communication network 2 such as the Internet and a telephone network, and a wireless base station 3. And a telephone 10. The server 1 is installed in, for example, a distribution center. The installation location of the server 1 may be installed in various places such as a head office or an Internet service provider (ISP), and is not limited to the distribution center.

  The server 1 receives and / or transmits an e-mail from the mobile phone 10 and received data of the e-mail received from the mobile phone 10 (the image data of the slip and the image data are partially character-recognized. Database function for registering and managing character data (such as slip number and payment amount)), key input function for slip number, and database search by key input for slip number, reading the corresponding data, monitor, printer Inquiry output function is provided.

  The cellular phone 10 includes a key operation unit 11, a memory 12 as a storage unit, a call unit 14, a camera unit 15 as an imaging unit, a wireless communication unit 16, and a liquid crystal display display unit 17 (hereinafter referred to as an LCD display) as a screen display unit. A CPU 18 as control means.

The key operation unit 11 includes a power switch, a numeric keypad (also used as a character key), a direction key for moving a selection item displayed on the screen up / down / left / right, and a function key (F key or the like) for confirming the selection item and finalizing editing. ) Etc. The memory 12 is provided with a system area 12a as a first storage means and a user area 12b as a second storage means.
The system area 12a is a storage area in which information cannot be rewritten by the user, and includes an image shooting function (control of the camera unit 15 and the like) and a character recognition function (image data cutout and character recognition processing) of the mobile phone 10. OCR functions), interpretation functions of the format control file 20, voice communication functions (such as calls) and message communication functions (such as sending and receiving e-mails), display functions (such as screen display control of the LCD display unit 17), etc. A control program (firmware) for operating is stored.
The user area 12b is a storage area in which information can be rewritten by the user. A format control file 20 (see FIG. 2), which is a format control description expression including each function processing and hardware control of the mobile phone 10, is stored in the user area 12b. It is remembered.

  An example of the description of the format control file 20 is shown in FIG. The format control file 20 is a text file in which a control description is described in an XML language or the like. FIG. 2 describes a specific example in which the form for character recognition is a payment slip. The format control file 20 includes a display control description for displaying a frame for enclosing the outline outline of the payment slip (a frame for alignment), and a cutout range (area) of image data to be subjected to character recognition. For character recognition in a character image in an image cut out from a range specified by a display control description for displaying a frame indicating) and a frame indicating a cut-out range (region) of image data to be character-recognized. A character type control description (for example, “alphanumeric” is specified when recognizing a slip number or amount) and a description of an action for data obtained by character recognition are described. An example of an action description is to send an e-mail with the date, time, telephone number, character-recognized slip number, and character-recognized payment amount to a predetermined address (ocrceter@xxxx.co.jp) It is.

  The call unit 14 is composed of a microphone, a speaker, etc. and a control circuit thereof. The camera unit 15 includes a lens, a CCD camera, and a control circuit thereof, and captures an image in a direction in which the lens opening is directed. The number of effective photographing pixels is 310,000 pixels, for example, and it is even better if the number of pixels is more than this. The wireless communication unit 16 performs wireless communication with the wireless base station 3. The LCD display unit 17 includes various screens (function menu screen, form selection screen, OCR screen, correction screen, etc.), a character recognition area frame 34 that is a character recognition target frame line on the OCR screen, and a position for the user to perform positioning. A guide frame 35 or the like, which is a guide frame line, is displayed.

  The CPU 18 is stored in the user area 12b when a predetermined function such as an OCR function is selected and designated by the user while each function is operated by the firmware stored in the system area 12a in response to a user operation. It functions as a means for reading and interpreting the existing format control file 20. Further, the CPU 18 functions as a control means for controlling the image photographing means, the character recognition function, the voice and message communication function (e-mail sending means), and the screen display function based on the contents of the interpreted format control file 20.

  That is, the cellular phone 10 displays a camera unit 15 as an imaging unit that captures an image, and a character recognition area frame 34 as a first area to be recognized in the image captured by the camera unit 15. A memory 12 as a storage means for storing a format control file 20 that is format control information, an interpretation function for interpreting the format control file 20 stored in the memory 12, and a format control file interpreted by the interpretation function A character recognition function for cutting out a portion (first region) of the character recognition area frame 34 in the image captured by the camera unit 15 based on the description content 20 and performing character recognition processing, and recognition by this character recognition function And a screen display function for displaying the edited characters in an editable manner.

  In addition, the mobile phone 10 includes a camera unit 15 as an imaging unit that captures an image of the slip R1 as a form, and character recognition as a first area to be recognized in the image captured by the camera unit 15. A memory 12 as storage means for storing a format control file 20 which is format control information for displaying at least one of an area frame 34 and a guide frame 35 as a second area for image alignment; The character recognition area frame 34 in the image captured by the camera unit 15 based on the interpretation function for interpreting the format control file 20 stored in FIG. 12 and the description content of the format control file 20 interpreted by the interpretation function A character recognition function that cuts out a designated portion (first region) and performs character recognition processing; A display function for displaying the image of the slip R1 imaged by the camera unit 15, the guide frame 35, the character recognized by the character recognition function, and a message prompting the user to correct or confirm the character, and the image of the confirmed character. E-mail transmission means as a message communication function for transmitting the image of the slip R1 to the address of the server 1 which is a predetermined transmission destination is provided.

Hereinafter, the operation of the physical distribution management system will be described with reference to FIGS.
In the case of this physical distribution management system, a delivery person who is a user performs an operation for displaying a function menu on the key operation unit 11 when reading a slip R1 of an article to be delivered shown in FIG.
Then, the CPU 18 of the mobile phone 10 displays the function menu screen 31 as shown in FIG. 4 on the LCD display unit 17 according to the firmware stored in the system area 12a (S201).
This function menu screen 31 includes: 1. Send mailbox, 2. Receive mailbox; Melody box, 4. Photo box, 5. Selection items such as an OCR function box are displayed.

When, for example, “5.” is selected from these selection items (Yes in S202), the CPU 18 reads and interprets the contents of the format control file 20 stored in the user area 12b of the memory 12. If there are a plurality of formats, all the format control files 20 are read, and a form selection screen 32 for selecting a form corresponding to the read format control file 20 is displayed as shown in FIG. 5 (S203).
The form selection screen 32 includes: General documents, 2. 2. Company A mail order slip read. B company application form reading, 4. Selection items such as C company bar code reading are displayed.
When, for example, “2. Company A mail order slip reading” of “2.” is selected from these selection items (Yes in S204), the CPU 18 reads the format control file 20 created to read the selected form. The contents (see FIG. 2) are interpreted (S205).

  Then, after interpreting the contents of the format control file 20, the CPU 18 displays the OCR screen 33 and controls the camera unit 15 to start imaging as shown in FIG. On the OCR screen 33, a character recognition area frame 34, a guide frame 35, and a message such as “Please align with the frame on the form” are displayed (S206). The character recognition area frame 34 and the guide frame 35 are displayed at positions specified by the description of the format control file 20. The guide frame 35 is used by the user to change the position of the mobile phone 10 and match the outline of the form R1. These frames are simplified form styles.

As shown in FIG. 7, the slip R1 as a form has a slip number “13” printed in a predetermined position, for example, the upper left corner of the form face, and a payment amount description column printed in the lower right corner of the form face. In the column for the amount to be paid in, “24000” is written. On the other forms, the name, address, item, etc. of the payer are described. The name, address, item, etc. of the payer need only be read as image data.
On the OCR screen 33, the image of the form R1 imaged by the camera unit 15 is displayed in the background of the character recognition area frame 34 and the guide frame 35 in a full screen (S207). Since the images on the OCR screen 33 are taken at predetermined time intervals (every several milliseconds), when the mobile phone 10 is moved, the position is shifted accordingly.

According to the message displayed on the OCR screen 33, the user moves the mobile phone 10 so that the outline of the image of the form R1 displayed on the background of the OCR screen 33 overlaps the guide frame 35, and changes the position. When the F key is pressed at the position, the CPU 18 turns on the flash and performs a shutter operation (storage operation), that is, stores the captured image in the memory 12. In addition, a user's shutter operation can also be abbreviate | omitted by performing the following character recognition process, image-controlling the camera part 15 and imaging an image every several msec, and memorize | storing it in the memory 12.
The CPU 18 cuts out the image of the character recognition area frame 34 from the image stored in the memory 12 and executes the character recognition process (S208). When a character recognition result is obtained and the character recognition process is completed (Yes in S209), the CPU 18 generates a sound from the speaker of the calling unit 14 (S210), and notifies the user to that effect.

  As shown in FIG. 8, the CPU 18 displays the correction screen 36 (S211), and displays text data of the character recognition result and a message such as “Please collate with the contents described in the form” on the screen. To do. In this example, “13” is displayed on the right side of the slip number and “24000” is displayed on the right side of the payment amount as text data of the character recognition result. If the content of the correction screen 36 is correct, the user does not correct it (after correction by key operation if it is not correct), presses the F key to perform a confirmation operation.

Then, the CPU 18 determines that the character recognition result is confirmed (Yes in S212), and causes an action as post-processing according to the description content of the format control file 20.
In this example, the CPU 18 inserts text data (13, 24000) of the date, time, telephone number, and character recognition result into the mail body, and sends an e-mail attached with the read image data to a predetermined destination, in this case, a server. 1 is sent to the e-mail address (S213).
When the server 1 receives an e-mail from the mobile phone 10, the received mail data is dropped into a standard form and registered and managed in a database. Note that the post-processing performed on the mobile phone 10 side may be a simple tabulation process.

As described above, according to the physical distribution management system of this embodiment, the cellular phone 10 has the interpretation function of the format control file 20, stores the format control file 20 in the user area 12b of the memory 12, and selects this when recognizing characters. By using them, special processing such as reading of the delivery slip R1 and character recognition can be performed by a general commercially available mobile phone 10 with a camera function.
Further, a slip number (alphanumeric) and an amount (printed or printed at a predetermined position on the delivery slip R1 from an image read from the slip R1 using the camera function of the mobile phone 10 with a camera function that is commercially available. (Number) is recognized and sent to the server 1 by e-mail together with the captured image data, so that it is not necessary to process a large amount of slips at the distribution center, reducing the number of center personnel or abolishing the center processing. This can contribute to reduction of fixed costs and improvement of business efficiency.

Next, another example of format control in the above embodiment will be described with reference to FIGS.
As another example of the format control, as shown in FIG. 9, when an item “Application for Company B” of “3.” is selected from the selection items on the form selection screen 32, the CPU 18 Interpret the contents of the format control file corresponding to the selected form [B company application form].

  After interpreting the contents of the format control file, the CPU 18 displays the OCR screen 37 and controls the camera unit 15 to start imaging as shown in FIG. In the OCR screen 37, an ID field frame 38 is displayed according to the description of the corresponding format control file, and a message such as “Please adjust the ID field to the upper frame” is displayed. The ID field frame 38 also serves as a guide frame. The ID field frame 38 in this example is a character recognition frame used to determine whether, for example, it is the fourth D-form among application sheets that are a single form. That is, this example is used when identifying and determining the type of the form itself. The processing after character recognition from the image in the ID field frame 38 is the same as that in the above embodiment.

  As shown in FIG. 11, when the item “4.” [C company barcode reading] is selected from the selection items on the form selection screen 32, the CPU 18 selects the selected form [C Interpret the contents of the format control file corresponding to [Company Bar Code Read].

After interpreting the contents of the format control file, the CPU 18 displays an OCR screen 39 as shown in FIG. 12, and controls the camera unit 15 to start imaging. On the OCR screen 39, three marks 40 are displayed according to the description of the corresponding format control file, and a message such as “3 point mark being detected” is displayed. The three marks 40 also serve as guide frames, and the user moves the mobile phone 10 so that the marks printed on the form are aligned with the marks 40. In this example, when the mark of the form is aligned with the position of each mark 40, the CPU 18 performs a reading process of the barcode printed at a predetermined position on the lower right of the form. Note that the existing technology is used for the barcode reading process.
In this way, it is possible to perform character recognition or barcode reading of various forms simply by changing the description of the format control file.

In the above-described embodiment, an example in which a form is read by the mobile phone 10 carried by the delivery person is described. However, as a business model, in addition to this, an article requested for delivery by a customer is stored in a retail store such as a convenience store. In addition, since it is conceivable that the delivery person goes to pick up the route service, in this case, it is more efficient to read the slip with a mobile phone held by the retail store and convert it into electronic data in advance. In this case, electronic data is stored in a mobile phone of a retail store in a memory card or the like mounted in a slot of the mobile phone, and the data is transferred to the mobile phone 10 carried by the delivery person via the memory card. Data may be transmitted from the delivery person's mobile phone 10 to the server 1. Further, the data transfer medium is not limited to the memory card, and a mobile phone interconnection cable or the like may be used, or infrared rays or radio waves may be used.
In the above-described embodiment, the mobile phone 10 has been described as an example of the mobile information terminal. However, a personal digital assistant (PDA) or the like may be used.

The figure which shows the structure of a physical distribution management system. The figure which shows the format control file memorize | stored in the mobile telephone used with the physical distribution management system of FIG. The flowchart which shows operation | movement of a mobile telephone. The figure which shows the function menu screen of a mobile telephone. The figure which shows the form selection screen of a mobile telephone. The figure which shows the OCR screen of a mobile telephone. The figure which shows an example of a slip. The figure which shows the correction screen of a mobile telephone. The figure which shows the form selection screen of a mobile telephone. The figure which shows the OCR screen of a mobile telephone. The figure which shows the form selection screen of a mobile telephone. The figure which shows the OCR screen of a mobile telephone.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Server, 2 ... Internet, 3 ... Wireless base station, 10 ... Mobile phone, 11 ... Key layer operation part, 12 ... Memory, 14 ... Call part, 15 ... Camera part, 16 ... Wireless communication part, 17 ... LCD display Part, 18 ... CPU, R1 ... slip.

Claims (3)

An imaging means for capturing an image;
Storage means for storing a format control description formula for setting a first area for character recognition in an image captured by the imaging means;
A character recognition function for recognizing characters from the first area in the image captured by the imaging means based on the format control description formula;
A portable information terminal comprising: a transmission unit configured to transmit the captured image and the character recognition result to a predetermined transmission destination. The format control description formula includes
3. The portable information terminal according to claim 1, further comprising a description for displaying a guide for aligning images. In an image processing method by a portable information terminal provided with a photographing means, a character recognition function, a transmission means, and a storage means,
Capturing an image by the photographing means;
The first of the images captured by the imaging step based on a format control description formula for setting a first region for character recognition in an image captured by the imaging unit and stored in the storage unit. The character recognition function recognizes characters from one area;
An image processing method using a portable information terminal, comprising: a step in which the transmission unit transmits the image captured in the imaging step and the character recognition result to a predetermined transmission destination.

Images