JP2021144469A - Data input support system, data input support method, and program - Google Patents

Abstract

To solve a problem such that, in data input work, a user needs to check whether or not an item subjected to character recognition processing is correct, and the user conventionally has to select an item for which character recognition cannot be performed and manually input a correct item, resulting in complicated check work.SOLUTION: In data input support processing of the present invention, processing of extracting a character string from a scanned image is performed to display it on a check screen. When a user designates a character string that has failed in extraction on a check screen, a character string is newly extracted according to a cause of failure in extraction. As a result, it is possible to enhance efficiency of check work as compared with a case where the user himself/herself manually performs correction.SELECTED DRAWING: Figure 9

Description

The present invention relates to a data input support device, a data input support method, and a program for supporting a user confirmation work in a data input business.

Conventionally, data entry work has been performed in which items such as the amount of money written on a form are read using a scanner, the read items are extracted by character recognition processing such as OCR (Optical Character Recognition), and automatically registered in the system. be. However, if the item cannot be read because it is shaded, or even if it can be read, it cannot be extracted because the result of character recognition processing does not match the preset format rules. In some cases, automatic extraction fails. Therefore, for the data input work, it is necessary for the user to confirm whether the item whose character recognition processing is correct is correct, and if it is not correct, correct it.

Therefore, there is a data input support system for assisting the user in confirming the automatically extracted items.
However, there are various layouts depending on the document to be read. For example, considering an invoice, which is a type of form, the invoice is usually created by the issuer using a template unique to the issuer, and there are various layouts. For this reason, in the data input support system, it is necessary to register the position of the item for each layout of the document, and it is difficult to apply it to a form in which various layouts can exist.

As a technique for dealing with such a problem, Patent Document 1 discloses a form recognition device that recognizes a character string described in a form. In Patent Document 1, in order to deal with a form in which various layouts are mixed, an item name representing a data attribute and an item value representing the data are searched from the character recognition result of the image, and the positional relationship between the two is determined. The item value is extracted by associating the item name with the item value based on the item value.
Then, the character recognition result of the item value is displayed on the display screen, and the area of the item value to be recognized is displayed by being surrounded by a thick line frame. Further, Patent Document 1 describes that when there is an unrecognizable item value, the input of the selected item value is accepted through the operation unit.

Japanese Unexamined Patent Publication No. 2016-513393

However, in Patent Document 1, it is necessary for the user to select the item values that cannot be recognized by the form recognition device and manually input the correct item values by himself / herself, which complicates the confirmation work by the user. Further, even if the character recognition process is performed again for the item value that could not be recognized, there is a high possibility that the recognition will fail again unless the cause of the recognition failure is resolved.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a data input support system for improving the efficiency of confirmation work by a user.

The present invention comprises a first extraction means for performing a first extraction process for extracting a character string included in a plurality of items on an image, and a first extraction process for the plurality of items. A display means for displaying the result of 1, an input means for accepting the designation of one or a plurality of items among the plurality of items displayed as the first result on the display means, and the designated item. The data input is characterized by having a second extraction means for performing a second extraction process for extracting a character string included in the designated item under conditions different from the first extraction process. It is a support system.

According to the present invention, it is possible to streamline the confirmation work by the user in the data input work.

It is a figure which shows the hardware configuration of the data input support device. This is an example of the screen displayed on the operation panel. It is a figure which shows the software structure of the data input support device. This is an example of a form image. This is an example of item information obtained from the form image. This is an example of a confirmation screen. This is an example of a confirmation screen. This is an example of a form image after binarization processing. It is a flowchart which shows the data input support process of Embodiment 1. It is a flowchart which shows the extraction correction processing. This is an example of a confirmation screen. This is an example of a confirmation screen. It is a figure for demonstrating the method of determining the area of a touch coordinate. It is a flowchart which shows the data input support process of Embodiment 2.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, each embodiment described below is merely an example, and is not intended to limit the scope of the present invention to them. In addition, not all combinations of features described in the following embodiments are essential to the means of solving the present invention.

<Embodiment 1>
<Hardware configuration>
FIG. 1 is a diagram showing a hardware configuration of a data input support device 100 for realizing the data input support system according to the first embodiment.
The data input support device 100 is composed of, for example, an image processing device such as a multifunction device (MFP). The data input support device 100 includes a control unit 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, an input unit 106, and a scanner 107.
The control unit 101 is composed of a CPU or the like, reads a control program stored in the ROM 102, and executes various processes. The RAM 103 is used as a temporary storage area such as a main memory and a work area of the control unit 101. The HDD 104 stores various data, various programs, and the like. The functions and processes of the data input support device 100, which will be described later, are realized by the control unit 101 reading a program stored in the ROM 102 or the HDD 104 and executing this program.

The display unit 105 has a liquid crystal display and the like, and displays various information. The input unit 106 has a keyboard and a mouse, and accepts various operations by the user. The display unit 105 and the input unit 106 may be integrally configured as an operation unit like a touch panel. Further, the display unit 105 may project an image by a projector. In this case, the input unit 106 may recognize the position of the fingertip with respect to the projected image by the camera.
The scanner 107 reads the document and generates a scanned image. The scanner 107 is not limited to the contact type scanner, and may be a non-contact type scanner using a document camera, a smartphone, or the like.
In the present embodiment, the scanner 107 reads a document such as a form and generates a form image as a scanned image. Then, the generated form image is stored in a storage device such as HDD 104.

<UI>
FIG. 2 is an example of a screen displayed on the operation panel 201 as a UI (User Interface) constituting the display unit 105 and the input unit 106 of the data input support device 100 in the present embodiment.
The operation panel 201 includes a touch panel 202 and a numeric keypad 203. The touch panel 202 has a column 202a for displaying a logged-in user ID, a column 202b for displaying a main menu, and the like.
In the present embodiment, the UI is one means for providing the user with a form image to be processed, item information extracted from the form image, and the like, and these information are provided on the touch panel 202. The information is not limited to the touch panel 202, and may be provided on a display connected to the PC.

<Software configuration>
FIG. 3 is a block diagram showing a software configuration for realizing the data input support device 100.
As shown in FIG. 3, the software that realizes the data input support device 100 is composed of various modules 301 to 311. The program that realizes the software is stored in the ROM 102 or the HDD 104.

The control means 301 controls the program and gives instructions and management to various modules.
The display means 302 provides the display unit 105 with a UI for the user to operate the data input support device 100 and various processing results in accordance with the instruction from the control means 301.
The input means 303 accepts a user's operation on the data input support device 100.
The storage means 304 stores the program and other information managed by the program in the ROM 102 or the HDD 104.
The character recognition means 305 performs character recognition processing on characters or character strings such as numbers on a form image (hereinafter collectively referred to as "character strings") to extract character strings, and extracts their coordinates and character types. Identify.

The item information extracting means 306 extracts a character string as item information from the item to be the target of the data input business from the form image. The item information extraction means 306 has the following submodules 307 to 311. Here, the item refers to each unit including a set character string in the form image.
The item value area detecting means 307 detects an area in which a character string to be the target data of the data input business is described as the item value area from the form image.
The item name area detecting means 308 detects an area in which a character string representing the name of the item value is described as the item name area from the form image.
The item value acquisition means 309 acquires a character string in the item value area obtained by the character recognition means 305 as an item value.
The item name acquisition means 310 acquires the character string of the item name area obtained by the character recognition means 305 as the item name.
The extraction success / failure acquisition means 311 acquires extraction availability information regarding whether or not the item type, item value, and item name can be extracted, and if extraction is not possible, determines and acquires the cause of the extraction failure, which is the cause. Details of the method for determining the cause of extraction failure will be described later.

The item value area, item name area, item value, and item name can be obtained by using, for example, a known method as disclosed in Patent Document 1.
Further, the character recognition means 305 does not need to target the character string of the entire form image to the character recognition process, and may recognize the character string required by the item value acquisition means 309 and the item name acquisition means 310. For example, when the character recognition means 305 extracts an area in which a character string is entered and then determines that the character string is not an item name based on the position, size, layout between areas, etc. of the area, the character string is used. The amount of calculation can be reduced by not performing the process of specifying the character type.

<Item information>
FIG. 4 is a diagram showing an example of a form image 400 generated by reading a form used in the present embodiment. Further, FIG. 5 is item information 501 for each item acquired from the form image 400 using the item information extraction means 306.
Item information 501 has various information about one or more items (each line in FIG. 5). Various information includes item type, item value, item name (there may be more than one), normal form, information on extraction success / failure, and information on the vertex coordinates of the area for each item value / item name (not shown). be.

In the examples of FIGS. 4 and 5, No. 5 in FIG. The item information for the items 1 to 3 is extracted from the regions 401, 403, and 405 in FIG. 4, respectively. Similarly, No. 5 in FIG. The item information for the items 4 to 7 is extracted from each region 406 to 409 in FIG. 4, respectively.
In addition, No. 5 in FIG. Each item name 1 of the item information about 2 to 6 is extracted from the regions 403a, 405a, and 404a in FIG. 4, respectively. Similarly, No. 5 in FIG. The item name 2 of the item information about No. 3 is extracted from the area 405b in FIG.

In FIG. 5, the “item type” represents the type of extracted item information. In the example of FIG. 5, four types of items, "issue date", "billing amount", "telephone number", and "line subtotal", are extracted. Of the item types, the "line subtotal" item is item information indicating the breakdown of the billed amount.
The "item value" is a data value for the item type and is extracted from the character string described in the form image. In the example of FIG. 5, for example, “4,450” and “5,000” are extracted from the regions 406 and 408 in FIG. 4, respectively. However, from the area 407, the character recognition has failed, so the question is "?". Further, from the area 408, "(1,236)" with parentheses before and after the number is extracted.
The "item name" is a character string corresponding to the item type. In the example of FIG. 5, for example, "TEL" and "head office" are extracted as item names corresponding to "telephone number".

The "normal form" is a character string in which the item value is converted (normalized) by applying the format rule set for each item type. For example, for the "issue date", a format rule of "YYYYMMDD" is set. As a result, in the example of FIG. 5, No. The item value "March 8, 2019" for 1 is normalized to the character string "20190308". Similarly, for "billing amount" and "line subtotal", the format rule is "real number with two decimal places", and for "phone number", the format rule is "character string consisting only of numbers". Is set. As a result, it is possible to eliminate fluctuations in the notation of item values that differ for each form.

In the "extraction success / failure", information indicating whether the extraction of the item value was successful or unsuccessful, and the extraction success / failure information which is the cause (extraction failure cause) when the extraction fails are entered. For example, in the extraction success / failure column for the line subtotal item, when the item value (character string) which is the line subtotal is successfully extracted, "○" indicating success is entered. If the extraction fails, the cause of the extraction failure is entered.
The case where the extraction fails includes the case where the character string cannot be extracted in the first place and the case where the extracted character string does not match the set format rule. Specifically, in the present embodiment, the cause of the extraction failure includes, for example, "character extraction failure" and "type mismatch".

Among the causes of extraction failure, "character extraction failure" is entered when the character string included in the item cannot be extracted in the first place. For example, a character extraction failure occurs when the character string portion and other pixels cannot be separated in the area for the line subtotal item (line subtotal area). For example, in the line subtotal area, there may be writing, noise may be superimposed, shading may be performed, or the background color may be dark. In such a case, the character string may not be extracted by the character recognition process.

For example, in the example of the form image 400 shown in FIG. 4, the area 407 is shaded. When the form image 400 is binarized as the character recognition pre-processing, the area 407 is filled in black as shown by the area 801 in the post-binarized form image 800 of FIG. It may be an image.
In such a case, the character string cannot be extracted from the area 407. However, since the area 407 is presumed to be the line subtotal area from the positional relationship with other items in the form image 400, the No. 5 in FIG. As shown in 5, the item type is entered as "line subtotal?". Then, since the character string cannot be extracted, "?" Is entered in the item value, "?" Is entered in the normal form, and "character extraction failure" is entered in the extraction success / failure.

If the cause of the extraction failure is a character extraction failure, the correct character string is extracted by performing the character recognition pre-processing such as noise removal / binarization processing that is performed before the character recognition processing by changing the conditions. Sometimes.

On the other hand, among the causes of extraction failure, "type mismatch" is set in advance for the character string as the line subtotal, although the item value (character string) could be extracted from the line subtotal area. Filled in if it does not match the formatting rules that were used.
For example, for a character string as a line subtotal, it is assumed that a format rule "consisting only of numbers and commas, and a comma exists after a number in the thousands" is set in advance. In this case, in the example of FIG. 4, the area 409 in the form image 400 has a character extraction result of "(1,236)" and includes parentheses "(" and ")" other than numbers and commas. , Will not match the formatting rules.

In such a case, the area 409 is presumed to be the line subtotal area from the positional relationship with other items in the form image 400. As shown in 7, the item type is entered as "line subtotal?". Further, since the character string can be extracted, "(1,236)" is entered as the item value.
However, since the item value does not match the format rule, "?" Is entered in the normal form and "type mismatch" is entered in the extraction success / failure.
In addition to this, as an example of "type mismatch", there are writings such as check marks in the corresponding area, non-corresponding characters before and after the extracted character string, and no commas. There may be cases.

If the cause of the extraction failure is a type mismatch, the correct character string may be extracted by normalizing to match the format rules. It should be noted that the user can manually input the character string to correct the character string.

<Data entry support processing>
Next, the data input support process will be described with reference to FIG. FIG. 9 is a flowchart showing the flow of the data input support process of the present embodiment.
The processing in each step of this flowchart is performed by the control unit 101 composed of a CPU or the like reading and executing the control program stored in the ROM 102.

First, in step S901, the control unit 101 acquires the form image 400 stored in the HDD 104.
Next, in step S902, the control unit 101 performs character recognition processing on the form image 400. Before performing the character recognition processing, character recognition pre-processing such as binarization processing is performed as necessary. As a result, as a character recognition result, a character string or the like corresponding to each item information is acquired from the form image 400.
Next, in step S903, the control unit 101 acquires item information 501 from the form image 400 based on the character recognition result. The item information also includes information on the success or failure of extraction.

Next, in step S904, the control unit 101 generates and displays a confirmation screen in order for the user to confirm or correct the form image 400 and the item information 501.

FIG. 6 is an example of the confirmation screen 600 displayed in step S904.
Explaining the outline of the confirmation screen 600, a bird's-eye view image 601 in which the form image 400 is displayed as a thumbnail is displayed on the left side of the confirmation screen 600. On the right side of the confirmation screen 600, each item information extracted from the form image 400 is displayed. Further, on the right side of the confirmation screen 600, an end button 610 for ending the confirmation work and moving to the next screen is displayed at the lower part. The confirmation screen 600 receives a user's instruction on the screen by an operation using the pointer 611, an operation by directly touching with a finger, or the like.
The details of the confirmation screen 600 will be described later.

Next, in step S905, the control unit 101 determines whether or not the confirmation by the user has been completed.
When the user touches the end button 610 on the confirmation screen 600, the control unit 101 determines that the confirmation has been completed, and ends the data input support process of FIG.
On the other hand, when the user touches an area other than the end button 610 on the confirmation screen, the control unit 101 determines that the confirmation has not been completed because the item information needs to be corrected, and proceeds to step S906. Proceed.

In step S906, the control unit 101 acquires the coordinates (touch coordinates) of the position on the confirmation screen 600 designated by the user.
Next, in step S907, the control unit 101 determines whether or not the touch coordinates are within the area where the line subtotal item is displayed (line subtotal area).
When the user specifies any row subtotal area, that is, when the touch coordinates are within the row subtotal area, the control unit 101 advances the process to step S908.
On the other hand, when the touch coordinates are outside the row subtotal area, the control unit 101 advances the process to step S911. Then, the control unit 101 notifies the user that the touched position is an area not subject to the extraction correction process (described later), and returns to the display of the confirmation screen 600 in step S904.

Here, a method of determining whether or not the touch coordinates are within the row subtotal area on the confirmation screen 600 will be described with reference to FIG.
FIG. 13 shows an enlarged image 1300 which is an enlarged view of the bird's-eye view image 601 on the confirmation screen 600 of FIG.
In FIG. 13, the X direction and the Y direction of the coordinates are shown as 1301. Each rectangular coordinate surrounding the row subtotal areas 406 to 409 is represented as (left end X, right end X, upper end Y, lower end Y). That is, as shown in FIG. 13A, each row subtotal region 406 to 409 is (L1, R1, T1, B1), (L2, R2, T2, B2), (L3, R3, T3, B3, respectively. ), (L4, R4, T4, B4).

Here, when determining whether the touch coordinates are within the row subtotal area in the X direction, as shown in FIG. 13 (a), from the minimum value Xmin in the left ends L1 to L4, in the right ends R1 to R4. It is determined whether or not it is between the maximum value Xmax of.
Further, when determining whether or not the device is within the row subtotal region in the Y direction, as shown in FIG. 13B, the minimum value Ymin, which is the upper end T1 of the region 406 at the top of the row subtotal region, is the highest. It is determined whether or not it is between the maximum value Ymax which is the lower end B4 of the lower region 409.

Then, it is further confirmed whether the touch coordinates are included in the height range of the area regarded as the row subtotal. Specifically, as shown in FIG. 13C, first, the center coordinates m1 to m4 of each row subtotal region are acquired in the height direction. That is, the center coordinates m1 to m4 are acquired so that the intervals s1 to s3 between the center coordinates are substantially the same. Next, it is determined whether or not the touch coordinates are included in the range of h1 to h4 in the height direction with the center coordinates m1 to m4 as the center. In addition, each range may have a certain margin.

Returning to the description of the flowchart of FIG. 9, in step S908, the control unit 101 searches the item information 501 for the item corresponding to the touch coordinates, and acquires the extraction success / failure information of the item.
Next, in step S909, the control unit 101 performs extraction correction processing based on the acquired extraction success / failure information. The details of the extraction correction process will be described later with reference to FIG.
When the extraction correction process is completed, in step S910, the control unit 101 transmits the corrected item information to an external system (not shown) and registers it. As a result, the data input support process of FIG. 9 is completed.

<Confirmation screen>
Again, with reference to FIG. 6, the confirmation screen 600 generated in step S904 described above will be described.
The confirmation screen 600 is a screen for presenting the extracted item information 501 to the user so that the user can confirm or modify the item information 501. On the confirmation screen 600, as a confirmation operation, the user confirms whether the item value is detected from the correct area and whether the correct item value is extracted, and if there is an error, corrects the error. The confirmation screen 600 includes a bird's-eye view image 601, item type texts 602a to d, item value texts 603a to c, da, db, item images 604a to c, da, db, an end button 610, and the like.

The bird's-eye view image 601 is a thumbnail display of the form image 400. In the bird's-eye view image 601 of FIG. 6, the areas 401, 403, 403a, 405, 405a, 405b, 406, and 408 corresponding to the item value and the item name in the item information 501 are highlighted. Further, the areas 401, 403, 405, 406, 408 corresponding to the item values and the areas 403a, 405a, 405b corresponding to the item names are highlighted separately. The bird's-eye view image 601 can change the display position and the display magnification by performing a swipe operation or a pinch-in / pinch-out operation.

In the item type texts 602a to d, the name of the item type transmitted to the external system in step S910 described above is displayed. In the example of FIG. 6, the item type text 602a has an "issue date", the item type text 602b has a "billing amount", the item type text 602c has a "phone number", and the item type text 602d has a "line". "Subtotal" is displayed respectively.

In the item value texts 603a to c, da, and db, the item values corresponding to the areas highlighted in the bird's-eye view image 601 are displayed as text. User input is possible in the item value text field, and if there is an error in the character recognition result, the user can make corrections in this field.

In the item images 604a to c, da, and db, images extracted from the item values and item names corresponding to the areas highlighted in the bird's-eye view image 601 are displayed. When each item image is selected by the user, the item value text corresponding to the selected item image may be displayed on the confirmation screen 600 and may be further highlighted.

The end button 610 is a button for ending the confirmation screen 600. When the confirmation work of the item information 501 is completed on the confirmation screen 600, the user presses the end button 610 to close the confirmation screen 600.

<Extraction correction processing>
Next, the extraction correction process executed in step S909 of FIG. 9 will be described with reference to FIG. FIG. 10 is a flowchart showing the details of the extraction correction process.
The processing in each step of this flowchart is also performed by the control unit 101 composed of a CPU or the like reading and executing the control program stored in the ROM 102.

In step S1001, the control unit 101 acquires the extraction failure cause entered in the extraction success / failure of the item information 501 corresponding to the item touched by the user in the above-mentioned step S906, and whether the extraction failure reason is "type mismatch". Judge whether or not.
If the reason for the extraction failure is "type mismatch", the control unit 101 advances the process to step S1002. On the other hand, if the reason for the extraction failure is not "type mismatch", the process proceeds to step S1004.

In step S1002, the control unit 101 acquires an item value from the item information 501 corresponding to the item touched by the user in step S906, and generates and displays an updated confirmation screen.

Here, the confirmation screen 700 generated in step S1002 will be described with reference to FIG. 7. FIG. 7 is an example of the confirmation screen 700.
For example, as shown in FIG. 6, it is assumed that the user specifies the area A on the confirmation screen 600 in step S906. In this case, as shown in FIG. 7, on the confirmation screen 700, the bird's-eye view image 601 is converted into an enlarged image 701 in which the line subtotal area 707 is enlarged and displayed. Along with this, the touched column is fixed and the other columns are converted to a scrollable display method so that the comparison display with other columns such as "product name" can be performed. To scroll, the left button 709 or the right button 710 may be used.

Here, since the area A specified by the user is the line subtotal area 704, the cause of the extraction failure acquired in step S1001 is "type mismatch". In this case, since the character recognition process itself is performed, the character recognition result is displayed as it is on the item value text 705 on the confirmation screen 700. In addition, the corresponding item image 706 is also displayed.
In the example of FIG. 7, since the item value text 705 that has undergone character recognition processing contains "(" and ")", it is one of the preset format rules for the line subtotal "consisting of only numbers and commas". I haven't done it. Therefore, in order to notify the user in an easy-to-understand manner that the format rules are not matched, the item value text 705 is highlighted by coloring it.

Returning to the description of the flowchart of FIG. 10, in step S1003, the control unit 101 corrects the item value based on the correction instruction by the user performed on the confirmation screen 700.
In the example of FIG. 7, the item value text 705 is modified by deleting "(" and ")" so as to match the format rule of the line subtotal. When the correction is made, the extraction correction process of FIG. 10 ends.
If the extraction failure reason is not "type mismatch" in step S1001, in step S1004, the control unit 101 determines whether or not the extraction failure reason is "character extraction failure".
If the reason for the extraction failure is "character extraction failure", that is, if the character string cannot be extracted even though the touch coordinates are within the line subtotal area in step S906 described above, the control unit 101 performs processing. Proceed to step S1005. Otherwise, the process proceeds to step S1009.

In step S1005, the control unit 101 changes the conditions and performs character recognition preprocessing.
For example, as a result of performing binarization processing on the form image 400 as character recognition preprocessing, as shown in FIG. 8, the shaded area 407 is an image that is painted black like the area 801. Suppose that In such a case, in step S1005, the character recognition preprocessing is performed again by changing the conditions so that an image from which the character string can be extracted can be obtained. Examples of changing the conditions of the character recognition preprocessing include changing the threshold value in the binarization processing and changing the noise removal method.

In step S1006, the control unit 101 performs character recognition processing again on the form image 400 that has undergone character recognition preprocessing in step S1005. The character recognition process performed in step S1006 may be the same method as the character recognition process performed in step S902, or may be a different method.

In step S1007, the control unit 101 generates and displays an updated confirmation screen in order to make the user confirm whether or not the item information has been successfully extracted.
In step S1008, the control unit 101 determines whether or not the confirmation by the user has been completed.
When the user confirms, the extraction confirmation process of this flowchart ends. If the user does not confirm, the control unit 101 advances the process to step S1009.

Here, the confirmation screen 1100 displayed in step S1007 will be described with reference to FIG. FIG. 11 is an example of the confirmation screen 1100.
For example, as shown in FIG. 6, it is assumed that the user specifies the area B on the confirmation screen 600 in step S906. In this case, as shown in FIG. 11, on the confirmation screen 1100, the bird's-eye view image 601 is converted into the enlarged image 1101 in which the line subtotal area 707 is enlarged and displayed.

Here, since the area B specified by the user is the shaded line subtotal area 1102, the confirmation screen 1100 shows the characters after changing the conditions for the line subtotal area 1102 and performing the character recognition preprocessing. The result of the recognition process is displayed in the item value text 1103. The item value text 1103 is highlighted by painting or the like in order to facilitate confirmation by the user. In addition, the item image 1104 is also displayed corresponding to the item value text 1103.

Returning to the description of the flowchart of FIG. 10 again, in step S1009, the control unit 101 corrects the item value according to the input of the user. For example, when the user determines that the correct character string is not displayed in the item value text 1103, the user corrects it by manually inputting the correct character string.
This completes the extraction confirmation process.

In the flowchart of FIG. 10, the condition for the character recognition preprocessing is changed only once, but if the character string extraction is not successful, the condition is changed a plurality of times for character recognition. Preprocessing may be performed.
Further, in the present embodiment, an example in which the extraction confirmation processing is performed on the row subtotal has been described, but the target of the extraction confirmation processing is not limited to the row subtotal. For item information other than the line subtotal, if there is an item value that could not be extracted from the form image with the required item information, the same extraction confirmation process as described above can be applied.

As described above, in the present embodiment, when the extraction process for extracting the character string included in each item from the form fails, the condition is changed according to the cause of the extraction failure and the extraction process is performed again. conduct.
As a result, in the confirmation work in which the user confirms and corrects the extracted character string on the confirmation screen, the work efficiency is improved as compared with the case where the user finds the extracted character string by himself and manually corrects it. Can be planned.

<Embodiment 2>
In the first embodiment, an example in which the extraction correction process is performed on the row subtotal item for which extraction has failed and the confirmation screen is updated has been described.
In the second embodiment, the line subtotal item for which the extraction correction processing has been performed is further processed so as to be displayed at an appropriate position on the confirmation screen.

FIG. 12 is an example of the confirmation screen 1200 displayed in the present embodiment. In the example of FIG. 12, the confirmation screen 1200 shows the state after the extraction correction processing is performed, and the item values to be the target of the extraction correction processing are entered in the line subtotal areas 1201 and 1202. There is.

FIG. 14 is a flowchart of the data input support process according to the second embodiment. In the flowchart of FIG. 14, steps S1401 to S1403 are added to the flowchart of FIG. 9 described in the first embodiment. The processes from step S901 to step S909 are the same as those in the first embodiment.
Further, the processing in each step of this flowchart is also performed by the control unit 101 composed of a CPU or the like reading and executing the control program stored in the ROM 102.
Hereinafter, the processes from step S1401 to step S1403, which are different from the first embodiment, will be described.

In step S1401, the control unit 101 confirms whether or not there is another item value as the same item type as the item value for which the extraction correction process has been performed.
When another item value exists as the same item type, the control unit 101 advances the process to step S1402. If the item value text of the same item type does not exist, the process proceeds to step S1403.
In the example of FIG. 12, the extracted row subtotal areas 702 and 703 exist in the same column as the row subtotal areas 1201 and 1202 for which the extraction correction process has been performed.

In step S1402, the control unit 101 displays the item value for which the extraction correction process has been performed at an appropriate position on the right side of the confirmation screen 1200 so as to correspond to the position of the item value displayed on the confirmation screen 600. ..
In the example of FIG. 12, the item value text 1203a extracted from the line subtotal area 1201 is displayed between the item value text 603da and the item value text 603db, similarly to the line subtotal area shown on the enlarged screen 1205. At that time, the item image 1204a is also displayed corresponding to the item value text 1203a.
Further, the item value text 1203b and the item image 1204b are also displayed in the same manner as the item value text 1203a and the item image 1204a.

On the other hand, in step S1403, the control unit 101 adds and displays the item value for which the extraction correction process has been performed at the position instructed by the user. As a method of instructing the position by the user, for example, as shown in FIG. 12, there is a method of dragging the item value text 1203a using the pointer 1206 and moving the item value text 1203a to an appropriate position on the confirmation screen 1200.

As described above, according to the second embodiment, the character string subjected to the extraction correction processing is arranged and displayed at an appropriate position on the confirmation screen.
As a result, it is possible to further improve the efficiency of the confirmation work in which the user confirms / corrects the extracted character string on the confirmation screen.

<Other Embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
Further, the present invention may be applied to a system composed of a plurality of devices or a device composed of one device.
The present invention is not limited to the above-described embodiment, and various modifications can be made based on the gist of the present invention, and these are not excluded from the scope of the present invention. That is, all the configurations in which the above-mentioned configuration examples and modifications thereof are combined are also included in the present invention.

100 Data input support device 101 Control unit 400 Form image 600 Confirmation screen

Claims (14)

With respect to a plurality of items on the image, a first extraction means for performing a first extraction process for extracting a character string included in the item, and
A display means for displaying the first result of the first extraction process for the plurality of items, and
An input means that accepts the designation of one or a plurality of items among the plurality of items displayed as the first result on the display means.
It has a second extraction means for performing a second extraction process for extracting the character string included in the designated item under conditions different from the first extraction process for the designated item. A data input support system featuring. When the extraction of the character string included in the specified item fails in the first extraction process, the second extraction means is based on the conditions set according to the cause of the failure. The data input support system according to claim 1, wherein a second extraction process is performed. The second extraction means said that the cause of the failure was the first cause that the character string was not extracted, or the extracted character string did not match the set rule. The data input support system according to claim 2, wherein the second extraction process is performed based on a condition set according to the cause. The data input according to claim 3, wherein when the cause of the failure is the second cause, the character string that failed to be extracted in the first extraction process is converted so as to match the rule. Support system. The data input support system according to any one of claims 1 to 4, wherein the second extraction process includes a character recognition process and a preprocess performed before the character recognition process. .. The data input support system according to claim 5, wherein the preprocessing is a binarization process of the image. The data input support system according to claim 5 or 6, wherein the second extraction means performs the second extraction process in the pretreatment under conditions different from those of the first extraction process. When the extraction of the character string included in the specified item fails due to the first cause that the character string was not extracted in the first extraction process, the second extraction means is the above-mentioned front. The data input support system according to any one of claims 5 to 7, wherein the second extraction process is performed under conditions different from those of the first extraction process. The data input support system according to any one of claims 1 to 8, wherein a character string extracted by the first extraction process and the image are displayed on the display means. .. The data input support system according to claim 9, wherein the designated item is specified based on the coordinates received by the input means. The data input support system according to any one of claims 1 to 10, wherein the display means displays the second result of the second extraction process. The data input support system according to claim 11, wherein the display means displays the second result so as to correspond to the first result. For a plurality of items on the image, a step of performing a first extraction process for extracting a character string included in the item, and
A step of displaying the first result of the first extraction process for the plurality of items, and
A step of accepting the designation of one or a plurality of items among the plurality of items displayed as the first result, and
It is characterized by having a step of performing a second extraction process for extracting the character string included in the designated item under conditions different from the first extraction process for the designated item. Data entry support method. A program for causing a computer to execute the data input support method according to claim 13.

Images