JP2024067409A - Character recognition device, character recognition method, and program - Google Patents

Abstract

To provide a character recognition device which accurately recognizes characters described on a drawing sheet.SOLUTION: A character recognition device includes: a learning data acquisition unit which acquires first learning data obtained by attaching alternative characters as a correct answer label to image data of a displayed object described on a drawing sheet and not being a character; a learning unit which creates a character recognition model by learning the first learning data; a recognition object data acquisition unit which acquires recognition object image data including displayed objects being a character and not a character; and a recognition unit which recognizes characters from the recognition object image data on the basis of the character recognition model, and outputs residual characters obtained by deleting the alternative characters from the recognized characters as a recognition result.SELECTED DRAWING: Figure 1

Description

This disclosure relates to a character recognition device, a character recognition method, and a program.

In plant design work such as nuclear power plants, there is a lot of work to compare multiple drawings using symbols, equipment numbers, drawing numbers, etc. on the drawings. Currently, the type of equipment is determined manually, equipment numbers are read, and association is performed on paper drawings, which requires a lot of man-hours. To automate this work, it is necessary to read equipment numbers using OCR (Optical Character Recognition), determine equipment types using symbol recognition, and associate equipment numbers with equipment information. Patent Document 1 discloses a technology to read characters such as dimensions required for calculating the weight of architectural structures from architectural drawings. In Patent Document 1, columns, walls, steel frames, etc. of structures are recognized from architectural drawings and separated from characters, and recognition processing is performed on the character areas to improve the recognition accuracy of characters. Patent Document 2 discloses a technology to improve the recognition accuracy of symbols such as valves, pumps, and electronic elements drawn on drawings.

JP 2019-207530 A JP 2022-63599 A

One of the factors that reduces the accuracy of character recognition on drawings is erroneous recognition of characters due to symbols that appear on the drawings, but Patent Documents 1 and 2 do not disclose effective means to address erroneous recognition due to such factors. There is a need for a method to improve character recognition on drawings.

This disclosure provides a character recognition device, a character recognition method, and a program that can solve the above problems.

The character recognition device of the present disclosure includes a learning data acquisition unit that acquires first learning data in which alternative characters are assigned as correct labels to image data of non-character display objects shown in the drawings, a learning unit that learns the first learning data to create a character recognition model, a recognition target data acquisition unit that acquires image data of a recognition target including characters and the non-character display objects, and a recognition unit that recognizes characters from the image data of the recognition target based on the character recognition model, and outputs the remaining characters after deleting the alternative characters from the recognized characters as recognition results.

The character recognition method disclosed herein includes the steps of acquiring first learning data in which substitute characters are assigned as correct answer labels to image data of non-character objects depicted in drawings, learning the first learning data to create a character recognition model, acquiring image data of a recognition target including characters and the non-character objects, and recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after deleting the substitute characters from the recognized characters as recognition results.

The program disclosed herein causes a computer to execute the steps of acquiring first learning data in which alternative characters are assigned as correct labels to image data of non-character objects depicted in drawings, learning the first learning data to create a character recognition model, acquiring image data of a recognition target including characters and the non-character objects, and recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after removing the alternative characters from the recognized characters as recognition results.

The above-mentioned character recognition device, character recognition method, and program can improve the recognition rate of characters written in a drawing.

1 is a block diagram showing an example of a character recognition device according to an embodiment; FIG. 13 is a diagram illustrating an example of an effect of a trained OCR model according to an embodiment. FIG. 1 is a first diagram illustrating an example of erroneous recognition according to an embodiment. FIG. 11 is a second diagram illustrating an example of erroneous recognition according to the embodiment. 11A and 11B are diagrams illustrating a learning method for preventing erroneous recognition according to an embodiment. FIG. 11 is a first diagram showing another example of erroneous recognition according to the embodiment. FIG. 11 is a second diagram showing another example of erroneous recognition according to the embodiment. 10 is a flowchart illustrating an example of a process for improving a recognition rate according to the embodiment. 4 is a flowchart illustrating an example of a character recognition process of the drawings according to the embodiment. 10 is a flowchart illustrating an example of post-processing according to the embodiment. 11A and 11B are diagrams illustrating an example of an effect of post-processing according to the embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of the character recognition device according to the present embodiment.

<Embodiment>
The character recognition method of the present disclosure will be described below with reference to the drawings.
(composition)
1 is a block diagram showing an example of a character recognition device according to an embodiment. The character recognition device 10 is configured with one or more computers. The character recognition device 10 accurately recognizes characters shown in the drawings. As shown in the figure, the character recognition device 10 includes a data acquisition unit 11, an input reception unit 12, a processing unit 13, an output unit 14, and a storage unit 15.

The data acquisition unit 11 acquires image data including characters and/or non-character objects written on drawings of machines, plants, etc. For example, the data acquisition unit 11 acquires learning image data in which a correct answer label is attached to the image data, and image data for character recognition. Characters are symbols such as alphanumeric characters, kana, hiragana, kanji, parentheses, and hyphens (symbol characters to be read), and are characters to be read from the drawings. Non-character objects are symbols such as contact marks included in the drawings, and notations to be read, such as table lines. When characters on drawings are read using a general OCR model, non-character objects are often erroneously recognized as characters to be read. As will be described later, the character recognition device 10 has a function to prevent such erroneous recognition.

The input reception unit 12 is configured using input devices such as a keyboard, mouse, touch panel, and input interface, and receives various settings for improving the character recognition rate and instructions to execute character recognition processing, which are input using these input devices.

The processing unit 13 executes processes such as character recognition processing, creation of a deep learning model for character recognition (called an OCR model), etc. The processing unit 13 includes a learning unit 131, a character recognition unit 132, and a post-processing unit 133.
The learning unit 131 additionally learns a general OCR model (e.g., FOTS, MaskTextSpotter, etc.) provided for character recognition to create an OCR model specialized for character recognition of drawings. The OCR model is, for example, a deep learning model. Character recognition using a general trained OCR model trained on general images such as outdoor images cannot recognize characters around symbols on drawings or characters inside tables with practical accuracy. In addition, in addition to characters, various symbols, tables, figures, etc. are written on drawings, which are likely to induce erroneous recognition. In addition, characters written on drawings may have a significantly different shape from characters that are generally intended to be read, which reduces the recognition accuracy. In response to this, the learning unit 131 additionally learns the OCR model using two methods described later to create an OCR model specialized for character recognition on drawings, thereby improving the character recognition rate of drawings.

The character recognition unit 132 performs character recognition on the drawing using the OCR model created by the learning unit 131 .
The post-processing unit 133 performs post-processing on the character recognition results of the character recognition unit 132. The post-processing is a process for correcting misrecognition that occurs even when an OCR model specialized for character recognition of drawings is used. The post-processing unit 133 also performs an integration process for integrating the recognition results of characters on drawings obtained by multiple OCR models.

The output unit 14 outputs the result of the recognition of characters on the drawing by the processing unit 13 to a display device, another device, an electronic file, or the like.
The storage unit 15 stores various data necessary for character recognition. For example, the storage unit 15 acquires image data acquired by the data acquisition unit 11, setting information accepted by the input acceptance unit 12, and the like. The storage unit 15 also stores one or more OCR models. For convenience, an OCR model 1 is described in FIG. 1. For example, when two OCR models are stored, they are described as an OCR model 1 and an OCR model 2. When it is not necessary to distinguish between the OCR models 1 and 2, they are simply described as an OCR model. As described later, in this embodiment, as an example, two OCR models, an OCR model 1 and an OCR model 2, are combined (integration process) to improve the character recognition rate, but there is no limit to the number of OCR models to be combined. Three or more types of OCR models may be used, or only one type of OCR model may be used. (When only one type of OCR model is used, it is not possible to improve the character recognition rate by integration processing. However, as described later, in this embodiment, the character recognition rate of the drawing can be improved by a plurality of methods. Therefore, even when only one type of OCR model is used, the character recognition rate can be improved by those methods.) When a plurality of OCR models are used, the learning unit 131 and the character recognition unit 132 have functions (programs) for each OCR model. For example, when the learning unit 131 creates OCR model 1, the learning unit 131 performs additional learning using a function constructed for OCR model 1 (learning unit for OCR model 1), and when the character recognition unit 132 performs character recognition using OCR model 2, the character recognition unit 132 performs character recognition using a function constructed for OCR model 2 (character recognition unit for OCR model 2). In the following, instead of describing them as a learning unit 131 for OCR model 1 and a learning unit 131 for OCR model 2, it will be simply described as learning unit 131 creating OCR model 1, character recognition unit 132 performing character recognition using OCR model 2, but in reality, processing is performed using the functions for each OCR model as described above.

In this embodiment, two methods for improving character recognition rates are provided: a method for training a general OCR model to create a model specialized for character recognition in drawings, and a method for improving recognition accuracy by performing post-processing on the recognition results of the OCR model.

<Method of improving character recognition rate by learning OCR model>
First, a method for creating a model specialized for character recognition in drawings by training a general OCR model will be described.
(Method A) Learning data is created with a correct answer label attached to the characters to be read that are written on the drawing, and the learning unit 131 is made to learn using the created learning data (for example, fine-tuning of the deep learning model is performed). This allows a deep learning model specialized for drawings to be learned. By learning a deep learning model to be specialized for drawings, it is possible to improve the accuracy of character recognition itself, and it is possible to relatively reduce erroneous recognition of symbols, etc. FIG. 2 shows an example of the drawing character recognition rate of an OCR model created by learning learning data suitable for drawing character recognition. Graph 2a in FIG. 2 shows the character recognition rate before and after learning in OCR model 1. Graph 2b in FIG. 2 shows the character recognition rate before and after learning in OCR model 2, which is different from OCR model 1. As shown in the figure, in both OCR models 1 and 2, the character string detection rate and character string recognition rate are significantly improved by learning characters actually written on drawings.

(Method B) A deep learning model is trained by assigning substitute characters to display objects that are likely to induce misrecognition, such as symbols and lines in drawings, and is made to recognize the symbols, lines, etc. as substitute characters assigned. Examples of misrecognition are shown in Figures 3A and 3B. Ranges 3a and 3b in the figure represent the target range of character recognition. Figure 3A shows an example in which a general OCR model misrecognizes an "ABC symbol (a shape with lines extending upward from a circle, a contact symbol)" as "ABCO (the letter O)" when the "ABC" is written in the frame of a table in a drawing, and the general OCR model misrecognizes the ruled line on the left side of the table as "1" and misrecognizes it as "1ABC". In response to such misrecognition, a character that is not planned to be used in the drawing is assigned to the display object that induces the misrecognition. For example, "@" is assigned to the symbol in Figure 3A, and "|" is assigned to the ruled line in Figure 3B. In the method 2, first, the remaining misrecognition parts are identified from the result of performing character recognition of the drawing by the OCR model created by the method 1, and a target for learning a substitute character is specified. Here, it is assumed that the misrecognition explained in FIG. 3A and FIG. 3B is identified. FIG. 4 shows the process after identifying the target for learning a substitute character. 1. First, learning data is created with the contact marks and the ruled lines of the ticket as substitute characters. For example, the learning data used in the method 1 is processed to create learning data by attaching the correct answer label "ABC@" to the range 3a in FIG. 3A and attaching the correct answer label "|ABC" to the range 3b in FIG. 3B. It is desirable that the substitute character is a character that is not scheduled to be used in the drawing and has a shape similar to the display object that causes the misrecognition. By selecting a substitute character with a similar shape, the load of the next learning process can be reduced. 2. Next, the learning unit 131 causes the OCR model to learn the learning data created in 1. As a result, an OCR model is created that recognizes the character string in the range 3a of FIG. 3A as "ABC@" and the character string in the range 3b of FIG. 3B as "|ABC". 3. Next, the character recognition unit 132 is set to perform a process of deleting the substitute character from the recognition result of the OCR model created in 2. The character recognition unit 132 simply deletes the substitute character if, for example, the substitute character is at the beginning or end of the character string of the recognition result. For example, if the recognition result is "ABC@", the character recognition unit 132 deletes @ from the recognition result to make it "ABC". Also, if the substitute character is in the middle of the recognition result, the character recognition unit 132 divides the character string at the substitute character and then deletes the substitute character. For example, if the recognition result is "ABC@DEF", the character recognition unit 132 recognizes the recognition result as "ABC" and "DEF". For example, if the recognition result is "A|B|C", the character recognition unit 132 recognizes the recognition result as "A", "B", and "C". The character recognition unit 132 outputs the character or character string after removing the substitute character as the recognition result.

<Methods for improving character recognition rate through post-processing>
Next, a method for improving the recognition accuracy by performing post-processing on the recognition results of the OCR model by the post-processing unit 133 will be described. The above method can eliminate the influence of symbols and lines that are easily misrecognized, but it is difficult to prevent misrecognition of symbol characters such as parentheses and hyphens that have important meanings in drawings and are the objects to be recognized.

<Method 1>
When a pair of characters (e.g., parentheses) exists only at the beginning or end of the character string of the recognition result, the post-processing unit 133 adds the other pair of characters to the other end of the character string. For example, when one parenthesis is missing, the other parenthesis is complemented. FIG. 5A shows an example of a recognition result in which one parenthesis was not recognized. FIG. 5A shows an example in which "(ABC1)" is recognized by the character recognition unit 132 as "ABC1)". When one parenthesis is present at the end of a character string and there is no parenthesis at the beginning, the post-processing unit 133 complements the beginning parenthesis and corrects it to "(ABC1)". In this way, the post-processing unit 133 may complement the other parenthesis at the beginning of a character string when the shape of the parenthesis at the end of the character string is the shape of a closing parenthesis based on the shape of the parenthesis. Similarly, when a parenthesis (opening parenthesis) exists only at the beginning, the post-processing unit 133 can also complement the other parenthesis at the end of the character string. Moreover, the brackets to be complemented are not limited to those shown in the figure, and may have other shapes. Moreover, the objects to be complemented are not limited to brackets, and may be other pairs of characters. Moreover, the user may be able to arbitrarily set the brackets to be complemented and the pair of characters. For example, the user sets in the character recognition device 10 that "<" and ">" are a pair of characters. The input reception unit 12 receives the user's setting, and registers the setting in the storage unit 15. The post-processing unit 133 performs a process of complementing brackets, etc., based on the user's setting. For example, one side of a bracket can be complemented by the method 1. By complementing one side of a bracket, it is possible to reduce the recognition failure of the bracket.

<Method 2>
The post-processing unit 133 may compare the recognition result of the character recognition unit 132 with a character string registered in a dictionary, and if the recognition result is similar to the character string registered in the dictionary, replace the recognition result with the registered character string. Specifically, a dictionary created from a naming rule or similar characters is set to correct erroneous recognition. For example, assume that the name of a certain part is "ABC-D$" ($ represents any number) and this part name is written many times on a drawing. In this case, the user instructs the character recognition device 10 to register "ABC-D$" in the dictionary. The input receiving unit 12 receives this instruction and registers "ABC-D$" in the dictionary of the storage unit 15. The dictionary-registered "ABC-D$" means that if the character recognition unit 132 recognizes a character string similar to "ABC-D$", the post-processing unit 133 converts (corrects) the recognized character string to "ABC-D$". For example, when the character string "RBC-D1" is recognized by the character recognition unit 132, the post-processing unit 133 corrects the recognition result to "ABC-D1". Regarding what character string is determined to be similar to "ABC-D$", for example, a setting of similar characters may be registered in the storage unit 15, and the post-processing unit 133 may determine whether a certain character string is similar to "ABC-D$" based on the setting of similar characters. As an example of setting similar characters, "R" and "A" are registered in the storage unit 15 as similar characters ("R" is easily recognized as "A" and "A" is easily recognized as "R" by the OCR model). The post-processing unit 133 converts "RBC-D1" to "ABC-D1" by referring to this setting. The post-processing unit 133 compares the converted "ABC-D1" with "ABC-D$". In this case, since the characters match including the number "$", the post-processing unit 133 determines that "RBC-D1" and "ABC-D$" registered in the dictionary are similar, and corrects "RBC-D1" to "ABC-D1". For example, when the character string "FBC-D1" is recognized by the character recognition unit 132, since there is no setting of a similar character for any of "F", "B", "C", and "D", the post-processing unit 133 directly compares "FBC-D1" with "ABC-D$". In this case, since "FBC" and "ABC" are different, for example, the post-processing unit 133 determines that the recognized "FBC-D1" is different from the dictionary-registered "ABC-D$", and does not perform correction based on the dictionary registration for "FBC-D1". The dictionary registration may be performed by the user, or the output unit 14 may display the recognition result by the character recognition unit 132 on a display device and register it in the dictionary upon confirmation by the user. For example, if a drawing contains "ABC-D1,""ABC-D2," and "ABC-D3," and the OCR model recognizes these character strings, converting the numbers to $ results in three "ABC-D$." The output unit 14 presents this recognition result on the display device, and the user instructs registration in the dictionary. The input reception unit 12 then registers "ABC-D$" in the dictionary stored in the storage unit 15. This makes it possible to perform dictionary registration and similar character setting based on, for example, expert knowledge of drawings, and to correct erroneous recognition based on expert knowledge of drawings.

<Method 3>
If one or more character strings satisfy a predetermined condition, the post-processing unit 133 corrects the recognition result by adding a predetermined character to a predetermined position of the character string in the recognition result based on a rule that specifies that a predetermined character is included in a predetermined position of one or more character strings. For example, in a drawing, when a character string is written across multiple lines, a rule is assumed that a hyphen is displayed at the beginning of the character string except the top line. On the premise that such a rule exists, if a character string of multiple lines is recognized and there is no hyphen at the beginning of the second line or subsequent lines, the post-processing unit 133 adds a hyphen to the beginning of the second line or subsequent lines. (For example, when a character string of multiple lines is recognized, it is assumed here that the characters are not independent characters but are character strings written across multiple lines.) FIG. 5B shows an example in which "-D1" is written in the second line of a character string that spans two lines, and the character recognition unit 132 recognizes it as "D1". In such a case, the post-processing unit 133 adds "-" to the beginning of the second line. According to Method 3, by setting appropriate rules, it is possible to supplement a character string that is assumed to contain a hyphen (-), with a hyphen. Character recognition that corresponds to line breaks in equipment numbers that contain hyphens or character strings in drawings becomes possible, and the character recognition rate for long texts can be improved.

<Method 4>
The post-processing unit 133 integrates the recognition results of a plurality of OCR models. For example, when another OCR model recognizes a character that the OCR model with the highest recognition rate could not recognize, the post-processing unit 133 selects the character recognized by the OCR model with the highest recognition rate among the other OCR models as the recognition result. For example, if the character recognition rate of the OCR model 2 is higher than that of the OCR models 1 and 2, the post-processing unit 133 adopts the character recognized by the OCR model 2 as the recognition result for the character that the OCR model 2 recognized, regardless of whether the OCR model 1 recognized the character, and adopts the character recognized by the OCR model 1 as the recognition result for the character that the OCR model 2 could not recognize, if the OCR model 1 recognized the character. This makes it possible to respect the recognition result of the OCR model 2 with the high recognition rate, while complementing the character that the OCR model 2 cannot recognize with the recognition result of the OCR model 1.
The combination of OCR model 1 and OCR model 2 may be a combination of models created based on different OCR models to be specialized for character recognition in drawings (for example, OCR model 1 is a model created based on FOTS, and OCR model 2 is a model created based on MaskTextSpotter), or a combination of models created by training the same OCR model under different conditions (for example, OCR model 1 is a model created by training FOTS with "training data 1", and OCR model 2 is a model created by training FOTS with "training data 2").

(motion)
Next, the process flow of the character recognition device 10 will be described.
First, with reference to FIG. 6, a preparatory process for character recognition in a drawing will be described.
FIG. 6 is a flowchart illustrating an example of a process for improving a recognition rate according to the embodiment.
The learning unit 131 learns learning data for drawing characters and creates an OCR model specialized for character recognition of drawings (step S11). This process corresponds to the above method A. Next, the learning unit 131 learns substitute characters for displayed objects that cause erroneous recognition, such as symbols and ruled lines (step S12). In addition, in accordance with the learning of substitute characters, the character recognition unit 132 is set to delete the substitute characters from the recognition result. This process corresponds to the above method B. Next, the user sets the post-processing related settings for the post-processing unit 133 (step S13). This process is preparation for the above methods 1 to 4. For method 1, characters such as parentheses to be completed are set. For method 2, dictionary registration and similar characters are set. For method 3, rules such as conditions for adding hyphens, etc. (when spanning two lines and there is no - at the beginning of the second line), characters to be added ("-"), and positions to add (the beginning of the second line) are set. For method 4, the order of character recognition rates is set for the multiple OCR models to be used. The input receiving unit 12 receives these settings and registers them in the storage unit 15. It is only necessary for the user to set, among methods 1 to 4, the method to be executed.

Next, the process of recognizing characters on a drawing will be described with reference to FIGS.
FIG. 7 is a flowchart illustrating an example of a character recognition process in the drawings according to the embodiment.
First, the data acquisition unit 11 acquires image data to be recognized (step S21). The data acquisition unit 11 records the image data in the storage unit 15. Next, the character recognition unit 132 performs processing to recognize characters from the image data using OCR models 1 and 2. The character recognition unit 132 performs character recognition on the image data recorded in the storage unit 15 using OCR model 1 (step S22). The character recognition unit 132 acquires the character recognition result of OCR model 1 and deletes substitute characters (step S23). The character recognition unit 132 outputs the result after deleting the substitute characters to the post-processing unit 133. Next, the post-processing unit 133 performs post-processing (step S24).

An example of post-processing according to the embodiment is shown in FIG. 8. The post-processing unit 133 acquires the recognition result from the character recognition unit 132 (step S31). Next, the post-processing unit 133 performs post-processing on the character string including parentheses (step S32). For example, the post-processing unit 133 corrects the recognition result "ABC" to "(ABC)". This processing corresponds to the above-mentioned method 1. Next, the post-processing unit 133 performs post-processing based on the dictionary (step S33). For example, if "RBC-D1" is registered in the dictionary and "A" and "R" are set as similar characters, the post-processing unit 133 corrects the recognition result "RBC-D1" to "ABC-D1". This processing corresponds to the above-mentioned method 2. Next, the post-processing unit 133 performs post-processing on the character string including a hyphen (step S34). For example, the post-processing unit 133 corrects the recognition result "D1" in the second line to "-D1".

Similarly, the character recognition unit 132 uses OCR model 2 to perform character recognition on the image data recorded in the storage unit 15 (step S25). The character recognition unit 132 obtains the character recognition result from OCR model 2 and deletes the substitute characters (step S26). The character recognition unit 132 outputs the result after deleting the substitute characters to the post-processing unit 133. Next, the post-processing unit 133 performs post-processing (step S27). The post-processing is the same as that described for OCR model 1 using FIG. 8. Note that the processing from step S22 to step S24 and the processing from step S25 to step S27 may be performed in parallel, or one of them may be performed first.

Next, the post-processing unit 133 integrates the recognition results obtained by performing the processes of steps S22 to S24 and the recognition results obtained by performing the processes of steps S25 to S27 (step S28). The post-processing unit 133 integrates the recognition results by performing a process in which the recognition result by OCR model 2, which has a higher recognition rate, is regarded as correct, and when OCR model 1 recognizes a character that OCR model 2 was unable to recognize, the recognition result by OCR model 1 is adopted. Next, the output unit 14 outputs the recognition result after the integration process to a display device or the like (step S29).

In the explanation of step S13 in FIG. 6, the order of OCR models is set in descending order of character recognition rate, but after steps S23 and S26 have been executed or after steps S24 and S27 have been executed, it may be verified which OCR model has the higher recognition rate, and steps S28 and onward may be executed based on the verification results. Also, post-processing methods 1 to 4 may be configured not to be executed at all, or may be configured to be executed only in part. Also, the processing order of steps S32 to S34 in FIG. 8 may be arbitrary.

Figure 9 shows an example of the process by which the character string recognition rate is improved by post-processing. Referring to Figure 9, it can be seen that the recognition accuracy of the drawing characters is improved each time the post-processing of steps S31 to S34 and step S28 is performed.

(effect)
As described above, according to the present embodiment, the recognition rate of characters written on drawings can be improved by learning characters on actual drawings, and by regarding symbols and lines specific to drawings as a type of character and performing character recognition once, and excluding substitute characters corresponding to the symbols specific to drawings from the character recognition results. In addition, the recognition rate of characters can be improved by performing character complementation and correction through post-processing after character recognition. Furthermore, by integrating the recognition results of multiple OCR models, it is possible to complement the advantages and disadvantages of multiple OCR models, and the recognition rate and robustness of the character recognition device 10 as a whole can be improved.

FIG. 10 is a diagram illustrating an example of a hardware configuration of a character recognition device according to an embodiment.
The computer 900 includes a CPU 901 , a main memory device 902 , an auxiliary memory device 903 , an input/output interface 904 , and a communication interface 905 .
The character recognition device 10 described above is implemented in a computer 900. The above-described functions are stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, loads it in the main storage device 902, and executes the above-described processing in accordance with the program. The CPU 901 also reserves a storage area in the main storage device 902 in accordance with the program. The CPU 901 also reserves a storage area in the auxiliary storage device 903 for storing data being processed in accordance with the program.

A program for implementing all or part of the functions of the character recognition device 10 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing by each functional unit. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. In addition, if a WWW system is used, the term "computer system" also includes a home page providing environment (or display environment). In addition, the term "computer-readable recording medium" refers to portable media such as CDs, DVDs, and USBs, and storage devices such as hard disks built into a computer system. In addition, if the program is distributed to a computer 900 via a communication line, the computer 900 that receives the program may load the program into the main storage device 902 and execute the above processing. In addition, the program may be for implementing part of the functions described above, or may be capable of implementing the functions described above in combination with a program already recorded in the computer system.

As described above, several embodiments of the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

<Additional Notes>
The character recognition device, the character recognition method, and the program described in each embodiment can be understood, for example, as follows.

(1) A character recognition device 10 according to a first aspect includes a learning data acquisition unit that acquires first learning data in which alternative characters are assigned as correct labels to image data of non-character objects as shown in the drawings; a learning unit that learns the first learning data to create a character recognition model; a recognition target data acquisition unit that acquires image data of a recognition target including characters and non-character objects; and a recognition unit that recognizes characters from the image data of the recognition target based on the character recognition model, and outputs the remaining characters after removing the alternative characters from the recognized characters as recognition results.
This makes it possible to perform character recognition while reducing the influence of display objects that may cause erroneous character recognition, such as symbols and marks unique to drawings and table lines, thereby improving the character recognition rate.
The data acquisition unit 11 is an example of a learning data acquisition unit. The OCR models 1 and 2 are examples of character recognition models. The data acquisition unit 11 is an example of a recognition target data acquisition unit. The character recognition unit 132 is an example of a recognition unit.

(2) A character recognition device 10 according to a second aspect is the character recognition device of (1), wherein the learning data acquisition unit further acquires second learning data in which a correct answer label is attached to image data of a character shown in the drawing, and the learning unit learns the second learning data to create the character recognition model.
Characters on drawings may have different shapes from those used by general OCR models, which reduces the character recognition rate. By learning characters written on actual drawings with correct answer labels, an OCR model specialized for character recognition on drawings can be created, which can significantly improve the character recognition rate on drawings (Figure 2).

(3) A character recognition device 10 according to a third aspect is a character recognition device according to (1) to (2), further comprising a post-processing unit that corrects the recognition result output by the recognition unit, and when one of a set of characters that forms a predetermined pair is included only at one of the beginning and end of a string of characters in the recognition result, the post-processing unit corrects the recognition result by adding the other of the set of characters that forms the pair to the other of the beginning and end.
As a result, for example, one of the parentheses can be completed. By completing one of the parentheses, it is possible to reduce oversight of parentheses recognition and improve the character recognition rate. The post-processing unit 133 is an example of a post-processing unit.

(4) A character recognition device 10 according to a fourth aspect is a character recognition device according to any one of (1) to (3), further comprising a post-processing unit that corrects the recognition result output by the recognition unit. The post-processing unit compares the recognition result with a registered character string registered in a dictionary, and if the recognition result is similar to the registered character string, corrects the recognition result by replacing the recognition result with the similar registered character string.
By registering a dictionary based on expert knowledge about drawings and setting similar characters, it becomes possible to correct misrecognition based on expert knowledge about drawings. The post-processing unit 133 is an example of a post-processing unit.

(5) A character recognition device 10 according to a fifth aspect is a character recognition device according to any one of (1) to (4), further comprising a post-processing unit that corrects the recognition result output by the recognition unit, and the post-processing unit corrects the recognition result by adding a specified character to the recognition result based on a rule that specifies that if one or more character strings satisfy a specified condition, the specified character is included in a specified position of the one or more character strings.
This makes it possible to supplement a character string that is assumed to include a "- (hyphen)" with a hyphen, for example. It also makes it possible to handle equipment numbers that include hyphens and line breaks in character strings in drawings, improving the character recognition rate for long texts, etc. The post-processing unit 133 is an example of a post-processing unit.

(6) A character recognition device 10 according to a sixth aspect is a character recognition device according to any one of (1) to (5), further comprising an integration unit that integrates the recognition results based on each of the character recognition models when the recognition unit recognizes a character from the image data of the recognition target based on each of the multiple character recognition models, and when another character recognition model is able to recognize a character that the character recognition model with the highest recognition rate was unable to recognize, the integration unit outputs the character recognized by the character recognition model with the highest recognition rate among the other character recognition models as the recognition result.
This makes it possible to complement the strengths and weaknesses of multiple character recognition models (OCR models, deep learning models for character recognition), thereby improving the recognition rate and robustness of the entire system (character recognition device 10). The post-processing unit 133 is an example of an integration unit.

(7) A character recognition method according to a seventh aspect includes the steps of acquiring first learning data in which an alternative character is assigned as a correct answer label to image data of non-character objects depicted in a drawing, learning the first learning data to create a character recognition model, acquiring image data of a recognition target including characters and non-character objects, and recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after deleting the alternative characters from the recognized characters as recognition results.

(8) The program according to the eighth aspect causes a computer to execute the steps of acquiring first learning data in which alternative characters are assigned as correct labels to image data of non-character objects depicted in a drawing, learning the first learning data to create a character recognition model, acquiring image data of a recognition target including characters and non-character objects, and recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after removing the alternative characters from the recognized characters as recognition results.

Reference Signs List 1, 2: OCR model 10: character recognition device 11: data acquisition unit 12: input reception unit 13: processing unit 131: learning unit 132: character recognition unit 133: post-processing unit 14: output unit 15: memory unit 900: computer 901: CPU
902: Main memory device 903: Auxiliary memory device 904: Input/output interface 905: Communication interface

Claims (8)

A learning data acquisition unit that acquires first learning data in which substitute characters are assigned as correct answer labels to image data of non-character display objects shown in the drawings;
a learning unit that learns the first learning data to create a character recognition model;
a recognition target data acquisition unit that acquires image data of a recognition target including characters and non-character display objects;
a recognition unit that recognizes characters from the image data of the recognition target based on the character recognition model, and outputs the remaining characters after deleting the substitute characters from the recognized characters as a recognition result;
A character recognition device comprising: The learning data acquisition unit further acquires second learning data in which a correct answer label is attached to image data of a character depicted in the drawing,
The learning unit learns the second learning data to create the character recognition model.
2. The character recognition device according to claim 1. a post-processing unit that corrects the recognition result output by the recognition unit;
Further equipped with
the post-processing unit, when one of a set of characters forming a predetermined pair is included only at one of the beginning and the end of the character string of the recognition result, corrects the recognition result by adding the other of the set of characters forming the pair to the other of the beginning and the end.
3. The character recognition device according to claim 1. a post-processing unit that corrects the recognition result output by the recognition unit;
Further equipped with
the post-processing unit compares the recognition result with a registered character string registered in a dictionary, and if the recognition result is similar to the registered character string, corrects the recognition result by replacing the recognition result with the similar registered character string.
3. The character recognition device according to claim 1. a post-processing unit that corrects the recognition result output by the recognition unit;
Further equipped with
the post-processing unit corrects the recognition result by adding a predetermined character to the recognition result based on a rule that specifies that if one or more character strings satisfy a predetermined condition, the one or more character strings contain a predetermined character at a predetermined position;
3. The character recognition device according to claim 1. an integration unit that integrates the recognition results based on each of the character recognition models when the recognition unit recognizes characters from the image data of the recognition target based on each of the plurality of character recognition models;
Further equipped with
when another character recognition model is able to recognize a character that the character recognition model with the highest recognition rate was unable to recognize, the integration unit outputs, as the recognition result, the character recognized by the character recognition model with the highest recognition rate among the other character recognition models.
3. The character recognition device according to claim 1. A step of acquiring first learning data in which an alternative character is assigned as a correct answer label to image data of a display object that is not a character depicted in a drawing;
creating a character recognition model by learning the first learning data;
acquiring image data of a recognition target including characters and non-character objects;
a step of recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after deleting the substitute characters from the recognized characters as a recognition result;
The character recognition method includes the steps of: On the computer,
A step of acquiring first learning data in which an alternative character is assigned as a correct answer label to image data of a display object that is not a character depicted in a drawing;
creating a character recognition model by learning the first learning data;
acquiring image data of a recognition target including characters and non-character objects;
a step of recognizing characters from the image data of the recognition target based on the character recognition model, and outputting the remaining characters after deleting the substitute characters from the recognized characters as a recognition result;
A program that executes the following.

Images