JP2005258577A - Character input device, character input method, character input program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a character input device which improves a character recognition rate even for a distinctive hand-written character, and also provide a character input method, a character input program and a recording medium. <P>SOLUTION: The character input method has the following steps: a display step which displays a hand-written character on a tablet; a conversion step which converts the string of hand-written characters to a character string and replaces the former with the latter; a voice input step which acquires voice data inputted corresponding a prescribed character string among character strings; a voice recognition step which performs voice recognition and converts the voice data to a character string; a character string replacement step which make replacement with the character string obtained by the result of the voice recognition; a character dictionary updating step which corrects dictionary data for character recognition based on the replaced character string and the string of the hand-written characters; and a voice dictionary updating step which corrects dictionary data for voice recognition based on the replaced character string and the voice data corresponding to the replaced character string. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technology for recognizing handwritten characters, and more particularly to a technology for increasing the recognition rate of handwritten characters in combination with audio information.

  In tablet PCs, handwritten character recognition is generally used. However, if there is a misrecognition of a character, it is necessary to rewrite the character and input it.However, if there is a flaw in the character, a misrecognition will occur repeatedly, so it is necessary to input it repeatedly. It was.

In order to solve this problem, there has been proposed a method of correcting misrecognition of handwritten characters based on the result of speech recognition by inputting speech together with writing. According to this method, since the erroneous recognition of handwritten characters is corrected by voice, an effect of improving the efficiency of the correction work can be obtained. (For example, refer to Patent Document 1 and Patent Document 2).
JP-A-6-52240 JP 2002-259908 A

  However, it is considered that handwritten characters of the same font will be misrecognized many times thereafter, so even if the inventions described in Patent Literatures 1 and 2 are used, the operation of recognizing and correcting each time is repeated. It will be.

  The present invention has been made in view of such circumstances, and a character input device and a character input method capable of increasing the recognition rate while using even a handwritten character having a habit that causes erroneous recognition. An object of the present invention is to provide a character input program and a recording medium.

  The character input device according to claim 1 according to the present invention for solving the above-described problem is a character input device that converts a character handwritten with a pen into a character character on the tablet. Display means for displaying on the display, and a conversion means for sequentially converting the input handwritten character string into a character character string based on the dictionary data for character recognition, and replacing the handwritten character displayed on the tablet with the character character; Selection display means for distinguishing and displaying a predetermined character string selected from among character strings converted by the conversion means and displayed on the tablet; and predetermined character characters displayed by the selection display means Voice input means for acquiring voice data input corresponding to the sequence; and voice authentication for the voice data acquired by the voice input means. Voice recognition means for voice recognition based on dictionary data for use and conversion to a character string, and a character string for replacing the character string on the tablet displayed by the selection display means with the character string obtained as a result of the voice recognition Based on the replacement means, the character string replaced by the character string replacement means, and the handwritten character string before the conversion means corresponding to the character string is replaced with the character character string, the dictionary data for character recognition is obtained. A voice for correcting the dictionary data for speech recognition based on the character dictionary updating means to be corrected, the character string replaced by the character string replacing means, and the voice data acquired by the voice input means corresponding to the character string Dictionary updating means.

  According to a second aspect of the present invention, there is provided the character input device according to the first aspect, wherein the selection display means selectively displays a predetermined character string in units of words or phrases.

  The character input device according to claim 3 of the present invention is the character input device according to the invention described above, wherein the selection display means has a selection range of the character character string selected on the tablet as a word or a phrase. If not, the selection range is automatically changed to be a word or a phrase.

  Moreover, the character input method according to claim 4 according to the present invention is a character input method for converting a character handwritten with a pen into a character character into a character character, and displaying the input handwritten character on the tablet, The input handwritten character string is sequentially converted into a character character string based on the dictionary data for character recognition, and the conversion step replaces the handwritten character displayed on the tablet with the character character. A selection display step for distinguishing and displaying a predetermined character character string selected from the character character strings displayed on the tablet, and an input corresponding to the predetermined character character string displayed in the selection display step A voice input step for acquiring the recorded voice data, and the voice data acquired by the voice input step. A voice recognition step for voice recognition based on dictionary data for voice recognition and converting it into a character string, and a character string on the tablet displayed in the selection display step in the character string obtained as a result of the voice recognition A character string replacement step for replacing the character string, the character string replaced in the character string replacement step, and the handwritten character string before the conversion means corresponding to the character string replaces the character string. The dictionary data for speech recognition based on the character dictionary updated step for correcting the dictionary data, the character string replaced in the character string replacement step, and the voice data acquired in the voice input step corresponding to the character string A speech dictionary updating step for correcting

  The character input program according to claim 5 of the present invention is a character input program for converting a character handwritten with a pen into a character character into a character character, and a display step for displaying the input handwritten character on the tablet. The input handwritten character string is sequentially converted into a character character string based on dictionary data for character recognition, and the conversion step replaces the handwritten character displayed on the tablet with the character character, and is converted in the conversion step. A selection display step for distinguishing and displaying a predetermined character character string selected from the character character strings displayed on the tablet, and an input corresponding to the predetermined character character string displayed in the selection display step A voice input step for acquiring the recorded voice data, and the voice input step A voice recognition step for voice recognition based on dictionary data for voice recognition to convert the voice data into a character string, and a character string obtained as a result of the voice recognition on the tablet displayed in the selection display step. Based on the character string replacement step for replacing the character string, the character string replaced in the character string replacement step, and the handwritten character string before the conversion means corresponding to the character string replaces the character string Based on the character dictionary update step of correcting the dictionary data for character recognition, the character string replaced in the character string replacement step, and the voice data acquired in the voice input step corresponding to the character string, the voice recognition And a voice dictionary update step of correcting the dictionary data.

  According to a sixth aspect of the present invention, there is provided a recording medium according to the present invention, wherein the input handwritten character is displayed on the tablet in a recording medium in which a character input program for converting a character input by handwriting on the tablet to a character character is recorded. A conversion step of sequentially converting the input handwritten character string into a character character string based on dictionary data for character recognition, replacing the handwritten character displayed on the tablet with the character character, and the conversion A selection display step for distinguishing and displaying a predetermined character string selected from the character character strings converted in step and displayed on the tablet; and the predetermined character string displayed in the selection display step. A voice input step for acquiring correspondingly input voice data; and the voice input step. A voice recognition step for voice recognition based on dictionary data for voice recognition to convert the voice data acquired by the voice recognition into a character string, and a tablet displayed in the selection display step with the character string obtained as a result of the voice recognition. Based on the character string replacement step for replacing the character string above, the character string replaced in the character string replacement step, and the string of the handwritten characters before the conversion means corresponding to the character string replaces the character string Based on the character dictionary updated step for correcting the dictionary data for character recognition, the character string replaced in the character string replacement step, and the voice data acquired in the voice input step corresponding to the character string. A voice dictionary updating step for correcting dictionary data for recognition, and a program for causing a computer to execute It was recorded.

  According to the present invention, it is possible to increase the recognition rate while using even a handwritten character having a habit that causes erroneous recognition.

[First Embodiment]
FIG. 1 is a diagram showing an appearance of an input terminal to which the character input method according to the first embodiment of the present invention is applied.

  The input terminal 1 includes a tablet 2, a pen 3, and a microphone 4. The tablet 2 is an input area for inputting handwritten characters, and is a display area for recognizing input characters and displaying them as characters (characters). The pen 3 is an input operation tool for handwriting characters on the display unit 2. The microphone 4 is an input means for inputting to the input terminal 1 by voice.

  Next, an outline of the character input method according to the first embodiment will be described.

  When a user handwrites a character on the tablet 2 using the pen 3, the handwritten character 5 is recognized and displayed as a type 6. When the user designates the area 7 of the character to be corrected using the pen 3, the area 7 is displayed in reverse video. Then, when the user inputs the characters in the region 7 from the microphone 4 by voice, the characters in the region 7 are replaced with the characters of the voice recognition result.

  When the user operates the save button 8 displayed on the tablet 2, the entire character-recognized document is saved. When the user operates the clear button 9 displayed on the tablet 2, the highlighted area 7 is erased.

  FIG. 2 is a diagram illustrating a configuration of the input terminal 1 to which the character input method according to the first embodiment of this invention is applied.

  The input terminal 1 includes a display unit 11, a pen input unit 12, a voice input unit 13, a character recognition unit 14, a voice recognition unit 15, a processing unit 16, a program memory 17 and an information memory 18.

  The display unit 11 is an interface that exchanges display data with the tablet 2. The pen input unit 12 is an interface for receiving input data operated with the pen 3. The voice input unit 13 is an interface for receiving voice data input from the microphone 4.

  The character recognition unit 14 recognizes a handwritten character input by the user. The voice recognition unit 15 recognizes voice input by the user. The processing unit 16 comprehensively controls the operation of each unit of the input terminal 1. The program memory 17 stores a program that operates in the processing unit 16. The information memory 18 stores data used by each program operating in the processing unit 16.

  FIG. 3 is a diagram illustrating a configuration of the character recognition unit 14. The character recognition unit 14 includes a graphic processing unit 21, a recognition unit 22, a character string correction unit 23, a character recognition user dictionary 24, a character recognition dictionary 25, and a language analysis dictionary 26.

  The graphic processing unit 21 executes image processing for handwritten characters such as rotation and size adjustment of written character strings, thinning, character segmentation, and stroke decomposition. The recognition unit 22 performs character recognition based on the stroke feature. The character string correcting unit 23 corrects the character string based on the word, phrase, and continuous string information.

  The character recognition user dictionary 24 stores information related to character stroke characteristics written by the user. The character recognition dictionary 22 stores information related to standard character stroke characteristics. The language analysis dictionary 26 stores information related to word, phrase, and series information of phrases.

  FIG. 4 is a diagram illustrating a configuration of the voice recognition unit 15. The speech recognition unit 15 includes a speech processing unit 31, a model generation unit 32, a recognition unit 33, a character string correction unit 34, a speech recognition user dictionary 35, a speech recognition dictionary 36, an acoustic model dictionary 37, and a language analysis dictionary 38. .

  The audio processing unit 31 performs processing such as noise removal, filter processing, and extraction of audio features from the input audio signal. The model generation unit 32 generates a comparative model to be recognized based on the pronunciation information and the sound information about the sound source and the series of sounds. The recognition unit 33 recognizes the input speech for each vocabulary based on the speech features of the input speech and the speech features of the comparison model, and converts them into character strings. The character string correcting unit 34 corrects the character string based on the word, phrase, and series of phrase information.

  The speech recognition user dictionary 35 stores information related to the user's pronunciation characteristics. The speech recognition dictionary 36 stores information related to the speech of the recognition target vocabulary. The acoustic model dictionary 37 stores information related to speech features in units of phonemes. The language analysis dictionary 38 stores information about words, phrases, and series information of phrases.

  FIG. 5 is a flowchart showing a schematic procedure of the character input method.

  In step S01, the process waits until an input operation with the pen 3 is performed. And in step S02-S03, when the writing to the tablet 2 is detected, a character recognition process (FIG. 6) is performed.

  In steps T01 to T02 of FIG. 6, the graphic processing unit 21 normalizes the inclination and size of the written character string (normalization of the writing pattern), detects a character break, and cuts out a written character pattern for each character. (Character separator detection). In steps T03 to T04, the recognition unit 22 decomposes each character of the handwritten character pattern into strokes (images) (stroke decomposition), the number of strokes constituting one character, and the position, direction, and length of each stroke. Based on the above, character candidates are selected (character shape determination). Next, in steps T05 to T06, the character string correcting unit 23 determines whether or not a character string composed of candidate characters before and after the recognition target character and already recognized characters can form a word or a phrase. (Character string determination), the candidate character that constitutes the most correct character string in the character string determination is determined as a recognition result (character determination), and the process returns.

  The result of character recognition is displayed as a type on the tablet 2 via the display unit 11. When the user confirms the displayed character recognition result and determines that the character is recognized correctly, the user executes the next handwritten character input operation. If it is determined that it is erroneous recognition, the area 7 of the erroneously recognized character string is selected by specifying with the pen 3.

  In step S04 to S06 in FIG. 5, when the type character string part is selected as erroneous recognition and the clear button 9 is selected, the selected character string part is deleted and the next input operation is performed. It becomes a waiting state to wait. On the other hand, when the type character string portion is selected as being erroneous recognition in steps S04 to S05, and voice is input, the correction process in step S07 (FIG. 7) is executed.

  In steps T10 to T11 in FIG. 7, the character string correction unit 34 checks whether or not the selected character string is a boundary between phrases (or words). The selected character string is enlarged to make a new character string a selection range. For example, if the user mistakenly selects “I am a person” for the character string “I am,” the selection range is expanded and “I am a person” becomes a new selected character string. In this embodiment, correction in phrase units is described, but word units may be used.

  In steps T12 to T13, the voice recognition process (FIG. 8) is executed based on the user's voice input detected via the voice input unit 13.

  In steps P01 to P02 of FIG. 8, the sound processing unit 31 performs sound processing such as noise and distortion removal of input sound and Fourier transform (voice analysis), and for example, sounds such as cepstrum features from the processed signals. Extract feature values (voice feature extraction). In steps P03 to P04, the model generation unit 32 reads the pronunciation information and acoustic information to be recognized from the voice recognition dictionary, the voice recognition user dictionary, and the acoustic model dictionary, and generates a model voice for comparison (model voice). Generation), the recognition unit 33 compares the feature quantity of the input voice and the feature quantity of the model voice, and selects a character string corresponding to the model voice with high similarity as a candidate (candidate character string generation). Next, in steps P05 to P06, the character string correction unit 34 determines whether or not the character string composed of the selected candidate character string and the already recognized characters can form a word, a phrase, or a sentence. (Character string determination), the candidate character that constitutes the most correct character string in the character string determination is determined as a recognition result (character determination) and the process returns.

  In step T14 of FIG. 7, the selected character string is replaced with the recognized result and displayed on the tablet 2. When the replaced character string portion is selected again by the user in step T15, steps T10 to T14 are repeated again. Execute. When the replaced character string portion is not selected by the user, the correction process is terminated and the process returns.

  After the character that has been misrecognized in character recognition is corrected by voice recognition, a learning process (FIG. 9) is executed in step S08 of FIG. In this learning process, learning is performed so that misrecognized handwritten characters can be recognized correctly thereafter.

  In step T21 of FIG. 9, the handwritten character of the character string selected as the correction portion, the character string of the recognition result after correction, and the handwritten pattern of the character string are stored in the information memory 18. In step T22, the voice input corresponding to the character string selected as the correction portion and the character string of the recognition result after the correction are stored in the information memory 18.

  Subsequently, in step T23, when the average CPU usage rate is equal to or less than a predetermined value, the following process is executed. In step T24, the user's handwritten character stored in the information memory 18 is decomposed in association with the reference handwritten character pattern registered in the character recognition dictionary. That is, the number of strokes of characters written by the user and the position, direction, and length of each stroke are extracted. Then, the extraction result is registered in the character recognition user dictionary 24 in association with the corrected recognition result character stored in the information memory 18.

  As a result of this processing, the user's handwritten font is registered in the user dictionary, and thereafter, when the same font character is input, the correct candidate characters are extracted by referring to the character recognition user dictionary 24. And the recognition rate can be increased.

  Next, in step T25, the pronunciation information and the acoustic information obtained from the user's voice input stored in the information memory 18 are registered in the voice recognition user dictionary 35 in association with the corrected recognition result characters.

  By this process, the model voice for comparison can be approximated by the voice of the user thereafter, so that the probability of extracting correct candidate characters is increased and the recognition rate can be increased. Furthermore, if the speech input recognition rate can be increased by this processing, the character recognition rate according to the present invention can be increased as a result.

  In order to increase the speech recognition rate more positively, if the speech recognition itself is misrecognized during the process of correcting the character recognition result, the character string selected for correction is deleted by selecting Clear and is recognized correctly. The handwriting input and the recognition may be repeated until it is recognized, and the voice and the correct character string may be stored in the information memory 18 when the processing proceeds to the next handwriting input after the recognition.

  After the learning process in step S08 in FIG. 5 is executed, a standby state is waited for the next input operation in step S01.

  In steps S10 to S11, when the save button 8 of the tablet 2 is operated, the character-recognized document is saved and the character input process is terminated.

  According to the present embodiment, in order to store the handwritten character pattern of the user who caused the misrecognition and the correct character in association with each other in the character recognition dictionary data, when the same handwritten character of the same font is input again, the character recognition dictionary Characters that are correctly recognized can be extracted by referring to the data.

  In addition, since the user's voice input that is input in association with the occurrence of the erroneous recognition is associated with the correct character and stored in the voice recognition data, the voice recognition data is referred to when the user's voice is input. Accordingly, it is possible to extract characters that are correctly recognized in consideration of the pronunciation of the user.

  Each function described in the above embodiment may be configured using hardware, or may be realized by reading a program describing each function into a computer using software. Each function may be configured by appropriately selecting either software or hardware.

  Furthermore, each function can be realized by causing a computer to read a program stored in a recording medium (not shown). Here, as long as the recording medium in the present embodiment can record a program and can be read by a computer, the recording format may be any form.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

The figure which shows the external appearance of the input terminal to which the character input method of the 1st Embodiment of this invention is applied. The figure which shows the structure of the input terminal to which the character input method of the 1st Embodiment of this invention is applied. The figure which shows the structure of a character recognition part. The figure which shows the structure of a speech recognition part. The flowchart which shows the procedure of the outline of a character input method. The flowchart which shows the general | schematic procedure of a character recognition process. The flowchart which shows the procedure of the outline of a correction process. The flowchart which shows the procedure of the outline of a speech recognition process. The flowchart which shows the outline procedure of a learning process process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Input terminal, 2 ... Tablet, 3 ... Pen, 4 ... Microphone, 5 ... Handwritten character, 6 ... Type, 7 ... Area | region, 14 ... Character recognition part, 15 ... Voice recognition part, 16 ... Processing part, 22 ... Recognition , 23 ... character string correction unit, 24 ... character recognition user dictionary, 25 ... character recognition dictionary, 33 ... recognition unit, 34 ... character string correction unit, 35 ... voice recognition user dictionary, 36 ... voice recognition dictionary.

Claims (6)

In a character input device that converts characters handwritten with a pen to a tablet into character characters,
Display means for displaying the input handwritten character on a tablet;
Conversion means for sequentially converting the input handwritten character string into a character character string based on dictionary data for character recognition, and replacing the handwritten character displayed on the tablet with the character character;
Selection display means for distinguishing and displaying a predetermined character string selected from among character strings converted by the conversion means and displayed on the tablet;
Voice input means for acquiring voice data input corresponding to the predetermined character string displayed by the selection display means;
Voice recognition means for voice recognition based on dictionary data for voice recognition and converting the voice data acquired by the voice input means into a character string;
Character string replacement means for replacing the character string on the tablet displayed by the selection display means with the character string obtained as a result of the speech recognition;
A character dictionary for correcting the dictionary data for character recognition based on the character string replaced by the character string replacement unit and the string of handwritten characters before the conversion unit corresponding to the character string replaces the character string Update means;
A voice dictionary updating means for correcting the dictionary data for speech recognition based on the character string replaced by the character string replacing means and the voice data acquired by the voice input means corresponding to the character string;
A character input device comprising:   2. The character input device according to claim 1, wherein the selection display means selectively displays a predetermined character string in units of words or phrases.   When the selection range of the character string selected on the tablet is not a word or a phrase, the selection display means automatically changes the selection range to be a word or a phrase. The character input device according to claim 1 or 2. In a character input method for converting a character handwritten with a pen into a character character into a tablet,
A display step of displaying the input handwritten character on a tablet;
A conversion step of sequentially converting the input handwritten character string into a character character string based on dictionary data for character recognition, and replacing the handwritten character displayed on the tablet with a character character;
A selection display step for distinguishing and displaying a predetermined character string selected from among the character strings converted in the conversion step and displayed on the tablet;
A voice input step for acquiring voice data input corresponding to the predetermined character string displayed in the selection display step;
A voice recognition step of converting the voice data acquired in the voice input step into a character string by voice recognition based on dictionary data for voice recognition;
A character string replacing step of replacing the character character string on the tablet displayed in the selection display step with the character string obtained as a result of the speech recognition;
A character dictionary that corrects the dictionary data for character recognition based on the character string replaced in the character string replacement step and the handwritten character string before the conversion means corresponding to the character string replaces the character string. An update step;
A voice dictionary update step of correcting the dictionary data for voice recognition based on the character string replaced in the character string replacement step and the voice data acquired in the voice input step corresponding to the character string;
The character input method characterized by having. In a character input program that converts characters handwritten on the tablet with a pen to character characters,
A display step of displaying the input handwritten character on a tablet;
A conversion step of sequentially converting the input handwritten character string into a character character string based on dictionary data for character recognition, and replacing the handwritten character displayed on the tablet with a character character;
A selection display step for distinguishing and displaying a predetermined character string selected from among the character strings converted in the conversion step and displayed on the tablet;
A voice input step for acquiring voice data input corresponding to the predetermined character string displayed in the selection display step;
A voice recognition step of converting the voice data acquired in the voice input step into a character string by voice recognition based on dictionary data for voice recognition;
A character string replacing step of replacing the character character string on the tablet displayed in the selection display step with the character string obtained as a result of the speech recognition;
A character dictionary that corrects the dictionary data for character recognition based on the character string replaced in the character string replacement step and the handwritten character string before the conversion means corresponding to the character string replaces the character string An update step;
A voice dictionary update step of correcting the dictionary data for voice recognition based on the character string replaced in the character string replacement step and the voice data acquired in the voice input step corresponding to the character string;
A program that causes a computer to execute. In a recording medium on which a character input program for converting a character handwritten with a pen to a tablet into a character character is recorded,
A display step of displaying the input handwritten character on a tablet;
A conversion step of sequentially converting the input handwritten character string into a character character string based on dictionary data for character recognition, and replacing the handwritten character displayed on the tablet with a character character;
A selection display step for distinguishing and displaying a predetermined character string selected from among the character strings converted in the conversion step and displayed on the tablet;
A voice input step of acquiring voice data input corresponding to the predetermined character string displayed in the selection display step;
A voice recognition step of converting the voice data acquired in the voice input step into a character string by voice recognition based on dictionary data for voice recognition;
A character string replacing step of replacing the character character string on the tablet displayed in the selection display step with the character string obtained as a result of the speech recognition;
A character dictionary that corrects the dictionary data for character recognition based on the character string replaced in the character string replacement step and the handwritten character string before the conversion means corresponding to the character string replaces the character string An update step;
A voice dictionary update step of correcting the dictionary data for voice recognition based on the character string replaced in the character string replacement step and the voice data acquired in the voice input step corresponding to the character string;
A recording medium on which a program for causing a computer to execute is recorded.

Images