JP2004062517A - Voice controller, voice control method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a convenient user interface enabling inputs to be received from user interfaces for both voice and GUI, thus using them alternatively according to situations. <P>SOLUTION: A voice controller includes a dictionary retaining means 3 for retaining a dictionary in which the rules of converting voice into character strings are stored; a voice recognition means 2 for converting input voice into characters by consulting the dictionary; a command conversion means 4 for converting the recognized character strings into command strings and setting the command strings in the GUI; a GUI supervising mans 5 for supervising the state of the GUI; a coordination data retaining means 6 for retaining coordination data that coordinates the state of the GUI recognized by the GUI supervising means 5 with the method of voice recognition and the dictionary which the voice recognition means 2 consults; and a coordination changing means 7 for reading the coordination data corresponding to the state of the GUI from the coordination data retaining means 6 and changing the method of voice recognition and the dictionary which the voice recognition means 2 consults. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a voice control device, a voice control method, and a program, and more particularly, to a user interface using a voice recognition technology.
[0002]
[Prior art]
As a conventional technique, there is known a technique in which windows and applications are controlled by voice, such as a "computer system" disclosed in Japanese Patent Application Laid-Open No. 10-222337. In this method, control commands for windows and applications are associated with keywords, and the keywords are registered as a dictionary for speech recognition. When the keyword is recognized from the voice input, a control command corresponding to the keyword is executed.
[0003]
In such speech recognition, a common dictionary and a dictionary for each application are prepared. By preparing a dictionary for each application, it becomes possible to assign a different operation to each application even for the same utterance. The dictionary for the application is configured to switch when the target application is activated by voice or when the window is switched.
[0004]
[Problems to be solved by the invention]
In the above-described method, the dictionary for the application is switched according to the recognition result of the input voice, instead of monitoring the state of the GUI. In such a case, if the selection state of the window or application is changed by a method other than voice input such as a mouse or a touch panel, switching to the corresponding dictionary does not occur. Therefore, the correspondence between the selection state of the application or the window and the dictionary for recognition is rubbed, so that there is a problem that it is impossible to continue the control by voice.
[0005]
In general, a user interface using voice has an advantage that, when one of multiple items is selected, a space for displaying a selection candidate and a selection procedure can be reduced. For example, when trying to realize selection from multiple items with a graphical user interface (GUI) such as a pull-down menu, a menu is given a hierarchical structure, and menu display and selection are repeated many times. It takes time to find the corresponding item from.
[0006]
An object of the present invention is to provide a voice control device, a voice control method, and a program that realize an easy-to-use user interface that can be properly used depending on the situation by enabling input from both voice and GUI user interfaces. It is to be.
[0007]
[Means for Solving the Problems]
A first voice control device of the present invention includes a graphical user interface (GUI);
Dictionary holding means for holding a dictionary storing rules for converting voice to character strings; voice recognition means for converting input voice to characters by referring to the dictionary; and a command for inputting a character string recognized by the voice recognition means. A command conversion means for converting into a column and setting the GUI, a GUI monitoring means for monitoring the status of the GUI, and associating the GUI status recognized by the GUI monitoring means with a voice recognition method or dictionary referred to by the voice recognition means. An association data holding unit that holds the association data; and an association change unit that reads the association data corresponding to the state of the GUI from the association data holding unit and switches a voice recognition method or a dictionary that is referred to by the voice recognition unit. It is characterized by the following.
[0008]
The second voice control device of the present invention includes, in the first voice control device, a GUI obtaining means for obtaining identification information of a selected GUI, and a method of setting a recognition result of the voice recognition means in the GUI. A setting method database that holds setting method data, association data that associates identification information of the GUI, a dictionary used when the GUI is selected, and a setting method for the GUI held in the setting method database are generated. An association unit for storing the data in the association data holding unit.
[0009]
A first voice control method according to the present invention includes a voice recognition step of converting a voice string input in reference to a dictionary storing rules for converting a voice into a character string, and a command for converting the character string recognized in the voice recognition step into a command. A command conversion step of converting into a column and setting the same in a graphical user interface (GUI); and a correspondence data holding unit for holding association data for associating a GUI state with a speech recognition method or a dictionary referred to by the speech recognition unit. And an association changing step of switching the dictionary or the voice recognition method referred to in the voice recognition step by reading the association data according to the state of the GUI.
[0010]
A second voice control method according to the present invention, in the first voice control method, comprises: a GUI obtaining step of obtaining identification information of a selected GUI; and a dictionary used when the GUI is selected and the GUI is selected. And a method of generating association data in which the method associates a method of setting a recognition result of the voice recognition step with the GUI and storing the association data in the association data holding unit.
[0011]
According to a first program of the present invention, a speech recognition step of converting a speech string input by referring to a dictionary storing rules for converting speech to a character string, and converting the character string recognized in the speech recognition step into a command string A command conversion step of converting and setting a GUI to a graphical user interface; and a mapping data holding means for holding mapping data for associating a GUI state with a speech recognition method or a dictionary referred to by the speech recognition means. The computer causes the computer to execute an association change step of reading the association data according to the state and switching the speech recognition method or dictionary referred to in the audio recognition step.
[0012]
According to a second program of the present invention, in the first program, a GUI acquisition step of acquiring identification information of a selected GUI, identification information of the GUI, a dictionary used when the GUI is selected, and the voice recognition And causing the computer to execute an associating step of generating association data that associates the method of setting the recognition result of the step with the GUI and storing the association data in the association data holding unit.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, a first embodiment of the present invention will be described in detail with reference to the drawings.
Referring to FIG. 1, a first embodiment of the present invention relates to a GUI 1 including input means such as a keyboard, output means such as a display device, and various graphical user interfaces (GUIs), and a voice input device such as a microphone. Recognizing means 2 for converting the voice data input through the PC into characters, dictionary holding means 3 for holding a dictionary for converting the voice data into characters, and converting the characters converted by the voice recognizing means 2 into commands. Command conversion means 4 to be input to the GUI 1, GUI monitoring means 5 for monitoring the status of the GUI 1, and association data for associating the status of the GUI 1 with a speech recognition method in the speech recognition means 2 and a dictionary to be activated. When the status of the GUI 1 changes, the association data holding unit 6 to be held is associated with the notification from the GUI monitoring unit 5. Load the association data corresponding to GUI1 that is currently active from over data holding means 6, constituted by the association changing unit 7 for switching a dictionary to recognize scheme or an active speech recognition unit 2.
[0014]
The operation of the present embodiment will be described in detail with reference to FIGS.
FIG. 2 is a diagram for explaining the operation of GUI1. The GUI 1 includes a plurality of GUIs corresponding to various interfaces with the user. FIG. 2 shows an example of the GUI, in which windows 21, 22, and 23 are activated on the screen, and the window 21 is activated. It indicates that The window 21 includes a text box 202, a list box 203, and a button 204.
[0015]
The window 21 shows an operation of inputting a station name in a text box 202 with a keyboard, selecting a time from a list box 203, and clicking a button 204 to perform a search.
[0016]
FIG. 3 is an example of data stored in the association data holding unit 6.
The association data stores rules for switching dictionaries according to the active GUI. In the example of FIG. 3, N pieces of association data of data 1 to data N are stored. Each association data includes an active application name, a window name, a GUI name, a dictionary name to be switched, and a command conversion method. The command conversion method means a method of controlling the GUI using the recognition result when the voice is recognized.
[0017]
Referring to FIG. 3, when the text box 202 on the window 21 of the application 1 is activated, the data 1 is switched to the general place name strengthening dictionary 1. When the above is executed, the recognition result is displayed in the text box 202. Means to be inserted into
[0018]
In the data 2, when the list box 203 on the window 21 of the application 1 is activated, the dictionary is switched to the time dictionary 2, and when the above is executed, the recognition result is displayed in the text box 202 according to the recognition result. It means that it can be switched.
[0019]
In the data 3, when the text box 202 and the list box 203 are both inactive in the window 21 of the application 1, the menu is switched to the menu / button dictionary 3, and when the above is executed, the button of the GUI name corresponding to the recognition result or the menu Means to click.
[0020]
The data N means that the application dictionary N is activated in a GUI state other than that specified by the data N, and when the above is activated, the application having the application name corresponding to the recognition result is activated.
[0021]
FIG. 4 is an example of a dictionary stored in the dictionary holding unit 3. FIG. 4 illustrates two examples of a dictionary when the speech recognition method is dictation and a dictionary when the speech recognition method is CFG.
[0022]
FIGS. 4A and 4B are examples of dictation dictionaries. Dictation dictionaries can recognize a relatively wide range of utterances by having word string data to record the reading, notation, part of speech, and appearance frequency of each word, and the appearance frequency between words. Become.
[0023]
FIG. 4A is an example of word string data for recording individual words such as the aforementioned reading, notation, and appearance frequency. As shown in FIG. 4A, the word string data includes a dictionary name, a registered part of speech, a reading, a notation, and a word appearance frequency.
[0024]
In this example, four types of parts of speech, “proper noun place name”, “noun”, “particle”, and “auxiliary verb” are registered in the word string data. As the "proper noun place name", three words having the readings of "Tokyo", "Osaka", and "Kyoto" are registered, and the notation is "Tokyo", "Kyoto", and "Osaka". I have. The frequency of appearance of these words is 2.0.
[0025]
As the "noun", three words "Eki", "Shi", and "Town" are registered, and the notation of each word is "Station", "City", and "Town". The frequency of appearance of these words is 1.0 for "Eki" and 0.5 for "shi" and "machi". Two words “de” and “ha” are registered as “particles”. Two words, “is” and “mas”, are registered as “auxiliary verbs”.
[0026]
The word appearance frequency is the ease of appearance for each word, and means that the larger the value, the higher the probability of appearance as a recognition result. In the above example, proper nouns are most likely to appear among the four parts of speech. Also, in nouns, "eki" means that it appears more easily than "machi" or "shi".
[0027]
FIG. 4B is a table showing the frequency of appearance between dictation parts of speech. In this table, the appearance frequency of “proper noun place name” and “noun” is 1.2, and the appearance frequency of auxiliary verbs and particles is 0.1. This indicates that the combination of “proper noun place name” and “place name” appears more easily than the combination of “auxiliary verb” and “particle”.
[0028]
FIGS. 4C and 4D are examples of a dictionary for CFG. In a dictionary for CFG, a word string and a word string are described in combination.
FIG. 4C is an example of a word string. This column shows an example of registration of three types of word strings, “time”, “prefix”, and “end”. Each word string is composed of a reading and a notation. In the word string "time", four words are registered, and each reading is "Rokuji", "Gozenkuro", "Asakuroku", "Asanokuroku", and the corresponding notation is " 6:00 ”.
[0029]
FIG. 4D shows an example of the grammar. The grammar is defined by a permutation of a recognizable word string. In this example, a permutation of three word strings is shown, and the grammar 1 indicates that a combination of the word string “prefix”, the word string “time”, and the word string “end” can be recognized. By defining, "Oh, it's Asano lottery" can be recognized. Similarly, grammar 2 indicates that a combination of the word string “prefix” and the word string “time” can be recognized, and by defining such a combination, it is possible to recognize “um, asanokuroku”. Can be. The grammar 3 indicates that a combination of the word string “time” and the word string “end” can be recognized. By defining such a combination, “please enter lottery” can be recognized.
[0030]
FIG. 5 is a flowchart for explaining the operation when the GUI is changed. The case where the active GUI of the window 21 is changed from the text box 202 to the list box 203 in the same window 21 will be described using the above specific example.
[0031]
The GUI monitoring means 5 compares the stored information of the GUI with the information of the currently active GUI, and if different, determines that the active GUI has been changed. The GUI information is described by an application name, a window name, a GUI name, and a GUI identification number. The identification number of the GUI is a number for identifying the running GUI, and if this value is recorded, it can be understood that the GUI has been changed even if there is a GUI having the same window name and the same GUI name. If there is no change in the GUI, the process ends (step A1).
[0032]
If the GUI has been changed, the current GUI information is sent to the GUI monitoring means 5 and the association changing means 7 and updated (step A2).
[0033]
In response to this, the association changing unit 7 reads the corresponding data from the association data holding unit 6 (step A3). In the above-described example, the data 2 as the association data corresponding to the case where the application name is the application 1, the window name is the window 21, and the GUI name is the list box 203 is read from the association data holding unit 6.
[0034]
The association changing unit 7 sends a dictionary change command to the voice recognition unit 2. In the dictionary change instruction, the dictionary name to be switched is also sent. In response to this, the voice recognition means 2 switches the dictionary (step A4). In the above-described example, the voice recognition unit 2 changes the time dictionary 2 of the data 2 to be selected.
[0035]
Further, the association changing unit 7 sends the command conversion method described in the association data read in step A3 to the command conversion unit 4, and the command conversion unit 4 changes the command conversion method (step A5). In the example described above, the command conversion method of the data 2 is sent to the command conversion means 4. Thus, when a value is sent from the voice recognition unit 2 as a recognition result, the command conversion unit 4 changes the display to the same name as the value.
[0036]
FIG. 6 is a flowchart for explaining an operation when a voice is input to the voice recognition unit 2. The case where the text box 202 is active according to the above-described example will be described as an example.
[0037]
When a voice is input to the voice recognition means 2, voice recognition is performed using the currently selected dictionary (step B1). If the result of the speech recognition does not yield a recognition result, the process ends.
[0038]
When the voice is recognized, the "reading" closest to the input voice is returned as a recognition result (step B2). Since a value corresponding to reading has been set, this value is obtained (step B3). As described above with reference to FIG. 4, when the dictionary being selected is the general place name strengthening dictionary 1 and the pronunciation is “Tokyo”, “Tokyo Station” is acquired as the value.
[0039]
The command conversion unit 4 reads the association data corresponding to the current GUI state through the association change unit 7 in advance. In response to the above value, the GUI is set to the GUI setting method described in the association data (step B4). To explain according to the above-described example, when the text box 202 is active, the data 1 is read into the command conversion unit 4. When the value acquired in step A3 is “Tokyo”, the command conversion unit 4 sets “Tokyo” in the text box 202 according to the data 1.
[0040]
Next, a second embodiment of the present invention will be described with reference to the drawings. The second embodiment relates to the generation of the association data in the first embodiment.
[0041]
As shown in FIG. 7, the configuration of the second embodiment of the present invention comprises a GUI 1, a GUI acquisition unit 702 for acquiring identification information of a GUI being selected, a dictionary holding unit 3 for speech recognition, A setting method database 703 storing a method of setting a result recognized by using a grammar in a GUI; a GUI identification information acquired by the above-described GUI acquiring means 702; a dictionary and a GUI used when the above-mentioned GUI is selected; And an associating data holding unit 6 that stores the created associating data.
[0042]
Next, the operation of the second embodiment will be described based on an example used in the first embodiment. FIG. 8 is a flowchart for explaining the operation of the second embodiment. First, an application and a window including a GUI for which association data is to be created are activated, and the target GUI is activated (step C1).
[0043]
As a specific example of the operation of the following flowchart, a method of creating association data corresponding to the text box 202, the list box 203, and the button 204 in the window 21 shown in FIG. 2 will be described. Upon receiving step C1, GUI identification information is obtained (step C2).
[0044]
FIG. 9 is a diagram for explaining the identification information of the GUI. The GUI identification information includes a window name, a GUI name, and a GUI type. When the text box 202 is activated, it means that the window name is the window 21, the GUI name is the text box 202, and the GUI type is the text box.
[0045]
Upon receiving the identification information of the GUI in step C2, the setting method data is acquired from the setting method database 703 (step C3). The setting method is a method of setting the result of the voice recognition means 2 for each type of GUI. The setting method data includes a setting method name, a GUI type to be set, and a setting method for the GUI.
[0046]
FIG. 10 shows an example of the setting method database 703. The setting method database 703 stores three setting method data from setting method 1 to setting method 3. In the setting method 1, when the type of the GUI to be set is a text box, the result of the voice recognition unit 2 is set in the text box. In the setting method 2, when the type of the GUI to be set is a list box, a notation that matches the result of the voice recognition means 2 is selected from the list and displayed. In the setting method 3, when the type of the GUI to be set is a button or a menu, a button or a menu that matches the result of the voice recognition unit 2 is to be executed. For example, when the GUI identified in step C2 of FIG. 9 is a text box type, the setting method 1 is selected.
[0047]
Next, a dictionary corresponding to the selected GUI is selected from the dictionary holding means 3 (step C4). The dictionary holding means 3 stores a plurality of dictionaries created in advance.
[0048]
Finally, the identification information of the GUI acquired in step C2, the setting method acquired in step C3, and the dictionary set in step C4 are combined as association data, and stored in the association data holding unit 6.
[0049]
【The invention's effect】
A first effect is that the input GUI can be controlled by voice so that the GUI can be controlled without contacting an input device such as a keyboard, a mouse, and a touch panel.
[0050]
The second effect is that a voice input function can be added to an existing application by monitoring all GUIs and having a program separately from the application to be monitored.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a first embodiment of the present invention.
FIG. 2 is an example of a GUI for describing an operation of the first exemplary embodiment of the present invention.
FIG. 3 is an example of data stored in association data 6 for explaining the operation of the first exemplary embodiment of the present invention.
FIG. 4 is an example of a dictionary stored in the dictionary holding means 3 for explaining the operation of the first exemplary embodiment of the present invention.
FIG. 5 is a flowchart for explaining the operation of the first exemplary embodiment of the present invention.
FIG. 6 is a flowchart for explaining the operation of the first exemplary embodiment of the present invention.
FIG. 7 is a configuration diagram of a second embodiment of the present invention.
FIG. 8 is a flowchart for explaining the operation of the second exemplary embodiment of the present invention.
FIG. 9 is an example of information of a GUI under selection for explaining the operation of the second exemplary embodiment of the present invention.
FIG. 10 is an example of data stored in a setting method database 703 for explaining the operation of the second exemplary embodiment of the present invention.
[Explanation of symbols]
1 GUI
2 voice recognition means 3 dictionary holding means 4 command conversion means 5 GUI monitoring means 6 association data holding means 7 association changing means 21, 22, 23 window 202 text box 203 list box 204 button 702 GUI acquisition means 703 setting method database 704 Correlation means

Claims (6)

A graphical user interface (GUI);
Dictionary holding means for holding a dictionary storing rules for converting speech to character strings;
Voice recognition means for converting the input voice to characters by referring to the dictionary,
Command conversion means for converting the character string recognized by the voice recognition means into a command string and setting the same in a GUI;
GUI monitoring means for monitoring the status of the GUI;
Association data holding means for holding association data for associating a state of the GUI recognized by the GUI monitoring means with a speech recognition method or dictionary referred to by the speech recognition means;
A voice control device comprising: an association changing unit that reads association data corresponding to a state of a GUI from an association data holding unit and switches a speech recognition method or a dictionary to be referred to by a speech recognition unit. GUI acquisition means for acquiring identification information of the selected GUI, a setting method database holding setting method data including a method of setting the recognition result of the voice recognition means in the GUI, GUI identification information and the GUI. An association unit that generates association data in which a dictionary used when selected and a setting method for a GUI held in a setting method database are associated, and stores the association data in the association data holding unit. The voice control device according to claim 1. A voice recognition step for converting a voice string into a voice character input by referring to a dictionary storing rules for converting a voice into a character string; and converting a character string recognized in the voice recognition step into a command string to a graphical user interface (GUI). A command conversion step to be set, and reading of association data according to the state of the GUI from an association data holding unit that holds association data for associating a GUI state with a speech recognition method or a dictionary referred to by the speech recognition unit. A voice recognition method, comprising: changing a voice recognition method and a dictionary to be referred to in the voice recognition step. A GUI acquisition step of acquiring identification information of the selected GUI, a method of setting identification information of the GUI and a dictionary used when the GUI is selected, and a method of setting the recognition result of the voice recognition step to the GUI. 4. An audio control method according to claim 3, further comprising an association step of generating association data and storing the association data in the association data holding unit. A voice recognition step for converting a voice string into a voice character input by referring to a dictionary storing rules for converting a voice into a character string; and converting a character string recognized in the voice recognition step into a command string to a graphical user interface (GUI). A command conversion step to be set, and reading of association data according to the state of the GUI from an association data holding unit that holds association data for associating a GUI state with a speech recognition method or a dictionary referred to by the speech recognition unit. A program for causing a computer to execute an associating change step of switching a speech recognition method or a dictionary referred to in the speech recognition step. A GUI acquisition step of acquiring identification information of the selected GUI, a method of setting identification information of the GUI and a dictionary used when the GUI is selected, and a method of setting the recognition result of the voice recognition step to the GUI. 6. The computer program according to claim 5, further comprising the step of: generating an association data and storing the association data in the association data holding means.

Images