EP1417678A1

EP1417678A1 - Method and device for recognising a phonetic sound sequence or character sequence

Info

Publication number: EP1417678A1
Application number: EP01974180A
Authority: EP
Inventors: Hans Geiger
Original assignee: Knittel Jochen
Current assignee: KEEP GMBH
Priority date: 2001-08-13
Filing date: 2001-08-13
Publication date: 2004-05-12
Also published as: US20040199389A1; US7966177B2; WO2003017252A1

Abstract

The invention relates to a method for recognising a phonetic sound sequence or a character sequence, e.g. according to the ASCII standard, comprising the following steps: a) the sequence is fed to a neuronal network, b) in said neuronal network, a sequence of characteristics is formed from the phonetic sequence or character sequence, by taking into consideration phonetic and/or lexical stored information and/or based on a character string sequence (blank characters), c) the character sequence thus formed is compared with a characteristic combination (H) that has a defined statement content, said combination being formed from stored lexical and semantic information, based on the characteristic sequence, d) step c) is repeated using new character combinations until, by the reduction of contradictions, a character combination is found that at least largely corresponds with the character sequence, e) the statement content of the character combination with the least number of contradictions is output as the result and/or an action assigned to the statement content is carried out.

Description

METHOD AND DEVICE FOR DETECTING A PHONETIC VOLUME OR STRING

The present invention relates to a method and an apparatus for recognizing a sequence of phonetic sounds or characters, e.g. a string of characters according to the ASC Il standard.

In many computer applications, control commands are entered in a predetermined manner, which is defined by the command language of the computer software. Entries that do not exactly correspond to the specified command sets are simply not recognized and processed. The input can be made either by entering a character string on the keyboard or by speaking a control command into a microphone that is connected to a speech recognition system. Regardless of what type of control command is entered, it is in any case necessary that the entered or recognized control command corresponds exactly to a control command specified by the computer software.

US Pat. No. 5,737,485 shows a device for speech recognition, in which feature extras are provided both behind a microphone for short-range communication and behind a microphone arrangement for room detection, which form a feature sequence from a phonetic sound sequence. The feature sequences of the proximity and spatial detection are fed to a neural network, which is intended to recognize correlations between disturbed features and largely trouble-free features in a learning phase and later replace speech-impaired features with largely trouble-free features in accordance with the knowledge learned, before the Features of a speech recognition are supplied. The disadvantage of this device is that _. Inability of the system to react dynamically to changes in sound generation. Detection continues indistinctly spoken words are not improved by the system.

DE 198 04 603 shows a speech recognition system that generates a feature sequence in the form of test signals from a phonetic sequence. These test signals are then compared with lexical knowledge in the form of reference signals, with sequences of reference signals representing words that are obtained from a training signal. Speech recognition is to be improved in that the features in the form of the test signals are not only compared with the vocabulary learned in this way but also with learned combinations of sentence elements that are more likely to occur. The accuracy of the

I use sprack detection for unclearly pronounced words

[not improved by this.

It is an object of the present invention to provide a method and a device which allow input of unformatted speech or commands, which enable improved recognition of the speech or commands even if they are spoken indistinctly, in a changed manner or pronounced by different people become.

The invention is achieved by a method according to claim 1 and by an apparatus according to claim 9. Advantageous developments of the invention are the subject of the associated subclaims.

According to the invention, the input character string or phonetic sound sequence is fed to a neural network in which the character string is broken down into individual features in the character sequence either based on phonetic, semantic and / or lexically stored information or merely based on separating marks (e.g. spaces, speech pause). This Characteristics can be short strings / words or certain sounds and combinations of sounds. Based on this sequence of features, the neural network compiles combinations of features that are combined with lexical, semantic and / or phonetic information, taking into account the information from a lexicon, in such a way that they have a defined meaningfulness. The neural network forms many of these combinations of features and compares them in the form of time-coded neurons with the sequence of features. Finally, the meaningfulness of the combination of features is used which, while minimizing the contradictions, largely corresponds to the sequence of features. The defined information content can then, if necessary, be transformed into an executable command according to a predefined command set of computer software, which causes an action such as output, query, etc.

If the sequential comparison of the feature sequence with the combination of features established by the neural network does not lead to a satisfactory result, it can be attempted to leave parts of the feature sequence that showed a relatively good agreement with a combination of features unchanged and, for example, those parts with the aid of fuzzy logic to change the feature sequence that showed little or no agreement with the combinations of features.

If a satisfactory result is not achieved overall when comparing all the combinations of characteristics, the sequence of characteristics formed in the neural network is redefined, for example by defining and combining the related sounds, ie characteristics, in a sequence of sounds. With the newly formed sequence of features, a comparison can again be made with those found on the part of the neural network Characteristic combinations are carried out.

If there is only a partial match of the feature sequence with a combination of features, the matching part can be shown as recognized in a display or output unit and asked for a new entry of the undetected part.

Preferably, a temporary storage area is provided in the storage area for the lexical and / or semantic knowledge, in which matching features and combinations of features recognized from the previous recognition activity are stored, which the neural network prefers to access when forming the combination of features. In this way, the system focuses on a specific topic or statement area. This considerably improves the recognition of the characters or sequence of sounds.

By means of the invention it is achieved that questions or commands can be entered into an input terminal completely informally and without knowledge of a command language or command terminology, and thus certain actions can be triggered controlled by software. The method according to the invention and the device according to the invention assume the function of the interface between the informal, arbitrary, subjective input and the set of commands required by the computer software. The invention functions as an intelligent interface, which adjusts itself to subjective idiosyncrasies in the input and independently focuses the recognition on specific topics or statements or command areas.

In the memory for lexical knowledge, character sequences or sound sequences can be stored together with their meaningfulness and semantic information, so that it is the neural It is possible for the network to generate meaningful information content of the combination of features in the compiled combination of features based on the semantic information, which comes close to the feature sequence in terms of sound or the character sequence. The proximity can be recognized by characteristic vowel sequences, consonant sequences or vowel / consonant combinations.

The device for recognizing the 3edeu or. The statement of the phonetic sound sequence or character string must include an input device into which a text can be spoken or, for example, entered via a keyboard. While a character string is entered, the sequence of features is separated by delimiters such as e.g. Whenever spaces are specified, the sequence of features must be formed by the neural network itself in the case of a phonetic sequence of sounds. It uses the stored lexical and semantic knowledge in the memory areas that the neural network uses.

The formation of the feature sequence can be carried out using the associative word recognition, whereby sound or character sequences are formed from sounds or character components, the existence of which in the sound or character sequence is represented by a vector. In addition, the vector can also make statements, e.g. semantic type, contain and / or statements in connection with given control parameters of a computer software. The recognition can then be realized by comparing the vector of the entered sound or character string with stored vectors.

When the combination of features is formed by the neural network, the combination of features can ultimately be formed immediately on the basis of a meaningful content which, as a predetermined control command of a computer software, promptly initiates a certain action. The feature comparison can, however, also be carried out with combinations of features that contain a meaningful statement at all. This matching combination of features can then be checked to determine whether this meaningful statement matches a given control command.

The invention is described below, for example, using the schematic drawing.

1 shows a recognition device 10 for phonetic sequences

| gene or strings that are entered via an input device 12 '. The input device can be, for example, a keyboard or | to be a microphone. The phonetic string. or character string ¹ is ^"ann a neural network 14 is supplied equipped with a

¹ 'j large knowledge memory 16 are connected, in which lexical | semantic knowledge as well as knowledge of the corresponding phonetic properties of the lexical terms are stored. A temporary memory 18 is contained in the memory 16, the meaning of which will be described in more detail later. The neural network is also connected to fuzzy logic 20, which serves to process features or combinations of features that are not recognized as matching. Furthermore, the neural network 14 is connected to an output interface 22, within which permissible command sets for desired actions, for example an output or a query or the like, are stored.

The operation of the recognition device from FIG. 1 is described below. The character string or phonetic string transferred into the neural network 14 is divided into a sequence of features. This structure can either be based on separators in a character string or, for example, due to pauses in speech in a phonetic sequence. consequences. However, the structure can also be based on information from the memory 16. The subdivided individual components of the sequence form features, so that the phonetic or character sequence is converted into a feature sequence that can be examined for its meaning in the neural network. This recognition of the content of meaning is realized by a comparison with time-coded neurons. The neural network creates hypotheses in the form of combinations of features that are similar to the sequence of features and have a defined meaning. In accordance with the semantic and lexical knowledge available from the memory 16, the neural network forms a large number of such combinations of features, which are compared in time with the feature sequence of the phonetic or character string. . ^'In Figure 2, such a comparison scheme is shown: this, the abscissa shows a listing Ü prepared provoking features, in the present example of ιl to 6, and the ordinate represents a time axis showing the comparison of the characteristic sequence of phonetic or character strings with Characteristic combinations of different hypotheses Hl to H4 shows. The number of contradictions to the features of the feature sequence and / or to the lexical / semantic knowledge and / or to the command set of the data interface 22 is determined for each hypothesis. In a first point in time, all four hypotheses H1 to H4 are compared with the sequence of features. After the first comparison, a hypothesis with most contradictions is eliminated. At the second point in time, the remaining hypotheses are compared again, possibly taking into account new parameters (command set compatibility, etc.). This in turn leads to the falling out of the most contradictory hypothesis. The comparisons are carried out until a hypothesis (combination of features) remains that shows good agreement with the feature sequence and few contradictions. As a statement of the sequence of features the meaningfulness is defined which is assigned to the remaining hypothesis. In the comparison it was already taken into account as a parameter whether the statement of the combination of features corresponds to a command from the output interface 16 or can be transformed into such a command.

If the feature comparison reveals that the combination of features only matches a hypothesis in partial areas, the unrecognized feature sequence can be tried by feeding the unrecognized part of the sequence into the fuzzy logic 20 in conjunction with the lexical and semantic knowledge from the memory 16 weighting the

To compare the sequence and the interaction of the features with similar known feature combinations, the sequence of the features can be taken into account as an additional parameter when weighting the similarity. In this way, unrecognized phonetic or character strings may still be recognized by interpreting the sequence. If this method does not lead to a result either, the recognized character or sound component can be shown on a display and the meaning, description or changed input of the undetected phonetic or character string can be asked. In this way, the operator is taught which statements they need to clarify.

Of course, only at least largely identical hypotheses lead to success, the statement of which corresponds to the instruction set of an output or action interface 22. This can either be taken into account by the fact that the neural network only sets up hypotheses whose information content corresponds to the instruction set of the output interface, or that hypotheses (combinations of features) recognized as matching or largely matching only can be used if a command from the command set of the output interface 22 can be assigned to the meaningfulness of these hypotheses in a subsequent comparison process. The output interface 22 can be the input area of a computer program or different computer programs, by means of which different actions can be initiated. For example, the recognition device 10 could be used in an information or information terminal of an airport or the train. Such a detection device can also be used to control terminals or computers in order to enable access to databases, the system being able to recreate assignments between different data based on the entered / spoken links. Such a terminal could therefore not only be used to output data, but also to put together new data or to generate new data. The device preferably contains a display for displaying the command known or derived from the character or phonetic sequence, which may have to be confirmed by the operator before the action is carried out. In this way it can be avoided that wrong actions are started.

Claims

claims

1. A method for recognizing a phonetic sound sequence or a character string, z.3. according to the ASC II standard, with the following procedural steps: a) the sequence is fed to a neural network, b) in the neural network, the phonetic sequence or character sequence is taken into account, taking into account phonetic and / or lexical stored information and / or based on a sequence of features (blank) a sequence of features is formed, c) the feature sequence formed is compared with a combination of features (H) of a defined meaningfulness, which is formed from stored lexical and semantic information on the basis of the feature sequence, d) step c) is continued repeated with new feature combinations until a combination of features that at least largely matches the feature sequence is found while minimizing contradictions, e) the information content of the feature combination with the fewest contradictions is then output as a result and / or the information content of this feature combination Action associated with the ination is carried out if this result is plausible, otherwise a new defined sequence of features according to b) is formed by the neural network and steps c) to e) are repeated.

2. The method according to claim 1, characterized in that the information content of a matching feature combination is only output as a result if this information content is contained in a stored area of permissible information content.

3. The method according to claim 1 or 2, characterized in that matched feature combinations recognized in previous recognition steps are stored in a temporary memory, and that the features and / or feature combinations temporarily stored in the temporary memory are preferably used for a feature comparison.

4. The method according to claim 3, characterized in that the information stored in the temporary memory on the basis of stored lexical information is assigned to a particular statement area or topic area, and primarily features from this statement area or topic area are used to form the combinations of features.

5. The method according to any one of the preceding claims, characterized in that if no matching combination of features is found, the non-matching part of the sequence is changed on the basis of a combination of features with as many feature matches as possible and the changed sequence of detection is supplied in step c).

6. The method according to claim 5, characterized in that the mismatched part of the feature sequence is changed using fuzzy logic, the change of the mismatched part of the feature sequence taking into account the similarity and sequence of the features with the sequence in feature combinations (H) defined information content is made, provided that this similarity exceeds a defined value or range of values.

7. The method according to any one of the preceding claims, characterized in that if no matching combination of features is found, the feature sequence in step b) redefined and the detection in step c) is supplied.

8. The method according to any one of the preceding claims, characterized in that in the formation of the feature sequence vectors are formed which are characteristic of the sequence and / or properties of phonetic or character sequences, and that the feature comparison is carried out on the basis of the comparison of these vectors.

9. Apparatus for recognizing the meaning of a phonetic string or a string, e.g. according to the ASC II standard, with the following components:

An input device (12) for voice input and / or for inputting characters, for example a keyboard, a neural network (14) which records the phonetic or character sequence entered in the input device, a memory (16) with lexical, semantic and / or phonetic information, a first level of the neural network, which, taking into account the information in the memory, forms a sequence of features from the phonetic or character sequence, a memory area (22) with permissible information or command sets, a second level of the neural network, which uses the information the memory forms combinations of features with a defined meaningful content, a comparison means for comparing the feature _¬ sequence with combinations of features, an evaluation device for the evaluation of Widersprü _¬ chen of prepared combinations of features with the shopping _¬ times sequence and / or stored in the memory lexical _¬ metallic / semantic information and, optionally, s command sets in the storage area (22 ), a device which, depending on the result, causes the first level of the neural network to form a new feature sequence, an output device for displaying the information content of the least contradicting feature combination and / or a device for performing an action associated with the information content of this combination of features.

10. The device according to claim 9, characterized by a fuzzy logic (20) for changing at least a part of a feature sequence taking into account the similarity and sequence of the features with feature combinations stored in the memory, which fuzzy logic is connected on the output side to the second level.

11. The device according to claim 9 or 10, characterized by a temporary memory that stores recognized combinations of features from the last detections, which temporary memory is connected on the output side to the second level of the neural network.

12. Device according to one of claims 9 to 11, characterized by a device which, depending on the result, causes the first level of the neural network to form a new sequence of features.