JP2007322523A - Voice translation apparatus and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice translation apparatus capable of determining language translation direction according to dialog flow in conversation attended by three or more persons. <P>SOLUTION: The voice translation apparatus comprises: a voice input section 10; a speaker identification section 20; a language determination section 30; a language control section 40; a speaker language recording section 50; a voice recognition section 60; and a machine translation section 70. Correspondence relation between a speaker and a spoken language is held, correspondence relation of speech out language is recorded with speech out person and while a pair of speakers who are currently speaking are dynamically changed by recording the correspondence relation between the speaker and the spoken language, the language translation direction is automatically determined, according to the dialog flow. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a speech translation apparatus and method for supporting communication between people who speak different languages.

  In recent years, research on speech processing technologies such as speech recognition and speech synthesis and language processing technologies such as machine translation has been actively conducted. Spoken language processing technologies such as speech translation that link speech processing and language processing are also being actively studied. In order to use speech translation in the real world, it is necessary to solve many problems, but it is put to practical use by restricting the usage situation well and drawing out the cooperation of users to cover technical issues. There are also products that have led to

  Such speech translation technology that supports communication between people with different native languages is expected to be applied in various situations against the background of increasing opportunities for international exchange such as overseas travel and international conferences. Many speech translation devices that are currently commercialized assume a face-to-face conversation, that is, a situation in which two persons who advance a conversation approach a close distance across the speech translation device and communicate with each other. However, there will be a variety of situations where translation is required in the future, such as conferences where more people gather in the same place, and remote conferences that are conducted remotely using various infrastructures.

  However, speech translation cannot be applied in a wide range by simply combining conventional techniques. Comparing the face-to-face dialogue and the conference as an example, there are various differences such as whether the terminal screen can be seen in common or whether the participant's utterance can be received with high quality. For this reason, not only the technical level required for implementation, but also the appropriate user interface varies depending on the use scene. In this way, it is important not only to simply combine conventional technologies, but also to introduce new technologies according to usage situations and to examine interfaces suitable for the situations.

  Let us consider the problem of applying speech translation technology to a meeting where three or more people who are native speakers of different languages gather at the same place. In this case, unlike the face-to-face conversation, there are many people, so it is not easy to set in advance in the speech translation device who speaks which language. Particularly in a scene where three or more different languages are exchanged, there is a problem that which language is input and which language should be output, that is, the language conversion direction is unknown. This is a problem that can be solved by preparing a conference room with facilities that can use dedicated input devices and translation devices for each participant and making settings that suit the participants in advance. Therefore, there is a problem in that it cannot be easily used in various scenes because the cost for the use is high and the places where it can be used are limited.

  In addition, since they are gathered in the same scene, there is a problem that the waiting time from inputting the voice to outputting the translation result cannot be increased. Even if some waiting time is allowed by devising the interface, such as using a screen that can be shared in face-to-face conversations, there will be people who can understand the speech without translation and those who can not do it if there are more than 3 conversations An unnatural state will occur. For this reason, unless the waiting time is reduced, a problem arises in that the flow of conversation deteriorates.

On the other hand, a method has been proposed in which the source language for speech recognition and the target language for machine translation are controlled according to the sound source direction in which the speech is input. Here, the speech translation device is equipped with a movable microphone and a microphone array to detect the sound source direction, and the language conversion direction is determined from the detected sound source direction according to the presetting. For example, the voice input from the front is converted from Japanese to English, and the voice input from the back is converted from English to Japanese. This method is very effective for face-to-face conversations with two people. If the target language is set only once in advance according to the person you are talking to, then the language conversion direction of the input speech is automatically changed without any special operation. It is determined. However, when this method is applied to a conference in which three or more people participate, it is not easy to associate the sound source direction with the input language because it is unknown which person speaks which language. As described above, in the conventional technique, there is a problem that it is not easy to associate the sound source direction with the input language in a conversation scene having three or more participants (see, for example, Patent Document 1).
On the other hand, a method has been proposed in which input speech is recognized by a multi-language speech recognition engine, likelihood is calculated, and a language having a high recognition likelihood is determined to be input. Here, no matter what direction is input from which direction, in order to perform processing assuming that a plurality of languages are input every time, it is not necessary to perform the presetting. Furthermore, there is an advantage that a person who participates in the conference can judge the language from each utterance. However, since this method performs language determination for all utterances, there is a problem that it takes a processing time, which causes a delay time and deteriorates the flow of conversation (for example, patent document). 2)
Furthermore, as a problem common to the prior art, there is a problem that it is not known which language the input speech should be translated into. In face-to-face conversations, it was only necessary to translate to another language different from the input language. However, in the case of a conversation with three or more people, especially a conversation that proceeds in three or more languages, which language is selected according to the flow of the conversation. You should decide whether or not you should preferentially translate, but there is no specific way to solve this. For example, a method of translating and outputting to all possible languages is conceivable, but it is not practical because there is a restriction that a large delay time cannot be taken because participants are gathered in the same place. As described above, the conventional technique has a problem that it is not easy to determine which language the input language should be translated into in a conversation scene with three or more participants.
JP 2005-141759 A JP 2004-347732 A

  Thus, conventionally, in a speech translation device that supports communication in a conversation in which three or more people participate, the language in which the input speech is to be recognized and translated, that is, the language conversion direction is determined. There was a problem of not knowing.

  In addition, there is a problem that a delay time for determining the language conversion direction of the input voice occurs.

  Therefore, the present invention has been made in view of such circumstances, and provides a speech translation apparatus and method for determining a language conversion direction according to the flow of a conversation in a conversation in which three or more people participate. is there.

  The present invention relates to a speech translation apparatus between three or more speakers, a speech input unit for inputting the speech of each speaker, a speaker identification unit for analyzing the speech and identifying a speaker, A language determination unit that analyzes each voice and determines a spoken language; a speaker-of-speech determination unit that determines which speaker is the speaker of interest among the speakers; and the identified speaker And a speaker language recording unit that records a correspondence relationship between the judgment language and the correspondence relationship and the attention speaker. (1) When the input speech is the attention speaker, the attention speech A language other than the input language is determined as an output language, and (2) when the input speech is not the attention speaker, the determination language of the speaker who inputs the speech is determined The input language, and the judgment language of the speaker of interest A language control unit for determining a speech language, a speech recognition unit for recognizing the input speech as the input language, and a machine translation unit for translating the speech recognition result from the input language to the output language. This is a speech translation apparatus characterized by

It is.

  According to the present invention, the translation language is automatically switched in accordance with the flow of the conversation regardless of which direction is input from any direction in the conversation scene in which three or more people participate. Support can be realized.

  Hereinafter, a speech translation apparatus according to an embodiment of the present invention will be described with reference to the drawings.

(First embodiment)
A speech translation apparatus according to the first embodiment of the present invention will be described with reference to FIGS.

(1) Configuration of Speech Translation Device FIG. 1 is a schematic configuration diagram of a speech translation device according to this embodiment.

  As shown in FIG. 1, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, and a machine translation unit 70. Is done.

  The voice input unit 10 passes voice data input from, for example, a microphone to the speaker identification unit 20, the language determination unit 30, and the voice recognition unit 60. If necessary, audio data descrambling processing, decoding processing, format conversion processing, rate conversion processing, and the like may be performed.

  The speaker identification unit 20 analyzes the voice data input from the voice input unit 10 to extract feature quantities for speaker identification, and who is the utterance in the analysis section based on the extracted feature quantities And the identified speaker information is output to the language control unit 40. As a method for identifying a speaker, for example, when the utterance for each speaker is available in advance, the speaker with the largest similarity is determined by comparing the reference model learned for each speaker with the feature amount series. If there is a method for identification and utterances for each speaker cannot be obtained in advance, clustering of speaker vectors that are similarities with reference models created for a large number of speakers available in advance is performed. There is a technique to identify the speaker. Thus, the speaker can be identified using various known means. Note that the speaker information output to the language control unit 40 may be information that can be distinguished from other speakers even if the individual speaker is not specified. For example, the voice input unit 10 is configured by a microphone array. The sound source direction or position of the speaker may be output as speaker information.

  The language determination unit 30 analyzes the voice data input from the voice input unit 10 and extracts a feature amount necessary for determining in which language the utterance in the analysis section is uttered. With reference to the learned dictionary, the acoustically most similar language is output to the language control unit 40 as a determination result. Various known means such as a mixed Gaussian distribution can be used as the language determination method for calculating the similarity between the dictionary learned in advance and the input speech.

  The language control unit 40 receives the identified speaker from the speaker identification unit 20 and the determination language from the language determination unit 30, and records the identification speaker and the determination language in the speaker language recording unit 50 in association with each other. Furthermore, based on the correspondence between the recorded identified speaker and the judgment language, the language conversion direction that is a pair of the input language and the output language is determined as to which language the input speech should be recognized and translated into. decide. The determined input language is output to the speech recognition unit 60 and the machine translation unit 70, and the determined output language is output to the machine translation unit 70.

  The speech recognition unit 60 analyzes the speech data input from the speech input unit 10 to extract feature amounts necessary for recognition, and selects a dictionary learned in advance based on the input language received from the language control unit 40. The most acoustically similar word or word sequence is output to the machine translation unit 70 as a recognition result. As a recognition method for calculating the similarity between the dictionary learned in advance and the input speech, various known means such as a hidden Markov model, a neural network, and DP matching can be used.

  The machine translation unit 70 receives a character string of the source language from the speech recognition unit 60 using the input language received from the language control unit 40 as the source language and the output language as the target language, and converts it into the target language. Various existing methods such as a rule-based translation method and a sentence-based translation method can be used.

(2) Operation of Speech Translation Device Next, detailed operation of the speech translation device will be described using a specific example.

  FIG. 2 shows an example of a conversation scene of a plurality of people. Among the participants in the conference, speakers A to D speak English, and speakers E and F speak Japanese. The speech translation apparatus is owned by only the speaker F, and the speech of the speaker is received by a microphone provided in the speech translation apparatus, thereby realizing speech translation.

  The operations of the speaker identification unit 20, the language determination unit 30, and the language control unit 40 will be described in detail using the conversation scene example of FIG.

(2-1) Speaker identification unit 20
The speaker identification unit 20 analyzes the voice data input from the voice input unit 10 to extract feature quantities for speaker identification, and who is the utterance in the analysis section based on the extracted feature quantities And the identified speaker information is output to the language control unit 40. Hereinafter, a method for identifying a speaker will be described in detail.

  As a method for identifying a speaker, a certain amount of participants' utterances are recorded in advance, a reference model such as a vector quantization dictionary is created for each speaker, and the speaker is extracted from the section to be identified. The most common method is to identify a speaker from a reference model having the largest similarity between a feature vector sequence and a reference model.

  However, registering the utterances of participants in advance in an actual meeting is costly and impractical. Therefore, in the present embodiment, a reference model is created for each of a large number of speakers that can be obtained in advance even if they are different from the participants, and the feature vector is converted into a speaker vector that is a similarity series with the created reference model. A speaker identification method will be described with reference to FIGS. 3 and 4 by taking an example of a method of performing clustering by conversion.

  In this embodiment, a Mel frequency cepstrum coefficient (hereinafter abbreviated as MFCC) is used as an example of a feature quantity for speaker identification. However, any feature quantity that can be used for speaker identification is used. Also good.

  FIG. 3 shows the distribution of the feature vector space. An ellipse in the feature vector space represents a reference model created for each speaker collected in advance. The ellipse of the speaker X in the figure shows the distribution of the reference model learned from the utterances of the speaker X collected in advance different from the participants. Input (a) in FIG. 3 represents a feature vector obtained by dividing the voice input from the voice input unit 10 into, for example, analysis frame lengths (for example, set to 10 mS), and analyzing and extracting each frame. . For the sake of explanation, the description will proceed assuming that the number of dimension of the feature vector is 3 and the number of reference models is 3. However, the number of dimension of the feature vector is actually as large as several tens, and the number of reference models is also several hundred to several thousand. There are many preparations.

  As shown in FIG. 3, since the conference model does not include participants in the conference, it is often the case that a feature vector having a low similarity with the reference model is input. Thus, the speaker vector space illustrated in FIG. 4 represents the input by performing conversion using the similarity with each of the previous speakers as a new feature vector. By converting the feature vector into the speaker vector in this way, the utterance that is not directly similar to the specific pre-speaker can express the speaker nature of the input speech by the distance relationship with the plurality of pre-speakers.

  For example, in the example of FIGS. 3 and 4, the input (b) and the input (c) have an intermediate voice quality between the speaker X and the speaker Y, and this is expressed by a close vector in the speaker vector space. Yes. By classifying speaker vectors in such a speaker space by a clustering method such as an LBG algorithm, it is possible to identify speakers of the input speech. For example, if the inputs (b) and (c) merged into the same class by the clustering method are classified as class (1), utterances classified into class (1) in the future can be identified as the same speaker. In order to simplify the explanation, the explanation is made such that the determination is performed in units of frames. However, in actuality, one speaker is used in one unit of utterance by using the speech section detection used in the front end of speech recognition. It is considered effective to determine the unit.

  In addition, the speaker identification method using the speaker vector has been described here. As described above, the speaker information to be identified can be distinguished from other speakers even if the individual speaker cannot be identified. For example, the voice input unit 10 may be configured by a microphone array, and the sound source direction or position of the speaker may be output as speaker information.

(2-2) Language determination unit 30
The language determination unit 30 analyzes the voice data input via the voice input unit 10 and extracts a feature amount necessary to determine in which language the utterance of the analysis section is uttered, A language that is acoustically most similar to the dictionary learned in advance is output to the language control unit 40 as a determination result. Hereinafter, a method for identifying a language will be described in detail.

  As a method for identifying a language, it is easier to collect a speech set of the language in advance compared to the case of identifying a speaker (at least for the acoustic model learning for speech recognition, the speech of the language A large amount of data is collected), and a reference model such as a mixed Gaussian distribution model is created for each language, and the reference model with the highest similarity between the feature vector sequence extracted from the section in which the language is to be identified and the reference model Can be used to identify the language.

  In this embodiment, a mel frequency cepstrum coefficient (hereinafter abbreviated as MFCC) is used as an example of the feature quantity for language identification. However, any existing feature quantity may be used as long as the feature quantity enables language identification.

  FIG. 5 shows the distribution of the feature vector space. An ellipse in the feature vector space represents a reference model created for each language collected in advance. The input (a) in the figure represents a feature vector obtained by dividing the voice input via the voice input unit 10 into, for example, analysis frame lengths (for example, set to 10 mS) and analyzing and extracting each frame. Yes. For the sake of explanation, the dimension number of the feature vector is 3 and the number of reference models is 3. However, in practice, the dimension number of the feature vector is as large as several tens. Here, the similarity between the feature vector and the model of each language is calculated, and the language with the highest similarity is determined. In the example of FIG. 5, it is determined that the input (a) is close to the Japanese model and the inputs (b) to (d) are close to each other and the similarity is relatively large. Here, for the sake of simplicity, the description has been made so that the determination is performed in units of frames. However, in actuality, one utterance is made in one unit of utterance by using the speech section detection used in the front end of speech recognition. It is considered effective to determine the language.

(2-3) Language control unit 40
Next, details of the language control unit 40 will be described.

  FIG. 6 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 6, when the speaker F, the speaker A, and the speaker B speak in this order from the beginning of the conversation, the speakers are classified into three classes by the speaker identification unit 20, and language determination is performed. The correspondence relationship when the language is correctly determined by the unit 30 is shown. Note that the uncategorized in FIG. 6 is displayed for the sake of convenience in order to make it easy to understand speakers who have not spoken so far, and the actual correspondence is the class (1) ( 2) Only (3). The operation of the language control unit 40 when the utterance of the speaker C (that is, the speaker of interest) is newly input while referring to the correspondence relationship in FIG. 6 will be described in detail.

  When a new utterance of the speaker C is input to the voice input unit 10, a new class (4) is output from the language identification unit 30 as an unknown speaker is input from the speaker identification unit 20. .

  The language determination unit 40 refers to the correspondence relationship shown in FIG. 6 and determines whether the input identified speaker and determination language are uttered by an existing speaker or a new speaker. In this case, since class (4) is an unknown speaker, the speaker is registered as a new entry in the correspondence relationship, and the correspondence relationship is updated as shown in FIG. At this time, it is recorded in the speaker attribute that the current speaker is class (4).

  The language control unit 40 operates to determine the input language and the output language based on the newly updated correspondence relationship in FIG.

  Here, as an example, consider a case where the language control unit 40 operates based on the simplest rule of determining a language different from the input language as the output language among the languages included in the correspondence relationship.

  In that case, English and Japanese are listed in the correspondence relationship in FIG. 7, and it can be understood that the current speaker is speaking English by referring to the speaker attribute. Determine the word as the output language. Based on the input language and the output language determined in this way, the speech recognition unit 60 recognizes the input speech as English, and the machine translation unit 70 translates the English recognition result into Japanese. By operating, the language conversion direction is automatically determined.

(3) Effect As described above, according to the present embodiment, the correspondence between the speaker and the utterance language is maintained, the correspondence between the utterer and the utterance language is recorded, and the speaker pair currently proceeding with the conversation is moved. The language conversion direction can be automatically determined according to the flow of dialogue.

  Therefore, the translation language is automatically switched according to the flow of the conversation regardless of which direction is input from which direction in the conversation scene in which three or more people participate, so that it is possible to realize the conversation support by translation without hindering the flow of the conversation. .

  In addition, since the correspondence relationship between the speaker and the spoken language is recorded, there is no need to perform a new language determination process for an existing speaker, and the effect of partially reducing the delay time due to speech translation is also produced.

(Second Embodiment)
Next, a speech translation apparatus according to the second embodiment of the present invention will be described with reference to FIGS. The speech translation apparatus according to the present embodiment determines the language conversion direction by using the owner of the speech translation apparatus as an attention speaker.

(1) Configuration of Speech Translation Device The configuration of the speech translation device according to this embodiment will be described.

  FIG. 8 is a schematic configuration diagram of the speech translation apparatus according to the present embodiment.

  As shown in FIG. 8, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and a possession. The person determination unit 80 is configured.

  In FIG. 8, the same numbers are assigned to portions that perform the same operations as those in the first embodiment, and a description thereof is omitted.

  The owner determination unit 80 detects the who is the owner of the speech translation device or the speaker who mainly uses the speech translation device in cooperation with the language control unit 40, and the correspondence between the identified speaker and the owner. Is recorded in the speaker language recording unit 50. For example, it is considered that the owner can easily operate with the speech translation device at hand, so when the owner speaks, the owner can press the special button on the device. If the utterance is synchronized with the timing when the button is pressed, it is easy to grasp the correspondence.

(2) Operation of Speech Translation Device Next, detailed operation of the speech translation device according to the second embodiment will be described using a specific example.

  FIG. 9 shows an example of a conversation scene of a plurality of people. Among the participants in the conference, speaker G speaks English, speaker H speaks Chinese, and speaker I speaks Japanese. The speech translation apparatus is owned only by the speaker I, and the speech translation is realized by receiving the utterance of the speaker by the microphone provided in the speech translation apparatus.

  The operation of the language control unit 40 will be described in detail using the conversation scene example of FIG.

  FIG. 10 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 10, the speakers G, H, and I have been uttered a plurality of times so far, and the speakers are classified into three classes by the speaker identification unit 20, and the language is correctly determined by the language determination unit 30. The correspondence relationship is shown when is determined. In addition, it is assumed that it has been found by the processing so far that the utterance of the speaker I is the owner by the owner determination unit 80. The operation of the language control unit 40 when a speaker G, speaker H, or speaker I newly utters in this state will be described in detail.

  FIG. 11 shows an operation flowchart of the language control unit 40.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S100, it is acquired from the correspondence shown in FIG. 10 that the language of the current speaker G is English.

  In step S101, it is acquired from the correspondence relationship that the language of the owner I is Japanese.

  Next, in step S102, it is determined whether the current speaker G and the owner I match. Since this case is different, the process proceeds to step S103 to compare the language of the current speaker with the language of the owner. In this case, since English differs from Japanese, in step S104, the input language is determined to be English and the output language is determined to be Japanese, and the processing of the language control unit 40 is completed.

(2-2) Utterance of speaker H in Chinese Next, consider the case where speaker H utters.

  In step S100, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese.

  In step S101, it is acquired from the correspondence relationship that the language of the owner I is Japanese.

  Next, in step S102, it is determined whether or not the current speaker H and the owner I match. Since this case is different, the process proceeds to step S103, where the language of the current speaker and the language of the owner are compared. In this case, since it differs between Chinese and Japanese, in step S104, the input language is determined to be Chinese and the output language is determined to be Japanese, and the processing of the language control unit 40 ends.

(2-3) Speech of Speaker I in Japanese Finally, the operation of the language control unit 40 when the speaker I who is the owner speaks will be described.

  In step S100, it is acquired from the correspondence relationship that the language of the current speaker I is Japanese.

  In step S101, it is acquired from the correspondence relationship that the language of the owner I is Japanese.

  Next, in step S102, it is determined whether the current speaker I and the owner I match. If they match, the process proceeds to step S106.

  Next, in step S106, all languages other than Japanese (in this case, English and Chinese) are acquired. In step S107, the input language is determined to be Japanese and the output language is determined to be English and Chinese. The process of the control unit 40 is finished.

  In addition, although not appearing in the dialog scene of FIG. 9, when a speaker who speaks the same language as the owner is included in the conversation, the input language and the output language are set in step S105 of the flowchart of FIG. Not. That is, it operates so that the input speech is not recognized and machine translated.

(3) Effect As described above, in the present embodiment, the utterance in a language other than the owner's language is translated into a language understandable by the owner, and the utterance of the owner is translated into all languages other than the owner. Thus, translation of all language pairs is not performed unnecessarily, and speech translation can be operated only when the owner cannot understand. As a result, it is possible to realize speech translation that supports the understanding of the owner while reducing interruption of the conversation due to speech translation.

(Third embodiment)
Next, a speech translation apparatus according to the third embodiment of the present invention will be described with reference to FIGS. 9, 12, 13, and 15. FIG. The speech translation apparatus according to the present embodiment determines the language conversion direction with the immediately preceding speaker, with the immediately preceding speaker as the attention speaker.

(1) Configuration of Speech Translation Device The configuration of the speech translation device according to this embodiment will be described.

  FIG. 12 is a schematic configuration diagram of a speech translation apparatus according to the present embodiment.

  As shown in FIG. 12, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and an utterance. A history recording unit 90 is included.

  In FIG. 12, parts that perform the same operations as in the first embodiment are given the same numbers, and descriptions thereof are omitted.

  The utterance history recording unit 90 records past utterance histories in cooperation with the language control unit 40. As shown in FIG. 13, an example of the utterance history is to hold speakers within a certain number of times in the past in time series. In addition, utterance attributes such as utterance time may be recorded in the utterance history.

(2) Operation of Speech Translation Device Next, detailed operation of the speech translation device according to the third embodiment will be described using a specific example.

  FIG. 9 shows an example of a conversation scene of a plurality of people. Among the participants in the conference, speaker G speaks English, speaker H speaks Chinese, and speaker I speaks Japanese. The speech translation apparatus is owned only by the speaker I, and the speech translation is realized by receiving the utterance of the speaker by the microphone provided in the speech translation apparatus. The operation of the language control unit 40 will be described in detail using the conversation scene example of FIG.

  FIG. 15 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 15, the speakers G, H, and I have spoken a plurality of times so far, and the speakers are classified into three classes by the speaker identification unit 20, and the language determination unit 30 correctly speaks the language. The correspondence relationship is shown when is determined. Consider the case where the utterance history up to immediately before is shown in FIG. In this case, it is found from the utterance history that the immediately preceding speaker is H, and the immediately preceding speaker is entered in the speaker attribute of speaker H in FIG.

  FIG. 14 shows an operation flowchart of the language control unit 40.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S200, it is acquired from the correspondence shown in FIG. 15 that the language of the current speaker G is English.

  Next, in step S201, it is acquired from the correspondence relationship that the language of the immediately preceding speaker H is Chinese.

  Next, in step S202, it is determined whether the current speaker G and the previous speaker H match. Since this case is different, the process advances to step S203 to compare the language of the current speaker with the language of the immediately preceding speaker. In this case, since English differs from Chinese, in step S204, the input language is determined to be English and the output language is determined to be Chinese, and the processing of the language control unit 40 is completed.

(2-2) Utterance of speaker H in Chinese Next, consider the case where speaker H utters.

  In step S200, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese.

  Next, in step S201, it is acquired from the correspondence relationship that the language of the immediately preceding speaker H is Chinese.

  Next, in step S202, it is determined whether the current speaker H and the previous speaker H match. If they match, the process proceeds to step S206.

  Next, in step S206, all languages other than Chinese (in this case, English and Japanese) are acquired, and in step S207, the input language is determined to be Chinese and the output language is determined to be English and Japanese. The process of the control unit is finished.

(3) Effects As described above, in this embodiment, the language conversion direction is preferentially translated so that the two languages that are engaged in the conversation are preferentially translated under the assumption that the previous speaker and the current speaker are proceeding with the conversation. By controlling, speech translation can be operated without losing the flow of dialogue as much as possible.

(Fourth embodiment)
Next, a speech translation apparatus according to a fourth embodiment of the present invention will be described with reference to FIGS. 9, 12, 13, 16, and 17. The speech translation apparatus according to the present embodiment determines the language conversion direction as the subject speaker with the subject who has the most speech as the target speaker.

(1) Configuration of Speech Translation Device The configuration of the speech translation device according to this embodiment will be described.

  FIG. 12 is a schematic configuration diagram of a speech translation apparatus according to the fourth embodiment.

  As shown in FIG. 12, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and an utterance. A history recording unit 90 is included.

  In FIG. 12, parts that perform the same operations as in the first embodiment are given the same numbers, and descriptions thereof are omitted.

  In the third embodiment, the method of extracting the immediately preceding speaker from the utterance history and determining the language conversion direction is shown. However, in the present embodiment, the main utterance that has mainly spoken from the utterance history in the past certain interval. The third embodiment is different from the third embodiment in that the language conversion direction is determined so as to prioritize communication between the main speaker and the current speaker.

(2) Operation of Speech Translation Device Hereinafter, details of the operation of the present embodiment will be described using the conversation scene shown in FIG. 9 and the utterance history shown in FIG. 13 as examples.

  FIG. 17 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 17, the speakers G, H, and I have so far spoken a plurality of times, and each speaker is classified into three classes by the speaker identification unit 20, and the language determination unit 30 correctly speaks the language. The correspondence relationship is shown when is determined.

  FIG. 16 shows an operational flowchart of the language control unit 40 in the present embodiment.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S30, the main speaker is detected from the utterance history shown in FIG. As a method for detecting the main speaker, for example, a method of selecting a speaker having the largest number of utterances or a speaker having the longest utterance time from among the speakers stored in the utterance history can be considered. Since the speaker G is selected as the main speaker in any case in the utterance history of FIG. 13, the fact that the speaker is the main speaker is recorded in the speaker attribute of the speaker G in the correspondence shown in FIG.

  Next, in step S300, it is acquired from the correspondence relationship that the language of the current speaker G is English, and in step S301, the language of the main speaker G is acquired in English.

  Next, in step S302, it is determined whether the current speaker G and the main speaker G match. In this case, since they match, in step S306, all languages other than English (in this case, Chinese and Japanese). In step S307, the input language is determined to be English and the output languages are determined to be Chinese and Japanese, and the processing of the language control unit 40 is completed.

(2-2) Utterance of speaker H in Chinese Consider the case where speaker H speaks instead of speaker G.

  In step S300, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese, and in step S301, the language of the main speaker G is English.

  Next, in step S302, it is determined whether the current speaker H and the main speaker G match. However, since this case is different, the process advances to step S303 to compare the language of the current speaker with the language of the main speaker. In this case, since English and Chinese are different, in step S304, the input language is determined to be Chinese and the output language is determined to be English, and the processing of the language control unit 40 is completed.

(3) Effect As described above, in the present embodiment, priority is given to the two languages that are engaged in the conversation on the assumption that the main speaker who speaks mainly for a certain section and the current speaker are proceeding with the conversation. By controlling the language conversion direction so that translation is performed, it is possible to operate speech translation without losing the flow of dialogue as much as possible.

(Fifth embodiment)
Next, a speech translation apparatus according to a fifth embodiment of the present invention will be described with reference to FIG. 9, FIG. 12, FIG. 13, FIG. The speech translation apparatus according to the present embodiment determines the language conversion direction based on the speaker-to-history by setting the immediately preceding speaker as the attention speaker. That is, the speaker with the most speech is set as the attention speaker, and the speaker with the next most speech is set as the talker.

(1) Configuration of Speech Translation Device The configuration of the speech translation device according to this embodiment will be described.

  FIG. 12 is a schematic configuration diagram of a speech translation apparatus according to the fifth embodiment.

  As shown in FIG. 12, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and an utterance. A history recording unit 90 is included.

  In FIG. 12, parts that perform the same operations as in the first embodiment are given the same numbers, and descriptions thereof are omitted.

  In the third embodiment and the fourth embodiment, the method of determining the language conversion direction by extracting the previous speaker or the main speaker from the utterance history is shown. The point which devised so that the history of the speaker pair who talked alternately in the inside may be acquired, and a language conversion direction may be determined according to the presence or absence of a speaker pair differs from previous embodiment.

(2) Operation of Spoken Translation Device The details of this embodiment will be described below taking the conversation scene shown in FIG. 9 and the utterance history shown in FIG. 13 as examples.

  FIG. 19 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 19, the speakers G, H, and I have spoken a plurality of times so far, and the speakers are classified into three classes by the speaker identification unit 20, and the language determination unit 30 correctly speaks the language. The correspondence relationship is shown when is determined.

  FIG. 18 shows an operation flowchart of the language control unit 40 in the present embodiment.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S40, a table of speaker pairs is created from the utterance history shown in FIG. FIG. 20 shows a pair of speakers created based on the utterance history of FIG. The table counts the frequency of (speaker just before, speaker just after) for each pair of consecutive speakers.

  Next, in step S400, it is acquired from the correspondence relationship that the language of the current speaker G is English. Next, in step S401, a list of speakers that are speaker pairs with the current speaker G is picked up from the speaker pairs illustrated in FIG. Referring to FIG. 20, a speaker having a history of a speaker pair with the current speaker G becomes a speaker H and a speaker I, so Chinese and Japanese are acquired as the language of the speaker pair.

  Next, in step S402, it is determined whether or not the speaker pair matches. In this case, since it exists, in step S403, the input language is determined to be English and the output language is determined to be Chinese and Japanese. The process of the unit 40 is finished.

(2-2) Utterance of speaker H in China Consider the case where speaker H speaks instead of speaker G.

  In step S400, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese.

  Next, in step S401, referring to FIG. 20, the speaker having the history of the speaker pair with the current speaker H becomes the speaker G, so English is acquired as the language of the speaker pair.

  Next, in step S402, it is determined whether or not a speaker pair exists. In this case, since it exists, in step S403, the input language is determined to be Chinese and the output language is determined to be English. Finish the process.

(3) Effect As described above, in the present embodiment, the language conversion direction is such that the two languages that advance the conversation based on the speaker pair that has exchanged with the current speaker during a certain section are preferentially translated. By controlling, speech translation can be operated without losing the flow of dialogue as much as possible.

(Sixth embodiment)
Next, a speech translation apparatus according to a sixth embodiment of the present invention will be described with reference to FIGS. 9, 12, 13, 19, 20, and 21. FIG. The speech translation apparatus of this embodiment prioritizes and determines a plurality of language conversion directions.

(1) Configuration of Speech Translation Device The configuration of the speech translation device according to this embodiment will be described.

  FIG. 12 is a schematic configuration diagram of a speech translation apparatus according to the sixth embodiment.

  As shown in FIG. 12, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and an utterance. A history recording unit 90 is included.

  In FIG. 12, parts that perform the same operations as in the first embodiment are given the same numbers, and descriptions thereof are omitted.

  In the third to fifth embodiments, when there are a plurality of output languages from the utterance history, the priority order is not particularly determined. However, in the present embodiment, when there are a plurality of output languages, The point of prioritizing the language to be output from the utterance history in the past fixed section is different from the previous embodiment.

(2) Operation of Spoken Translation Device The details of this embodiment will be described below taking the conversation scene shown in FIG. 9 and the utterance history shown in FIG. 13 as examples.

  FIG. 19 shows an example of the correspondence stored in the speaker language recording unit 50. In the example of FIG. 19, the speakers G, H, and I have spoken a plurality of times so far, and the speakers are classified into three classes by the speaker identification unit 20, and the language determination unit 30 correctly speaks the language. The correspondence relationship is shown when is determined.

  FIG. 21 shows an operation flowchart of the language control unit 40 in the present embodiment.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S50, a table of speaker pairs is created from the utterance history shown in FIG. FIG. 20 shows a pair of speakers created based on the utterance history of FIG. The table counts the frequency of (speaker just before, speaker just after) for each pair of consecutive speakers.

  In step S500, it is acquired from the correspondence relationship that the language of the current speaker G is English.

  Next, in step S501, a list of speakers that are speaker pairs with the current speaker G is picked up from the speaker pairs illustrated in FIG. Referring to FIG. 20, a speaker having a history of a speaker pair with the current speaker G becomes a speaker H and a speaker I, so Chinese and Japanese are acquired as the language of the speaker pair.

  Next, in step S502, it is determined whether or not the speaker pair matches. In this case, since it exists, the process proceeds to step S503.

  Next, in step S503, it is determined whether or not there are a plurality of speaker pairs. In this case, since there is a speaker pair, priority is given to output languages in step S504. As a method of assigning priorities, for example, referring to FIG. 20, a method of giving priority to a speaker having a high frequency as a speaker pair can be considered. In this case, since the speaker H and the speaker are paired many times, the speaker H's language, Chinese, is given priority over the speaker I's Japanese.

  Next, in step S505, priority is given to English as the input language and Chinese as the output language, and then it is determined to output Japanese, and the processing of the language control unit 40 ends.

(2-2) Utterance of speaker H in Chinese Consider the case where speaker H speaks instead of speaker G.

  In step S500, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese.

  Next, in step S501, referring to FIG. 20, the speaker having the history of the speaker pair with the current speaker H becomes the speaker G, so English is acquired as the language of the speaker pair.

  Next, in step S502, it is determined whether or not there is a speaker pair. In this case, since it exists, the process proceeds to step S503.

  Next, in step S503, it is determined whether or not there are a plurality of speaker pairs. In this case, since there is no speaker pair, the input language is determined to be Chinese and the output language is determined to be English.

(3) Effect As described above, in the present embodiment, when there are a plurality of output languages determined based on the utterance history during a certain interval, language conversion is performed so that the languages of the two persons who proceed with the conversation are preferentially translated. By controlling the direction, it is possible to operate speech translation without losing the flow of dialogue as much as possible.

(Seventh embodiment)
Next, a speech translation apparatus according to a seventh embodiment of the present invention will be described with reference to FIGS. The speech translation apparatus of this embodiment does not output until the language direction is made.

(1) Configuration of Speech Translation Device FIG. 22 is a schematic configuration diagram of a speech translation device according to the seventh embodiment.

  As shown in FIG. 22, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and a speech. The storage unit 100 is configured.

  In FIG. 22, the same number is assigned to the portion that performs the same operation as in the previous embodiment, and the description thereof is omitted.

  The voice storage unit 100 controls whether the voice input from the input voice unit 10 is immediately output to the voice recognition unit 60 according to the control of the language control unit 40 or is stored once and recognized later. If the purpose is to support the conversation by translating the speech of the participants in the conversation at the same place, basically it is better not to accumulate speech in order not to disturb the flow of conversation.

  However, if the input language of the input speech and the output language to be converted cannot be determined, the speech can be stored once and the input language or output language can be determined using the processing result of the speech input later It is only necessary to output the voice that has been stored together in.

(2) Operation of Speech Translation Device FIG. 23 shows an example of the correspondence stored in the speaker language recording unit 50.

  In the example of FIG. 23, only the speaker G has been uttered so far, and it is not known which language speaks in the conversation. In this situation, since it is not known to which language the speech of the speaker G can be translated, the speech is accumulated when the speech of the speaker G continues.

  Thereafter, for example, the utterance of the speaker H is input, that is, the number of utterances exceeds a specified number (for example, once) and a new speaker class is set in the speaker identifying unit 20, and the language determining unit 30 in Chinese Is correctly determined, the utterances of the speaker G accumulated so far are collectively converted from English to Chinese.

(3) Effect As described above, according to the present embodiment, even when the input language and the output language cannot be determined at the stage where the voice is input, the voice data is accumulated, and the number of utterances of the speaker is equal to or more than the predetermined number. By controlling to output the translation result when the language conversion direction is determined, the voice uttered at the beginning of the conversation can be properly transmitted to the partner without losing.

(Eighth embodiment)
Next, a speech translation apparatus according to an eighth embodiment of the present invention will be described with reference to FIGS. 9, 10, 24, 23, 18, and 20. The speech translation apparatus according to this embodiment outputs a plurality of languages using different media and display methods.

(1) Configuration of Speech Translation Device FIG. 24 is a schematic configuration diagram of a speech translation device according to the eighth embodiment.

  As shown in FIG. 24, the speech translation apparatus includes a speech input unit 10, a speaker identification unit 20, a language determination unit 30, a language control unit 40, a speaker language recording unit 50, a speech recognition unit 60, a machine translation unit 70, and results. The output unit 110 is configured.

  In FIG. 24, the same reference numerals are assigned to parts that perform the same operations as in the previous embodiment, and a description thereof is omitted.

  The result output unit 110 presents the translation result input from the machine translation unit 70 to the participant using a display method such as speech synthesis or screen display. In this embodiment, the display method of the translation result is switched by the language control unit 40, which is different from the previous embodiments.

(2) Operation of Speech Translation Device The details of this embodiment will be described below taking the dialog scene shown in FIG. 9 as an example.

  FIG. 10 shows an example of the correspondence relationship stored in the speaker language recording unit 50. In the example of FIG. 10, the speakers G, H, and I have been uttered a plurality of times so far, and the speakers are classified into three classes by the speaker identification unit 20, and the language is correctly determined by the language determination unit 30. The correspondence relationship is shown when is determined.

  FIG. 25 shows an operation flowchart of the language control unit 40 in the present embodiment.

(2-1) Utterance of speaker G in English First, consider the case where speaker G utters.

  In step S600, it is acquired that the language of the current speaker G is English from the correspondence shown in FIG.

  Next, in step S601, the fact that the language of the owner I is Japanese is acquired from the correspondence relationship.

  In step S602, it is determined whether the current speaker G and the owner I match. Since this case is different, the process proceeds to step S603, where the language of the current speaker is compared with the language of the owner. In this case, since English and Japanese are different, in step S604, the input language is determined to be English and the output language is determined to be Japanese. In step S608, the output method is determined to be a screen display, and the processing of the language control unit 40 is performed. Finish.

(2-2) Utterance of speaker H in Chinese Next, consider the case where speaker H utters.

  In step S600, it is acquired from the correspondence relationship that the language of the current speaker H is Chinese.

  Next, in step S601, the fact that the language of the owner I is Japanese is acquired from the correspondence relationship.

  Next, in step S602, it is determined whether or not the current speaker H and the owner I match. However, since this case is different, the process advances to step S603 to compare the language of the current speaker with the language of the owner. In this case, since Chinese and Japanese are different, in step S604, the input language is determined to be Chinese and the output language is determined to be Japanese. In step S608, the output method is determined to be a screen display. Finish the process.

(2-2) Utterance of Speaker I in Japanese Finally, the operation of the language control unit 40 when the speaker I who is the owner utters will be described.

  In step S600, it is acquired from the correspondence relationship that the language of the current speaker I is Japanese.

  Next, in step S601, the fact that the language of the owner I is Japanese is acquired from the correspondence relationship.

  Next, in step S602, it is determined whether the current speaker I and the owner I match. If they match, the process proceeds to step S606.

  Next, in step S606, all languages other than Japanese (in this case, English and Chinese) are acquired. In step S607, the input language is determined to be Japanese and the output language is determined to be English and Chinese. In step S609, the output method is determined to be speech synthesis, and the processing of the language control unit is finished.

(3) Effect As described above, in this embodiment, if control is performed so that the speech of the owner is speech synthesis and the speech of the other speakers is output on the screen display, the conversation is interrupted by interruption of the conversation by the synthesized sound. It is possible to operate speech translation while reducing the risk of impairing the flow.

(Example of change)
The present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist thereof.

  In each of the above embodiments, the operation under the assumption that the determination results of the language determination unit 30 and the speaker identification unit 20 are always correct has been described. However, in practice, it is not easy to obtain 100% determination accuracy, and there is a risk of malfunction if determination errors are not taken into consideration.

  In order to consider the possibility of a determination error, for example, a measure such as determining the result of language determination or speaker identification by a majority decision of a plurality of utterance sections can be considered. By introducing multiple judgments, there is a risk that the response of speech translation will be delayed at the initial stage of the conversation, but by allowing only the delay at the initial stage, stable operation is possible later, so there is an advantage. It can be said that it is bigger.

It is a figure showing the schematic structural example of the speech translation apparatus in connection with the 1st Embodiment of this invention. It is a figure which shows an example of the conversation scene of several people in 1st Embodiment. It is an example of the feature vector space in the speaker identification part in the block diagram of FIG. 1 in connection with 1st Embodiment. It is an example of the speaker vector space in the speaker identification part in the block diagram of FIG. 1 in connection with 1st Embodiment. It is an example of the feature vector space in the language determination part in the block diagram of FIG. 1 in connection with 1st Embodiment. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 1 related to the first embodiment. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 1 related to the first embodiment. It is a figure showing the schematic structural example of the speech translation apparatus in connection with the 2nd Embodiment of this invention. It is a figure which shows an example of the conversation scene of several people in 2nd Embodiment. It is an example of the correspondence stored in the speaker language recording unit in the configuration diagram of FIG. 8 according to the second embodiment. It is an operation | movement flowchart regarding the language control part in the block diagram of FIG. 8 in connection with 2nd Embodiment. It is a figure showing the schematic structural example of the speech translation apparatus in connection with the 3rd, 4th, 5th and 6th embodiment of this invention. It is an example of the utterance history stored in the utterance history recording unit in the configuration diagram of FIG. 12 according to the third embodiment. 13 is an operation flowchart relating to a language control unit in the configuration diagram of FIG. 12 according to the third embodiment. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 12 relating to the third embodiment. 13 is an operation flowchart relating to a language control unit in the configuration diagram of FIG. 12 according to the fourth embodiment. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 12 concerning 4th Embodiment. FIG. 13 is an operation flowchart regarding the language control unit in the configuration diagram of FIG. 12 according to the fifth embodiment. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 12 relating to the fifth embodiment. It is an example of the utterance history stored in the utterance history recording unit in the configuration diagram of FIG. 12 according to the fifth embodiment. 13 is an operation flowchart relating to a language control unit in the configuration diagram of FIG. 12 according to the sixth embodiment. The figure showing the schematic structural example of the speech translation apparatus in connection with the 7th Embodiment of this invention. It is an example of the correspondence stored in the speaker language recording part in the block diagram of FIG. 22 relating to the seventh embodiment. It is a figure showing the schematic structural example of the speech translation apparatus in connection with the 8th Embodiment of this invention. It is an operation | movement flowchart regarding the language control part in the block diagram of FIG. 12 in connection with 8th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Voice input part 20 Speaker identification part 30 Language determination part 40 Language control part 50 Speaker language recording part 60 Speech recognition part 70 Machine translation part 80 Owner determination part 90 Speech history recording part 100 Voice storage part 110 Result output part

Claims (12)

In a speech translation device between three or more speakers,
A voice input unit for inputting the voice of each speaker;
A speaker identification unit that analyzes each voice and identifies a speaker;
A language determination unit that analyzes each voice and determines a spoken language;
An attention speaker determination unit that determines which speaker is the attention speaker among the speakers; and
A speaker language recording unit that records a correspondence relationship between the identified speaker and the determination language;
Based on the correspondence and the speaker of interest, (1) when the input speech is the speaker of interest, the determination language of the speaker of interest is the input language and a language other than the input language is output (2) When the input speech is not the target speaker, the determination language of the input speech is determined as the input language, and the determination language of the target speaker is determined as the output language. A language control unit,
A speech recognition unit that recognizes the input speech as the input language;
A machine translation unit that translates the speech recognition result from the input language into the output language;
A speech translation apparatus comprising: The speech translation according to claim 1, wherein the attention speaker determination unit determines which speaker is the owner among the speakers, and sets the determined owner as the attention speaker. apparatus. The speech translation apparatus according to claim 1, wherein the attention speaker determination unit records a speaker's utterance order as an utterance history, and uses the immediately-speaker recorded in the utterance history as the attention speaker. The attention speaker determination unit records the number of utterances or the utterance time of the speaker as an utterance history, and the number of utterances or the utterance time within a predetermined time based on the number of utterances or the utterance time recorded in the utterance history. The speech translation apparatus according to claim 1, wherein a speaker having the largest number of voices is a noted speaker. The attention speaker determination unit records the number of utterances or the utterance time of the speaker as an utterance history, and the number of utterances or the utterance time within a predetermined time based on the number of utterances or the utterance time recorded in the utterance history. The speaker who has the most
The language control unit is configured to interview the speaker of interest with the number of utterances or utterance time next to the speaker of interest within the predetermined time based on the number of utterances or utterance time recorded in the utterance history. (1) When the input speech is the attention speaker, the determination language of the attention speaker is set as an input language, and the language of the talker is determined as an output language; (2) When the input voice is the talker, the language of the talker is set as an input language, the determination language of the noticeable talker is set as an output language, and (3) the input voice is also the noticeable talker. 2. The speech translation apparatus according to claim 1, wherein, if the speaker is not the talker, the determination language of the input speech is determined as an input language, and the determination language of the attention speaker is determined as an output language. . The speech translation apparatus according to claim 1, wherein the speech recognition and the translation are not performed when the input language and the output language match. The language control unit, when there are a plurality of output languages, prioritizes the output languages and performs the speech recognition and the translation in the order of the priority. Speech translation device. An audio output unit for outputting the translation result and an image output unit;
The speech translation apparatus according to claim 1, wherein the language control unit selects the speech output unit or the image output unit for each output language and outputs the translation result. A voice storage unit for storing the input voice;
The input speech is stored in the speech storage unit until the language controller cannot determine the input language until the number of utterances of a speaker of the input speech exceeds a predetermined number. The speech translation device described in 1. A recognition result storage unit for storing the result of the speech recognition;
The recognition result is stored in the recognition result storage unit until the number of utterances of speakers other than the speaker of interest reaches a predetermined number or more when the language control unit cannot determine the output language. The speech translation apparatus according to 1. In the speech translation method between three or more speakers,
Input the voice of each speaker,
Analyzing each voice to identify the speaker,
Analyzing each voice to determine the spoken language,
An attention speaker determination unit that determines which speaker is the attention speaker among the speakers; and
Record the correspondence between the identified speaker and the judgment language,
Based on the correspondence and the speaker of interest, (1) when the input speech is the speaker of interest, the determination language of the speaker of interest is the input language and a language other than the input language is output (2) When the input speech is not the target speaker, the determination language of the input speech is determined as the input language, and the determination language of the target speaker is determined as the output language. And
Recognizing the input voice as the input language,
The speech translation method, wherein the speech recognition result is translated from the input language to the output language. In a speech translation program between three or more speakers,
A voice input function for inputting the voice of each speaker;
A speaker identification function for analyzing each voice and identifying a speaker;
A language determination function for analyzing each voice and determining a spoken language;
An attention speaker determination function for determining which speaker is the attention speaker among the speakers;
A speaker language recording function for recording a correspondence relationship between the identified speaker and the determination language;
Based on the correspondence and the speaker of interest, (1) when the input speech is the speaker of interest, the determination language of the speaker of interest is the input language and a language other than the input language is output (2) When the input speech is not the target speaker, the determination language of the input speech is determined as the input language, and the determination language of the target speaker is determined as the output language. Language control function to
A speech recognition function that recognizes the input speech as the input language;
A machine translation function for translating the speech recognition result from the input language to the output language;
A speech translation program characterized by being realized by a computer.

Images