JPH04176244A - Sound information processor - Google Patents

Abstract

PURPOSE:To allow the user to hear only a keyword sound estimated to be in an original sound by extracting at least a pitch structure (pattern) from an object original sound, estimating a position at which a keyword sound exists from output information of a stress information extraction means, copying an estimate keyword sound waveform for the estimated period and storing the sound in the lump. CONSTITUTION:A stress information extraction means 30 of a sound processing means 12 calculates a short-time power at a prescribed interval from a fetched sound data 24, regards a period when the state of the short-time power in excess of a prescribed threshold level is consecutive for a prescribed time or over as the period in which the sound is in existence and obtains a pitch frequency with respect to the sound for the period. When a short period powder at the head of each period is a large power to a degree or over and consecutive for a prescribed time or over, the possibility of a keyword is higher. A period making the sound period before and after the period satisfying the condition above longer by a prescribed length is outputted as an estimate position of the keyword. Then the result made correspondent to the original sound 24 is stored in a storage means 24.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to improvements in voice information processing devices that handle voice signals themselves, such as voice mail systems.

[Conventional technology]

It is said that in the human brain, the thinking center and the voice processing center are almost identical. For this reason, voice expressions are very convenient for users because they can simply say what comes to mind, eliminating the need to re-enter written text using a keyboard.

Therefore, voice information processing devices such as voice mail systems that take advantage of such advantages of voice are being used.

For example, a voice mail system handles the voice signal itself, records the input voice in memory, and plays it back as needed to listen to the contents of the memo. In some cases, the order may be rearranged, a part of the data may be discarded, or a part of the data may be edited based on manual instructions from outside.

Furthermore, a device has been proposed that is associated with a separately input electronic document or the like and used as a voice memo (annotation) or the like.

Now, on the other hand, when it comes to documents written in text, keywords are usually visible just by looking at them, and it is possible to judge whether or not the document is of interest. On the other hand, due to the characteristics of audio, it is impossible to judge whether it is important or not unless you listen to all of the audio in sequence. Therefore, with voice mails and voice memos, it is not easy to know whether each message is something you should listen to, or whether you can ignore or postpone it. Even for long content consisting of audio sentences as described above, one has no choice but to listen to the entire thing, which is a waste of time, and moreover, one has no choice but to spend most of the time in real time.

As a method to solve this problem, a method has been proposed in which, in addition to the target voice, subject information (keywords, etc.) is input separately as text information from the keyboard of a data terminal, etc., as in Document (1) JP-A-2-2022.58. However, it requires a separate data terminal keyboard, etc., and cannot be used for voice sent by simple input such as from a telephone, and the person inputting the information must also enter it as text information. There was.

[Problem to be solved by the invention]

The purpose of the present invention is to perform advanced processing that reaches the language level, such as speech recognition and understanding, without inputting subject information (keywords, etc.) separately from the keyboard of a data terminal as text information in addition to the target speech. It is an object of the present invention to provide an easy-to-use audio information processing device that improves these drawbacks by using a signal level processing function without having to perform the above operations.

[Means to solve the problem]

To achieve this objective, the present invention provides means for extracting intonation information, means for estimating the location of the keyword speech (one or more), and estimating the interval of the estimated keyword speech (one or more). means for copying and collectively storing one or more of the estimated keyword speech waveforms in the estimated section; means for associating the original speech with the stored estimated keyword speech waveform; and reading out the copied and stored estimated keyword speech waveform. Means for outputting and reproducing as audio 2
and means for reading out the original audio waveform and outputting and reproducing it as audio.

[Effect]

As can be seen by comparing long sentences produced by early synthetic speech with the same sentences spoken by humans, synthetic speech can be used to express important words (
Keywords) and other words are spoken in the same tone, so you have to listen carefully to all the sounds in order to understand what they are saying, whereas the human voice is likely to be doing something else to some degree. I can understand it. This is not only due to the lack of phonological clarity in synthesized speech, but also because in the human voice, information indicating the important parts is provided, whereas in early synthetic speech, information is uniform throughout. This is because it is synthesized into

Information about the important parts of a person's voice is mainly given in the structure of intonation. Intonation here is also called intonation, and is the structure of temporal changes in the pitch of the voice, the structure of temporal changes in the volume of the voice, and the temporal structure of each phoneme that makes up the voice. Refers to the structure of temporal changes (such as duration).

When uttering important words, as a result of human's natural and unconscious psychology, the parts are slow (temporal structure), loud (temporal change structure of loudness), and high (temporal change structure of pitch). ) observed to be uttered vocally (
Literature (2) “Syntax estimation using prosodic information and conversational speech understanding method using word spots” Komatsu, Idai, Ichikawa,
IEICE Transactions, J71-D, 7, p p
1218-1228, (3) "Sentence structure estimation method for spoken conversation sentences using prosodic information" Daita, Komatsu, Ichikawa, IEICE Transactions A, J72-A, 1, pp, 23-31,
etc.). As shown in the above documents, important word parts have at least one of these characteristics. In particular, the pitch structure (bang) plays a major role. Furthermore, as shown in the above-mentioned document, it is possible to automatically estimate the positions of important words.

The present invention actively utilizes the characteristics of voice described above. Keeping in mind the characteristics of voice explained above,
The operation of each means will be explained below.

The means for extracting intonation information has a function of extracting at least pitch structure (bang) from the target original speech.

The keyword existence position estimating means has one keyword voice from the output information of the intonation information extracting means.

The means for estimating the estimated keyword speech (one or more) section estimates the estimated keyword speech (one or more) section based on the output information of the keyword speech presence position estimation means.

The copy storage means copies and collectively stores one or more estimated keyword voice waveforms of the sections estimated by the estimated keyword voice section estimation means.

The associating means associates the target original speech with the copied and stored estimated keyword speech waveform.

The keyword 1 audio waveform output reproduction means has a function of reading out the copied and stored estimated keyword audio waveform and outputting and reproducing it as audio.

The original audio waveform output reproduction means has a function of reading out the target original audio waveform based on the information possessed by the association means and outputting and reproducing it as audio.

A person using the device according to the present invention can listen to only the keyword voice estimated to be included in the original voice by the keyword voice waveform output reproduction means, and can listen to only the keyword voice estimated to be included in the original voice, and from the outputted keywords connected or more. If it is determined that the entire original audio should be heard, it becomes possible to listen to it using the function of the original audio waveform output reproduction means.

In addition, if it is determined that it is not necessary to listen to the entire original audio, it is sufficient to move on to the operation of listening to the keyword audio that is estimated to be included in the next (different) original audio as necessary, and the processing can proceed efficiently. becomes possible.

〔Example〕 An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram illustrating an embodiment of an audio information processing device according to the present invention.

In this embodiment, the hardware configuration includes an information processing means 1 that controls the entire system and creates an application system for the user, and a means 2 that stores various data used for the information processing and the application system. , means 5 for selecting an input from an audio input means such as a microphone 3 or a telephone 4; and means 6 for amplifying and frequency filtering the analog audio signal from the selection means;
an encoding means 7 for sampling the output signal of the means and converting it into a digital quantity; a decoding means 8 for converting the audio signal expressed in the digital quantity into an analog audio signal;
means 9 for amplifying and frequency filtering the output signal of the means; means for selecting an audio output means such as a speaker 10 or a telephone]-1; and an audio processing means for controlling audio input/output and processing audio information. 12, an audio processing storage means 13 for storing audio signals etc. expressed in digital quantities at -1, and means 14 for storing audio processing conditions provided in the audio processing storage means 13. Means for storing management information of a plurality of audio data 15. Means 16 for storing audio reproduction conditions, and analysis means 17 for analyzing the output signal of the means 6 for amplifying and frequency filtering the analog audio signal and outputting the loudness of the sound and the voiced/unvoiced determination results at regular intervals.
means 18 (
Memory access conflict prevention means 18).

an interface section 21 consisting of an access means 19 from the voice processing means 12 etc. to the information processing means 1 or the storage means 2 etc., an access means 20 from the information processing means 1 to the voice processing means 12 etc., and a network. The correspondence between the communication means 23° audio data 24 and the extracted keyword audio 25° consists of a file 27 in which information 26 is recorded. Further, each means is connected by a main bus 28 or an audio bus 29 consisting of data including audio data, address, and control lines. Note that the means for performing audio processing will be collectively referred to as the audio processing system 22.

Further, it is assumed that the information processing means 1 and the audio processing means 12 are constituted by general microprocessors.

In addition, in this embodiment, each processing means that is the core of the present invention is realized as a software unit that mainly uses the above-mentioned hardware, and is realized by the above-mentioned information processing means 1 and audio processing means 12, which are composed of general microprocessors. ing.

Means for extracting intonation information 30. Means for estimating the location of keyword sounds (one or more) 31. Means for estimating the estimated keyword audio (one or more) sections 32. Each piece of software constituting the means 33 for copying and collectively storing one or more estimated keyword speech waveforms in the estimated section is mainly used in the speech processing means 12, and also for copying the original speech and the stored estimated keyword speech waveform. Means for associating 34. The software constituting the means 35 for reading out and reproducing the output of the estimated keyword speech waveform and the means 36 for reading out the original speech waveform and reproducing the output as speech are mainly composed of the information processing means 1 and the storage means 2 described in "II". It is configured as part of the system's control software.

The operation of this embodiment will be explained below with reference to the drawings.

The audio that has arrived via the communication means 23 is stored in the file 27 as audio data 24. In the following embodiment, it is assumed that the audio is digitized. It goes without saying that analog formats can be easily digitized by passing them through a normal analog/digital converter, so this premise does not limit the invention in any way. In addition to the communication means 23, the audio input path includes a means 6 for amplifying and frequency filtering analog audio signals from voice input means such as a microphone 3 and a telephone 4, and a means 6 for sampling the output signals of the means and converting them into digital quantities. Encoding means for converting into 7. Access means 1 in the interface section 21
9 and may be stored as audio data 24 in file 27.

The following processing is automatically started when the audio data 24 is stored in the file 27, and is configured to be processed in batch.In the case of normal use such as reading out the audio data 24 at a later time, The keyword audio has already been extracted and is configured so that it can be used immediately (if the audio data 24 is read out immediately after storage is completed, the keyword audio has not been extracted yet).
.

Of course, there is also a function for configuring the process to be activated according to instructions from the outside. As to which of the two is to be operated, an instruction may be set in the means 14 for storing audio processing conditions.

The intonation information extraction means 30 on the speech processing means 12 is a means 14.3 for storing speech processing conditions provided in the speech processing storage means 1.3 according to control information by the information processing means 1 which controls the entire system. The audio data 24 in the file 27 is taken in according to the conditions set in the means 15 for storing management information of a plurality of audio data, and the intonation of the audio data 24 is analyzed.

An example of intonation information extraction processing in the intonation information extraction means 30 will be briefly explained using FIGS. 2 and 3. For details, please refer to the above-mentioned documents (2) and (3), which are reported in detail.

In Figure 2, the short-term power of the captured audio data 24 is calculated at fixed intervals (usually 10 to 20 milliseconds), and the state in which the short-term power exceeds a certain threshold continues for a certain period of time or more. It is assumed that audio is present in the interval, and the pitch frequency is determined for the audio in that interval. The determination conditions are set in a means 14 for storing conditions for voice processing provided in the memory means 13 for voice processing. Note that there are many widely known methods for determining the short-time power, pitch frequency, and voice interval, and the details thereof will be omitted since it is not related to the essence of the present invention. For example, document (4) "Speech Recognition" by Niimi, Kyoritsu Shuppan,
Learn more about Information Science Course E.19.3.

Hereinafter, the shape of the fundamental frequency will be determined using a set of broken lines according to the flowchart in FIG. When the average difference between the approximate straight line and the analyzed value is more than a certain value, it is assumed that it consists of multiple utterance sections, and the position with the maximum error is divided as a dividing point candidate, and the sections before and after it are designated as sections A and B. In the figure, the coupling coefficient Ri
(A-B) is obtained, and when the value satisfies a constant value condition, each of them is regarded as a new section and the same process is repeated. The meaning of each variable for calculating the coupling coefficient is explained in the third section.
As shown in the figure.

Next, the processing of the keyword voice presence position estimating means 31 will be explained.

When the division has progressed to approximately the level of phrases and it is judged that there is no need to divide any more, the division is stopped. In many cases, a bunsetsu consists of an independent word and multiple attached words, so it can be assumed that the probability that a keyword exists at the beginning of each section is high.

Furthermore, since it is natural for important words to be uttered loudly and slowly, if the short-term power at the beginning of each section is at least a certain value and continues for a certain period of time, then the keyword The probability that this is the case becomes higher. The audio sections before and after the section that satisfies these conditions are set to a certain length (20
The section lengthened by approximately 0 to 400 milliseconds is output as the estimated position of the keyword.

Next, the processing of the keyword voice section estimating means 32 will be explained with reference to FIG. In this embodiment, the keyword speech interval estimating means 32 uses a well-known continuous dynamic
Programming (DP) matching method is used. The continuous DP matching method is also described in detail in the above-mentioned document (4), and can be easily realized by referring to the above-mentioned document (4) for details.

The keyword voice section estimating means 32 converts the section whose position has been estimated by the keyword voice presence position estimating means 31 into a continuous DP.
Regarded as the standard bang of the matching method, audio data 24
Matching is performed continuously by regarding the audio as input. In this case, the well-known LPC system (also described in the above-mentioned document (4)) is used as the voice feature parameter for matching.
(described in detail in ) are calculated and determined by the program. At this time, for example, a function for recording the matching path as shown in Document (5) JP-A-57-207297 can be added to each analysis position on the standard button (the keyword voice presence estimation section) side. A register is prepared to record the number of matchings for each match. At the same time, if the number of times in which the degree of matching quality is a certain value or more is greater than a certain value when matching with the entire audio data r17) 24 is completed, it is included in the keyword audio presence estimation interval. The possibility that the voice that appears is the keyword voice becomes even higher. At this time, the section where the value of the register that records the number of matchings for each analysis position, which is prepared at each analysis position on the keyword voice presence estimation section side, is a certain value or more may be a stable part as a keyword voice. would also be extremely high. This section is defined as a keyword voice section. Such processing is executed for all the keyword voice presence estimation sections.

In addition, when outputting the keyword voice as voice by the keyword voice waveform output reproducing means described later, a weight that gradually increases monotonically from 0 to 1 before and after such an interval,
By adding and outputting a waveform multiplied by a weight that monotonically decreases from 1 to 0 (fade-in, fade-out function), the influence of errors in section estimation can be reduced.

The keyword voice section information obtained for all the keyword voice presence estimation sections is the original voice and the stored estimated keyword, which is configured as part of the system control software on the information processing means 1 and storage means 2. The storage means 2 is associated with the original audio 24 according to a predetermined format by the audio waveform associating means 34.
recorded above. The configuration of the program that executes these processes is extremely simple, and there is no need to describe how to implement it.

The keyword audio waveform output reproducing means uses a simple program configured on the information processing means 1 and the storage means 2 as part of the control software of the system to generate the keyword audio section information recorded on the storage means 2. is used to add the aforementioned fade-in and fade-out functions to the audio of the corresponding section of the original audio 24 recorded on the storage means 2 and read it out, converting the audio signal expressed in digital quantity into an analog audio signal. a decoding means 8 for converting;
The output signal of the means is amplified, passes through means 9 for frequency filtering, and is output as sound from an audio output means selected by means 11 for selecting an audio output means such as a speaker 1o or a telephone.

It goes without saying that the weighting process used for the fade-in and fade-out functions described above can be easily realized on the audio processing means 12.

The means for outputting the original audio waveform 24 similarly outputs a plurality of audio data recorded on the storage means 15 of the storage means 17 according to a program configured on the information processing means 1 as part of the system control software. Using the management information of
A decoding means 8 reads out the original audio 24 recorded on the storage means 2 and converts the audio signal expressed in digital quantity into an analog audio signal, and amplifies the output signal of the means and performs frequency filtering. Means 1 for selecting a sound output means such as a speaker 10 or a telephone via means 9 for selecting
This is a series of processes for outputting audio from the audio output means selected in step 1.

〔Effect of the invention〕

As explained above, according to the present invention, it is possible to listen to only the keyword voice that is estimated to be included in the original voice.
From the estimated keywords that have been output or more,
If it is determined that the entire original audio should be heard, it becomes possible to listen to it using the function of the original audio waveform output reproduction means. In addition, if it is determined that there is no need to listen to the entire original audio, it is sufficient to move on to listening to the keyword audio that is estimated to be contained in another original audio as necessary, and the content of the original audio can be estimated in a short time. This makes it possible to proceed with processing that uses audio efficiently.

[Brief explanation of the drawing]

FIG. 1 is a diagram for explaining the configuration of an embodiment of the present invention, FIG. 2 is a flowchart for explaining the processing of the intonation information extraction section, and FIG. 3 is a diagram for explaining the symbols of the formula used in the intonation information extraction process. figure,
FIG. 4 is a flowchart illustrating the keyword voice segment estimation process. 1... Information processing means, 2... Storage means, 3... Microphone, 4... Telephone, 5... Voice input means selection, 6
...Analog audio signal amplification, frequency filtering means, 7° analog/digital conversion and quantification means, 8.Digital/analog signal conversion and decoding means, 9.Amplification, frequency filtering means, 10.Speaker, ]1.
. . . Audio output means selection means, 12 . . Audio input/output control audio processing means, ], 3 . . . Audio processing storage means, 14
. . . Audio processing condition storage means, 1.5 . . . Multiple audio data management information storage means, 16 . . . Audio reproduction condition storage means, 17 . 19... Access means, 20... Access means, 21... Interface section, 22... Audio processing system, 23... Network communication means, 24... Audio data, 25... Keyword audio, 26...・Correspondence is information, 27...File, 28...Main bus, 29.
...Voice bus, 30...Intonation information extraction means, 31...
・Keyword voice existence position estimation means, 32... Estimated keyword voice interval estimation means, 33... Estimated keyword voice waveform copy storage means, 34... Means for associating the estimated keyword voice with original voice, 35... Voice Output playback means, 3
6...Original audio waveform output reproducing means.

Claims (1)

[Claims] 1. Means for extracting intonation information of the original voice; means for estimating the position where one or more keyword voices exist;
means for estimating the one or more estimated keyword voice sections; means for copying and collectively storing the one or more estimated keyword voice waveforms in the estimated sections; and the original voice and the stored estimated keyword voice. An audio information processing device comprising at least a means for associating a waveform with a waveform, a means for outputting and reproducing the copied and stored estimated keyword audio waveform as read audio, and a means for outputting and reproducing the original audio waveform as a read audio. 2. The audio information processing apparatus according to claim 1, wherein keyword audio extraction processing is started upon completion of inputting the original audio. 3. The audio information processing apparatus according to claim 1, wherein when the keyword audio extraction instruction is set, the keyword audio extraction process is started upon completion of the input of the original audio. 4. The audio information processing device according to claim 1, wherein the extracted keyword audio is provided with a window and a fade-in/fade-out function. 5. The audio information processing device according to claim 1, wherein the keyword audio is estimated using a continuous DP method.